On-Road Evaluation of the SAVE-IT Vehicle Prototype \--

Task 14C Final Report for the

SAfety VEhicles using adaptive Interface Technology (SAVE-IT) Project

Volume 1

**Prepared by**

**University of Michigan Transportation Research Institute**

**for**

**Delphi Corporation**

**and the USDOT Volpe National Transportation Systems Center**

**February 15, 2008**

#  {#section .SaveIT-H1}

**AUTHORS**

The authors of this report are David LeBlanc, David W. Eby, Zevi
Bareket, and Jonathon M. Vivoda of the University of Michigan
Transportation Research Institute.

TABLE OF CONTENTS

[EXECUTIVE SUMMARY 7](#executive-summary)

[14.0 Program Overview 9](#program-overview)

[14.1 Introduction 10](#introduction)

[14.1.1 Objectives 10](#objectives)

[14.1.2 Overview of Work in Context of Project
11](#overview-of-work-in-context-of-project)

[14.1.3 Organization of Report 11](#organization-of-report)

[14.2 SAVE-IT Prototype System Description
12](#save-it-prototype-system-description)

[14.2.1 Baseline (Non-Adaptive) Collision Warning System
13](#baseline-non-adaptive-collision-warning-system)

[14.2.2 Adaptive Collision Warning System
15](#adaptive-collision-warning-system)

[14.2.3 Distraction Alert 17](#distraction-alert)

[14.2.4 Distraction Mitigation System
17](#distraction-mitigation-system)

[14.3 Method 20](#method)

[14.3.1 Overview 20](#overview)

[14.3.3 Experimental Data Archive 22](#experimental-data-archive)

[14.3.2 Recruitment 22](#recruitment)

[14.3.3 Experiments on the Track 23](#experiments-on-the-track)

[14.3.3.1 Track Site and Surrogate Target
25](#track-site-and-surrogate-target)

[14.3.3.2 Vehicle Instrumentation 26](#vehicle-instrumentation)

[14.3.4 Experimental Method for On-Road Testing
29](#experimental-method-for-on-road-testing)

[14.4 Results 31](#results)

[14.4.1 Potential Safety Impacts of the Adaptive Mechanisms
31](#potential-safety-impacts-of-the-adaptive-mechanisms)

[14.4.1.1 FCW Surprise Trial Conduct 31](#fcw-surprise-trial-conduct)

[14.4.1.1 Results of Track Experiments
33](#results-of-track-experiments)

[14.4.1.2 Results from On-Road Observations
36](#results-from-on-road-observations)

[14.4.1.3 Head Pose as a Surrogate for Attention: Analyzing FOT Data
49](#head-pose-as-a-surrogate-for-attention-analyzing-fot-data)

[14.4.2 Acceptance of the Adaptive Mechanisms for Crash Warning Systems
63](#acceptance-of-the-adaptive-mechanisms-for-crash-warning-systems)

[14.4.2.1 Acceptance on the Van der Laan Scale
63](#acceptance-on-the-van-der-laan-scale)

[14.4.2.2 Post-Drive Questionnaire Responses
65](#post-drive-questionnaire-responses)

[14.4.2.2 Video Review 81](#video-review)

[14.4.2.3 Final Questions Regarding Adaptive and Non-Adaptive Crash
Warning Systems
84](#final-questions-regarding-adaptive-and-non-adaptive-crash-warning-systems)

[14.4.3 Acceptance of Adaptive Mechanisms for Distraction Mitigation
86](#acceptance-of-adaptive-mechanisms-for-distraction-mitigation)

[14.5 Discussion of Results 89](#discussion-of-results)

[14.5.1 Overview 89](#overview-1)

[14.5.2 Safety and Performance Features of the Adaptive Crash Warning
System
90](#safety-and-performance-features-of-the-adaptive-crash-warning-system)

[14.5.2.1 Lane Departure Warning 90](#lane-departure-warning)

[14.5.2.2 Forward Crash Warning 93](#forward-crash-warning)

[14.5.3 Acceptance of the Adaptive Features of the Crash Warning Systems
96](#acceptance-of-the-adaptive-features-of-the-crash-warning-systems)

[14.5.4 Driver Acceptance of Adapting Available IVIS Features with the
Distraction Mitigation System
98](#driver-acceptance-of-adapting-available-ivis-features-with-the-distraction-mitigation-system)

[14.6 References 99](#references)

[14.7 Acknowledgments 100](#acknowledgments)

[14.8 Appendix A: Pre-Drive Questionnaire and Driver Responses **Error!
Bookmark not defined.**](#__RefHeading___Toc190855316)

[14.9 Appendix B: Case Study Table **Error! Bookmark not
defined.**](#__RefHeading___Toc190855317)

[14.10 Appendix C: Informed Consent Forms **Error! Bookmark not
defined.**](#__RefHeading___Toc190855318)

[14.10.1 Informed Consent for Test-Track Trials to Assess Driver
Interface Technology **Error! Bookmark not
defined.**](#__RefHeading___Toc190855319)

[14.10.2 Informed Consent for On-the-Road Trials to Assess Driver
Interface Technology **Error! Bookmark not
defined.**](#__RefHeading___Toc190855320)

[14.10 Appendix D: Procedures for Test Track Subject Running **Error!
Bookmark not defined.**](#__RefHeading___Toc190855321)

[14.12 Appendix E: Dana Test Track **Error! Bookmark not
defined.**](#__RefHeading___Toc190855322)

[14.13 Appendix F: List of Recorded Data Signals and Other Data
Acquisition Details **Error! Bookmark not
defined.**](#__RefHeading___Toc190855323)

[14.13.1 List of Recorded Data Signals **Error! Bookmark not
defined.**](#__RefHeading___Toc190855324)

[14.13.2 Details of Video Recording Approach **Error! Bookmark not
defined.**](#__RefHeading___Toc190855325)

[14.13.3 Details of Audio Recording Approach **Error! Bookmark not
defined.**](#__RefHeading___Toc190855326)

[14.14 Appendix G: Procedures for On-the-Road Subject Running **Error!
Bookmark not defined.**](#__RefHeading___Toc190855327)

[14.15 Appendix H: Post-Track-Testing Questionnaire **Error! Bookmark
not defined.**](#__RefHeading___Toc190855328)

[14.16 Appendix I: Post-Road-Testing Questionnaire **Error! Bookmark not
defined.**](#__RefHeading___Toc190855329)

[14.16.1 Questionnaire for Non-Adaptive and Adaptive LDW Systems
**Error! Bookmark not defined.**](#__RefHeading___Toc190855330)

[14.16.2 Questionnaire for Non-Adaptive and Adaptive FCW Systems
**Error! Bookmark not defined.**](#__RefHeading___Toc190855331)

[14.17 Appendix J: Driver Responses to Post-Drive Questionnaires
**Error! Bookmark not defined.**](#__RefHeading___Toc190855332)

[14.18 Appendix K: Drivers' Ratings of Alert Utility Based on Reviewing
Video Clips **Error! Bookmark not
defined.**](#__RefHeading___Toc190855333)

[Script: **Error! Bookmark not defined.**](#__RefHeading___Toc190855334)

[14.18.1 Analysis of Responses **Error! Bookmark not
defined.**](#__RefHeading___Toc190855335)

[14.19 Appendix L: Drivers' Responses to Final General Questions
Regarding Adaptive and Non-Adaptive Crash Warning Systems **Error!
Bookmark not defined.**](#__RefHeading___Toc190855336)

[14.20 Appendix M: Willingness to Engage in IVIS Tasks during Driving:
Cumulative Plots of Driver Ratings **Error! Bookmark not
defined.**](#__RefHeading___Toc190855337)

[14.21 Appendix N: Additional On-Road FCW Alerts Introduced by the
Adaptive Mechanisms **Error! Bookmark not
defined.**](#__RefHeading___Toc190855338)

[14.22 Appendix O: On-Road FCW Alerts during Adaptive-Mode Driving That
Are Common to the Non-Adaptive System **Error! Bookmark not
defined.**](#__RefHeading___Toc190855339)

[Figure 14.41. Subject 121, 0.8 seconds earlier **Error! Bookmark not
defined.**](#__RefHeading___Toc190855340)

# EXECUTIVE SUMMARY {#executive-summary .SaveIT-H1}

Driver distraction is a major contributing factor to automobile crashes
in the U.S. The problem of driver distraction is expected to grow as the
number and complexity of in-vehicle information systems and nomadic
devices brought into the vehicle increases rapidly. This report
documents an evaluation of a vehicle prototype that uses real-time
measurements of driver head pose to modulate lane departure warning
(LDW) and forward crash warning (FCW) decisions. The system also uses
real-time estimates of the demands of driving to allow, advise against,
or prohibit driver interactions with certain in-vehicle information
system tasks. These adaptive features are collectively known as the
SAVE-IT system (SAfety VEhicle(s) using adaptive Interface Technology).

The evaluation investigates driver acceptance of these adaptive
features, potential safety implications, and reports on real-world
system performance. This is done using data collected from experiments
with 26 drivers on a closed-course track, 2,000 miles of on-road driving
by 12 of those drivers on public roads, analysis of existing field
operational test data to overlay the adaptive mechanisms on an
independent set of data, and a series of subjective instruments.

Test-track experiments in which drivers were surprised by lead vehicle
braking indicate that the SAVE-IT mechanisms may provide inattentive
drivers with additional time to respond in forward-crash scenarios.
On-road and field operational test data show a dramatic reduction in the
overall alert rate associated with LDW (88 percent reduction or more)
and FCW (60 to 70 percent reduction) systems. These data indicates there
will be false positives (unnecessary alerts) and likely false negatives
(unduly delayed alerts for FCW) with the system. A major contributor to
false positives is the challenge of tracking the drivers' head poses
during times when they are moving their head back and forth from the
forward scene. False negatives are a larger concern for safety; these
are likely inevitable given the current state of the art in detecting
driver attention.

Drivers were found to be largely accepting of the adaptive features,
although there was no clear preference for adaptive over non-adaptive
forms when they were directly asked that question. However, there were
six areas where drivers favored one or both adaptive crash warning
systems to the non-adaptive form, whereas there was only one area where
the reverse was true. The limited number of drivers in the experiment
and their limited exposure to the SAVE-IT vehicle may have reduced the
significance of these findings.

The distraction mitigation system's approach to advising against, or
preventing, driver use of certain features in on-road driving was found
to be quite compatible with drivers' view of their own sense of safety
with performing those tasks at that same moment. Finally, a second
system that detects drowsiness or micro-sleep may be advisable with the
current SAVE-IT design to ensure that LDW alerts are not suppressed when
a drowsy driver is drifting over a lane boundary.

# 14.0 Program Overview {#program-overview .SaveIT-H1}

Driver distraction is a major contributing factor to automobile crashes.
The National Highway Traffic Safety Administration (NHTSA) has estimated
that approximately 25 percent of crashes are attributed to driver
distraction and inattention (Wang, Knipling, and Goodman, 1996). The
issue of driver distraction may become worse in the next few years as
more electronic devices (e.g., cell phones, navigation systems, and
wireless Internet and email devices) are brought into vehicles. In
response to this situation, the John A. Volpe National Transportation
Systems Center (VNTSC), in support of NHTSA\'s Office of Vehicle Safety
Research, awarded a contract to Delphi Electronics & Safety to develop,
demonstrate, and evaluate the potential safety benefits of adaptive
interface technologies that manage the information from various
in-vehicle systems based on real-time monitoring of the roadway
conditions and the driver\'s capabilities. The contract, known as SAfety
VEhicle(s) using adaptive Interface Technology (SAVE-IT), is designed to
mitigate distraction with effective countermeasures and to enhance the
effectiveness of safety warning systems.

The SAVE-IT program serves several important objectives. Perhaps the
most important objective is demonstrating a viable proof of concept that
is capable of reducing distraction-related crashes and enhancing the
effectiveness of safety warning systems. Program success is dependent on
integrated closed-loop principles that not only include sophisticated
telematics, mobile office, entertainment, and safety warning systems,
but also incorporate the state of the driver. This closed-loop vehicle
environment will be achieved by estimating the driver's state, assessing
the situational threat, prioritizing information presentation, providing
adaptive countermeasures to minimize distraction, and optimizing
advanced collision warnings. The SAVE-IT project includes research,
design, and evaluation phases. This report addresses evaluation of a
SAVE-IT prototype implementation onboard a vehicle.

# 14.1 Introduction {#introduction .SaveIT-H1}

This report describes the motives, methodology, and results of an
evaluation of a SAVE-IT prototype system implemented onboard a vehicle
platform. The main adaptive features of the SAVE-IT prototype are:

-   An adaptive crash warning system, comprised of logic imbedded in
    lane departure warning and forward crash warning systems, that uses
    the driver's head pose to sometimes suppress or delay alerts, or
    present the alerts earlier. The adaptive mechanism considers whether
    the driver is attentive to the driving task, which in turn is
    estimated through real-time tracking of driver head pose.

-   A distraction mitigation system, which is an adaptive feature that
    modulates the driver's ability to use certain features within an
    in-vehicle information system (IVIS), in order to mitigate the
    distraction that may result from that use. This adaptation is done
    by monitoring aspects of the current roadway, traffic, and/or
    environmental conditions, and thereby computing an overall demand on
    the driver to safely perform the current driving task.

This in-vehicle evaluation effort involves recruiting drivers from the
general public and putting them behind the wheel of the SAVE-IT vehicle
prototype, both on the test track and on public roads. Drivers are
exposed to both a baseline version of the crash warning systems as well
as the adaptive SAVE-IT version. Furthermore, their perceptions are
gathered regarding key features of the distraction mitigation system.
The vehicle itself is instrumented in order to monitor the driving
events, the system performance, and driver actions. Furthermore, an
extensive set of subjective instruments including questionnaires,
interviews, and driver review of video is used to gather subjective
feedback concerning the adaptive mechanisms.

## 14.1.1 Objectives {#objectives .SaveIT-H2}

The objectives of this in-vehicle evaluation are three-fold:

-   to assess the impact of the adaptive SAVE-IT mechanisms on driver
    acceptance

-   to look for effects of the SAVE-IT system that have the potential to
    impact the safety of the driver, relative to non-adaptive forms of
    crash warning and IVIS systems

-   to gain insight into the potential of the adaptive systems,
    especially through observations of the driver-system performance in
    the field and in post-hoc analysis of previously-collected field
    operational test data

## 14.1.2 Overview of Work in Context of Project {#overview-of-work-in-context-of-project .SaveIT-H2}

This document describes the in-vehicle portion of the evaluation phase
of the SAVE-IT project. Other evaluation tasks that use moving-base
driving simulators with the SAVE-IT system are reported elsewhere,
including separate work activities done at the University of Iowa and at
the Ford Motor Company. The approaches used in the in-vehicle and
simulator portions of the evaluation were designed to complement one
another with each taking advantage of the strengths of the respective
environments. In-vehicle testing may be the most useful environment for
studying driver acceptance, since drivers are able to spend more time in
the vehicle and the environment is more natural to them. Vehicle testing
also allows insight into the real-world performance of the system, such
as the impact of the adaptive mechanisms on the rate of nuisance alerts
as well as missed alerts. Simulator activity is ideally suited for
studying some aspects of safety impact, since drivers can be placed in
virtual situations that allow study of driver-system interaction in
near-crash situations. Simulators can also be more efficient
environments for studying distraction in driving.

The results and conclusions of the in-vehicle evaluation are reported
here as a separate set of findings, without reference to the other
simulator-based testing activities. The final SAVE-IT report will
assimilate the findings of both the in-vehicle and the simulator
evaluation efforts to provide an overall understanding of the promise
and challenges of these adaptive interface concepts.

## 14.1.3 Organization of Report {#organization-of-report .SaveIT-H2}

This report first provides a description in Section 14.2 of the SAVE-IT
prototype system, as implemented onboard a vehicle testbed. The
experimental design and method for both test-track and on-road testing
are presented in Section 14.3. The results and findings of individual
testing activities and studies are presented in Section 14.4. These
results are discussed and condensed into a set of findings in Section
14.5. Citations for references and an extensive set of appendices
follow.

# 14.2 SAVE-IT Prototype System Description {#save-it-prototype-system-description .SaveIT-H1}

This section describes the system that was evaluated using in-vehicle
testing. The SAVE-IT prototype system was installed on a 2002 Buick
LeSabre test vehicle (see Figure 14.1), and consisted of the following
functionalities:

-   An integrated crash warning system consisting of a lane departure
    warning (LDW) system and a forward crash warning (FCW) system, as
    described in Sections 2.1 and 2.2

-   A distraction alert to assist the driver in avoiding long periods of
    visual distraction while driving (Section 2.3)

-   A distraction mitigation system (DMS) to help the driver responsibly
    manage the use of the in-vehicle information system (IVIS), as
    presented in Section 2.3

-   An IVIS installed in the vehicle platform especially for this
    project with functions intended to represent interactive telematics
    devices expected to become available in the next several years

The following subsections address each of these components to provide
context for the experimental methods and findings in subsequent
sections.

![](media/image1.jpeg){width="5.482638888888889in"
height="3.1256944444444446in"}

Figure 14.1. The SAVE-IT vehicle prototype

The functions evaluated in the in-vehicle portion of evaluation
activities are essentially identical to those evaluated in simulators by
other organizations. The SAVE-IT systems were developed and integrated
into the prototype vehicle by the Delphi Corporation. This section is
not intended to be a technical specification or detailed description of
the system, but rather to provide enough description to support this
report on in-vehicle evaluation.

## 14.2.1 Baseline (Non-Adaptive) Collision Warning System {#baseline-non-adaptive-collision-warning-system .SaveIT-H2}

The collision warning system operates in either non-adaptive or adaptive
mode. The difference is that the adaptive mode includes the use of
real-time measurement of the driver's head pose (azimuth angle) to
suppress alerts or provide them earlier or later than alerts provided in
non-adaptive mode. The non-adaptive mode does not use head pose in its
decisions, and so is representative of most systems that are currently
on the market or have been the subject of previous and ongoing research
by the U.S. DOT and others (Ervin et al., 2005, LeBlanc et al., 2006).

Both the non-adaptive and adaptive systems consist of LDW and FCW
systems. For the non-adaptive mode, the lane departure warning system is
intended to provide alerts to help the driver avoid unintentionally
leaving the lane. The LDW alert in the SAVE-IT prototype is issued when
the vehicle crosses a perceived lane boundary without a turn signal
being activated. LDW alerts are suppressed for speeds less than 45 mph.
LDW uses computer vision to track painted lane markings and other
persisting visual features, such as pavement edges, curb cuts, and other
features. LDW presents alerts when the tires are at, or less than
several centimeters beyond, the lane boundary. The alerts given by LDW
include a haptic vibration on the seat, a non-directional audio tone
from speakers mounted on the B-pillar (just behind the driver), and a
flashing red spot on the windshield. The red spot is the reflection of
an LED mounted within the dash (the LED itself was out of the driver's
view). Figure 14.2 shows the approximate location and relative size of
the projected flashing red spot when an alert is issued. Table 14.1
summarizes the driver cues associated with the alert, as well as those
associated with FCW and the distraction alert.

![](media/image2.wmf){width="3.845833333333333in"
height="1.854861111111111in"}

Figure 14.2. Visual flashing alert (image from Delphi Corporation)

The haptic seat was a bottom-and-back cushion seat cover installed over
the OEM driver's seat (see Figure 14.3). Several vibration actuators
positioned within the seat provided the localized sensory input to the
driver.

![](media/image3.png){width="2.2131944444444445in"
height="2.8826388888888888in"}

Figure 14.3. Haptic seat (image from Delphi Corporation)

Table 14.1 Driver displays for the SAVE-IT prototype vehicle

  --------------- ------------------- ------------------- ----------------
                  **Audio cues**      **Visual cues**     **Haptic cues**

  Lane departure  Emulation of rumble Short-duration      Directional
  warning         strip from speakers flashing red spot   haptic vibration
                  on B-pillar         on windshield       on driver seat
                  (non-directional)                       pan

  Forward crash   Short tone sequence Same as LDW         None
  warning         from forward                            
                  speakers                                

  Distraction     None                Same as LDW, except None
  alert                               amber color         
  --------------- ------------------- ------------------- ----------------

The FCW system is intended to provide alerts to help drivers avoid
colliding with the rear end of same-direction vehicles. The FCW on the
SAVE-IT prototype was designed primarily to support the experiment, so
that an emphasis was to provide an adequate number of alerts. To this
end, the SAVE-IT developers provide alerts such that if the driver
applies a step input of deceleration (of assumed magnitude) after an
assumed response time, then the vehicle bumpers would just touch
(assuming the lead vehicle deceleration remains constant). See Figure
14.4 for a simplified algorithm; see Brunson et al. (2002) and LeBlanc
et al. (2001) for more information about this class of FCW algorithms.

> FCW alert is to be issued when the range becomes less than a threshold
> alert range.
>
> Threshold alert range is defined as the minimum distance between the
> vehicles under current speed and acceleration conditions, such that
> the range goes to zero exactly when the closing speed goes to zero.
>
> Assume lead vehicle deceleration will remain constant, unless the lead
> vehicle is expected to come to rest, in which case it remains at rest.
>
> Non-adaptive FCW assumptions:
>
> 1\. Assumed driver deceleration response to an alert is a step input
> of\
> approximately 0.5 g.

2.  Delay time, DT = 2.6 sec (for driver not currently braking), or 0.6
    sec (driver is braking).

Figure 14.4. Basic alert timing algorithm for non-adaptive FCW

The SAVE-IT FCW algorithm is intentionally relatively early. This allows
drivers to experience FCW alerts in the limited on-road testing that is
reported here. The SAVE-IT FCW does not produce alerts when vehicle
speed is less than 25 mph. Driver braking does not suppress alerts, but
does reduce the assumed driver response time value significantly.

The FCW alert consists of a short series of tones from the front
speakers, along with a visual cue similar to that described for the LDW.
To focus on the question of whether the use of head pose was a useful
adaptive mechanism, the system did not issue FCW alerts for targets that
had never been seen to move. The concern was that an excessive number of
false alerts (a common side effect of algorithms that allow long-range
alerts to stopped vehicles) would color drivers' views of FCW enough to
overshadow the difference between adaptive and non-adaptive modes.

FCW also includes an optional tailgating alert whereby a steady red
light is visible on the windshield when the time headway to the
preceding vehicle is less than a threshold. The tailgating component was
not enabled during in-vehicle testing.

## 14.2.2 Adaptive Collision Warning System  {#adaptive-collision-warning-system .SaveIT-H2}

The adaptive mode of the SAVE-IT prototype's collision warning systems
uses the measurement of head pose to decide whether and when to warn the
driver. The purpose of using head pose to suppress, delay, or move
alerts forward in time is to reduce nuisance alerts and false alerts and
thereby improve driver acceptance of collision warning systems.
Furthermore, head pose is sometimes used to issue FCW alerts earlier.

Figure 14.5 shows a module mounted on the steering column that includes
a camera and infrared illumination to detect head pose. Head pose in
this case is the driver's head position in the azimuth direction, i.e.,
in the direction of horizontal motion to the left or right. The head
elevation (whether the driver is looking up or down) is not a factor in
the SAVE-IT system's decision-making, except if head elevation
complicates the system's ability to measure azimuth. The head pose is
measured using a Delphi system that uses a camera mounted on top of the
steering wheel column. An approximate rule is that the SAVE-IT system
considers head pose to be forward when the driver's nose is pointed
toward any point on the left side of the windshield. The actual
boundaries do not coincide exactly with this, but this is a useful
approximation for this report.

![](media/image4.jpeg){width="4.944444444444445in"
height="2.816666666666667in"}

Figure 14.5. Column-mounted module for sensing head pose and
illuminating driver head

Since drivers sometimes move their eyes without moving their heads, the
driver's actual eye gaze will occasionally be forward when the head pose
is not forward, and vice versa. Furthermore, the use of head pose
measurement to adaptively alter crash alert decisions implies an
assumption that head pose is a surrogate for a general state of
attention and awareness.

In its adaptive mode, the SAVE-IT system will suppress LDW alerts when
the head pose has been forward at least once during the last 2 seconds
(see adaptive mode logic in Figure 14.6).

Logic of the non-adaptive LDW applies, except that:

If (driver head pose has been forward within the last 2 secs), then,\
Suppress LDW alert\
Else\
Issue LDW alert

Figure 14.6. SAVE-IT adaptive mechanism for LDW

The effect of the head pose measure on FCW (in adaptive mode) is more
complicated and involves modulating the assumption of driver delay time
to an alert. See Figure 14.7 for an explanation of the adaptive
mechanism for FCW. The result of this mechanism is that if the driver is
looking forward, or has been looking forward within the last half second
or so, the FCW alert will be delayed somewhat. This will often lead to
effectively suppressing the alert altogether, as oftentimes the driver
begins to brake during this half second, or one or both vehicles
maneuver laterally such that the paths are not considered to cross. If
the driver has not been looking forward for approximately one second,
then the adaptive FCW timing is about the same as the non-adaptive
timing. If the driver has not been looking forward for over a second,
then the adaptive FCW alert occurs sooner than the non-adaptive alert.
This case can sometimes lead not only to earlier alerts, but also to new
alerts that would not occur in non-adaptive mode. This will be seen
later in the results.

The adaptive logic also suppresses FCW alerts when the driver is braking
and the head pose is considered to be forward.

> The logic of Figure 14.6 also applies to adaptive mode, except that
> the delay time, DT, is modulated depending on head pose estimates and
> brake switch state, as follows:
>
> If brake is not being applied, then:\
> If (head pose is forward), then DT = 0.6 sec
>
> If (head pose is not forward for 0.1 sec), then DT = 1.6 sec
>
> If (head pose is not forward for 0.1 + t sec), then DT = 1.6 + t sec
>
> If (head pose is not forward for more than 2 secs), then DT = 3.6 sec
>
> If brake is being applied, then:\
> If (head pose is forward and brake is on), then alert is suppressed\
> If (head pose is not forward or unknown and brake is on), then\
> DT = 0.6 sec

Figure 14.7. SAVE-IT adaptive mechanisms for FCW alert timing

## 14.2.3 Distraction Alert  {#distraction-alert .SaveIT-H2}

The distraction alert component of the SAVE-IT prototype system provides
a flashing visual display on the windshield when the driver's head pose
has not been forward for a long time. The display is intended to capture
the driver's attention via the driver's peripheral vision. The display
is the same size and location as that used for LDW and FCW alerts (see
Figure 14.2), only it is amber instead of red and its flashing duration
is longer. The distraction alert has no audio or haptic cues.

## 14.2.4 Distraction Mitigation System  {#distraction-mitigation-system .SaveIT-H2}

The distraction mitigation system (DMS) refers to features of an
experimental IVIS system that are intended to reduce driver use of IVIS
features when that use may have negative safety impact. The IVIS itself
was created for the SAVE-IT project and consists of several features:

-   Radio (mocked-up feature, plays pre-recorded music)

-   MP3 player (actual feature)

-   Satellite radio (mocked-up feature, plays pre-recorded music)

-   Navigation system, including destination entry interface (with
    driver entry on a touchscreen) and map graphics and instructions
    (mocked-up)

-   Text messaging (mocked-up)

-   Phone (mocked-up, with no actual phone connection)

The IVIS is mounted in the top of the center console and has a
touchscreen as well as press buttons and knobs (see Figure 14.8).

![](media/image5.jpeg){width="4.555555555555555in"
height="2.7666666666666666in"}

Figure 14.8. The in-vehicle information system (IVIS) installed in the
SAVE-IT prototype vehicle

The DMS locks out or advises drivers against using certain features when
the perceived demand of driving is too high for the feature. Driving
demand is computed in real time using several variables addressing the
current driving situation (e.g., traffic density surrogate, wiper
state), as reported in another SAVE-IT report (Smith et al., 2007).
Driving demand is estimated to be at one of three levels: low, medium,
or high. Based on the demand level, the DMS allows, advises against, or
locks out certain features of the IVIS (see Table 14.2). The IVIS
changes the color of the feature's text on the screen to inform the
driver of these actions. For example, in Figure 14.9, the allowed
features have white text on dark background, the features that drivers
are advised against using have orange text on dark background, and the
blocked features have white text on gray background.

The only activity in this in-vehicle evaluation that involves addressing
and evaluating the DMS was an investigation into whether the driving
demand states are consistent with driver's perceptions as they are
traveling on the road.

Table 14.2. Distraction mitigation system:\
Selected adaptive actions in response to estimates of driving demand

  ----------------- ----------------- ----------------- -----------------
  **Estimated       **Radio tuning**  **Phone dialing** **Navigation
  demand of                                             destination
  driving**                                             entry**

  Low               Allows            Allows            Advises against

  Moderate          Allows            Advises against   Does not allow

  High              Advises against   Does not allow    Does not allow
  ----------------- ----------------- ----------------- -----------------

![](media/image6.jpeg){width="5.158333333333333in"
height="2.545138888888889in"}

Figure 14.9. Examples of features that have been allowed, locked out, or
advised against\
(Color image required for clarity, otherwise please see text) (courtesy
Delphi Corp.)

# 14.3 Method {#method .SaveIT-H1}

## 14.3.1 Overview {#overview .SaveIT-H2}

In-vehicle testing consisted of exposing test subjects to the SAVE-IT
system on both a test track and a public road circuit. The main research
questions that were addressed in the in-vehicle testing activities are
listed in Table 14.3. Two questions address safety while three address
driver acceptance. The in-vehicle testing provided a unique opportunity
to assess aspects of driver acceptance that would be impacted by the
adaptive mechanisms developed within the SAVE-IT project. This was
because drivers were able to experience the system in a vehicle
prototype with each driver spending approximately six hours behind the
wheel on public roads as well as at a test track. The in-vehicle work,
however, was less suited to placing drivers in truly safety-critical
situations, so that much of that work was conducted in simulator
experiments conducted by other organizations and reported elsewhere. The
bulk of in-vehicle testing focused on the driver acceptance and safety
impacts of the SAVE-IT adaptive mechanisms associated with the crash
warning systems. Only a minor portion of the in-vehicle work addressed
the distraction mitigation system, since the use of the in-vehicle
information system (IVIS) in real traffic posed safety concerns that
were not addressable with the scope of this evaluation effort.

The high-level questions in Table 14.3 will be broken down into more
specific questions and issues throughout this report. The evaluation
objectives led, through a series of discussions with Delphi Corporation,
to an approach that encompassed both test-track work and on-road work.
Twenty-six naïve drivers recruited from southeast Michigan were tested
in the prototype vehicle at a test track, and then 12 of those drivers
were further tested with on-road drives on public roads in the greater
Detroit metropolitan area. The two main objectives of the test-track
experiments were to collect data on the potential of the adaptive FCW
system to provide additional safety benefits to distracted drivers, and
to expose drivers to the LDW and FCW systems, the adaptive mechanisms,
and the concept of nuisance alerts.

One of the main objectives of the on-road work was to expose a subset of
the drivers to real-world conditions and performance of the SAVE-IT
system, in order to allow them to provide subjective feedback about the
adaptive mechanisms. An important second objective of on-road activities
was to learn about the performance of the SAVE-IT system, in terms of
appropriate and inappropriate suppression, delay, or advancing in time
of the LDW and/or FCW alerts. A minor aspect of the on-road tests was to
collect drivers' in-vehicle opinions about the suitability of conducting
certain IVIS tasks while driving in environments that involved differing
levels of driving demand. By overlaying those opinions on SAVE-IT's
internal rules for moderating driver access to those tasks, indirect
data was collected regarding the correlation of driver opinions with the
system and, hopefully, insight into the likely acceptance of those rules
by drivers.

A challenge in this evaluation was that there was a breadth of research
questions and, of course, limited scope for the research. The tactic
chosen was to expose drivers to as many of the issues in comparing
adaptive and non-adaptive systems as possible, to expose the adaptive
systems themselves to real-world driving circumstances, and to capture
driver feedback and to observe system performance in order to both look
for and report clear indications of possibilities and challenges. While
this results in a set of data that addresses many issues, there will be
circumstances in which the amount of data is limited enough that
statistical significance cannot be achieved. Hence when no significant
difference between adaptive and non-adaptive systems is uncovered in
this data, the cause may be a fundamental one or it may be that the data
is too limited.

The remainder of this section on experimental methods includes
subsections describing the overall data archive collected in the
in-vehicle evaluation, recruitment methods, the experimental method for
test-track activities, vehicle instrumentation and associated equipment,
and the experimental approach for on-road activities. This section does
not address the analyses that was conducted using field operational test
data collected during previous projects; that analysis is described in
Section 14.4.1.3.

Table 14.3. Primary research questions for the in-vehicle evaluation of
SAVE-IT

+-----------------------+---------------+------------------------------+
| **Research question** | **Test        | **Test method**              |
|                       | environment** |                              |
+-----------------------+---------------+------------------------------+
| Safety: Is there      | Test track    | Surprise lead vehicle        |
| evidence that the     |               | slowing trials               |
| earlier FCW alerts in |               |                              |
| adaptive mode provide |               |                              |
| additional safety for |               |                              |
| drivers?              |               |                              |
+-----------------------+---------------+------------------------------+
| Safety: Does test     | On-road       | Review of suppressed alerts  |
| data suggest          | testing       |                              |
| potential safety      |               |                              |
| concerns due to the   |               |                              |
| adaptive approach?    |               |                              |
+-----------------------+---------------+------------------------------+
| Acceptance: Would the | On-road       | Expose SAVE-IT to driving in |
| adaptive features     | testing       | both accompanied and         |
| reduce the number of  |               | unaccompanied driving by     |
| false and/or nuisance | Analysis of   | naïve subjects.              |
| alerts?               | data from     |                              |
|                       | previous      | Identify existing FOT alerts |
|                       | UMTRI FOTs    | that a head-pose system      |
|                       |               | would suppress.              |
+-----------------------+---------------+------------------------------+
| Acceptance: Do the    | Combined      | Questionnaires               |
| adaptive mechanisms   | exposures to  |                              |
| improve driver        | test-track    | Driver ratings of alerts     |
| acceptance?           | and on-road   | during post-drive video      |
|                       | driving       | review                       |
+-----------------------+---------------+------------------------------+
| Acceptance: Is        | On-road       | Driver responses to          |
| drivers' on-road      | testing       | questions during drives      |
| perception of driving |               |                              |
| demand similar to     |               |                              |
| SAVE-IT's estimates   |               |                              |
| of demand?            |               |                              |
+-----------------------+---------------+------------------------------+

## 14.3.3 Experimental Data Archive {#experimental-data-archive .SaveIT-H2}

Table 14.4 gives an overview of the data elements collected during the
in-vehicle evaluation activity. Details regarding each of these elements
are provided in later sections.

Table 14.4. Overview of data collected

  --------------- ------------- ------------------------------------------
  **Test          **Objective   **Data collected**
  environment**   data**        

  N/A             Objective     Driver age, gender

  Test track      Objective     Surprise FCW trial data

  Test track      Objective     Baseline (driver-directed) braking
                                decisions

  Test track      Subjective    Pre-drive questionnaire

  Test track      Subjective    Post-track-test questionnaire

  On-road testing Objective     On-road data (alerts, etc.) - for adaptive
                                mode

  On-road testing Objective     On-road data -- for non-adaptive mode

  On-road testing Subjective    Driver responses to questions asked during
                                on-road drives regarding willingness to
                                engage in IVIS tasks

  On-road testing Subjective    Post-road-test questionnaire -- for
                                adaptive mode

  On-road testing Subjective    Post-road-test questionnaire -- for
                                non-adaptive mode

  On-road testing Subjective    Post-road-test driver ratings of alert
                                utility using video review

  On-road testing Subjective    Final general questions addressing the
                                driver's general conclusions regarding the
                                adaptive systems

  FOT data        Objective     Head pose near alerts observed in the FOTs
  analysis                      
  --------------- ------------- ------------------------------------------

## 14.3.2 Recruitment {#recruitment .SaveIT-H2}

Subjects were recruited through an advertisement in a local newspaper.
During the recruitment process, attempts were made to create a
gender-balanced subject pool. The advertisement specified that potential
subjects must have a valid driver license and be between 35 and 55 years
of age. Before doing any driving, subjects completed a pre-drive
questionnaire (see Appendix A). A target number of 24 subjects was
recruited for the test-track portion of the study, which lasted about
two hours. Twenty-six subjects actually completed the track testing
procedures and produced a full data set for the track activities. This
included 15 males and 11 females with ages from 35 to 57.

Twelve of the 26 drivers (six males and six females) also completed
on-road testing a few weeks later. One of the 12 on-road subjects
(female) experienced only one of the two on-road drives due to a vehicle
failure during the second drive, which also left insufficient time to
conduct the post-drive review of alert videos with this driver. The ages
and genders of the drivers and the protocols they experienced are shown
in Appendix B.

## 14.3.3 Experiments on the Track {#experiments-on-the-track .SaveIT-H2}

During the test-track portion of the study, subjects drove trials at a
closed test track for approximately two hours. Upon arrival, subjects
were greeted by the experimenters and taken to a subject room. Subjects
were given the test-track informed consent document (see Appendix C)
and, following completion, were asked to fill out a survey regarding
their driving history, self-reported driving behavior, demographic
information, and experience with technology. Next, the subject was shown
to the instrumented vehicle and an explanation of the technology in the
vehicle was given. The scenarios and a brief description of on-the-track
activities are shown in Table 14.5. A complete description of test-track
procedures, including a matrix of activities, scripts, and surveys, is
available in Appendix D.

Subjects first drove two practice laps (one at 35 mph and one at 50 mph)
around the track to get acclimated to the vehicle and the test track.
When these laps were completed, the experimenter began asking the
subject to engage in various driving scenarios to allow the subject to
experience the vehicle warning systems. (See Appendix D for details of
the scenario kinematic conditions.) The first set of scenarios were
designed to expose each subject to the differences in how the adaptive
and non-adaptive modes affect the way the lane departure warning system
works. First, subjects were asked to drift out of the lane to experience
the system in non-adaptive mode. Next, the mode was changed to adaptive
and subjects were again asked to drift from the lane, to see that the
alert was now suppressed. Finally, with the system still in adaptive
mode, subjects were asked to look to the side and drift from the lane to
again experience the alert.

Table 14.5. Test-track driving scenarios

  ------------------------- ---------------------------------------------
  **Scenario**              **Description**

  Practice Laps             Two laps completed at 35 mph; two completed
                            at 50 mph

  Lane Departure Warning    Adaptive and non-adaptive scenarios

  Distraction Alert         Several scenarios were completed

  Practice with Lead        Laps completed at 35 and 50 mph with 2 sec.
  Vehicle                   headway

  Forward Crash Warning     Distracted driver, lead-vehicle braking
  (FCW) - Surprise Braking  scenario; adaptive/non-adaptive
  #1                        counter-balanced

  Driver Response to Lead   Three trials of "normal but firm" braking in
  Vehicle Braking           response to a slowing lead vehicle.

  FCW - Experimenter Demos  Experimenter demonstrates adaptive and
                            non-adaptive modes

  Priming for 2^nd^         Distracting task without lead vehicle braking
  Surprise Braking          

  FCW - Subject Scenarios   Adaptive and non-adaptive scenarios (no
                            surprise trials)

  Priming for 2^nd^         Distracting task without lead vehicle braking
  Surprise Braking          

  FCW - Surprise Braking #2 Distracted driver, lead-vehicle braking
                            scenario; adaptive/non-adaptive
                            counter-balanced
  ------------------------- ---------------------------------------------

The next scenario exposed the subject to the distraction alert. During
this scenario, subjects were asked to continue driving, but turn their
head to the side until the alert was triggered. Subjects repeated this
process at least one more time to get a good understanding of the timing
of the alert as well as how it was displayed. Following the distraction
alert scenario, the experimenter radioed the lead-vehicle driver and the
subject completed two additional practice laps following that vehicle.
These laps allowed each subject to get acclimated to driving behind the
lead-vehicle while keeping a consistent distance of two seconds of
headway between the two vehicles.

Once these practice laps were completed, the next group of scenarios was
designed to expose the subject to the forward crash warning system, as
well as to gather driving information about the subject. The first of
these scenarios was a surprise braking trial. During this scenario, the
driver was asked to continue driving, but begin to interact with the
IVIS in the vehicle by entering an address into the navigation system.
During address entry, the experimenter covertly signaled the lead
vehicle to begin to brake. Whether the system was in adaptive or
non-adaptive mode during this scenario was counter-balanced across
subjects. The next scenario was designed to gather a measurement of the
driver's response. For this trial, the lead vehicle began to brake and
the subject was asked to delay braking until the last moment and to use
"normal but firm" braking. This trial was repeated three times per
subject.

The next group of scenarios exposed the subject to differences in how
the forward crash warning system worked in adaptive versus non-adaptive
mode. The experimenter drove the vehicle and demonstrated the difference
between the two modes while looking away (simulating distraction).
During these scenarios, the experimenter delayed braking until the alert
went off (unless suppressed), so the subject could experience the timing
of the alerts in the two modes. For each mode of the system, the
experimenter demonstrated two scenarios, one with the lead vehicle
braking in front and another with the lead vehicle braking and leaving
the lane (similar to a vehicle slowing to make a right turn).

When the experimenter demonstrations were complete, the subject was
asked to complete another task using the navigation system. While
entering this address, the lead vehicle did not begin to brake. Since
the experimental plan called for a second surprise braking trail during
a later scenario, this trial was designed to prime the subject so a
connection between entering an address into the navigation system and
the lead-vehicle braking was not necessarily made.

Next, the subject completed several forward crash warning scenarios
similar to those demonstrated earlier by the experimenter. Subjects were
asked to try to wait for the alert before delay braking, but not to
depend on the alert, and to brake when they felt they must. Scenarios
were completed in each mode of the system, some with the lead vehicle
braking in front and some with the lead vehicle braking and moving out
of the lane.

For the final set of scenarios, the subject was asked to enter an
address into the navigation system. The subject was allowed to
completely enter two addresses without the lead vehicle braking to once
again prime the driver for the surprise trial. While entering the third
address, the experimenter covertly signaled the lead vehicle to brake in
an attempt to surprise the subject.

Following the track driving, subjects were taken back to the subject
room and were asked to fill out another questionnaire regarding their
assessment of the vehicle\'s systems. Once completed, subjects were paid
\$50 for participation in this portion of the study.

### 14.3.3.1 Track Site and Surrogate Target {#track-site-and-surrogate-target .SaveIT-H3}

Track testing was done at Dana Corporation's Technical Resource Park in
Ottawa Lake, Michigan. The 1.75-mile track is comprised of two banked
half-circle curves that are connected by two parallel straightaways (see
detailed drawing in Appendix D).

To provide a sense of reality for the subject while reducing the risk of
injury or property damage, a surrogate target was employed during some
track activities (Figure 14.10). The surrogate target is a lightweight
trailer that is towed behind a vehicle and is constructed to appear
similar to a passenger vehicle. The trailer body is a fiberglass shell
shaped as the rear portion of a Ford Taurus sedan. This method was first
used in 1998 and has since been used by several organizations. Drivers
have been found to be very reluctant to impact the surrogate target,
confirming that this device is a useful surrogate for some aspects of
real-world behavior.

The surrogate target is towed by a bar that consists of three aluminum
tubes. The tubes are able to slide inside one another in a telescopic
fashion if the force along the tow bar exceeds a threshold. This
provides approximately 6 ft (1.9 m) of telescopic motion should the test
subject's vehicle strike the rear end of the surrogate target. In
addition, the buck on the trailer is lightweight and therefore provides
an additional collapse zone in case there is impact.

![](media/image7.jpeg){width="5.378472222222222in"
height="2.2381944444444444in"}![](media/image8.jpeg){width="5.393055555555556in"
height="2.245833333333333in"}

Figure 14.10. Surrogate target -- rear and side views

### 14.3.3.2 Vehicle Instrumentation  {#vehicle-instrumentation .SaveIT-H3}

A data acquisition system (DAS) was installed by UMTRI into the SAVE-IT
vehicle prototype. The DAS was installed in the vehicle's trunk and an
associated display and keyboard were installed in the rear-seat area for
an experimenter to monitor the SAVE-IT status and performance, as well
as to control data collection (see Figure 14.11).

Three data types were collected during the in-vehicle evaluation:
numerical data, video data, and audio data. The numerical data refers to
data signals including:

-   Data available on the project CAN bus installed by Delphi, including
    information from the forward radar, lane-tracking system, vehicle
    motion sensors, driver control actions, SAVE-IT intermediate data,
    IVIS activity logging, head pose estimation data, and more

-   Experimenter inputs such as trial number, subject number, etc.

-   GPS data collected by the DAS

![](media/image9.jpeg){width="4.30625in" height="3.234722222222222in"}

Figure 14.11. Rear-seat experimenter station

This data is summarized in Appendix F. Data from six video cameras
installed by Delphi were also collected. Two of these images were from
the SAVE-IT system, specifically the forward lane-tracking system camera
and the head-pose tracking camera. Sample images from these cameras are
shown in Figure 14.12.

![](media/image10.jpeg){width="5.673611111111111in"
height="3.0708333333333333in"}

Figure 14.12. Forward and face cameras and images

Four other cameras installed by Delphi provided coverage of the outside
of the vehicle as well as a view of the center stack and steering wheel
area (see Figure 14.13). All video streams were coded in the DAS with a
time stamp, so that it could be played back synchronized with any of the
digital data channels.

During all testing, both on the track and on the road, audio was
recorded in the vehicle. Similar to the video data, the audio was also
encoded with a time stamp, so that any stream of video and any segment
of the digital data could later be examined. In addition to verbal
comments made by either the driver or the experimenter, the audio served
as another method to confirm that audio warnings were issued.

![](media/image11.jpeg){width="5.565277777777778in"
height="2.4555555555555557in"}

Figure 14.13. Four additional cameras and associated images

In addition to the DAS elements, a passenger-side brake was also
installed. The purpose of this brake was to add another layer of safety
during the test-track trials: While the supervising experimenter manned
the rear-seat monitoring station, a second experimenter was in the
passenger seat, ready to brake and avoid hitting the surrogate target in
case the driver failed to do so in a timely manner.

## 14.3.4 Experimental Method for On-Road Testing {#experimental-method-for-on-road-testing .SaveIT-H2}

During on-the-road driving, subjects drove the instrumented vehicle
along a pre-determined route during two sessions: one session with the
vehicle's warning systems in adaptive mode and one session with the
systems in non-adaptive mode. The order of the modes (morning versus
afternoon) was counterbalanced across subjects. A complete description
of on-the-road procedures, including scripts and surveys, is available
in Appendix G.

Upon arrival, subjects were greeted by the experimenter and taken to a
subject room. Next the subject was given the on-road informed consent
document (Appendix C), and the schedule for the day was described. The
experimenter described the various warning systems and modes installed
in the vehicle, and led the subject to the vehicle to begin the morning
drive. The morning and afternoon drives used the same route of
approximately 84 miles of public roads, passing from Ann Arbor into the
Detroit area and back. About half of this distance was freeway and half
a mix of major and minor arterials, including urban, suburban, and rural
roadways. Prior to departure on the morning drive, however, subjects
were asked to practice three IVIS tasks: tuning the radio, dialing a
phone number, and entering an address into the navigation system. All of
these required use of the IVIS touchscreen, and the tasks were selected
to represent increasing levels of driver involvement. Once practice with
the IVIS was complete, the experimenter moved to the back seat of the
car, where the car's systems could be monitored, and the subject began
to drive the route.

Subjects were instructed to avoid conversing with the experimenter
during the drives unless a question arose regarding one of the vehicle's
systems. As the subject drove, the experimenter gave route instructions
whenever a turn was coming up. The route took over two hours. During
each drive, as the SAVE-IT system identified different levels of driving
demand (as described in Section 14.2), the experimenter queried the
driver about the relative safeness of performing each of the three IVIS
tasks at that moment in time. Responses were recorded by the
experimenter on a form. The driver was not asked to actually perform the
task, since it was considered beyond the scope of this evaluation to
create a safe and realistic method to combine a highly involving task
and high driving demand. During each drive, subjects were asked twice
about each different task for every driving demand scenario, for a total
of 12 questions per task.

Following each drive, subjects were accompanied back to the subject room
and asked to complete a questionnaire about the mode of the system they
experienced during the immediately preceding drive. After the second
drive, subjects were also shown video from their two drives, as well as
some pre-scripted driving scenarios. Each video was approximately 20
seconds long and centered around either an alert that was received
during the drive or an alert suppressed by the adaptive system. While
watching these videos, subjects were asked about whether a warning
(either received or suppressed) was, or would have been, beneficial
given the circumstance. Overall impressions of each system and any other
comments were also solicited. Responses were videotaped and recorded on
a form by the experimenter. Each session lasted between three and
three-and-a-half hours for a total of six to seven hours per subject. A
break for lunch was scheduled between the sessions, and subjects were
paid \$170 for their time.

# 14.4 Results {#results .SaveIT-H1}

The presentation of results is divided into results that address safety
and those that focus on driver acceptance. Section 14.4.1 focuses on
safety and Section 14.4.2 addresses driver acceptance. Both sections
include substantial discussion of the results and their significance.
The next section -- Section 14.5 -- summarizes the results and provides
a set of conclusions.

## 14.4.1 Potential Safety Impacts of the Adaptive Mechanisms {#potential-safety-impacts-of-the-adaptive-mechanisms .SaveIT-H2}

Results that address the potential safety impacts of the SAVE-IT
mechanisms are presented in a series of subsections that follow on test
track trials and equipment, on-road work, and analysis of field
operational test data.

### 14.4.1.1 FCW Surprise Trial Conduct  {#fcw-surprise-trial-conduct .SaveIT-H3}

A total of 26 subjects participated in the test-track trials. As
described in Section 14.3.3, the activities on the test track included
several sets of trials to acclimate drivers to the concepts of FCW, LDW,
nuisance alerts, and the SAVE-IT adaptive approach. The experimental
portion of track activities also included exposing each driver to two
FCW surprise alerts (one trial each with the system in an adaptive mode
and non-adaptive mode), as well as some baseline trials of
driver-discretionary braking in order to calibrate their performance.
The purpose of the surprise FCW trials was to determine whether there
was any demonstrable safety advantage introduced by the adaptive
systems' advancing of the FCW alerts when the drivers' heads faced away
from the forward scene for a prolonged period of time.

The desired sequence of events for a surprise-trial is shown in Figure
14.14. As described in Section 14.3.2.2, the driver was asked to enter a
navigation destination on the IVIS and, as they began the task, the lead
vehicle was signaled to begin a 0.4 g deceleration. Ideally, the FCW
alert would occur while the driver was still engaged in the task and had
not noticed that the lead vehicle was braking. Drivers did, however,
occasionally survey the forward road scene while doing such tasks and so
some drivers were not surprised by the alert itself.

![](media/image12.wmf){width="5.781944444444444in"
height="2.7645833333333334in"}

Figure 14.14. FCW surprise alert trial -- presumed sequence

Under ideal execution of the plan, a total of 52 successful surprise
alerts would have occurred (two for each driver). However, in 20 of
these cases the driver braked prior to the point of issuing the FCW
alert. In addition, from examining the video of the remaining 32 FCW
surprise trials, in five of the braking events the driver was looking
forward at the time of the alert. Hence they cannot be counted as true
surprises in regards to the FCW alert.

Of the remaining 27 trials, 14 were paired. That is, seven drivers were
successfully surprised in both the adaptive and the non-adaptive modes.
Thus, 13 other drivers were surprised only once. Table 14.6 provides the
breakdown of the surprise alerts. During testing, subjects drove the
system at an alternating and balanced order. The right column of Table
14.6 associates the trial order with the count of subjects surprised.
Note the unexpected result that the rate of successfully surprising
drivers was higher in the driver's second surprise trial (15 of 26
trials, or 59 percent), compared to the success rate for driver's
initial surprise trials (12 of 26 trials, 47 percent). It is possible,
however, that the second surprise may not catch all drivers as unaware
as the first surprise attempt.

Table 14.6. Number of FCW trials with successfully surprised drivers

  ----------------- --------------------- ------------------- ------------
                    **Non-Adaptive**      **Adaptive**        **Total**

  1^st^ Attempt     9                     3                   **12**

  2^nd^ Attempt     8                     7                   **15**

  **Total**         **17**                **10**              **27**
  ----------------- --------------------- ------------------- ------------

### 14.4.1.1 Results of Track Experiments {#results-of-track-experiments .SaveIT-H3}

When a driver's head is considered "not forward" for a certain period of
time, the FCW algorithm assumes a longer driver response time to an
alert, so that FCW alerts will occur earlier. The experimental portion
of the test track protocols investigates whether this effect provides a
potential safety gain by helping inattentive drivers respond sooner to
unexpected lead vehicle decelerations.

Table 14.7 considers four different sets of surprise trials and, for
each set of trials, the effect of the adaptive mechanism on each of four
different metrics. The four sets of trials are:

-   All surprise trials (two trials each for 26 drivers)

-   All surprise trials where an FCW alert occurred (32 trials). FCW
    alerts do not occur when the driver has noticed the decelerating
    lead vehicle and performs braking action such that the FCW criteria
    are never satisfied.

-   Driver-surprised trials (27 trials). These are identified using
    review of video of the driver's face. A surprise is when the driver
    has not looked up and appears unaware of the lead vehicle braking
    until the alert is given.

-   Twice-surprised driver trials (14 trials). Seven drivers were
    surprised on both their first and second surprise trials. This set
    allows a within-subjects comparison.

Table 14.7. Differences in driver responses to adaptive and non-adaptive
FCW in surprise trials

+----------+----------+------------+-----------+----------+----------+
| **       |          | *          |           |          |          |
| Trials** |          | *Metrics** |           |          |          |
+----------+----------+------------+-----------+----------+----------+
|          | Trials\  | RT: Alert  | RT: Lead  | Time to  | Required |
|          | (non-ad  | onset to   | vehicle   | c        | dece     |
|          | aptive,\ | subject    | brake     | ollision | leration |
|          | non-a    | brake      | onset to  | at       | at       |
|          | daptive) | onset      | subject   | subject  | subject  |
|          |          |            | brake     | brake    | brake    |
|          |          |            | onset     | onset    | onset    |
+----------+----------+------------+-----------+----------+----------+
| All      | 52:\     | Not        | Not       | Not      | Not      |
| surprise | 26 NA,   | s          | si        | sig      | sig      |
| trials   | 26 A     | ignificant | gnificant | nificant | nificant |
+----------+----------+------------+-----------+----------+----------+
| Surprise | 32:\     | Not        | Ap        | Not      | Not      |
| trials   | 19 NA,   | s          | proaching | sig      | sig      |
| with an  | 13 A     | ignificant | sign      | nificant | nificant |
| FCW      |          |            | ificance\ |          |          |
| alert    |          |            | p=0.107\  |          |          |
|          |          |            | Adaptive  |          |          |
|          |          |            | has       |          |          |
|          |          |            | shorter   |          |          |
|          |          |            | RT        |          |          |
+----------+----------+------------+-----------+----------+----------+
| Trials   | 27:\     | A          | ***Signif | App      | Not      |
| with     | 17 NA,   | pproaching | icant***\ | roaching | sig      |
| apparent | 10 A     | sig        | p\<0.05;  | signi    | nificant |
| driver   |          | nificance\ | DF=1;     | ficance\ |          |
| surprise |          | p=0.191\   | F=5.24\   | p=0.195\ |          |
| at alert |          | Adaptive   | Adaptive  | Adaptive |          |
|          |          | has        | has       | has      |          |
|          |          | shorter RT | sign      | longer   |          |
|          |          |            | ificantly | TTC      |          |
|          |          |            | shorter   |          |          |
|          |          |            | RT        |          |          |
+----------+----------+------------+-----------+----------+----------+
| Twice-s  | 14:      | Not        | Ap        | Not      | Not      |
| urprised |          | s          | proaching | sig      | sig      |
| driver   | 7 NA, 7  | ignificant | sign      | nificant | nificant |
| trials   | A        |            | ificance\ |          |          |
|          |          |            | p=0.115\  |          |          |
|          |          |            | Adaptive  |          |          |
|          |          |            | has       |          |          |
|          |          |            | shorter   |          |          |
|          |          |            | RT        |          |          |
+----------+----------+------------+-----------+----------+----------+

Two of the four metrics used in this analysis are reaction times: (a)
the time from the FCW alert onset to the brake switch transition
indicating subject brake application, and (b) the time from the
beginning of lead vehicle braking until the subject's brake onset. The
other two metrics are conflict metrics computed using the kinematic
conditions at the time of the subject's brake onset. Time to collision
is defined here as distance between vehicles divided by closing speed,
and required deceleration is the constant deceleration level needed to
just avoid impact.

Analysis of variance (ANOVA) tests were conducted on each of the 52
surprise trial attempts, and there is no statistically significant
difference in any of the metrics, nor are there indications of
differences (defined as p \< 0.20) between the adaptive and non-adaptive
cases. The latter is also true for the 32 trials in which an FCW alert
was presented. The absence of differences for these sets is somewhat
expected. For drivers who are not surprised, their braking decisions are
assumed to be determined primarily by their own sense of the kinematics
and expectations of the situation, and not influenced substantially by
an FCW alert trial. Ervin et al. (2005) reported that in a field
operational test of FCW, the reaction time (alert to brake onset) and
the braking level of drivers was unchanged when comparing presented FCW
alerts and silent, background FCW alerts. Drivers do not typically
respond to FCW alerts with reflexive braking, at least in the field in
real-world driving. It is assumed that that behavior carries over to the
track.

The third set of trials considered in Table 14.7 does show a difference
and indications of other differences. When discussing these, it is first
assumed that the initial headways at the time of lead vehicle braking,
as well as the lead vehicle braking levels, are similar between trials
with non-adaptive and adaptive modes. Then the adaptive alert will occur
sooner in time, and at greater distances, because the driver's head is
away from the forward scene and the adaptive mechanism is to assume a
larger driver response time in the calculation of FCW warning range.
Table 14.7 and Figure 14.15 show that the differences in reaction time
of the subject from alert onset to the subject's brake onset approached
significance (shorter for adaptive; p = 0.191). Shorter reaction times
of this type are not expected; one might assume a driver's response to
any FCW alert is approximately the same, at least when the situation has
some level of forward conflict such that the driver will be compelled to
brake. Given that adaptive alerts occur sooner, and that the reaction
times to adaptive alerts may be shorter, it is not surprising that the
reaction time from the onset of lead vehicle braking to the subject's
brake onset is significantly shorter.

![](media/image13.wmf){width="4.444444444444445in"
height="2.1083333333333334in"}

Figure 14.15. Reaction time (from alert to driver brake onset) --
comparing surprise trials

The indication of longer time to collisions associated with adaptive
alerts is also consistent with this pattern. However, the fact that
there is no indication of required deceleration with adaptive alerts is
perhaps inconsistent. The results are obviously dependent on a small
amount of trials, and the control of the conditions of the trials is
imperfect. Figure 14.16 shows the variability in lead vehicle
deceleration (at alert onset), which may contribute to the lack of
significance in the required deceleration at alert onset.

![](media/image14.wmf){width="6.497916666666667in"
height="3.4743055555555555in"}

Figure 14.16. Variability in lead vehicle deceleration at the time of
alert onset

When considering the final row in Table 14.7 (the set of matched trials
for drivers who were surprised twice), the differences between driver
response to adaptive and non-adaptive alerts are barely perceptible.
This introduces the possibility that there is little difference between
driver responses to adaptive and non-adaptive FCW alerts for this
particular kinematic condition. This may be caused by any or all of
three reasons: the small number of trials prevents statistical
significance, the variation in initial headways and consistency of lead
vehicle braking introduces errors into the data, and/or the kinematic
conditions themselves are benign enough that drivers do not feel
compelled to take advantage of the earlier adaptive alerts.

All told, these results show only weak indications that the adaptive FCW
alerts can let drivers begin braking sooner to lead vehicle deceleration
situations when they have been looking away from the road, in this
particular kinematic condition. Thus the SAVE-IT mechanism may or may
not allow distracted drivers an additional safety margin in these
situations. Nevertheless, it seems safe to assume that an earlier alert
is better, so long as the nuisance potential does not increase, which
the next section shows, it does not.

### 14.4.1.2 Results from On-Road Observations {#results-from-on-road-observations .SaveIT-H3}

Each of the 12 drivers was asked to drive the SAVE-IT prototype twice
over a public road route, with one traversal using the non-adaptive
crash warning system and the other traversal using the adaptive version.
The order in which the two systems were activated was counterbalanced.
The length of the on-road route was approximately 86 mi (138 km). Table
14.8 shows that over the course of the on-road experiment, data was
collected from a total driving distance of 1,993 miles (3208 km) or,
equivalently, 47 hours of driving. Although 12 drivers began the
experimental drive, one of the drivers did not complete the afternoon
drive (adaptive mode). This was because concerns arose during the trip
regarding noise that seemed to be associated with the steering system of
the vehicle and the test was stopped. Because of the time delay needed
to retrieve the vehicle, the subject, and the experimenter, the subject
was not asked to complete the post-drive questionnaire or to review
video. (The vehicle issue was resolved before continuing the
experiment.)

Table 14.8. Distance and time of travel during on-road experiments

+-----------+------------------+------------------+------------------+
|           | **Non-adaptive   | **Adaptive\      | **\              |
|           | mode**           | mode**           | Total**          |
+-----------+------------------+------------------+------------------+
| Drivers   | 12               | 11               | 12               |
+-----------+------------------+------------------+------------------+
| Distance  | > 1,038 mi\      | > 955 mi\        | > 1,993 mi\      |
|           | > 1,671 km       | > 1,537 km       | > 3,208 km       |
+-----------+------------------+------------------+------------------+
| Time      | > 24.4 hr        | > 23.0 hr        | > 47.3 hr        |
+-----------+------------------+------------------+------------------+

14.4.1.2.1 Number of On-Road Alerts and Corresponding Alert Rates

This section provides a quantitative summary of the LDW and FCW alerts
experienced by drivers during the on-the-road testing phase. Subsequent
sections will address effects of the adaptive mechanisms on the
circumstances in which alerts occur.

Table 14.9 shows the number of FCW and LDW alerts presented to each
driver during the on-road drives. The alerts are broken down into
warning subsystem (FCW and LDW) as well as whether the alert occurred
while the system was in adaptive or non-adaptive mode. Consider first
the 103 LDW alerts presented to drivers: 93 during the non-adaptive
drives (12 drivers) and only 10 during the adaptive drives (11 drivers).
The alert rate for LDW was 9.0 alerts per 100 miles for non-adaptive
mode and 1.0 alerts per 100 miles for adaptive mode. Thus the LDW alert
rate is 88 percent lower in adaptive mode. All 11 drivers who drove in
both modes experienced a reduction in the number of alerts. The
difference between LDW alert rates in the two modes is very large and
will be shown to be caused primarily by the suppression effects of the
SAVE-IT adaptive mechanism (which is described in Section 14.2.2). There
could be other causes as well, such as drivers behaving differently with
the adaptive mode. Behaviors such as turn signal use and tolerance of
minor lane excursions could be influenced by the system itself, but
these behavioral influences are thought to be unlikely to cause much of
the difference in alert rates.

Table 14.9. Effect of adaptive mechanisms on the number of alert events
and alert rate\
during on-road testing

![](media/image15.wmf){width="6.463888888888889in"
height="2.936111111111111in"}The alert rate in non-adaptive mode is
similar to that observed in LeBlanc et al. (2006), which reported a rate
of 10 alerts per 100 miles for a lane departure system. However this
comparison is not very meaningful because the LDW alert rate can be
significantly affected by the roadways traversed as well as the overall
lane tracking availability (the fraction of travel time that the
vision-based lane tracking is successful in confidently locating lane
edges). Nevertheless, Section 14.4.1.3 describes that an analysis of
data from that field test suggests that the SAVE-IT adaptive approach
for LDW could be even more powerful in reducing alert rate than the 88
percent reduction seen in the on-road drives described in this document.

Table 14.9 also shows that when the SAVE-IT mechanism was enabled, the
alert rate for FCW decreased 60 percent from 6.0 to 2.4 alerts per 100
miles. Nine drivers experienced fewer alerts in adaptive mode, one
driver experienced the same number of alerts in both modes, and one
driver experienced a higher number of alerts in adaptive mode (two
alerts) than in non-adaptive mode (one alert). This same broad and
substantial reduction in alert rate suggests a systemic difference that
again would be assumed to carry over into a large-scale test or
deployment. The alert rates for FCW in both non-adaptive and adaptive
modes are in fact higher than were seen in the final FCW system design
addressed in the NHTSA Automotive Collision Avoidance System (ACAS)
project (Ervin et al., 2005). That project fielded a system that over
101,000 miles resulted in a rate of 0.90 alerts per 100 miles, and more
relevant to a comparison with SAVE-IT, a rate of 0.6 per 100 miles for
alerts triggered by targets observed to have moved. Thus the adaptive
and non-adaptive drives in this SAVE-IT evaluation had rates that were
4.2 and 10.0 times greater than seen in ACAS. This is likely due in
large part to the SAVE-IT FCW system being intentionally conservative,
in order to provide suitable numbers of FCW alerts to allow drivers in
these evaluations to respond to questions. There may also be an
influence of the particular route chosen.

This comparison with FOT alert rates is introduced because the fraction
of FCW alerts that are suppressed in this experiment may be higher than
the fraction of FCW alerts that would be suppressed if these same
approaches were applied in a commercial deployment. This hypothesis is
based on a presumed relationship between the overall trends between a
driver's likelihood of looking away and the level of forward conflict.
Consider the hypothetical curve shown in Figure 14.17. The driver is
assumed to be more likely to look away during periods when the forward
conflict is lower. If the FCW threshold can be considered in the
conflict dimension as shown, with a conservative alert timing being used
on the SAVE-IT FCW so that it is active at lower conflict levels than a
commercial system, then these assumptions imply that the suppression
rate will be higher for the SAVE-IT FCW than for a commercial system.
However, a rough estimate will be obtained in Section 14.4.1.3 when data
from ACAS is analyzed with the objective of estimating the suppression
rate of SAVE-IT techniques.

![](media/image16.wmf){width="4.743055555555555in"
height="2.263888888888889in"}

Figure 14.17. Hypothetical relationship between forward conflict level
and the likelihood\
of a driver looking away

14.4.1.2.2 Relation between Adaptive Mechanisms and Circumstances of
Alerts

While the previous section addressed the number of alerts, this section
addresses the circumstances in which alerts occurred. This provides
information about the difference in the driver's experience in adaptive
mode vs. non-adaptive mode.

LDW Alerts

This subsection addresses the influence of the SAVE-IT adaptive
mechanisms on specific alert events. Section 14.2.2 explained that the
adaptive mechanism for LDW is that the alert is given only if the
driver's head pose has been continuously away from the forward scene for
at least 2 seconds. For LDW, there were a total of 10 alerts presented
to the 11 drivers who completed the adaptive mode circuit. In
non-adaptive mode, there were 93 alerts presented for the 12 drivers.
For the purposes and the scope of this evaluation effort, the internal
data signals within the SAVE-IT system (or the evaluation team's
understanding of these signals) were inadequate to reliably provide
information about what the adaptive mode would have done during the
non-adaptive drives, and vice versa. Thus we compare the 10 alerts in
adaptive-mode driving to the 93 alerts observed in non-adaptive-mode
driving.

The 103 LDW alerts presented to the drivers were reviewed by analysts to
assign a driving scenario label to each event. Ten different scenario
labels, were developed using previous experience (LeBlanc et al., 2006)
and by viewing a subset of the SAVE-IT drive alerts. Table 14.10 shows
the 10 scenario labels, as well as four categories of LDW scenarios.
Table 14.11 shows the number of alerts that fall into each category.
Upon review of the event videos, seven of the 10 LDW alerts presented to
the drivers in adaptive mode involved situations in which the driver's
head pose was actually forward during some portion of the one-second
period immediately preceding the alert. In an eighth case, the head pose
was forward at a time about 1.3 seconds before the alert onset. The
conclusion is that most alerts (eight of 10) within this small set of
remaining alerts were due to errors in head pose estimation. The
remaining two LDW alerts in adaptive mode involved a driver whose head
was indeed not forward during the two-second period before the alert,
but who was monitoring the forward scene with his eyes. In one case, the
driver was preparing for a lane change and positioned his head toward
the inside rearview mirror, with glances forward. In the other case, the
head was pointed toward the rearview mirror.

Table 14.10. LDW alert scenarios and scenario categories

+-----------------+----------------------------------------------------+
| **LDW scenario  | **LDW scenario**                                   |
| category**      |                                                    |
+-----------------+----------------------------------------------------+
| False alert     | -   Lane tracking incorrectly locks on visual      |
|                 |     feature other than a lane edge.                |
+-----------------+----------------------------------------------------+
| Purposeful and  | -   Lane change without use of turn signal.        |
| complete        |                                                    |
| departure of    | -   Changes lanes without turn signal while moving |
| all wheels from |     into (or out of) a freeway ramp or turn lane   |
| lane            |     (i.e., "cuts over a changing lane marker").    |
+-----------------+----------------------------------------------------+
| Purposeful but  | -   Crosses lane boundary while cutting to inside  |
| incomplete      |     of a curve.                                    |
| departure from  |                                                    |
| lane (only two  | -   Crosses lane boundary while moving to one side |
| wheels leave    |     of a lane to stay away from another vehicle,   |
| lane)           |     construction items, etc.                       |
|                 |                                                    |
|                 | -   Cuts over a transitional bend in a lane        |
|                 |     marker, e.g., at the beginning or end of a     |
|                 |     turn lane near an intersection.                |
|                 |                                                    |
|                 | -   Moves over a lane marker during a merge, i.e., |
|                 |     an "early" merge.                              |
|                 |                                                    |
|                 | -   Crosses lane boundary while moving around a    |
|                 |     slower vehicle that is waiting to turn.        |
+-----------------+----------------------------------------------------+
| Incidental and  | -   Drifts over a lane boundary on a straight road |
| incomplete      |     section for reasons other than cited in        |
| departure from  |     earlier categories.                            |
| lane            |                                                    |
|                 | -   Moves over a lane boundary due to a delayed    |
|                 |     steering correction after completing a curve   |
|                 |     or turn.                                       |
+-----------------+----------------------------------------------------+

Table 14.11. Reduction in LDW alerts by scenario category

  ------------------------------- -------------------------- ---------------------- ---------------
  **\                             **Non-adaptive-presented   **Adaptive-presented   **Change in LDW
  Scenario category**             LDWs**                     LDWs**                 alerts/mile**

  False alert                     12                         1                      -91%

  Purposeful and complete         32                         7                      -76%
  departure                                                                         

  Purposeful and incomplete       33                         1                      -97%
  departure                                                                         

  Incidental departure            16                         1                      -93%

  **All LDW alerts**              **93**                     **10**                 **-88%**
  ------------------------------- -------------------------- ---------------------- ---------------

FCW Alerts

Recall that the adaptive mechanism for FCW is that the assumed driver
reaction time is varied in response to head pose (see Section 14.2.2).
Relative to non-adaptive FCW, this adaptive mechanism can result in an
FCW alert that is earlier, later, or at the same time as the
non-adaptive FCW system would provide. When the driver's head pose
continues to be away from the forward scene for a certain period of time
(about 1.5 seconds or more) , the FCW alert may occur earlier in
adaptive mode. At times, this will result in an adaptive-mode alert that
would not have occurred in non-adaptive mode. This is because events
subsequent to the adaptive FCW -- but prior to the time at which a
non-adaptive alert would occur -- can inhibit the non-adaptive FCW. This
may be due to driver braking, one or both vehicles changing lanes, or
the lead vehicle ceasing to decelerate or even speeding up. Conversely,
by delaying an alert, the adaptive FCW can lead to the suppression of an
FCW alert when events that occur after the non-adaptive timing and
before the adaptive timing preclude an alert from occurring.

Table 14.12 lists some driving scenarios in which FCW alerts were
observed and groups them into three categories: the host vehicle and the
preceding vehicle share the same travel lane throughout the episode; one
of the two vehicles changes lanes or turns; and false alerts triggered
by objects or vehicles that are never in the travel lane of the host
vehicle. Table 14.13 shows the number of alerts observed in these three
categories during both adaptive-mode and non-adaptive-mode driving. The
difference in alert rates is shown (alert rates are used for comparison
since the number of drivers is different with and without the adaptive
system).

In order to gain insight into the effect of the adaptive mechanisms on
the FCW performance, significant effort was invested in reviewing the
data collected during the adaptive-mode drives. This section considers
the driving scenarios and circumstances of those alerts, and
specifically the nature of alerts that are effectively suppressed or
introduced by the adaptive mechanisms.

Table 14.12. FCW alert scenarios and scenario categories

+-------------------+--------------------------------------------------+
| **FCW scenario    | **FCW scenario**                                 |
| category**        |                                                  |
+-------------------+--------------------------------------------------+
| Same-lane: Both   | -   Approaching a slower lead vehicle            |
| the host and the  |                                                  |
| target vehicle    | -   A lead vehicle decelerates                   |
| share the same    |                                                  |
| lane throughout   | -   Tailgating                                   |
| episode           |                                                  |
+-------------------+--------------------------------------------------+
| Multi-lane:       | -   Approaching a lead vehicle that is turning   |
| Either the host   |     or leaving the lane                          |
| or the target     |                                                  |
| vehicle change    | -   Encountering a lead vehicle that has cut in  |
| lanes, turn,      |     front of the driver's vehicle                |
| merge, or exit    |                                                  |
+-------------------+--------------------------------------------------+
| False alerts:     | -   FCW is triggered by a vehicle in the         |
| Alert is          |     adjacent lane, and neither vehicle is        |
| triggered by an   |     changing lanes                               |
| adjacent lane     |                                                  |
| vehicle           |                                                  |
+-------------------+--------------------------------------------------+

Table 14.13. Reduction in FCW alerts by scenario category

  --------------------------------- -------------------------- ---------------------- ---------------
  **\                               **Non-adaptive-presented   **Adaptive-presented   **Change in FCW
  Scenario category**               FCWs**                     FCWs**                 alerts/mile**

  Same-lane: Both the host and the  21                         15                     -22%
  target vehicle share the same                                                       
  lane throughout episode                                                             

  Multi-lane: Either the host or    29                         6                      -77%
  the target vehicle change lanes,                                                    
  turn, merge, or exit                                                                

  False alerts: Alert is triggered  12                         2                      -80%
  by an adjacent lane vehicle                                                         

  **All FCW alerts**                **62**                     **23**                 **-58%**
  --------------------------------- -------------------------- ---------------------- ---------------

The left half of Figure 14.18 shows a Venn diagram describing the system
performance in adaptive-mode driving. There were 23 alerts presented to
the driver. Using a background data signal from the SAVE-IT prototype
system that indicated the threshold for FCW alert range, it was possible
to identify alerts that *would have been* presented, if the system had
been in non-adaptive mode during those drives. Figure 14-18 shows that
71 alerts would have been presented, hence the adaptive mechanisms
reduced the total number of alerts by 68 percent from 71 to 23 alerts.
The alert events corresponding to the three regions of the left-side
Venn diagram were studied in some detail, in order to gain insight into
the adaptive mechanisms for FCW.

The left half of Figure 14.18 shows that only five of the 23 alerts
presented to the driver were unique to the adaptive mode, i.e., these
five would not have occurred in non-adaptive mode. These alerts are due
to longer assumed driver reaction times that the adaptive system applies
when drivers' head pose is perceived to be away from the forward scene
for approximately one second or longer. Upon review of these events, all
five involved approaches to a vehicle ahead during a lane change, with
drivers looking at mirrors or directly out windows to assess the gap in
the adjacent lane. Three of these occurred during one driver's drive;
this driver appeared attentive but was very active in looking at mirrors
or directly out the side windows during lane changes. Upon detailed
review of the events, four of the five alerts were associated with
missed detection of short periods in which the driver's head had moved
forward briefly (0.3 to 0.6 seconds). In the fifth case, the head pose
was not forward for two seconds, but the driver's eyes were occasionally
monitoring the forward scene. See Appendix N for examples of these
alerts.

![](media/image17.wmf){width="6.498611111111111in"
height="2.5097222222222224in"}Figure 14-18. FCW alerts presented and/or
suppressed during on-road adaptive-mode driving

Figure 14-18 shows that 18 alerts that were presented during
adaptive-mode driving would have still occurred had the system been in
non-adaptive mode. The timing of most of these alerts, however, would
have been different in the two modes. Figure 14.19 shows the relative
timing for adaptive and non-adaptive versions of the 15 alerts that are
common to the two modes. During the adaptive drives, 11 events would
have been delayed, relative to the time that they would have occurred in
non-adaptive mode. Another four were presented earlier in adaptive mode
than they would have been in the non-adaptive mode. Three FCW alerts
were presented at the same time in both modes.

![](media/image18.wmf){width="5.277083333333334in"
height="2.6770833333333335in"}

Figure 14.19. Relative timing of 18 on-road FCW alerts that are common
to the adaptive\
and non-adaptive systems

Note that for two of the 18 events, the adaptive mechanism delays the
FCW alert by more than a second. One event involved a constant-speed
approach to a turning lead vehicle. The adaptive alert was presented 1.9
seconds after the non-adaptive alert would have been presented. Although
the time to collision (distance divided by closing speed) was only 1.8
seconds at the time of alert onset, the delay in the adaptive alert did
not pose a safety risk in the actual event, since the driver was
attentive and the lead vehicle was clearly intending to turn at a
signalized intersection with no impediments to turning. The second event
involved another driver passing a turning vehicle. In this second case,
the time to collision associated with the adaptive alert was 3.8 seconds
at the time of alert onset, a fairly reasonable timing for FCW (see
Figure 6.81 in Ervin et al., 2005). Examples of alerts that are common
to the adaptive and non-adaptive versions are illustrated in Appendix O.
This includes one case in which the alert occurs earlier in adaptive
mode, and another that occurs later in adaptive mode.

The 53 events that did not trigger an alert in adaptive-mode driving
(but would have in non-adaptive mode) were also individually reviewed.
In all cases, the driver appeared to be attentive to the driving task.
In one of these events, however, the driver was changing lanes on a
freeway to move around a slower and decelerating vehicle. This driver
(the same one mentioned earlier who moves his head rapidly back and
forth from the forward scene to looking out side windows to find and
confirm gaps available for changing lanes) appeared to leave himself
vulnerable to the lead vehicle increasing its deceleration during this
period, as his eyes were often off the forward scene in the few seconds
of this maneuver. But the lead vehicle did not do so, and the lane
change was achieved without event.

For the record, the driving scenarios associated with all 53 events that
were effectively suppressed by the adaptive mechanisms are summarized in
Table 14.14.

Table 14.14. Scenario of FCW suppressed alerts in adaptive on-road
drives

  --------------- ----------------------------------------- ---------------------
  **Scenario      **Driving scenario**                      **Non-adaptive-only
  category**                                                "alerts"**

  Same-lane\      Vehicle ahead decelerating                17
  (21 alerts,\                                              
  or 40%)                                                   

                  Approaching a slower vehicle              3

                  Tailgating                                1

  Multi-lane\     Passing a slower or slowing vehicle       12
  (28 alerts,\                                              
  or 53%)                                                   

                  Other vehicle cuts in front               2

                  Vehicle on cross street cuts across lane  1

                  Lead vehicle turning left or right        10

                  Lead vehicle changing lanes               3

  False alert\    False alert (adjacent lane traffic)       **4**
  (4 alerts,\                                               
  or 8%)                                                    

  **All                                                     **53**
  suppressed FCW                                            
  alerts in                                                 
  adaptive-mode                                             
  driving:**                                                
  --------------- ----------------------------------------- ---------------------

Caveat on Expert Review

The reader is likely aware that analyst judgments of event severity or
alert appropriateness have a basic caveat: There is rather little
established criteria for making such judgments. The closest basis for
such judgments are crash risk factors such as eyes-off-road duration,
time to collision, and so on (Dingus et al., 2006). The joint
distribution of these is, however, unknown and the predictive quality is
not established. When expert analysts view the events themselves,
however, there is often disagreement over the true potential crash risk.
Instead, as is true in this evaluation effort, an analyst who has
decided to judge the appropriateness of an alert or the risk potential
of an event is essentially deciding, for each event, whether the
probability -- over a set of similar hypothetical events -- exceeds an
unstated threshold of carrying forward to a crash. In doing so, the
reviewer is implicitly considering small variations in either the
driver's attentiveness or awareness, the circumstances of the events
(e.g., shoulder condition, traffic signal state), or the events that are
unfolding (e.g., lead vehicle deceleration, traffic signal state), and
deciding whether they could lead to a near crash or crash. This is done
without the rigor (in this study, at least) of specifying the variables
and their distributions.

Thus the outcome of this judgment will vary from analyst to analyst due
to differences in the assumptions of variability in the different
variables (What if the driver had not looked up as soon? What if the
lead vehicle had continued to decelerate?), and the level of risk
tolerance by the analyst (How conservative should the alert function
be?).

Despite this, the exercise of expert judgment makes sense in some cases,
especially when labeling situations as not risky or labeling alerts are
not required. It is asserted here that a judgment that an event is
risky, or that an alert is necessary, is less reliable when based on
expert review than judgments of benignancy. This is because experts have
more intimate experience with benign driving than with crash events.
That said, the description of individual events and the reporting of
analyst judgments is a significant part of this report's analysis.

Conclusions of System Performance during On-Road Testing

The SAVE-IT prototype vehicle's adaptive mechanisms greatly reduce the
number of alerts presented to the driver, by 58 percent for LDW alerts
and 88 percent for FCW alerts. This may encourage improved driver
acceptance of LDW and/or FCW. Most alerts that do occur in adaptive mode
appear to be caused by errors in the head pose system, in particular
during events in which the driver repeatedly moves his or her head away
from the forward scene and then back forward. The SAVE-IT system has
been observed to occasionally fail to detect (or use the detection) of
brief periods of looking forward that are imbedded in between
away-looking episodes. Also, in a few cases, the driver's head is turned
away but their eyes are monitoring the scene. The false positive alerts
may challenge the driver's sense of system performance consistency,
which could be a negative factor in driver acceptance.

All alerts that are effectively suppressed during these experiments by
the adaptive mechanisms appear to have occurred in circumstances in
which the driver was attentive and there was no clear risk of crash or
serious conflict. That is, there were no observations of SAVE-IT alerts
that were potentially beneficial in helping the driver notice, or react
to, a conflict with potential to lead to a crash. Hence, no negative
unintended consequences of suppressing alerts was directly observed
(although see the subsequent subsections). Failure to observe negative
effects of delaying or suppressing alerts is not a surprising outcome,
nor necessarily a definitive one, given the limited scope of testing and
the lack of near-crashes or crashes in this evaluation experiment.

### 14.4.1.3 Head Pose as a Surrogate for Attention: Analyzing FOT Data {#head-pose-as-a-surrogate-for-attention-analyzing-fot-data .SaveIT-H3}

The SAVE-IT concept of using head pose to sometimes suppress LDW alerts
and to vary the timing of FCW alerts is intended in part to reduce the
nuisance associated with alerts given to attentive drivers, and in the
case of FCW, provide inattentive drivers with additional safety margin.
A concern arises, of course, when suppressing or delaying alerts, and
this evaluation must consider whether suppression or delay of alerts
could prevent an alert from being given in a situation where the driver
would benefit from an alert. This section presents an analysis of
existing databases of LDW and FCW alerts that occurred in recent field
operational tests (FOTs) in order to get an initial view of how many
alerts could be suppressed with the SAVE-IT concept, and whether there
is evidence that this suppression would have prevented useful alerts
that may have been observed in the FOTs. The study also makes a simple
correlation between the driver head pose logic that is used in the
SAVE-IT prototype and an expert's judgment of the driver's visual
engagement in the driving task at the time of the FOT alert events. The
two subsections that follow address, respectively, LDW alerts and FCW
alerts.

The approach taken in these studies is to review events in the FOT data
in which alerts were provided by the systems under test in those
experiments. By overlaying the SAVE-IT mechanisms for suppressing LDW
alerts or advancing/delaying FCW alerts, an estimate is made of the
percent of the FOT alert events that may have been suppressed, delayed,
and/or advanced if the SAVE-IT mechanisms had been in place in those
tests. Furthermore in this analysis, an expert makes a judgment of the
driver's visual engagement in driving for each of the reviewed events.
Given these judgments, it is then possible to correlate the
visual-engagement judgment with the estimate of SAVE-IT mechanisms on
the suppression, delaying, and/or advancing of the alerts.

Regarding the use of visual engagement in driving, this is intended to
represent visual attention but cannot be construed to represent either
cognitive attention to driving or full awareness of the driving
environment. As Klauer et al. (2006) note, visual engagement in driving
does not necessarily mean that the driver is aware of a potential crash
threat. For example, a driver may be looking in their rearview mirror
when a vehicle ahead abruptly begins to brake; the driver is engaged,
but not aware. The SAVE-IT philosophy, however, is similar to most other
mature crash avoidance warning systems in that it seems intended to
assist drivers who are temporarily not engaged visually in driving.

14.4.2.1 Overlaying a Head-Pose Suppression Logic on LDW Alerts from the
RDCW FOT

To estimate the effect that a head-pose suppression mechanism might have
on the number of LDW alerts experienced by a driver, and to get an
initial view of whether the suppression may have negative safety
impacts, an analysis of the Road Departure Crash Warning System Field
Operational Test (RDCW FOT) data was performed. The RDCW FOT involved 78
subjects each driving a vehicle equipped with an LDW and a curve-speed
warning system for four weeks. During the first week, the alerts were
not presented in order to capture each driver's baseline performance.
For the SAVE-IT project, an analysis was conducted in which the data and
video associated with 236 sampled LDW alerts from that baseline period
were reviewed. The baseline data were used since driver behavior with an
adaptive LDW system is assumed to be, in terms of lane keeping and turn
signal use, closer to the RDCW FOT baseline behavior than to the
behavior with RDCW FOT enabled. This is because the adaptive system will
have an alert rate that is much closer to zero (i.e., baseline) than to
the RDCW FOT enabled state.

Three values were coded for each of the sampled RDCW FOT events:

-   Head pose in the period leading up to the LDW alert

-   Whether the alert was obviously valuable in returning the driver\'s
    attention to a lane drift

-   An analyst\'s judgment of whether the driver was visually engaged in
    driving in the period leading up to the LDW alert

These three values are shown in Table 14.14, along with the values that
each could be assigned to the variables.

Table 14.14. Variables coded during review of RDCW FOT data

+--------------------+-------------------------------------------------+
| **Variable coded** | **Choices for coding the variable**             |
+--------------------+-------------------------------------------------+
| Head pose during   | -   Pose forward always                         |
| the period from 2  |                                                 |
| seconds before an  | -   Pose forward sometimes                      |
| alert to the time  |                                                 |
| of the alert       | -   Pose forward never                          |
|                    |                                                 |
|                    | -   Pose unknown                                |
+--------------------+-------------------------------------------------+
| Driver's visual    | -   Visually engaged.                           |
| engagement with    |                                                 |
| driving during the | -   Driver has glances away, but it is          |
| 2 (or more)        |     difficult to determine whether they are or  |
| seconds before the |     are not related to driving.                 |
| alert.             |                                                 |
|                    | -   Driver has glances away that are not        |
|                    |     related to driving.                         |
|                    |                                                 |
|                    | -   Not visually engaged (gross inattention)    |
|                    |                                                 |
|                    | -   Video does not support a decision by        |
|                    |     analyst                                     |
+--------------------+-------------------------------------------------+
| Analyst's view of  | -   Driver was unaware of lane drift and alert  |
| potential safety   |     may have returned their attention to the    |
| impact of LDW      |     situation.                                  |
| alert              |                                                 |
|                    | -   Driver was aware of lane drift, or if       |
|                    |     unaware, alert would not have had any       |
|                    |     impact.                                     |
+--------------------+-------------------------------------------------+

The head pose was considered during a two-second period leading up to
the LDW alert in the RDCW FOT. Head pose was considered to be forward if
the driver's nose was thought to be pointed in the azimuth direction
between the midpoint of the windshield and a point on the windshield
just inside of the A-pillar. See Figure 14.20 for examples. Using the
SAVE-IT adaptive mechanism perspective, the coding considered whether
the head pose was forward always, sometimes, or never during that
two-second period. Note that for several of the reviewed events, the
reviewer needed to make a judgment in some cases about whether the
driver's nose was pointed to the left or right of the windshield center.
The sensitivity of the results to this judgment is thought to be rather
limited since there were very few instances where a driver may have not
looked forward at least during some portion of the two seconds. Even
when there was question about the head pose at a particular moment, the
fact that drivers looked forward rather often led to some period of the
two second window showing the driver clearly looking forward. Whether
they were looking forward during some subset of that time was not
relevant to this analysis.

![](media/image19.wmf){width="5.84375in" height="4.166666666666667in"}

Figure 14.20. Examples of driver head pose and glances from the RDCW FOT
data set

The second variable that was coded involved the reviewer noting whether
the driver appeared to be visually engaged in the driving task. This
judgment was made based mostly on the two-second period before the
alert, but often it was necessary to review earlier data (several
seconds) to confirm or modify decisions made using the two-second
period. In most instances, the driver appeared to be using their
eyesight for the driving task. In some instances, however, the driver
was seen to glance away from the forward scene or the rearview mirrors.
In this case, an attempt was made to determine whether the driver's
glances were related to driving (such as looking at a speedometer or a
road sign) or were associated with non-driving activities, such as
looking at scenery or looking at rear-seat passengers in the rear-view
mirror as they conversed. If the glances away were part of the driving
task, e.g., looking at rearview mirrors, then the driver was still
considered to be visually engaged in the task. If the driver was looking
at scenery, the center console (where the radio and heating/cooling
controls are), or at another passenger, that was considered glancing
away for non-driving reasons. In none of the 236 cases was a driver so
grossly inattentive in a visual sense that he or she did not spend at
least some fraction of the two seconds looking forward. Images from the
RDCW FOT video showing various head pose and driver glance examples are
shown in Figure 14.20.

The third variable was intended to identify any events in which an LDW
alert may have provided a positive safety benefit. The purpose of
identifying such events was to report any obvious evidence that the
SAVE-IT adaptive mechanisms might suppress LDW alerts that were in fact
valuable alerts. A potential positive safety benefit from a given LDW
alert was considered to occur if the alert may have helped the driver
return attention to the forward scene to correct a lane drift. To
determine that such a safety benefit was possible, the analyst was
required to decide that the driver had not been paying attention to the
forward scene, and that the alert brought their eyes forward at the time
that a lane departure was occurring or had recently occurred. However,
for the 236 sampled LDW alerts, such a clear and obvious case was not
found.

The 236 LDW alerts that were reviewed were selected from a set of over
800 LDW alerts that were previously reviewed and coded for other
analyses during the RDCW FOT effort. The 236 sampled alerts were chosen
to include a subset of the 285 previously-coded alerts that occurred
during the baseline period of the RDCW FOT (the first week of driving,
per test participant). A subset was used in this analysis to provide the
same balance of cautionary and imminent LDW alerts as were received
overall by drivers in the baseline portion of the FOT (58.9 percent of
the FOT alerts were imminent alerts). Thus all 139 baseline imminent
alerts were reviewed for this analysis, and then 97 cautionary alerts
were randomly selected so that the ratio of imminent to cautionary was
58.9 percent to 41.1 percent, which totals 236 alerts.

Figure 14.21 shows the overall results for the observation of head pose
during the two-second period leading up to an LDW alert. In 59 percent
of the alerts reviewed, the driver's head was always forward within that
two-second window. In the remaining 41 percent of cases, the driver's
head was forward for part of the two-second window, and not-forward
during the rest of the window. In the 236 alerts sampled, there were no
cases in which the driver's head was never forward during those two
seconds. This means that all 236 of these alerts would have been
suppressed by a system with perfect sensing of head position, with no
latencies, and with an algorithm that suppresses LDW alerts if the
driver's head is forward at some moment in the two seconds before an LDW
alert decision is made. The reality of implementing such a system could
of course lead to some alerts occurring. In addition to the errors in
tracking driver head pose possibly introducing an alert, it is possible
that in this sample that a very few instances may have occurred in which
a driver was looking away for the latter portion of the two-second
window (e.g., the latter 1.5 seconds), and continued to look away for a
short period after the alert (e.g., 0.6 seconds). In this case, it is
possible a system would delay the alert but finally allow it based on a
continuous two-second period of the head pose never being forward.
Nevertheless, the use of head pose over a two-second window to suppress
an LDW alert is very powerful and is capable of appropriately
suppressing almost all LDW alerts.

![](media/image20.wmf){width="4.040972222222222in"
height="3.2194444444444446in"}

Figure 14.21. Driver head pose during a two-second window leading up to\
236 sampled alerts from the RDCW FOT

To study whether head pose is a useful surrogate for visual engagement
in driving for the purposes of lane-drift warnings, a correlation of the
video-coded head pose variable and the visual-engagement variable was
performed for the 236 events. (See Table 14.4 again for the variable
definitions.) Specifically, the visual engagement is compared between
the two cases of head always forward and head sometimes forward, which
address the two seconds preceding an LDW alert.

During the two seconds leading up to each of the 236 LDW events, the
reviewer made a determination of whether there were eye glances away
from the forward scene that were not related to driving. (Some eye
glances away occur without head pose changing.) Figure 14.22 shows the
frequency of driver glances away from the driving scene for non-driving
reasons. This includes glances at scenery, at passengers, glances toward
center console controls such as the radio, phone dialing, grooming, and
so on. Figure 14.22 shows that for those where the head pose was forward
during the entire two seconds leading up to the alert (i.e., the lower
bar in the figure), in 84 percent of the events, the driver did not make
any glances for non-driving reasons during this period. Within this
subset of cases, there were glances toward mirrors or scans of the
forward scene, but any such glances were considered to be very likely
for the purposes of driving. The figure also shows that nine percent of
the head-always-forward cases did involve glances that were for
non-driving reasons. Another 6 percent of these cases involved glances
that were difficult to assign as either driving or non-driving glances.
A common case of the difficult-to-assign cases is when a driver looks in
the rearview mirror when a rear-seat passenger was visible. At times it
was clear the driver was looking as part of a conversation; at other
times it seemed clearly related to driving; and other times it was
ambiguous. Finally, only 1 percent of the cases involved video data that
was washed out or otherwise compromised enough to prevent analyst
judgments from being made.

![](media/image21.wmf){width="5.9847222222222225in"
height="3.9159722222222224in"}

Figure 14.22. Correlation of head pose status in the two seconds before
LDW alerts in an FOT experiment and the visual engagement of the driver

A conclusion from this figure is that head pose measurement is not a
perfect surrogate for detecting visual distraction in drivers. This was
never thought to be possible, of course, but the data confirms the fact.

The upper bar in Figure 14.22 shows the same analysis for the 97 cases
in which the head pose was both forward and not-forward during the
two-second window leading to the LDW alert. In this case, the percentage
of cases in which non-driving-related glances were observed was 21
percent, or over twice the percentage seen when the head pose was always
forward. Thus a head-not-forward measurement is more suggestive of
visual distraction than a measurement where head pose remains forward.
The level of suggestiveness, however, is modest.

The third variable coded for the LDW events in the field operational
test data was to identify rare and possibly dramatic cases in which the
LDW alert may have had potential to prevent a dangerous lane departure.
Because only the baseline-period alerts were reviewed (in which the
alert displays were not presented), there was no opportunity to observe
a driver responding to an alert. However, none of the 236 sampled events
involved a lane departure that seemed to entail any danger of running
off the road or crashing into adjacent-lane traffic. The conclusion is
that this small data set does not contain evidence that head pose
suppression would have inhibited potentially useful alerts. Note,
however, that there are cases in which drivers fall asleep and run off
the road, so that the effectiveness of LDW alerts that are suppressed
when the driver's head is forward in an azimuthal sense can be
conjectured to be minimal in many of these cases.

14.4.2.2 Overlaying a Head-Pose Suppression Logic on FCW Alerts from the
ACAS FOT

To estimate the effect that a head-pose suppression mechanism might have
on the number of FCW alerts experienced by a driver, and to get an
initial view of whether the suppression may have negative safety
impacts, an analysis of the Automotive Collision Avoidance System Field
Operational Test (ACAS FOT) data was performed. The ACAS FOT involved 96
subjects each driving a vehicle equipped with adaptive cruise control
(ACC) and forward crash warning (FCW) for four weeks. During the first
week, the alerts were not presented in order to capture each driver's
baseline performance. The remaining three weeks of each driver's
experience comprised the ACAS-enabled period.

For the SAVE-IT project, an analysis was conducted using 200 FCW alerts
from the ACAS project. This included 169 alerts sampled randomly from
the baseline period. The baseline period data were used to avoid the
possibility that the ACAS function influenced driver behavior. This
sample set was also drawn only from periods during which the cruise
control system was not being used, since FCW alerts in cruise mode are
typically quite different in their timing since the presence of FCW
suggests that conventional cruise will not be used, and adaptive cruise
will be employed. Adaptive cruise control includes a conflict management
effect that substantially impacts the occurrence of conflicts and driver
behavior (Ervin et al., 2005).

An additional 31 alerts that were labeled as potentially useful by the
organizations that performed the original analyses of the ACAS data (13
alerts were labeled as useful by UMTRI in a targeted review of a subset
of ACAS FCW alerts, and the U.S. DOT Volpe National Transportation
Center agreed with the judgment for eight of those same alerts, plus
another 18 alerts that were reviewed in the larger set of ACAS alerts).
All of the events in which the alert was originally rated as potentially
useful occurred during the ACAS-enabled period, during which alerts were
actually presented.

Five variables were coded for each of the sampled ACAS FOT events. These
variables and the choice of values that each could be assigned are
presented in Table 14.15. The variables were designed to provide
information about three topics: (1) whether the SAVE-IT mechanisms may
have suppressed, delayed, or advanced in time the occurrence of ACAS FOT
alert events, had the SAVE-IT mechanisms been in effect; (2) whether the
driver appeared to be visually engaged in the driving process at the
time of the alert; and (3) a review to identify events that were
associated with any near-crash events.

Table 14.15. Variables coded during review of ACAS FOT data

+----------------------------+-----------------------------------------+
| **Variable coded**         | **Choices for coding the variable**     |
+----------------------------+-----------------------------------------+
| Head pose at the time that | -   Pose forward always during this     |
| ACAS sent commands to the  |     period                              |
| display hardware (plus the |                                         |
| head pose at the time 0.1  | -   Pose forward for at least one 0.1   |
| sec before the display     |     sec sample during this period       |
| command),                  |                                         |
|                            | -   Pose never forward during the       |
|                            |     period                              |
|                            |                                         |
|                            | -   Pose either not forward or close to |
|                            |     not-forward during the entire       |
|                            |     period                              |
|                            |                                         |
|                            | -   Pose unknown                        |
+----------------------------+-----------------------------------------+
| Head pose during the       | (Same as above)                         |
| period beginning 1.5 sec   |                                         |
| before the display command |                                         |
| is given and ending 0.2    |                                         |
| sec before the display     |                                         |
| command is given           |                                         |
+----------------------------+-----------------------------------------+
| Head pose during the       | (Same as above)                         |
| period beginning 0.1 sec   |                                         |
| *after* the alert command  |                                         |
| is given and continuing    |                                         |
| until 1.0 sec after the    |                                         |
| alert command              |                                         |
+----------------------------+-----------------------------------------+
| Driver's visual engagement | -   Visually engaged.                   |
| with driving during the    |                                         |
| 1.5 sec (or more) before   | -   Driver has glances away, but it is  |
| the alert.                 |     difficult to determine whether they |
|                            |     are or are not related to driving.  |
|                            |                                         |
|                            | -   Driver has glances away that are    |
|                            |     not related to driving.             |
|                            |                                         |
|                            | -   Not visually engaged (gross         |
|                            |     inattention)                        |
|                            |                                         |
|                            | -   Video does not support a decision   |
|                            |     by analyst                          |
+----------------------------+-----------------------------------------+
| Analyst's view of          | Baseline alerts:                        |
| potential safety impact of |                                         |
| FCW alert                  | -   Yes/No: Driver was unaware of       |
|                            |     forward conflict and alert may have |
|                            |     returned their attention to the     |
|                            |     situation.                          |
|                            |                                         |
|                            | ACAS-enabled alerts:                    |
|                            |                                         |
|                            | -   Yes/No: Alert clearly returned an   |
|                            |     inattentive driver's attention to   |
|                            |     the forward conflict scene.         |
+----------------------------+-----------------------------------------+

The first three variables listed in Table 14.15 address the driver's
head pose in the moments before and following a baseline-period FCW
alert during the ACAS FOT experiment. Recall that baseline-period
"alerts" are not displayed to the driver, but are events flagged in the
data to indicate that if the driver displays had been enabled, an FCW
alert would have occurred. Head pose is defined in the same manner
described in the previous subsection on RDCW FOT alerts. Here, however,
variables are coded for three different periods, in an attempt to allow
better estimates of how the SAVE-IT FCW mechanisms function. The fourth
variable is the same as previously coded for the RDCW FOT drivers, and
addresses the level of apparent driver involvement in driving, at least
from the perspective of scanning in directions associated with safe and
attentive driving. Note that the period is only 1.5 seconds before the
alert, and not the 2.0 seconds used for the RDCW FOT data. The fifth and
final variable assigned during video review is again similar to a data
element described previously, except with the ACAS FOT data, both
baseline and display-enabled alerts are considered.

The SAVE-IT mechanisms for altering the FCW alert logic are more complex
than those used for LDW, as described at length in Section 14.2.2 (see
especially Figure 14.7). Extensive work was undertaken to find a means
of using video coding of head pose to estimate the likely change in FCW
alert timing for the ACAS FOT data. However, FCW alerts involve
comparing the current distance to a lead vehicle with an
internally-computed warning threshold distance, with additional
heuristics added in the FCW system to consider the likely paths, the
possible intentions of the driver, and so on. The threshold distance
used in the ACAS FOT, as well as the heuristics and other logic, are not
available or recorded in the data set. In order to determine whether,
and when, an FCW alert would occur then requires access to all the
detailed logic within the FCW system. This is much different from the
LDW case, since its own internal logic is significantly simper and the
SAVE-IT mechanism was simpler as well.

Thus, for FCW, the most useful mapping of driver head pose coding to the
likely impact of SAVE-IT's adaptive mechanism on the FCW alert observed
in the ACAS FOT data is as follows:

-   If the driver head pose was **not** forward for the 1.5 seconds
    preceding the alert, and was not forward at the moment of the alert,
    then a SAVE-IT-influenced FCW alert is **likely to be provided
    earlier** than the non-adaptive (ACAS FOT) alert.

-   If the driver head pose was **always** forward for the 1.5 seconds
    preceding the alert, and was still forward at the moment of the
    alert, then a SAVE-IT-influenced FCW alert is **likely to be
    provided later** than the non-adaptive (ACAS FOT) alert. Depending
    on the magnitude of delay, the alert is often **effectively
    suppressed**. The magnitude of the delay would vary according to the
    kinematic conditions before, during, and after the alert. Driver
    actions and vehicle motions are also important in this regard. In
    many (probably most) cases, these alerts would effectively be
    suppressed since conditions would have changed in the intervening
    period. This was seen in the on-road experiments with the SAVE-IT
    prototype in adaptive mode, in which 11 FCW alerts were delayed, but
    eventually presented to the driver, and 52 were effectively
    suppressed by a delay. This means that 83 percent of the SAVE-IT
    alerts that were delayed were effectively suppressed (52 of 63
    events).

-   If the driver's head pose was **not** forward for some portion of
    the period beginning 1.5 seconds before the alert and ending at the
    time of alert presentation, but was indeed forward for another
    portion of that same period, the SAVE-IT mechanism was considered to
    have an **unknown impact on the alert occurrence**. This is a
    simplified rule that recognizes the complexity of overlaying the
    SAVE-IT logic on the unknown ACAS FOT logic, and fortunately the
    results of the study are not seriously impacted because it will be
    shown that drivers rarely are seriously distracted during these
    events.

-   The SAVE-IT mechanism would very likely **introduce new FCW alerts**
    into the ACAS FOT data set, had the mechanism been implemented. This
    would occur when the driver was considered to be visually distracted
    (with head pose not-forward), and when a forward conflict was
    developing. This was seen in Section 14.4.1.2 when the adaptive FCW
    system introduced five new alerts that would not have occurred with
    the non-adaptive SAVE-IT system. No estimate is made of the number
    of alerts that would have been created in the ACAS FOT experiment.

Given even the simplified rules above, there is some uncertainty
associated with the impact on FCW timing that is hinted at by the labels
"likely later" and "likely earlier." The direct impact of head pose,
through the SAVE-IT mechanism, is that the assumed driver reaction time
used to compute the warning threshold distance is decreased or increased
(see Section 14.2.2). But while head pose is sufficient to know whether
the reaction time is assumed to decrease or increase, there is in the
video coding an assumption that shorter or longer reaction times
generally result in later and earlier alerts, respectively. This is not
always true, as sometimes the conflict situations develop so rapidly
that an alert would occur simultaneously for a wide range of assumed
driver reaction times. Hence the labels become "likely later" and
"likely earlier."

The analysis of the 200 ACAS FOT alerts first addresses the random
sample of 169 baseline-period alerts. Table 14.16 shows the likely
impact of the SAVE-IT adaptive mechanism on the occurrence and timing of
this event sample, with the type of driving scenario represented by the
rows of the table and the likely effect of the adaptive mechanisms
represented by the columns. Of the 169 alert events, there are 143 alert
events that would likely be delayed (or effectively suppressed) by the
SAVE-IT mechanisms (85 percent delayed or suppressed). Recall that 52 of
63 delayed alerts observed in the adaptive drives of the SAVE-IT test
subjects resulted in effective suppressions of those alerts (85 percent
of delayed alerts were suppressed). If this were true for the sample of
169 baseline-period ACAS FOT alerts, an estimate of the number of alerts
remaining would be 47, for a total reduction in alert rate of 72
percent. This must be taken as an estimate, but clearly the mechanism is
capable of powerful reduction in the number of alerts.

Table 14.16 indicates that there is only one event in the set of 169
events in which the driver's head pose is in the not-forward direction
for the entire 1.5-second period leading up to the alert, so that there
is only one obvious case in which the alert is very likely to be
earlier. There are likely some alerts that would be earlier, if the
actual SAVE-IT mechanism were combined with the actual ACAS FOT logic,
but the data is not available to clearly support that on an
event-by-event basis.

Table 14.16 shows that there are 25 ACAS FOT alert events (15 percent of
the data set) for which the effect of SAVE-IT on the FCW alert timing is
difficult to estimate. These events are those in which the driver looked
away during some, but not all, of the 1.5-second period before, so that
the actual effect of the SAVE-IT mechanisms depends on the details of
the kinematics and the ACAS algorithm.

Table 14.16 also breaks down these results by driving scenario class
(see Table 14.12 for more details of scenario class definitions). The
rightmost column of Table 14.16 shows that the rate of delay/suppression
is distributed fairly evenly across the three classes of driving
scenarios, except there is less of this effect for out-of-path targets.
Consistent with this is the observation that 18 of the 25 events in
which the driver looked away for some period of the 1.5 seconds leading
to the alert we associated with a false FCW alert that was triggered by
out-of-path objects. Together this suggests that drivers may be more
likely to look away in situations without a forward conflict (i.e., in a
false FCW alert scenario) than when there is a possible forward
conflict. Further, this may imply that a mechanism that suppresses
alerts based on driver head pose may result in a system that has a much
lower overall alert rate, but a higher percentage of those remaining
alerts may be false alerts. This may have some negative impact on the
driver's sense of the system performance, but how this is balanced by
the hypothesized improvement in overall driver acceptance that results
from the lower overall alert rate is not known. The SAVE-IT system did
not respond to stopped objects, so that the rate of false alerts
triggered by out-of-path stationary objects (the primary source of FCW
false alerts in the current state of the art) was not part of the
SAVE-IT subject's experience.

Table 14.16. Likely changes in relative timing of FCW as function of
driving scenario

  ---------------- -------------- ------------ ------------ ---------------
                   Likely timing                            
                   and occurrence                           
                   of adaptive                              
                   alert,\                                  
                   relative to                              
                   non-adaptive                             
                   alert -                                  

  Scenario type    Alerts would   Alerts would Uncertain    Percent of
  for baseline     likely be      likely be    effect on    alerts likely
  (ACAS) alert     later, if ever earlier      alert timing delayed or
                                                            suppressed

  Same-lane        34             1            3            89%
  scenarios                                                 

  Multi-lane       41             0            4            91%
  scenarios                                                 

  Out-of-lane      68             0            18           79%
  targets                                                   

  Subtotal         143            1            25           85%
  ---------------- -------------- ------------ ------------ ---------------

Table 14.17 addresses whether the suppression, delay, or advancing of
FCW alerts is consistent with the intentions of suppressing or delaying
alerts only when the driver is visually engaged, and advancing the alert
in time when the driver is not "in the loop" visually. The columns of
Table 14.17 are identical to the previous table and represent estimates
of the impact of the adaptive mechanism on the timing or occurrence of
the alert. The rows are associated with the coding of driver visual
engagement for the alerts. Table 14.17 shows that drivers were judged to
be visually engaged in driving for 140 of the 169 baseline-period FCW
events (the first row of the table sums to 140 events, or 83 percent of
the events). The SAVE-IT mechanism would most likely delay or suppress
at least 130 of the 140 events in which the drivers were visually
engaged (93 percent). This is consistent with the intentions of the
SAVE-IT system. It is possible that some of the remaining 10 events
would also be suppressed or delayed.

Table 14.17. Correlation of driver's visual engagement in the driving
task and the likely change\
in relative timing of FCW alerts with the SAVE-IT adaptive mechanism

  ---------------------- -------------- ----------- ----------- -------------
                         **Likely                               
                         timing and                             
                         occurrence of                          
                         adaptive                               
                         alert,\                                
                         relative to                            
                         non-adaptive                           
                         alert**                                

  Driver appears to be   Alerts would   Alerts      Uncertain   Percent of
  visually engaged in    likely be      would       effect on   alerts likely
  driving                later, if ever likely be   alert       delayed or
                                        earlier     timing      suppressed

  Yes                    130            0           10          93%

  No                     0              1           2           \--

  Maybe                  13             0           10          57%

  Subtotal               143            1           22          85%
  ---------------------- -------------- ----------- ----------- -------------

For three of the events (2 percent), the driver was judged to be *not*
visually engaged in the driving task, although two of these were
associated with out-of-lane false alerts (roadside objects). The effect
of the SAVE-IT adaptive mechanism is uncertain in these cases. For the
sole vehicle-related case, the driver was involved in grooming while
looking in the rear-view mirror and looked back toward the forward scene
0.1 seconds after the alert (which was not presented in the baseline
period). In this case, the likely effect of SAVE-IT would have been to
provide the FCW alert earlier, a successful case of enhancing the
potential safety contribution of the FCW system.

There was uncertainty in the driver's visual engagement in driving for
another 22 of the events (14 percent), although one event was due to
poor video quality and 12 others were associated with roadside objects
or construction zone objects. Thus, when roadside objects and other
false alerts are removed, there is only one event out of 169 FCW alerts
in which the driver was not visually engaged with driving and a
potential forward crash conflict developed, and 10 events associated
with vehicle targets where there was some uncertainty about visual
engagement (6 percent of the 169-event set). The issue of the adaptive
mechanism's effect on timing with potentially distracted drivers is
pursued further in the next set of results.

Finally, we consider the likely effect that the SAVE-IT adaptive
mechanism would have had on those ACAS alert events that were previously
labeled by the independent evaluator of the ACAS FOT project (U.S. DOT's
Volpe National Transportation Systems Center) or UMTRI (during its role
as conductor and analyzing organization of the project) as being events
in which an FCW alert that was actually given to a driver may have
provided some utility for that event (Najm et al., (2006), Ervin et al.,
(2005)). Although there was 31 such events, this analysis considers only
30 since face video was not collected during one of the events (since no
FCW alert occurred). Ideally, the adaptive mechanisms would lead to
these alerts being earlier -- or at least no later -- than the actual
alert that occurred. The reader is reminded that no actual safety
analysis has been carried out in this evaluation effort, so that the
change in alert timing can only be taken as suggestive of safety
benefits or dis-benefits.

Upon re-review of these events for the SAVE-IT project, only eight of 30
events appeared to have met the standards used in coding during the
SAVE-IT analysis for "clear potential to enhance safety" during that
event (see Table 14.15 for a description of variables). This is
significant since it indicates that the 30 events do not provide an
ironclad set of events in which FCW alerts are necessarily required.
There is no further distinction in this report between the eight events
identified and the remaining 22 of the 30-event set being discussed
here.

Table 14.18 shows that 16 of the 30 FCW alert events would likely have
been delayed by an adaptive mechanism such as is used in the SAVE-IT
prototype. Only one would have been provided earlier, and the effect is
unknown for the remaining 13 events (43 percent of the 30-event set).

Table 14.18. Correlation of driver's visual engagement in the driving
task and the likely change\
in relative timing of "valuable" FCW alerts with the SAVE-IT adaptive
mechanism

  ---------------------- -------------- ----------- ----------- -------------
                         **Likely                               
                         timing and                             
                         occurrence of                          
                         adaptive                               
                         alert,\                                
                         relative to                            
                         non-adaptive                           
                         alert**                                

  Driver appears to be   Alerts would   Alerts      Uncertain   Percent of
  visually engaged in    likely be      would       effect on   alerts likely
  driving                later, if ever likely be   alert       delayed or
                                        earlier     timing      suppressed

  Yes                    11             0           2           85%

  No                     0              1           2\*         \--

  Maybe                  5              0           9           35%

  Subtotal               16             1           13          60%
  ---------------------- -------------- ----------- ----------- -------------

\* These two events were false alerts triggered by roadside objects.
Some other cells include other such events, too.

The adaptive mechanism is thus likely to lead to later FCW alerts in a
substantial number of driving circumstances in which an FCW alert is
judged post hoc by analysts to be have safety-enhancing potential. The
reference (Ervin et al., 2005) noted that a driver can be expected to
strike the rear end of another vehicle once every 117 years. Thus, given
that the ACAS-enabled period of testing represented about eight driving
years, there is a very good chance that none of the 30 events would have
led to a real rear-end crash. Thus the "safety-enhancing potential" must
be taken with some reservations, since clearly it is very unlikely that
all 30 events were leading to a real crash. However, due to the fact
that at least half of the analyst-judged alerts with this property would
have been delayed by the adaptive mechanisms, there is some evidence
that there is a possibility of loss of safety margin due to this
adaptive mechanism. Since this is one of many factors in evaluating the
adaptive mechanism, this subject will be brought up again in the
conclusions, after the driver acceptance results are presented.

Incidentally, in many cases in which the adaptive mechanism would delay
the FCW alert, the driver can be expected to sometimes react and brake
before the FCW alert occurs, effectively inhibiting the alert. The lack
of any alert may have a negative effect on the driver's perception of
the system performance reliability, or there may be a positive effect on
acceptance for those events in which the driver was actually aware of
the situation and did not feel an alert was called for.

######### 

## 14.4.2 Acceptance of the Adaptive Mechanisms for Crash Warning Systems {#acceptance-of-the-adaptive-mechanisms-for-crash-warning-systems .SaveIT-H2}

This section addresses the results of investigating the relative
improvement in driver acceptance derived from the adaptive mechanisms
associated with the crash warning systems. These results are based on
driver responses to the subjective instruments that were administered
throughout this evaluation activity, as previously listed in Table 14.4.
Also earlier in the report, the hypotheses were posed that the adaptive
mechanisms would result in improved driver acceptance (through reduction
in nuisance alerts), and that the driver could generate and maintain a
mental model that was consistent with system performance.

Five subsections provide results addressing, respectively, acceptance
ratings on the Van der Laan scale, post-drive questionnaire responses,
driver feedback from post-drive video review, final driver comments on
the adaptive and non-adaptive warning systems, and a summary of the
acceptance results. A separate section addresses data and results from
an exercise that examined driver perceptions that are likely related to
acceptance of the distraction mitigation system that controls access to
the in-vehicle information system in the SAVE-IT vehicle.

### 14.4.2.1 Acceptance on the Van der Laan Scale  {#acceptance-on-the-van-der-laan-scale .SaveIT-H3}

Throughout the testing, drivers were asked to provide numerical ratings
and open-ended written and verbal responses addressing aspects of their
likely acceptance of adaptive and non-adaptive crash warning systems.
One set of nine questions was used to compute values of usefulness and
satisfaction using the Van der Laan scale. The Van der Laan scale is
used to measure driver acceptance of new technology along the dimensions
of perceived usefulness and satisfaction of the technology (Van der
Laan, Heino, & de Waard, 1997). This scale has been used in several
previous studies of driver assistance systems, including Ervin et al.
(2005) and LeBlanc et al. (2006). The questions used to compute the Van
der Laan responses are given in Appendix H.

The responses to the nine questions are used to assign ratings on a
usefulness scale and a satisfaction scale, each ranging from -2 to +2,
with positive ratings corresponding to more positive perceptions. Mean
responses from those scales are provided below for each system and
SAVE-IT mode. Table 14.19 shows the mean values of these ratings from
the SAVE-IT test subjects.

Table 14.19 Mean values of Van der Laan scale results from Task 14C and
previous field operational tests

+---------+-----+-------+-----+-------+-----+--------+-----+-------+
|         | >   |       |     |       | *   |        |     |       |
|         | **T |       |     |       | *On |        |     |       |
|         | est |       |     |       | the |        |     |       |
|         | > t |       |     |       | roa |        |     |       |
|         | rac |       |     |       | d** |        |     |       |
|         | k** |       |     |       |     |        |     |       |
+---------+-----+-------+-----+-------+-----+--------+-----+-------+
|         | N   |       | Ada |       | N   |        | Ada |       |
|         | on- |       | pti |       | on- |        | pti |       |
|         | Ada |       | ve\ |       | Ada |        | ve\ |       |
|         | pti |       | N = |       | pti |        | N = |       |
|         | ve\ |       | 26  |       | ve\ |        | 12  |       |
|         | N = |       |     |       | N = |        |     |       |
|         | 26  |       |     |       | 11  |        |     |       |
+---------+-----+-------+-----+-------+-----+--------+-----+-------+
|         | Use | Satis | Use | Satis | Use | Sati   | Use | Satis |
|         | ful | fying | ful | fying | ful | sfying | ful | fying |
+---------+-----+-------+-----+-------+-----+--------+-----+-------+
| LDW     | 1.5 | 0.8   | 1.8 | 1.1   | 1.1 | 0.4    | 1.0 | 0.8   |
+---------+-----+-------+-----+-------+-----+--------+-----+-------+
| FCW     | 1.6 | 0.9   | 1.9 | 1.2   | 1.0 | 0.5    | 1.1 | 0.7   |
+---------+-----+-------+-----+-------+-----+--------+-----+-------+
| Dist    | \-  | \-\-- | 1.5 | 1.1   | \-  | \-\--  | \-  | \-\-- |
| raction | \-- |       |     |       | \-- |        | \-- |       |
| alert   |     |       |     |       |     |        |     |       |
+---------+-----+-------+-----+-------+-----+--------+-----+-------+

Results from the questionnaire administered after the on-road driving
are considered the primary results, since these drivers (though a
smaller sample) had a more complete exposure to the systems. All
ratings, whether from track or on-road studies, are generally positive
for FCW, LDW, and the distraction alert. This is true for the
non-adaptive and adaptive modes. Usefulness ratings appear higher than
satisfaction ratings (without statistical comparison). This was also
seen in the FOT drivers who had each system in its active state for
three weeks of naturalistic driving. In the SAVE-IT tests, there is
little difference in the ratings among the three warning functions (FCW,
LDW, and distraction alert), although no statistical comparisons were
made among the subsystems.

Four ANOVA tests were performed using Van der Laan ratings compiled from
SAVE-IT subject responses. Comparisons were made between adaptive and
non-adaptive forms of each warning system (FCW and LDW) within each test
environment (track and road). The only statistically significant result
is that following test track activities, the adaptive FCW system was
rated as more useful than the non-adaptive version (F~1~ = 6.71p\<0.05).
This result is not considered very meaningful, however, as driver
experience on the track is thought to be less realistic than the
combined experiences on the track and road. It is assumed that the
bottom-line result would come from the responses following the on-road
tests. There was no significant difference seen in these results, but
there were only 11 matched pairs of data from drivers from on-road
testing, and 26 match pairs of data from the track activity.

These ratings are based on six hours of driving by each subject,
including two hours of highly scripted test-track experience, and thus
must be considered with care. As a benchmark, Table 14.20 shows the mean
values of Van der Laan ratings reported in two field operational tests
that addressed forward crash warning with 66 drivers (Ervin et al.,
(2005)) and lane departure warning with 78 drivers (LeBlanc et al.,
(2006)). The average travel time per driver with the system enabled for
those tests was roughly 28 hours (FCW) and 22 hours (LDW). Although
those systems were different from the SAVE-IT crash warning systems, it
may be useful to compare the results of those systems with the
non-adaptive on-road ratings provided by the SAVE-IT subjects. The LDW
system in the FOT scored higher than the SAVE-IT system, but the FCW
system scores in the FOT are very close to those in this SAVE-IT Task
14C evaluation effort. The conclusion is that for at least two
high-level metrics of acceptance, the SAVE-IT results are not very
different from ratings provided by drivers having longer exposures.

Table 14.20. Mean values of Van der Laan scale results from previous
field operational tests

  ------------------------- ------------------- --------------------------
                            **Field Operational 
                            Tests**             

                            Useful              Satisfying

  LDW (N = 78)              1.34                0.78

  FCW (N = 66)              0.90                0.50
  ------------------------- ------------------- --------------------------

### 14.4.2.2 Post-Drive Questionnaire Responses {#post-drive-questionnaire-responses .SaveIT-H3}

In addition to the nine questions that were the basis for the Van der
Laan ratings, the post-drive questionnaires also addressed the topics
outlined in Table 14.21. The full questionnaires are in Appendices H and
I. The responses to the questions in Table 14.21 are presented
throughout this section. Many of the responses are tallied for each
driver in Appendix B. Highlights are presented below for each question.

Table 14.21. Post-drive questionnaire contents

+-----------+---+-------------------------------------------+-------------+
| *         | * |                                           | **Response  |
| *Question | * |                                           | type**      |
| type**    | Q |                                           |             |
|           | u |                                           |             |
|           | e |                                           |             |
|           | s |                                           |             |
|           | t |                                           |             |
|           | i |                                           |             |
|           | o |                                           |             |
|           | n |                                           |             |
|           | * |                                           |             |
|           | * |                                           |             |
+-----------+---+-------------------------------------------+-------------+
| Track and |   |                                           |             |
| on-road   |   |                                           |             |
+-----------+---+-------------------------------------------+-------------+
| Van der   |   | First questions in post-drive surveys.    | 5-point     |
| Laan      |   |                                           | Likert      |
+-----------+---+-------------------------------------------+-------------+
| Wi        |   | If it were available, I would want a      | 5 choices   |
| llingness |   | \[warning system type\] on my next car.   | for each    |
| to        |   |                                           | question    |
| purchase  |   | What is the most you would pay to have a  |             |
|           |   | \[warning system type\] on your next car? |             |
+-----------+---+-------------------------------------------+-------------+
| System    |   | How might the \[warning system\] be       | Open-ended  |
| im        |   | improved?                                 | written     |
| provement |   |                                           | response    |
| su        |   |                                           |             |
| ggestions |   |                                           |             |
+-----------+---+-------------------------------------------+-------------+
| On-road   |   |                                           |             |
| only      |   |                                           |             |
+-----------+---+-------------------------------------------+-------------+
| Alert     |   | How often did the \[warning system\]      | 7-point     |
| approp    |   | function as you expected?                 | Likert      |
| riateness |   |                                           | scales      |
| and       |   | How often did the \[warning system\] give |             |
| nuisance  |   | a warning you felt was appropriate?       |             |
|           |   |                                           |             |
|           |   | How often did the \[warning system\] give |             |
|           |   | a warning that you felt was unnecessary?  |             |
|           |   |                                           |             |
|           |   | When the \[warning system\] gave a        |             |
|           |   | warning that you considered unnecessary,  |             |
|           |   | what level of annoyance did you feel?     |             |
|           |   |                                           |             |
|           |   | How often did the \[warning system\]      |             |
|           |   | alert you to a potential crash?           |             |
+-----------+---+-------------------------------------------+-------------+

**If it were available, I would want a*(n) \[insert warning system\]* on
my next car.**

Drivers rated FCW and LDW after test-track activities and after each of
the two on-road drives using the scale in Table 14.22. Again, results
from on-road testing are considered the most meaningful data, although
both sets are presented here (track data is in Figure 14.23 and on-road
data is shown in Figure 14.24). Following the on-road testing, 67
percent and 81 percent of the responses were positive when this question
addressed the non-adaptive and adaptive versions of LDW, respectively.
The only negative responses were from two of 12 respondents in regard to
the non-adaptive LDW. For FCW, 67 percent and 91 percent of the
responses to the non-adaptive and adaptive versions were positive
following on-road testing. Again, two of the 12 respondents provided
negative responses in regard to the non-adaptive mode of FCW. The
distraction alert was presented and evaluated only during test track
testing. Seventy-six percent of drivers indicated a positive response.

Table 14.22. Scale for 5-point Likert responses

  --------------------- -------------------------------------------------
  **Value**             **Rating**

  1                     Strongly agree

  2                     Mildly agree

  3                     Agree/disagree equally

  4                     Mildly agree

  5                     Strongly disagree
  --------------------- -------------------------------------------------

ANOVAs were conducted to compare responses addressing the non-adaptive
versus the adaptive versions of the system. This was done within a given
test environment (e.g., test-track results for adaptive vs.
non-adaptive), but not between test environments. The only significant
difference was seen with the test-track data. The adaptive FCW rated
higher than the non-adaptive FCW (means 1.27 and 1.69; F~1~ = 4.37;
p\<0.05).

![](media/image22.wmf){width="5.471527777777778in"
height="3.877083333333333in"}

Figure 14.23. Driver ratings of the warning systems following test track
activities

![](media/image23.wmf){width="5.913888888888889in" height="3.1625in"}

Figure 14.24. Driver ratings of the four warning systems tested in the
on-road portion of testing

**What is the most you would pay to have *\[insert warning system\]* on
your next car?**

By asking drivers about the cost they would pay for LDW and FCW (each in
adaptive and non-adaptive mode), researchers obtain a data point that
addresses the possible market penetration of these systems.

Figure 14.25 shows results for drivers who responded after the on-road
experiments. Over 60 percent of responses identified \$0 to \$500 as the
amount they would pay. In the short term, during introduction of these
systems, the costs are likely to be much higher than this. If systems
move into the larger-volume models, say, within seven to eight years,
costs may still be higher than \$500, but perhaps not twice that amount.
If these costs are in fact accurate representations of the actual market
willingness, the introduction of systems could be difficult but may be
sustainable in the mid-term.

Note that three of 12 drivers responded with a willingness to pay over
\$1,000 for LDW (three drivers would pay that amount for non-adaptive
LDW, and two of those would pay that amount for adaptive LDW). The same
driver who reported a willingness to pay \$1,000 for non-adaptive LDW
reported the same range for non-adaptive FCW. That was the only driver
who reported willingness to pay that much for either FCW. There were no
statistically significant results for this question, when comparing an
adaptive version of a warning system to a non-adaptive system.

Responses to this question that were gathered after test-track work are
not presented in the main body of this report , but also contain no
statistically significant differences. See Appendix B for the responses.

![](media/image24.wmf){width="4.635416666666667in"
height="3.470833333333333in"}

Figure 14.25. Driver responses regarding a willingness to purchase crash
warning systems

**How often did the *\[insert warning system\]* function as you
expected?**

This question attempted to address whether the driver felt that the
system behavior was predictable. This was done because an initial
question about the SAVE-IT adaptive approach is whether suppressing,
delaying, or advancing alerts may lead to some drivers perceiving the
system as inconsistent in its behavior.

Drivers were to use the seven-point Likert scale shown in Table 14.23
for their response; Figure 14.26 shows the distribution of drivers'
responses to the question with regard to LDW and FCW, each in
non-adaptive and adaptive mode. Figure 14.27 shows the *cumulative*
percentage on the vertical axis of the figure, which is the sum of the
percentages of drivers selecting the response shown on the horizontal
axis, as well as those who select responses to the left. These plots are
more effective in seeing the trend of responses -- whether drivers are
choosing predominately responses on the "right side" of the scale, or
the left side.

Table 14.23. Scale for 7-point Likert responses

  --------------- -------------------------------------------------------
  **Value**       **Rating**

  0               Did not ever receive a warning

  1               Never

  2               Hardly ever

  3               Occasionally

  4               Quite often

  5               Frequently

  6               Nearly all the time
  --------------- -------------------------------------------------------

Figure 14.27 may suggest that a few of these drivers had some difficulty
predicting the behavior, especially of the FCW system. The proportion of
respondents replying in the three most positive categories -- that the
system either "quite often," "frequently," or "nearly all the time"
functioned as they would expect -- is:

-   Eight of 12 drivers (75 percent) - LDW non-adaptive

-   Seven of 11 drivers (64 percent) - LDW adaptive

-   Four of 12 drivers (58 percent) - FCW non-adaptive

-   Five of 11 drivers (55 percent) - FCW adaptive

There is, however, no statistical significance when comparing the
results of adaptive and non-adaptive versions of the same system. The
lack of statistical significance may be related to the small number of
subjects (12 respondents for the non-adaptive systems and 11 for the
adaptive systems). Furthermore the exposures of drivers to the systems
are limited. However the results are not inconsistent with previous
projects in which a higher percentage of drivers reported an ability to
understand the logic of LDW systems than the percentage of drivers
reporting the same for FCW (LeBlanc et al., 2005). Note, of course, that
there is very little difference between the responses to adaptive and
non-adaptive. This may suggest that the test subjects did not have
sufficient experience to form opinions well enough to distinguish
between the adaptive and non-adaptive systems.

![](media/image25.wmf){width="6.4944444444444445in"
height="3.623611111111111in"}

Figure 14.26. Driver responses addressing whether the system performed
as expected\
(following on-road testing)

![](media/image26.wmf){width="6.492361111111111in"
height="3.8201388888888888in"}

Figure 14.27. Cumulative distribution: driver responses addressing
whether crash warning systems performed as expected

**How often did the *\[insert warning system\]* give a warning that you
felt was appropriate?**

Drivers were asked to use the seven-point scale in Table 14.23 for this
question. The intention was to ask drivers about the proportion of
alerts they felt were appropriate. Regarding the question itself, a
concern arose during analysis that some drivers may have interpreted the
question as "how many times" an appropriate alert occurred, in an
absolute sense. The difference, of course, is that if a driver assumed
the latter meaning, a confounding factor in the results would be the
number of alerts a driver experienced.

Nevertheless, the scale of Table 14.23 was used once again, and the
results following on-road driving are shown in Figure 14.28. An ANOVA
conducted on these data found a statistically significant difference
where the non-adaptive version of LDW elicited more positive responses
than the adaptive version. The mean value of responses was 4.25 for
non-adaptive and 2.36 for adaptive (F~1~= 5.31; p\<0.035). Hence,
drivers may have felt that non-adaptive LDW gave appropriate warnings
more often than adaptive LDW.

![](media/image27.wmf){width="6.498611111111111in"
height="3.0256944444444445in"}

Figure 14.28. Cumulative distribution: driver responses regarding the
frequency with which crash warning systems provided appropriate alerts

**How often did the *\[insert warning system\]* give a warning that you
felt was unnecessary?**

This question probed drivers' perceptions of the frequency of
unnecessary alerts from each of the systems. The hypothesis, of course,
is that adaptive versions of LDW and FCW will be perceived as providing
fewer unnecessary alerts. Figure 14.29 shows the results from the
drivers after on-road testing was completed. Once again, the scale of
Table 14.23 was used, and the traces on the figure show the cumulative
percent of drivers whose ratings lie at, or to the left of, the label on
the horizontal axis. Traces that are farther to the right on the graph
mean that more drivers are selecting responses on the right side of the
rating scale, i.e., that unnecessary alerts are more common.

When comparing the adaptive and non-adaptive versions of FCW and LDW,
the traces in Figure 14.29 suggest that drivers' perceptions are indeed
that adaptive versions (dashed traces on the figure) provide fewer
unnecessary alerts. However, the ANOVAs conducted on these data revealed
that only the result for the LDW system were statistically significant
(means 3.25 and 1.27, F~1~ = 12.26; p\<0.01). When comparing mean
ratings for FCW, there was no significance between adaptive and
non-adaptive systems.

![](media/image28.wmf){width="5.997916666666667in"
height="3.1381944444444443in"}

Figure 14.29. Cumulative distribution: driver responses addressing which
crash warning systems produced unnecessary alerts

**When the warning system gave a warning that you considered
unnecessary, what level of annoyance did you feel?**

The objective of this question was to compare the nuisance level
associated with alerts from the adaptive and non-adaptive systems. The
rating scale is shown in Table 14.24. Both the percentage of drivers
choosing each rating, and the cumulative percentage of drivers choosing
ratings at, or to the left of, each rating value are shown for responses
collected after the on-road tests (see Figure 14.30). On the lower
(cumulative percentage) graph, traces that lie to the right are
associated with ratings indicating a higher annoyance level. The chart
suggests that the adaptive versions are less annoying, however, the
ANOVAs conducted with these data found that only the ratings for the LDW
system were statistically significant in this regard. The mean value of
ratings were 2.25 and 1.09 for the non-adaptive and adaptive LDW
systems, respectively (F~1~ = 10.68; p\<0.01).

Table 14.24. Scale for 6-point Likert responses addressing annoyance
level

  --------------- -------------------------------------------------------
  **Value**       **Rating**

  0               Did not ever receive a warning

  1               None

  2               Mild

  3               Moderate

  4               A lot

  5               Extreme
  --------------- -------------------------------------------------------

![](media/image29.wmf){width="6.085416666666666in"
height="3.1284722222222223in"}

![](media/image30.wmf){width="6.008333333333334in"
height="3.098611111111111in"}

Figure 14.30. Histogram and cumulative distributions: driver responses
regarding perceptions of annoyance associated with alerts deemed to be
unnecessary

**How often did the \[warning system\] alert you to a potential crash?**

Recall each driver accumulated approximately 170 miles of on-road
driving in the SAVE-IT prototype vehicle. The responses following the
on-road driving segments are shown in Figure 14.31 (with both a
histogram of responses and the cumulative form of responses). Note that
there were two different drivers who reported some experience in which a
system alerted them to a potential crash. (See Appendix B for details on
how individual drivers responded to this, or any other, question.) The
other drivers, as expected, did not report strong perceptions of such an
event.

There was no statistically significant difference between the adaptive
and non-adaptive forms of the LDW or FCW.

![](media/image31.wmf){width="5.897916666666666in"
height="2.7333333333333334in"}

![](media/image32.wmf){width="5.996527777777778in"
height="3.097916666666667in"}

Figure 14.31. How often did the system alert you to a potential forward
crash?

**How might the *\[warning system\]* be improved?**

For each warning system, the last question on the questionnaire was
open-ended to allow drivers to express other concerns or comments.
Appendix J provides a list of all comments provided, sorted into six
topical areas for ease in browsing. A summary of those that may be of
most interest is provided here.

Regarding the LDW system, selected feedback includes:

-   Some positive comments about adaptive LDW, versus non-adaptive LDW.

-   Suggestions about the displays (various, some contradictory).

-   Request for improved functionality that is not specific to the
    SAVE-IT system, e.g., ability to suppress LDW alerts when the driver
    is making deliberate, un-signaled lane changes, and improved lane
    tracking for higher system availability.

-   Following the track testing, two drivers expressed concern about the
    possibility of "glitches" in the technology required for the
    adaptive system and the possible performance implications of such
    issues. These comments may have induced by the pre-track activity in
    which the experimenter confirmed the head-tracking system
    performance and occasionally asking a driver to adjust the seat to
    improve the performance.

-   Request to warn sooner.

Regarding the FCW system, selected feedback includes:

-   Preference for adaptive FCW is cited by two drivers.

-   Requests for earlier alerts in both adaptive and non-adaptive FCW,
    despite the fact that the FCW timing in the prototype is actually
    earlier than the NHTSA algorithm developed in the ACAS FOT project.
    (This feedback is not unexpected, as drivers are unaware that the
    major challenge of reducing nuisance alerts pushes designers and
    standards-developers toward timings that are uncomfortable for
    drivers, but allow effective responses in most conflict scenarios.)

Feedback regarding the driver distraction alert that was used in the
track activities:

-   Two drivers felt the distraction alert was not as useful as the
    other systems (LDW, FCW).

-   Concern and advice about the displays. A few drivers mentioned that
    placing visual displays on the sides of the cabin would allow
    drivers who were looking well away from the forward scene to notice
    the alert. (Developers likely did not choose such a strategy since
    it could induce drivers to look at the side displays and not return
    their gaze forward.) This comment was made by one driver relative to
    FCW as well.

-   Two drivers suggested faster triggering of the distraction alert,
    i.e., displaying the alert after a shorter period of looking-away
    time.

Overall, responses to this set of open-ended questions seem to suggest
that at least some drivers have an appreciation of the adaptive system,
whereas no driver suggested that the non-adaptive systems were
preferable. Furthermore, there may be questions to be researched further
regarding the use of the particular alert display suite that was used.
The unique feature in the alert display approach used in the SAVE-IT
prototype is the method of presenting a forward visual alert by the use
of a light source reflected from the windshield. The drivers' concerns
provided in this evaluation may suggest further examination of that
approach.

The other comments summarized above do not seem to contribute
significantly to an understanding of whether or how drivers will
perceive the adaptive forms of LDW or FCW.

### 14.4.2.2 Video Review {#video-review .SaveIT-H3}

After each subject had completed both on-road drives (adaptive system
and non-adaptive system) and the written questionnaire, he or she was
asked to review video of several events from his or her on-road drives,
as well as clips from several scripted events with an UMTRI driver at
the wheel. These events represented potential LDW or FCW alerts, and the
purpose was to collect a database of driver responses regarding the
usefulness of alerts in certain scenarios, as well as whether the events
were expected to elicit alerts. This database was intended to provide
further data regarding driver's acceptance of adaptive systems, since it
is known which events would be presented or suppressed by the adaptive
or non-adaptive systems. Table 14.25 shows that 214 events were reviewed
and coded, including 104 of drivers' own events from their on-road
drives as well as 11 drivers rating each of 10 scripted events (110
total ratings).

![](media/image33.wmf){width="3.036111111111111in"
height="1.163888888888889in"}Table 14.25. Number of potential alert
events reviewed by drivers using video

The method for choosing the alerts was to randomly select six LDW alerts
and four FCW alerts from among events in that driver's on-road
experience in which the non-adaptive LDW and FCW systems did (or would
have) issued an alert. Note that given the high rate of suppression of
alerts when driving in adaptive mode, most of these events were not
actually presented to the driver, but were suppressed alerts. Drivers
saw their own face video as well as the forward scene, but neither heard
nor saw any indication of whether or when an alert occurred. Obviously,
some drivers would remember the alert events and some may have forgotten
them.

Each driver furthermore reviewed five LDW and five FCW alerts that were
scripted by UMTRI staff. The clips included:

-   Two lane crossing or lane drift situations in which LDW was
    suppressed, based on driver attentiveness

-   Two LDW alerts that were allowed based on driver inattention
    (looking to left or right)

-   One lane drift event with a driver "nodding off" while head pose
    continued to be forward, and during which the LDW was suppressed

-   Three appropriate delays/suppressions of FCW alerts, based on driver
    attentiveness

-   One appropriate FCW alert with the driver approaching a turning lead
    vehicle while not looking forward for several seconds

-   One FCW alert that would have been suppressed or delayed, but with
    ambiguity regarding the driver's attentiveness (head not forward,
    but occasional eye glances forward in a rapidly developing scenario)

Subjects then rated how useful they found the alert (or would have found
the alert, for those that were suppressed), as well as their
expectations about receiving an alert in each instance. The first
statement was: "I felt that the system provided me with useful
information during this segment of my drive." If the alert had not,
however, been experienced by the driver, the statement became: "If I had
received an alert in this situation, I feel that the system would have
provided me with useful information during this segment of my drive."
The second statement was: "I expected an alert from the system during
this segment of my drive." The scale for each of these questions was as
follows:

![](media/image34.wmf){width="6.499305555555556in"
height="1.0180555555555555in"}

After reviewing video from their own driving, subjects rated the
warnings issued during their own adaptive drives as more useful than
those issued during the non-adaptive drives (with results combined for
FCW and LDW). The mean values were 2.33 and 3.37 for adaptive and
non-adaptive alerts, respectively. The ANOVAs conducted on these data
revealed that the differences between the usefulness ratings of two
modes of the system were statistically significant F~1~ =
12.79;(p\<0.001).

When ANOVAs were conducted on data from when only the LDW alerts that
occurred during drivers' own experience, there was still a significantly
higher rating for the usefulness of the adaptive alerts (mean values
2.16 for adaptive and 3.27 for non-adaptive, F~1~ = 9.71; p\<0.01). More
analysis results are available in Appendix K.

For the 104 events that were pulled from the drivers' own SAVE-IT
driving, another analysis focused on the difference in driver's
responses as a function of the driving scenario. The motivation for this
was to provide a database of driver ratings, as a function of scenario,
which could inform the decision whether to use the adaptive mechanisms.

Tables 14.26 and 14.27 present the mean value of driver responses to FCW
and LDW episodes, respectively, as well as indications of the *p* value
that connotes statistical significance. An ANOVA was also conducted on
these data. Only one result was significant at the p\<0.05 level: For
FCW, the multi-lane scenarios (in which one or both vehicles is changing
lanes, turning, or otherwise leaving the original travel lane) are
considered less useful than the same-lane scenarios (in which both
vehicles remain in the same, shared travel lane throughout the event).
This finding is consistent with results in Ervin et al. (2005). Other
findings approach significance, meaning that p\<0.10. These include four
items shaded in Tables 14.26 and 14.27.

The failure to have more findings that corroborate with earlier findings
of Ervin et al. (2005) and LeBlanc et al. (2005) is very likely due to
the small number of events within each scenario cell.

Table 14.26. Comparing responses to drivers' own video-reviewed driving
events\
as a function of FCW scenario

+---------+------+------+-----------+---------+-----------+-----------+
| **\     | **Sa |      | **Mu      |         | **Out of  |           |
| Compare | me-l |      | lti-lane\ |         | path\     |           |
| to:**   | ane\ |      | (14       |         | (5        |           |
|         | (21  |      | events)** |         | events)** |           |
|         | e    |      |           |         |           |           |
|         | vent |      |           |         |           |           |
|         | s)** |      |           |         |           |           |
+---------+------+------+-----------+---------+-----------+-----------+
|         | **E  | **U  | **        | **Us    | **        | **        |
|         | xpec | sefu | Expect=** | eful=** | Expect=** | Useful=** |
|         | t=** | l=** |           |         |           |           |
|         |      |      | **4.36**  | *       | **4.20**  | **3.20**  |
|         | **3. | **2. |           | *4.09** |           |           |
|         | 70** | 75** |           |         |           |           |
+---------+------+------+-----------+---------+-----------+-----------+
| Sa      |      |      | M         | Mul     | D         | D         |
| me-lane |      |      | ulti-lane | ti-lane | ifference | ifference |
|         |      |      | less      | less    | not       | not       |
|         |      |      | e         | useful, | si        | si        |
|         |      |      | xpected?\ | F~1~=   | gnificant | gnificant |
|         |      |      | p=0.086   | 5.91;   |           |           |
|         |      |      |           | p\<0.05 |           |           |
+---------+------+------+-----------+---------+-----------+-----------+
| Mul     |      |      |           |         | D         | D         |
| ti-lane |      |      |           |         | ifference | ifference |
|         |      |      |           |         | not       | not       |
|         |      |      |           |         | si        | si        |
|         |      |      |           |         | gnificant | gnificant |
+---------+------+------+-----------+---------+-----------+-----------+
| Out of  |      |      |           |         |           |           |
| path    |      |      |           |         |           |           |
+---------+------+------+-----------+---------+-----------+-----------+

Table 14.27. Comparing responses to drivers' own video-reviewed driving
events\
as a function of LDW scenario

  ------------- -------------- ----------- -------------- ------------ -------------- ------------- ------------- -------------
  **Compare     **Purposeful               **Purposeful                **Accidental                 **False alert 
  to:**         and complete               but partial                 and partial                  -- lane       
                lane                       lane                        lane                         tracking      
                departure\                 departure\                  departure\                   error\        
                (12 events)**              (28 events)**               (8 events)**                 (16 events)** 

                **Expect\      **Useful\   **Expect\      **Useful\    **Expect\      **Useful\     **Expect\     **Useful\
                =3.45**        =3.00**     =2.86**        =2.21**      =3.50**        =2.88**       =3.82**       =2.36**

  Purposeful,                              Difference not Partial      Difference not Difference    Difference    Difference
  complete                                 significant    purposeful   significant    not           not           not
                                                          less                        significant   significant   significant
                                                          useful?\                                                
                                                          p=0.088                                                 

  Purposeful,                                                          Difference not Difference    False alert   Difference
  partial                                                              significant    not           less          not
                                                                                      significant   expected?     significant
                                                                                                    p=0.062       

  Accidental,                                                                                       Difference    Difference
  partial                                                                                           not           not
                                                                                                    significant   significant

  False alert                                                                                                     
  ------------- -------------- ----------- -------------- ------------ -------------- ------------- ------------- -------------

### 14.4.2.3 Final Questions Regarding Adaptive and Non-Adaptive Crash Warning Systems {#final-questions-regarding-adaptive-and-non-adaptive-crash-warning-systems .SaveIT-H3}

Following the video review, subjects were asked about their overall
impressions of the systems. The script used by experimenters and the
resulting subject responses are given below:

![](media/image35.wmf){width="5.751388888888889in"
height="3.5805555555555557in"}.

An overview of the responses to each question is provided below. Every
individual response to the first three questions is provided in Appendix
L.

What are the Advantages, Disadvantages, and Concerns with the
Non-Adaptive and Adaptive Crash Warning Systems?

The first three questions above address drivers' perceptions of the
relative strengths and drawbacks of the adaptive system in comparison to
the non-adaptive system. Appendix L lists the driver's verbal comments,
as captured by an experimenter's notes. Drivers' comments identified
some perceived advantages of the adaptive versions, including:

-   Reduction in nuisance alerts (mentioned by four of 11 drivers)

-   Fewer alerts would prevent drivers from being desensitized to
    alerts.

-   One driver said, "if drowsy -- very helpful," which is presumably
    due to a mistaken notion that the adaptive system would sense
    drowsiness.

Regarding perceived advantages of the non-adaptive system, drivers'
comments included:

-   System keeps drivers "on toes," or if your mind is "out to lunch"
    (presumably referring to the non-adaptive system's ability to
    provide alerts when the driver's head is forward but the driver is
    cognitively distracted).

-   "Makes you more aware of your driving behavior -- hopefully improve
    your driving."

-   One driver expressed concern with the adaptive system in the drowsy
    driver scenario (the adaptive system would suppress LDWs if the head
    pose of a drowsy driver were to remain forward).

-   One driver stated that he would be interested in the non-adaptive
    system if it were less expensive.

**Which would you prefer in a vehicle that you or your family would
purchase: adaptive or non-adaptive?**

Table 14.28 shows the results of the 11 drivers' responses to this
question. Drivers did not exhibit a preference for either adaptive or
non-adaptive crash warning systems. In fact, only four of the 11 drivers
offered a clear choice: two preferred non-adaptive and two preferred
adaptive. Six picked both, and one wanted a system that could be toggled
between adaptive and non-adaptive. Of the drivers choosing both systems,
one also said, "If you have a teen, you might want it in non-adaptive
\[mode\]." Another driver stated, "I might switch between the two on a
long drive or at night. If I had to chose, I'd pick non-adaptive."

The responses may reflect a real ambivalence about the
adaptive/non-adaptive choice. The responses may also reflect an
experiment in which the drivers were quickly exposed to advantages and
disadvantages of each system, and yet they did not have enough
first-hand experience to develop a preference for themselves.

Table 14.28. Driver preferences for adaptive versus non-adaptive systems

  -------------------------------------- ------------------ --------------
  **Choice**                             **Drivers**        **Percent**

  Non-Adaptive                           2                  18.2

  Adaptive                               2                  18.2

  Neither                                0                  0.0

  Both                                   6\*                54.5

  Don't know                             0                  0.0

  Other                                  1                  9.1
  -------------------------------------- ------------------ --------------

## 14.4.3 Acceptance of Adaptive Mechanisms for Distraction Mitigation {#acceptance-of-adaptive-mechanisms-for-distraction-mitigation .SaveIT-H2}

During each of their on-road drives, the subjects were asked how safe
they would feel engaging in each of three IVIS tasks. The experimenter,
who was able to monitor the SAVE-IT-estimated level of driving demand,
collected 36 responses from each driver as follows:

-   Two drives: morning and afternoon

-   Three levels of driving demand: low, moderate, and high

-   Three IVIS tasks: radio tuning, phone dialing, and destination
    entry, all involving the use of the IVIS touchscreen

-   Two queries for each triad of drive, demand level, and task\\

See Section 14.2.4 for information about the IVIS and distraction
mitigation systems, which are capable of allowing, advising against, or
prohibiting the use of these three tasks and other IVIS functions as the
level of estimated driving demand varies. See Section 14.3.2.3 for
details on the training and procedures for this task.

Specifically, subjects were asked to rate the safety of engaging in the
IVIS task:

> *Please rate how safe you feel it would be to \[insert task here\]
> right now. If you forget the scale as we go along, it is also posted
> on the dashboard.*

![](media/image36.wmf){width="5.406944444444444in"
height="0.5444444444444444in"}

The data gathered from the 11 drivers who completed both on-road drives
is presented next, but first it may be convenient to duplicate an
earlier-presented table showing the SAVE-IT mapping from estimated
driving demand to distraction-mitigation activities, at least as it
affects the three tasks addressed in this study. Table 14.29 shows that,
for instance, the distraction mitigation system would advise against
destination entry when the driving demand is estimated to be low, but
would not allow destination entry when the estimated demand is moderate
or high.

Table 14.29. SAVE-IT distraction mitigation system actions relevant to
this evaluation that may influence the driver's ability to engage in the
targeted IVIS tasks

  ------------------ --------------- ----------------- ------------------
  **IVIS task**      **Driving                         
                     demand level**                    

                     **Low**         **Medium**        **High**

  **Radio tuning**   Allowed         Allowed           Advised against

  **Phone dialing**  Allowed         Advised against   Not allowed

  **Destination      Advised against Not allowed       Not allowed
  entry**                                              
  ------------------ --------------- ----------------- ------------------

Table 14.30 shows the mean of the driver ratings for each of the three
tasks, broken down by the level of driving demand when the questions
were posed. Generally, the mean ratings in Table 14.30 validate the way
in which the SAVE-IT design varies its actions according to task
complexity and driving demand level, as represented in Table 14.29. This
is a key validation of the distraction mitigation approach since it
suggests that the driver's sense of appropriateness of engaging in a
task depends in the same way on the two dimensions of complexity and
demand that are used by the SAVE-IT system.

Table 14.30 Driver-reported willingness to engage in IVIS tasks during
on-road drives\
(cells shown mean ratings)

  ------------------ --------------- ----------------- ------------------
  **IVIS task**      **Driving                         
                     demand**                          

                     **Low**         **Medium**        **High**

  **Radio tuning**   1.8             1.9               2.5

  **Phone dialing**  2.4             3.1               3.7

  **Destination      3.1             3.7               4.1
  entry**                                              
  ------------------ --------------- ----------------- ------------------

Consider, however, Table 14.31, which provides the percent of driver
responses that rated a *task X demand* condition as "very safe,"
"somewhat safe," or "neither safe nor unsafe." The term "not unsafe"
will be applied to this set of ratings. The table shows that, for
instance, 93 percent of drivers rate the radio tuning task as "not
unsafe" in low demand conditions. This percentage drops to 76 percent
when the demand is high, but still three of four drivers do not see the
task as patently unsafe. It seems appropriate that SAVE-IT allows this
task at all driving demand levels. SAVE-IT does, however, advise against
the task in high demand conditions from a safety perspective. However,
given the ratings, many drivers may consider the system to be overly
cautious.

Consider also the phone dialing and destination entry tasks and the data
in Table 14.30. There are three task-demand combinations that the
distraction mitigation system advises against. For these, 57, 65, and 76
percent of drivers feel these advised-against tasks are "not unsafe."
Thus a majority of drivers may disagree with the distraction mitigation
system advice. For the tasks that are prohibited by the SAVE-IT
distraction mitigation system, the percent of drivers who feel the tasks
are still not unsafe are 18, 31, and 37 percent. For two of these tasks
that are prohibited -- destination entry with moderate driving demand
and phone dialing with high driving demand \-- over 30 percent of the
ratings indicate that some drivers did not feel they were unsafe tasks.
In these situations, there will be a significant minority of drivers who
may be displeased at the inability to engage in these tasks.

It is important to note that the destination entry task at high driving
demand is rated as "not unsafe" by only 18 percent of the drivers. The
SAVE-IT prototype system's decision to inhibit this task in these
conditions is consistent with 82 percent of the driver ratings in this
study.

Overall, the magnitude of intervention by this prototype distraction
mitigation system may be greater than drivers prefer. If unchanged in
its calibration, the system may be too cautious for drivers.

Table 14.31. Percent of driver ratings indicating the IVIS task is very
safe, somewhat safe, or neither safe nor unsafe at the indicated driving
demand level.\
SAVE-IT interventions are shown in brackets.

  ----------------- ----------------- ----------------- ------------------
  **IVIS task**     **Driving demand                    
                    level**                             

                    **Low**           **Medium**        **High**

  **Radio tuning**  93%               98%               76%\
                                                        \[Advised
                                                        against\]

  **Phone dialing** 93%               57%\              37%\
                                      \[Advised         \[Not allowed\]
                                      against\]         

  **Destination     65%\              31%\              18%\
  entry**           \[Advised         \[Not allowed\]   \[Not allowed\]
                    against\]                           
  ----------------- ----------------- ----------------- ------------------

See Appendix M for cumulative plots of the driver-rated "safeness" as a
function of task and demand level. Those plots capture the raw data.

# 14.5 Discussion of Results {#discussion-of-results .SaveIT-H1}

## 14.5.1 Overview {#overview-1 .SaveIT-H2}

This evaluation effort addressed the two types of adaptive mechanisms
that are part of the SAVE-IT system: adaptive crash warning systems with
decisions that are modulated by measurements of driver head pose, and a
distraction mitigation system that adaptively adjusts driver access to
features of an in-vehicle information system (IVIS) according to driving
conditions. This evaluation directly addresses driver acceptance of each
of these two subsystems, evaluates potential safety impacts associated
with selected aspects of the adaptive crash warning system, and provides
observations regarding system performance elements of the adaptive crash
warning system.

The specific objectives in this evaluation are pursued as though the
issues of acceptance and safety are decoupled. In reality, the
introduction and development of many types of automotive safety
technologies often involves tradeoffs between driver or acceptance (as a
surrogate for early viability of the concept) and the level of safety
benefits that may be realized. Driver acceptance is critical for a
significant deployment of these technologies on U.S. highways. A simple
model for computing overall safety benefit of a crash warning system is
to multiply the number of system-equipped vehicles by the system
effectiveness, defined as the relative reduction in some harm metric per
exposure to an event in which the system would be active (Najm et al.,
2006). Thus the best strategy to maximize safety impact is often to
ensure higher driver acceptance of a system with significant safety
benefits, and avoid introducing more ideal -- but unpopular---systems.
Thus driver acceptance affects achievable safety benefits.

Another coupling is that, in the longer term, drivers' willingness to
continue supporting certain safety technologies requires a continued
perception that the system provides safety and is minimally credible
performance. Thus, systems sold as safety features need to achieve the
basic functions that drivers expect, or they may not remain on the
market long. Hence, safety benefits can affect continued acceptance.

This evaluation, however, seeks not to answer the overall coupled
question of whether and how the SAVE-IT features represent positive or
negative features, but rather, the evaluation provides data about the
acceptance and safety elements.

Section 14.5.2 addresses the safety and performance aspects of the
adaptive crash warning system by summarizing the analyses of the
on-board data collected during the evaluation, as well as the FOT data.
Section 14.5.3 presents the high-level results of driver acceptance of
the crash warning systems, especially the adaptive features. Section
14.5.4 addresses the consistency of driver decision-making with the
rules imbedded in the distraction mitigation system of the IVIS system,
which controls driver access to the IVIS features.

## 14.5.2 Safety and Performance Features of the Adaptive Crash Warning System {#safety-and-performance-features-of-the-adaptive-crash-warning-system .SaveIT-H2}

This section summarizes results from track testing, on-road testing, and
the analysis of FOT data that address how the SAVE-IT adaptive crash
warning systems affect the alerts that are provided to drivers, and
whether there is a potential that safety is impacted. The results are
discussed separately for LDW and FCW, and for each system, are organized
into the following topics:

-   Change in overall alert rates,

-   Potential of false positives from the SAVE-IT system (instances when
    the system perceives that the driver is inattentive when in fact he
    or she appeared to be attentive), and

-   Potential of false negatives from SAVE-IT (instances when SAVE-IT
    delays or suppresses alerts when the driver appears to possibly be
    inattentive or when the driver may have benefited from the alert).

In addition, for the FCW system only, the following points are
discussed:

-   Safety-related findings from the FCW surprise trials, and

-   Comparing adaptive FCW alerts to non-adaptive FCW alerts.

This evaluation addressed the safety impact of the adaptive features of
the crash warning systems in three ways. First, observations of system
performance during on-road testing provide a first look at the
inevitable delay or suppression of alerts that would occur when the
system perceives the driver as being visually attentive, when in fact he
or she is not. Second, test-track trials were conducted to look at the
potential of adaptive FCW to provide additional safety benefits to
visually inattentive drivers. Third, an indirect way in which safety is
addressed is through the driver acceptance results in the next section.
Earlier the statement was made that safety gains may be realized by
fielding systems that are more acceptable to the driver, accelerating
deployment and also serving to maintain acceptance so that deployment
remains stable and substantial. Acceptance is discussed in the next
section.

Since the alert rates are dramatically reduced with SAVE-IT, and since
there are inevitably false positives and false negatives associated with
the adaptive mechanisms, the questions of how these results together
impact driver acceptance are addressed in the following section.

### 14.5.2.1 Lane Departure Warning {#lane-departure-warning .SaveIT-H3}

**Reduction in overall alert rate**: The on-road drivers provide an
opportunity to get a sense of the effectiveness of the SAVE-IT adaptive
mechanisms in reducing the overall alert rates associated with LDW and
FCW, as well as suggest the types of alerts that may be more likely to
be reduced. For the LDW systems, a large reduction in the alert rate was
observed. The drives with non-adaptive LDW yielded 93 alerts, or 9.0
alerts per 100 miles traveled for 12 drivers. With the adaptive system,
10 alerts were recorded for 11 drivers, or 1.0 per 100 miles. This
represents an 88 percent reduction in the LDW alert rate from the
non-adaptive LDW. Eight of the 10 alerts that remained would not have
occurred with ideal head tracking performance, so that an alert rate
reduction of over 95 percent would have resulted. A video review of each
of these instances confirmed that all of the alert suppressions were
justified, and none would have resulted in a removal of a warranted
alert.

An approximate version of the SAVE-IT mechanism for suppressing LDW
alerts was also overlaid onto the preexisting data set from the Road
Departure Crash Warning System Field Operational Test (RDCW FOT). In
this analysis, 236 LDW alerts sampled from the baseline period of the
RDCW FOT data set were reviewed individually (video and data) to
determine how the SAVE-IT adaptive mechanism would have affected the LDW
alert experience. With ideal sensing of the driver's head pose, all of
the 236 alerts would have been suppressed by the SAVE-IT algorithm, so
that the overall alert rate in this data set would be reduced by 100
percent. Similar to those alerts suppressed during the on-road testing,
the video review confirmed that these alert suppressions did not occur
in instances in which the driver appeared at risk due to leaving the
lane.

**False positives from the adaptive mechanism**: False positives are
cases where the system perceives that the driver is visually inattentive
when in fact he or she is attentive. For LDW, that means alerts are
provided to attentive drivers, possibly negatively impacting driver
acceptance. The 10 adaptive LDW alerts that occurred during the on-road
drives with the SAVE-IT system were reviewed. In eight of these cases,
the driver's head was indeed forward during some period just before the
time of the alert, causing false positives due to imperfect head
tracking. The other two adaptive LDW alerts were cases in which the
drivers appeared to be paying attention to the forward scene, yet had
their head turned toward the rearview mirror (one side mirror, one
inside mirror), as they were moving their eyes back and forth from the
forward scene to the mirror. In these two situations, the SAVE-IT system
is performing as designed, but there is a violation of the assumption
that head pose is a surrogate for driver attention. That such violations
occur is, of course, expected, but in the near term, tracking the
driver's head is more feasible technically than tracking the driver's
eye glances.

As explained above, since the expert reviewing the video provides ideal
sensing of the driver's head pose, all of the alerts in the FOT data
would likely have been suppressed. Hence there were no false positives
of the adaptive mechanism in the sample FOT data for the LDW system. It
is noteworthy that there were no cases in which the driver was visually
attentive and yet the driver's head pose was not forward for the two
seconds that is the trigger for the SAVE-IT system. Thus the data
analyzed in this evaluation of LDW supports the SAVE-IT assertion that
head pose is a useful and practical surrogate for visual attention of
the driver.

**False negatives from the adaptive mechanism**: False negatives are
events in which the driver is not visually attentive, but the system
sensors and algorithms interpret the driver as attentive. Two sources of
error can lead to false negatives: head-tracking errors and violations
of the assumption that a driver whose head is turned away from a forward
position for two seconds is attentive. The analysis focused on the
latter.

The 236 events from the RDCW FOT in which a baseline LDW alert was
triggered were broken down into three groups, based on whether the
driver's head pose during the two seconds preceding the LDW alert was
forward the entire period (59 percent), forward for a portion of the
period (41 percent), or never forward in that period (no instances).

Determining whether a driver is attentive involves a subjective judgment
about the driver's awareness of the current driving tasks and readiness
for events that have some likelihood to occur. In reality, drivers
routinely manage to travel while talking to passengers, looking at
scenery, tuning the radio, and so on, all the while occasionally turning
their heads and ultimately reaching their destination without event.
Furthermore, they exercise some judgment in choosing when to engage in
secondary (non-driving) behavior based on the current driving demand and
their expectations for changes in that demand in the near future. This
is why the SAVE-IT system assumes that short-lived head turns are not
sufficient evidence to conclude that the driver is inattentive or
unaware of potential conflicts. When reviewing events, then, it is
difficult to assign a label of "inattentive" unless there is a patent
failure to look forward enough to complete the current driving task, or
the vehicle is clearly drifting dangerously with the driver not looking.
No such event was observed during the on-road drives with the SAVE-IT
prototype or with the sample set of FOT data. Thus the basic result is
that there were no observed false negatives for LDW introduced by the
SAVE-IT system.

As a data point for insight, each event was also annotated for the
presence of driver glances away from the forward scene for purposes
*other* than driving. This includes looking at passengers, etc.
Twenty-one percent of the events in which the driver's head was not
forward at some point within the two seconds before an alert involved
driver glances that were not related to driving. These included moving
their eyes and sometimes their head to the rearview mirror as part of a
conversation, looking down while dialing or eating, and so on. For
events in which the driver's head was forward during the entire
two-second period, the fraction of events in which non-driving glances
away from the forward scene was 12 percent. Thus the head pose metric
used by the SAVE-IT system to allow or suppress LDW alerts does capture
some portion of the looking-away behavior.

False negatives of the adaptive LDW could be important. Many road
departure crashes have been attributed to driver fatigue, but the
approach of using the azimuth angle of the head would presumably not
capture much of the driver fatigue behavior. Thus, if the SAVE-IT system
were the only technology on a vehicle, a driver could fall asleep and
there is a real chance that the SAVE-IT system would suppress the alert.
This concern becomes moot if there are supplementary technologies, such
as eye-closure detection or systems that sense a decay in a driver's
capabilities by monitoring the vehicle path and driver pedal and
steering wheel inputs.

### 14.5.2.2 Forward Crash Warning {#forward-crash-warning .SaveIT-H3}

**Reduction in overall alert rate**: The SAVE-IT adaptive mechanisms
reduced the FCW alert rate during on-road drives from 6.0 alerts per 100
miles with the non-adaptive version to 2.4 alerts per 100 miles with
adaptive FCW (a 60 percent reduction). A set of 169 events from the
Automotive Crash Avoidance System Field Operational Test (ACAS FOT) was
also reviewed to determine the effects of the SAVE-IT adaptive FCW
mechanisms on a different and larger data sample. Compared to the LDW
adaptive mechanism, the effect of the FCW SAVE-IT mechanism was more
difficult to estimate since it is more complicated (modulation of
assumed driver reaction time) and because the algorithm behind the ACAS
FOT alerts is not available to the SAVE-IT team. Furthermore, FCW as
implemented in both data sets includes a computation that is simple in
concept but complicated in practice. A true overlay of the SAVE-IT
mechanism would require knowledge and re-computation of the ACAS FOT
algorithm with the SAVE-IT mechanism in place. It would also require
manual coding of head pose state at every video frame within several
seconds of each ACAS alert. Together these requirements were beyond the
scope of this evaluation effort.

However, this report described how an estimate of the alert rate change
was generated. It was estimated that 85 percent of the ACAS FOT alerts
examined would have been delayed in time, had the adaptive mechanism
been in place. Based on observations from the SAVE-IT system, an
estimate was made that roughly 70 percent of all the ACAS FOT alerts
that were examined might have been eliminated by the application of the
SAVE-IT adaptive mechanism. This is tempered slightly by the fact that
another small set of new alerts would likely be introduced by this same
mechanism, since it employs a more safety-cautious set of parameters
when drivers have been looking away. The on-road drives with SAVE-IT
found that there were five new alerts in the adaptive drives that were
due to this aspect of the adaptive mechanism (while 53 were suppressed).
The FOT data analysis supports the results from the smaller data set
from the SAVE-IT on-road experience, with both analyses suggesting that
well over half of the FCW alerts could be eliminated with a head-pose
algorithm similar to SAVE-IT's.

14.2.5.3.1 Track Testing: Safety Implications from the Surprise FCW
Trials

Surprise FCW trials were attempted on the test track with each driver,
using both adaptive and non-adaptive FCW alerts. Four metrics of driver
response and potential safety impact were considered: driver delay times
to the lead vehicle braking, delay times to the alert itself, the time
to collision, and the required deceleration at the moment of driver
brake onset. Since adaptive FCW provides earlier alerts when the
driver's head has been away from the forward scene for more than one
second, it can be presumed that the adaptive system provides
safety-positive, or at least safety-neutral, results relative to the
non-adaptive system, at least when the driver's head has been away for
more than one second.

The results from the trials are somewhat inconsistent, but positive in
that adaptive FCW may allow an extra safety margin. Drivers may have
reacted more quickly to the adaptive FCW alert than the non-adaptive FCW
alert, thereby possibly allowing longer time to collisions at the time
of brake onset. This finding is only significant at the p \< 0.05 level
when comparing the time lag between the beginning of the lead vehicle
deceleration and the onset of the subject vehicle braking. The data
suggesting that the adaptive FCW led to shorter reaction time (from
alert onset to brake onset) and longer time to collisions only
approaches significance (p between 0.1 and 0.2).

However, there is no result approaching significance when comparing the
required braking levels required at driver braking onset following
adaptive and non-adaptive FCW. There is no obvious mechanism explaining
why driver reaction time would be shorter for the adaptive system versus
the non-adaptive system, as the driver displays are identical. The
finding that the time to collision is longer with the adaptive system is
consistent with the earlier onset of the adaptive alerts for inattentive
drivers. However, a similar finding was not made with the required
deceleration. The variability in initial headway and the braking level
of the decelerating vehicle may explain this inconsistency. Recall that
the number of trials in which drivers were actually surprised was modest
(27 trials), which may also explain why there was only weak evidence
that adaptive FCW may provide inattentive drivers with a slightly higher
safety margin.

14.2.5.3.2 On-Road Testing: Comparing Adaptive FCW to Non-Adaptive FCW

In this comparison only the adaptive-mode drives were considered
(section 14.4.1.2.2); the comparison is done between adaptive-mode FCW
alerts that were actually issued (23 alerts), and those non-adaptive
alerts that would have been issued under the very same conditions (71
alerts). In that comparison, the following was observed:

-   Five alerts were unique to the adaptive mode. These alerts would not
    have occurred had the system been operating in non-adaptive mode.
    Four of these alerts would not have occurred had the head pose
    estimation system been perfect. Instead, short-lived returns of the
    head to the forward scene were sometimes missed by the system.

-   Eighteen alerts that were issued by the adaptive system would have
    also been issued by the non-adaptive system. Eleven of these alerts,
    however, were given later by the adaptive system, and four alerts
    were issued earlier. (The other three occurred at the same time for
    both versions of the FCW.)

-   The 53 events that would have been issued uniquely by the
    non-adaptive system were all associated with attentive driving.

The reduction in FCW alerts was mostly due to eliminating false alerts
during multiple-lane scenarios, with much lower reduction in the
occurrence of alerts in same-lane scenarios. This seems fortunate, since
drivers rated false alerts and multiple-lane scenarios as having the
least value in a recent field operational test (Ervin et al., 2005).
None of the adaptive actions of effectively suppressing or delaying
alerts seemed to pose an observable risk to the driver, so there was no
evidence that suppression or delaying of alerts posed negative safety
impacts, relative to a non-adaptive system. This is not surprising,
given the relatively limited driving exposure and the infrequency of
real forward crash threats.

14.2.5.3.3 False Positives

False positives are events in which the FCW alert is given earlier than
intended (due to pose measurement errors) or when the adaptive algorithm
does not delay an alert, even though the driver is attentive. The result
of a false positive is not safety-negative, as the FCW alert is given a
little earlier than intended or needed.

The on-road FCW alerts and the FOT analysis data set were not examined
specifically for false positives, however a few cases of false positives
were identified while pursuing other objectives. The five alerts that
were unique to adaptive FCW during the on-road drives (described above)
are examples of this. False positives can be expected, as the head-pose
measurement system will occasionally miss short periods in which the
driver's head pose is not forward.

14.2.5.3.4 False Negatives

From the ACAS FOT data set, in addition to the 169 events already
mentioned, a second sample of 30 events was analyzed. These 30 events
were identified in previous research efforts as potentially useful in
assisting the driver to become aware and/or respond to a real developing
forward-crash conflict. The analysis of the FOT data suggested that 16
of these 30 events would have likely been delayed in time had the
SAVE-IT adaptive FCW mechanisms been in place. Upon closer examination,
eight of the 30 alerts were identified as having a higher potential to
be useful than the remaining 22 alerts. Four of those eight events that
may be considered high-priority FCW events might have been delayed by
the SAVE-IT mechanism, since the driver's head pose had been forward
shortly before the alerts occurred. Based on this, and on the obvious
fact that head pose is not always identical to visual attention, there
is of course reason to continue to hypothesize that the adaptive
mechanism may prove less effective than the non-adaptive FCW.

## 14.5.3 Acceptance of the Adaptive Features of the Crash Warning Systems {#acceptance-of-the-adaptive-features-of-the-crash-warning-systems .SaveIT-H2}

The in-vehicle evaluation of SAVE-IT mechanisms also acquired driver
feedback on the SAVE-IT system using a combination of questionnaires,
open-ended questions, debriefings, and video-assisted reviews of
scenarios by the drivers.

Drivers in this study, independent from the adaptive features, responded
to the generic functions of LDW and FCW in a similar way, as have
drivers in larger field tests with longer exposures. Ratings of LDW and
FCW on the Van der Laan scale were generally positive, with usefulness
ratings being more positive than satisfaction ratings. Drivers responded
favorably to the function concepts, with 70 to 80 percent of the on-road
drivers reporting a desire to have the features on their next vehicle.
However, over 60 percent of drivers reported a willingness to pay only
\$0 to \$500 for each system, which is likely less than near-term
systems will cost. There was concern with nuisance alerts (albeit
possibly induced by the experiment and its deliberate introduction of
the concept to the drivers).

More important, however, is comparing adaptive to non-adaptive versions
of the two systems. Based on the subjective feedback reported by the
drivers in this study, and the power of the adaptive method to reduce
the alert rate, adaptive systems may have advantages in driver
acceptance:. However the results from over 30 comparisons of adaptive
and non-adaptive systems have many inconclusive conclusions.

In open-ended questioning, drivers displayed an awareness of some basic
tradeoffs in incorporating this form of crash warning adaptation:

-   Drivers cited a reduction in nuisance alerts as a key advantage of
    the adaptive crash warning systems.

-   A few drivers preferred the non-adaptive form of the crash warning
    system. One driver stated an advantage of the non-adaptive versions
    was that it would keep drivers "on their toes" and would help if a
    driver's mind was "out to lunch."

-   Cost was a concern and one driver preferred the adaptive form, as
    long as it did not cost more than the non-adaptive form.

-   Two drivers mentioned some concern about the drowsy driver scenario,
    after becoming aware (through the experimental process) that the
    adaptive LDW may not always warn a drowsy driver.

When asked which system they would prefer on their own vehicle, only
four of 11 drivers gave a definitive preference (two for non-adaptive
and two for adaptive). The remaining seven drivers either had no
preference (six drivers) or wanted both (one driver). These seven
drivers mentioned the advantages and disadvantages of each system, and
one stated that a system that could be toggled between the two would be
useful.

Regarding the statistically significant results from the questionnaires,
there were six instances of preference for adaptive over non-adaptive:

1.  According to the Van der Laan scale, drivers who completed both
    track and on-road driving rated adaptive FCW as more useful than
    non-adaptive FCW.

2.  Based on their track exposure, the drivers indicated they were more
    inclined to want the adaptive system on their own vehicles than the
    non-adaptive system. The track results are based on 26 drivers'
    feedback, but without the experience of on-road driving.

3.  Drivers indicated that the adaptive versions of LDW and FCW,
    combined, gave fewer unnecessary alerts than the non-adaptive set.
    The statistical significance of this advantage for adaptive systems
    also held when adaptive LDW was compared to non-adaptive LDW.

4.  Drivers reported feeling less annoyance with unnecessary alerts
    given by adaptive LDW than with unnecessary alerts provided by the
    non-adaptive LDW. If true, this may suggest that there is a
    component of annoyance that results from an accumulation of
    unnecessary alerts, since the non-adaptive LDW provided many more
    alerts than the adaptive form.

5.  Following video review of events from their own drive, drivers found
    adaptive alerts more useful for LDW.

6.  The same result reported in the previous item was found again when
    comparing driver review of the combined set of LDW and FCW alerts.

Questionnaire results supporting the non-adaptive versions:

-   Drivers indicated that non-adaptive LDW gave appropriate warnings
    more often than adaptive LDW. One question about this result is
    whether drivers interpreted the phrase "more often" as meaning more
    events per unit time, or that given any alert, it was itself more
    likely to be appropriate with non-adaptive LDW than adaptive LDW.
    Hence there is some ambiguity in this result.

Indirect support of adaptive over non-adaptive versions:

-   Drivers reported the FCW multi-lane scenarios as less useful than
    the same-lanes scenarios. The analyses found that it is likely that
    the multi-lane scenario occurrences will be reduced by more than the
    same-lane scenario events.

There was no significant difference between the adaptive and
non-adaptive forms in drivers' willingness to purchase the system or in
the amount they would pay.

A conclusion from this study, then, is that certain aspects of the
adaptive forms of LDW and FCW are more acceptable than the non-adaptive
forms. The preference is not overwhelming and does not appear when
drivers are asked point-blank for their preference. The small data set
is certainly a complication in drawing conclusions, as is the
inexperience of the drivers with these systems. A positive aspect of the
weak findings in the acceptance areas is that at least drivers are not
so disturbed by the adaptive mechanisms that they reject them
altogether.

Regarding one novel aspect of the driver displays for LDW and FCW, there
was some indication that further research may be needed. The SAVE-IT
prototype used a light source that was reflected from the dashboard off
the windshield to create a visual spot that appears to the driver to be
superimposed on the forward scene. Drivers had some concerns with this,
with at least one report each of: difficulty in seeing the spot,
preference for a different color, steering wheel occlusion of the
signal, being disturbed by signal, fear that signal would not be seen if
truly distracted, and preferring a different location altogether. More
investigation regarding this approach -- not only effectiveness, but
driver perception of the approach -- may be warranted.

## 14.5.4 Driver Acceptance of Adapting Available IVIS Features with the Distraction Mitigation System  {#driver-acceptance-of-adapting-available-ivis-features-with-the-distraction-mitigation-system .SaveIT-H2}

The second feature of the SAVE-IT system is an in-vehicle information
system (IVIS) that allows, advises against, or does not allow certain
features to be accessed by the driver, depending on the system's
perceived level of driving demand. This adaptation is called the
distraction mitigation system (DMS), and it considers traffic density,
weather, vehicle motion, and speed, among other variables, in its
decisions. Twelve drivers were trained on three tasks of varying
complexity and then queried during their on-road driving about their
willingness to engage in those tasks as the driving circumstances
varied.

The driver responses correlated very well with the DMS's strategy of
intervening as a function of both the task complexity and driving
demand. This result is not surprising because one basis for the DMS
strategy was simulator-based data from drivers on their subjective sense
of task appropriateness, given video of driving situations. The level of
DMS interventions was consistent with the opinions of the majority of
the drivers in the current experiment. The calibration of the level of
interventions, however, is a possible area of further study because, in
two cases -- destination entry with moderate driving demand, and phone
dialing in high driving demand -- the DMS does not allow drivers to
access the feature, while more than 30 percent of the driver responses
indicated a perception that the task was not unsafe to perform in that
driving circumstance. Applying the right level of intervention to
mitigate driver distraction with IVIS systems, and yet creating an
intuitive and acceptable intervention strategy, will be an important
emerging area of design and research. This may be a topic for which
industry standardization, whether formal or de facto, may be helpful, so
that automakers can provide drivers with safeguards and not risk
competitive disadvantage in doing so.

# 14.6 References {#references .SaveIT-H1}

LeBlanc, D., Sayer, J., Winkler, C., Bogard, S., Devonshire, J. Mefford,
M., Hagan, M., Bareket, Z., Goodsell, R., and Gordon, T. (2006). "Road
departure crash warning system (RDCW) field operational test final
report." UMTRI reports no. 2006-9-1 and 2006-9-2.

Ervin, R., Sayer, J., LeBlanc, D., Bogard, S., Mefford, M., Hagan, M.,
Bareket, Z., and Winkler, C. (2005). "Automotive collision avoidance
system (ACAS) field operational test methodology and results." *US DOT
HS 809 901,* Washington, D.C.

Kiefer, R., LeBlanc, D. Palmer, M., Salinger, J., Deering, R., and
Shulman, M. (1999). "Development and validation of functional
definitions and evaluation of procedures for collision warning/
avoidance systems," *US DOT report HS 808 964*.

Van der Laan, J.D., Heino, A., & de Waard, D. (1997) A simple procedure
for assessment of acceptance of advanced transport telematics.
*Transportation Research, Part C***,** 5(1), 1-10.

Brunson, S.J., Kyle, E., Phamdo, N., and Preziotti, G (2002). "Alert
algorithm development program: NHTSA rear-end collision alert algorithm
final report." US DOT report HS 809 526, Washington, D.C.

LeBlanc, D., Kiefer, R., Deering, R., Shulman, M., Palmer, M., and
Salinger, J. (2001) "Forward Collision Warning: Preliminary Requirements
for Crash Alert Timing," Society of Automotive Engineers SAE paper
2001-01-0462.

Klauer, S. G., Dingus, T.A., Neale, V. L., Sudweeks, J.D., and Ramsey,
D. J. (2006). The impact of driver inattention on near-crash/crash risk:
an analysis using the 100-car naturalistic driving study data. USDOT
technical report DOT HS 810 594. Washington, D.C.

Dingus, T.A., et al. (2006). "The 100-car naturalistic driving study,
phase II -- results of the 100-car field experiment." *US DOT HS 810
593,* Washington, D.C.

Najm, W., Stearns, M. , Howarth, H., Koopman, J., and Hitz, J. (2006).
"Evaluation of an automotive rear-end collision avoidance system." US
DOT report HS 810 569, Washington, D.C.

# 14.7 Acknowledgments {#acknowledgments .SaveIT-H1}

This work was funded by the Volpe National Transportation System Center
of the US Department of Transportation, in conjunction with the National
Highway Transportation Safety Administration. Delphi Corporation serves
as the prime contractor with the University of Michigan, University of
Iowa, and Ford Motor Company as partners. Mary Stearns of the Volpe
Center served to oversee this project with support from Mike Perel
(NHTSA), J Pollard (Volpe) and Tom Sheridan. The authors wish to
acknowledge the guidance and assistance of direct sponsors Delphi
Corporation, particularly Matthew Smith, Gerry Witt, and Ray Prieto.
Additional assistance at UMTRI was provided by Mike Hagan, Reneé M. St.
Louis, Amy Neumeyer, Bob Goodsell, John Koch, and Ben Powell.