a

Utilizing Various Data Sources 

for Surface Transportation 

Human Factors Research

The Exploratory Advanced Research Program

WORKSHOP SUMMARY REPORT • November 6–7, 2013
b

Notice

This document is disseminated under the sponsorship of the U.S. Department of Transportation in the 

interest of information exchange. The U.S. Government assumes no liability for the use of the information 

contained in this document.

The U.S. Government does not endorse products or manufacturers. Trademarks or manufacturers’ 

names appear in this report only because they are considered essential to the objective of the document.

Quality Assurance Statement

The Federal Highway Administration (FHWA) provides high-quality information to serve Government, 

industry, and the public in a manner that promotes public understanding. Standards and policies are 

used to ensure and maximize the quality, objectivity, utility, and integrity of its information. FHWA 

periodically reviews quality issues and adjusts its programs and processes to ensure continuous quality 

improvement.

Cover photo illustration: 

© instructables.com, Federal Highway Administration and iStock.com
i

Technical Report Documentation Page
ii
iii

Executive Summary

On November 6–7, 2013, at the Turner–

Fairbank Highway Research Center 

in McLean, VA, the Federal Highway 

Administration’s (FHWA) Office of Safety 

Research and Development, with support 

from the Exploratory Advanced Research 

(EAR) Program, convened the workshop, 

”Utilizing Various Data Sources for Surface 

Transportation Human Factors Research.” 

The workshop addressed the increasing 

number of different datasets and multiple 

ways of collecting data—from naturalistic 

driving and simulator studies to eye trackers 

and surveys—that can be used to increase an 

understanding of human errors. 

Human errors are still a major cause of injuries 

and fatalities; however, a number of different 

datasets have recently become available to 

analyze human errors. These datasets point 

in different directions within different areas of 

interaction. Experts in human factors research, 

transportation safety, and driver behavior 

and performance analysis, met to discuss 

and determine which datasets were best and 

how one might resolve the differences. The 

information provided by the different datasets 

is sometimes complementary, sometimes 

competing, and sometimes confirmatory. 

The workshop brought together a panel of 

experts to share their research experience 

of using multiple methods to gain insights 

about different aspects of driver and traveler 

behavior and performance. 

During day one of this workshop, participants 

heard seven presentations on using various 

datasets from sources such as driving 

simulators, field studies and field 

operational tests, and naturalistic driving 

studies. The experts discussed various 

methods to study behaviors that lead to 

errors and shared strategies they have 

deployed to gain insightful information 

about what datasets to use to target one 

or more human factors or behavior issues. 

The workshop also presented the idea of 

using multiple data collection methods 

to “cross-reference” analysis results, 

validate conclusions, and enhance the 

understanding of behaviors.

On day two of the workshop, an expert 

panel discussed issues related to 

consolidating data from multiple types of 

collection methods. The experts discussed 

how datasets must be carefully examined 

when combined from different sources. 

For example, some data sources are 

contradictory, leaving researchers with 

the need to conduct additional research 

to resolve the controversies. Alternatively, 

other data sources can be complementary 

and provide information in the field and 

in the laboratory on driver behaviors 

that point in a similar direction. How 

best to create complementary datasets 

also needs to be carefully considered. 

In addition, very few data sources are 

comprehensive, and they do not provide 

information on both driver behavior and 

crashes. The ability to develop models 

that can link behavioral datasets with 

crash datasets, leading to comprehensive 
iv

datasets, is still in its infancy. The expert 

panel went on to identify several potential 

research topics to address the challenges 

that must be overcome to integrate data 

from multiple sources.

At the end of day two, the workshop sponsor 

divided the participants into three groups 

so that detailed discussion could be held 

to identify research gaps related to the 

following interactions of drivers: (1) with 

other road users, (2) with changing elements 

of the roadway and infrastructure, and (3) 

with their own vehicle. All three groups 

presented summaries of their discussion and 

recommendations to conclude this workshop.

Workshop panelists and participants noted 

two different ways of seeing how best to 

deal with multiple contradictory datasets, 

as follows:

 • Bottom up—It is possible to take various 

known instances in which there are 

contradictions across datasets and identify 

why these inconsistencies arise and what 

can be done to avoid them in the future.

 • Top down—A study across multiple sites 

would allow for the collection of various 

different types of data. It would then be 

possible to look for inconsistencies across 

sites in the same dataset and inconsistencies 

within sites across datasets.

Panelists were unanimous in recommending 

that there should be an attempt to understand 

how to use the different types of data in a 

study that includes the following components: 

 • Multiple sites (e.g. locations, geometries, 

traffic density, and environment).

 • Multiple types of data gathered at each 

site (e.g., survey, simulator, and field).

 • Multiple users (e.g., bicyclists, 

pedestrians, motorists, and drivers).

 • Multiple methods of analysis (e.g., 

descriptive and inferential statistics, 

and quantitative behavioural models).

As part of the final workshop 

recommendations, participants identified 

many areas of priority for human factors 

research that could make use of the 

expanding datasets now available and soon to 

be available. These included modeling, safety, 

roadway departure, urban intersections, 

vehicle, pedestrian and bicyclist interaction, 

and data analysis. Participants suggested a 

number of specific items for further research, 

as follows:

 • Evaluate the effectiveness of current 

signage used on roadways.

 • Research speed perception.

 • Develop solutions to improve roadway 

safety for pedestrians and bicyclists.

 • Evaluate current Intelligent 

Transportation System technologies for 

pedestrian and bicycle safety.

 • Develop a methodology to conduct 

research by using multiple data sources.

 • Construct methods to measure 

exposure of pedestrians and bicyclists.

To further understanding and use of multiple 

data types, participants recommended a 

study, possibly focused at intersections, which 

includes multiple sites, multiple data types 

gathered at each site, multiple user types, 

and multiple methods of analysis. This study 

could provide critical information on how to 

resolve contradictions among datasets, how 

to put together complementary datasets 

that describe risky behaviors, and how to 

generate comprehensive datasets that link 

behaviors and crashes.
v

Table of Contents

Introduction

Day One: Presentations

Driver–Driver and Other Road Users’ Data for Human Factors Research

 Presented by Dr. Marco Dozza

Driver and Pedestrian Recognition Behavior for Avoiding Conflict with 

Each Other at an Intersection

 Presented by Dr. Toru Hagiwara and Dr. Hidekatsu Hamaoka

Driver–Infrastructure and Roadway Data for Human Factors Research

 Presented by Dr. Michael Manser

Traffic Control Devices and In-Vehicle Systems

 Presented by Dr. Susan Chrysler

Triangulating Data Sources to Understand Driver–Vehicle Behavior

 Presented by Dr. John Lee

Driver–Vehicle Data for Human Factors Research

 Presented by Dr. Linda Ng Boyle

Day Two: Discussion and Summary

Expert Panel Discussion

 Moderated by Dr. Donald Fisher

Group Session Summary

Workshop Recommendations

Appendices

 Appendix A: Agenda

 Appendix B: About the Presenters

 About the EAR Program

1

2

3

9

14

21

28

33

40

41

45

49

50

51

53

55
vi

List of Figures

Figure 1. An instrumented traditional bicycle.

Figure 2. Average cycling behavior speeds.

Figure 3. Cyclists' speed profile.

Figure 4. An R-type accident.

Figure 5. Front-passing diagram.

Figure 6. Reproduced collision conflict between right-turning 

vehicle and pedestrian.

Figure 7. Pedestrian–vehicle dedicated short-range 

communications system.

Figure 8. Pedestrian crossing an experimental intersection.

Figure 9. Real-world research environments.

Figure 10. Expected research environments in terms of validity 

and experimental control.

Figure 11. Adjusted research environments in terms of validity 

and experimental control.

Figure 12. Tools and environments in relation to data sources.

Figure 13. Experimental signs used in laboratory study.

Figure 14. Trade-off between validity and experimental control.

Figure 15. Research methods.

Figure 16. Comparison of real maps and simulated scenarios.

Figure 17. Impacts on driver response when using adaptive cruise control.

Figure 18. Cluster analysis using simulation data.

4

5

6

9

10

11

12

13

15

16

16

17

18

22

26

29

34

35
vii

List of Acronyms and Abbreviations

General Terms

ACC adaptive cruise control

ASV-3 Advanced Safety Vehicle-3

CICAS-SSA cooperative intersection collision avoidance system–stop sign assist

DOT Department of Transportation

DSRC dedicated short-range communications

EAR Exploratory Advanced Research

FHWA Federal Highway Administration

FOT field operational test

GPS Global Positioning System

HAWK High-intensity Activated crossWalK

IEEE Institute of Electrical and Electronics Engineers

IMU inertial measurement units

ITS Intelligent Transportation System

IV-DSRC inter-vehicle dedicated short-range communications

LIDAR light detection and ranging

MUTCD Manual on Uniform Traffic Control Devices

NADS National Advanced Driving Simulator

NHTSA National Highway Traffic Safety Administration

NSF National Science Foundation 

OTL observed time lag

PI principal investigator 

PTL predicted time lag

PV-DSRC pedestrian–vehicle dedicated short-range communications

SHRP 2 second Strategic Highway Research Program

STRADA Swedish Traffic Accident Data Acquisition

TCD traffic control devices

TMC traffic management center

TxDOT Texas Department of Transportation

UMTRI University of Michigan Transportation Research Institute

USDOT U.S. Department of Transportation

V2V vehicle-to-vehicle

VehDAQ vehicle data acquisition

VOQ vehicle operator questionnaire

WTI Western Transportation Institute
viii
1

Introduction

Transportation safety is the top priority 

at the Federal Highway Administration 

(FHWA) and U.S. Department of 

Transportation (USDOT). A high percentage 

of transportation incidents and vehicle 

crashes are caused by human errors. As a 

result, it is important to continue investing in 

research resources to gain a comprehensive 

understanding of human errors and to try to 

answer the question, "Why do drivers and 

travelers do what they do?”

The motivation for this workshop was in large 

part a function of the increasing number 

of different datasets and multiple ways of 

collecting data—from naturalistic driving 

and simulator studies to eye trackers and 

surveys—that can be used to increase our 

understanding of human errors. Now is an 

ideal time to begin a discussion about how to 

resolve the differences and how to choose the 

best datasets for particular applications.

To initiate this discussion, FHWA’s Office 

of Safety Research and Development, with 

support from the Exploratory Advanced 

Research (EAR) Program, convened the 

workshop, “Utilizing Various Data Sources 

for Surface Transportation Human Factors 

Research,” on November 6–7, 2013. Experts 

in transportation safety analysis and 

driver behavior and performance, were 

invited to the Turner–Fairbank Highway 

Research Center in McLean, VA, to share 

their research experience of using multiple 

methods to gain insights into different 

aspects of driver and traveler behavior 

and performance. 

A primary question posed to researchers 

was how best to select the particular 

datasets most helpful for analyzing one 

of the following three major research 

topics: (1) the interaction between drivers 

and other road users, such as pedestrians 

and bicyclists; (2) the interaction 

between drivers and roadway and other 

transportation infrastructure; and (3) 

the interaction between drivers and their 

vehicles. This report captures highlights 

from the workshop and summarizes the 

discussions that took place.
2

Day One:

Presentations

Expert presentations are summarized 

in the following section.
3

Overview

Dr. Marco Dozza reminded workshop 

participants that safety is an ongoing 

concern as the complexity of the roadway 

environment continually increases. This 

complexity particularly jeopardizes cycling 

safety. Roadway space is commonly shared 

among cyclists and other road users, such as 

drivers, and the interaction between these 

different modes of transportation creates a 

high risk for crashes and potential injuries 

and fatalities. 

In Europe, 1,994 cyclists were killed in 2010 

and accounted for 6.8 percent of total road 

fatalities, compared with 2 percent of road 

fatalities in the United States.1, 2 Improving 

cycling safety is therefore crucial, because 

cycling is increasingly becoming a more 

popular mode of transportation. In addition, 

with an integrated electric motor available 

for propulsion, electric bicycles (e-bikes) 

heighten this concern because of their high 

speed and increasing prevalence. A better 

understanding of how cyclists behave in traffic 

is therefore needed to develop improved 

safety measures. This could be achieved by 

transferring existing methods of naturalistic 

data collection for cars and trucks to collect 

naturalistic cycling data. 

Dozza informed participants that the goal of 

this research is to understand how bicyclists, 

using traditional bicycles and e-bikes, 

behave in traffic and the extent to which 

safety–critical situations (i.e., crash and near 

crashes) are different for e-bikes compared 

with traditional bicycles. The researchers of 

this study collected and analyzed naturalistic 

cycling data and also acquired additional 

datasets from the Swedish Traffic Accident 

Data Acquisition (STRADA) database. 

Dozza informed workshop participants 

that, in Sweden, 70 percent of the bicycle 

crashes that occur are reported in accident 

databases. In accordance, cycling accidents 

within the STRADA database were isolated 

and combined with the former data to 

better address a number of issues. Dozza 

told workshop participants that this project 

is expected to provide the research and 

transportation industry with methods 

to gather naturalistic data, in particular 

naturalistic cycling data, to understand 

accident causation, to investigate 

cycling behavior, to inform regulations 

and infrastructure design, and to test 

intelligent systems. 

Naturalistic Cycling Data

Naturalistic data collection refers to data 

collected in traffic by road users performing 

their usual daily activities. Traditionally, 

naturalistic data are recorded from 

instrumented cars and trucks. There are many 

reasons researchers are interested in collecting 

naturalistic data, summarized as follows:

Driver–Driver and Other Road Users’ Data 

for Human Factors Research

Dr. Marco Dozza

Chalmers University of Technology

1. European Road Safety Observatory (2012). Traffic 

Safety Basic Facts. Retrieved July 7, 2014, from http://

ec.europa.eu/transport/road_safety/pdf/statistics/

dacota/bfs20xx_dacota-swov-cyclists.pdf.

2. National Highway Traffic Safety Administration (2012). 

Bicyclists and Other Cyclists. Retrieved July 7, 2014, from 

http://www-nrd.nhtsa.dot.gov/Pubs/811624.pdf.
4

 • Understanding accident causation—

Researchers for a 100-car naturalistic 

driving study concluded that off-road 

glances longer than 2 sec doubled 

accident risk.3 

 • Investigating driver behavior—Researchers 

for a 2009 commercial vehicle operation 

study showed that texting increases 

accident risk by 23 times.4 

 • Informing regulations and infrastructure 

design—Researchers for a Sweden–

Michigan naturalistic field operational test 

examined the “kangaroo effect” of speed 

cameras and the relationship between 

curb design and lane departures.5

 • Testing intelligent systems—Researchers 

for a 2011 study examining integrated 

vehicle-based safety systems and lanedeparture warning systems demonstrated 

that these systems improve lane keeping.6

Dozza suggested that the same reasons 

researchers collect naturalistic driving data 

can also be applied to bicycles. In addition, 

with the increase in cyclists, it is important for 

researchers to understand other road-user 

behavior. All road users have to contend with 

issues of distraction and obedience to road 

rules, in addition to adapting to the speed of 

the new e-bikes. 

Dozza highlighted that gathering naturalistic 

cycling data will make it possible to improve 

current regulations and road infrastructure 

in Europe. In 2012, there were 1.2 million new 

e-bikes on the road; however, bike lanes in 

Europe may not fully accommodate e-bikes, 

and they may require new infrastructure.7 For 

example, in Sweden, pedestrians and cyclists 

frequently share the same sidewalk. With 

naturalistic cycling data, researchers can test 

intelligent systems—such as new smartphone 

applications—that promise to help cyclists 

and test if they display destructive behavior. 

Overall, the study findings will contribute 

to the development of countermeasures to 

reduce cyclist trauma. 

Dozza informed participants that naturalistic 

cycling data collection requires a 

sophisticated network of sensor processing 

and recording systems. In accordance, 

equipment requirements for bicycles differ 

from those of cars, for example, weight and 

weather resistance requirements are more 

important for bicycles than they are for 

cars. In this study, the researchers recorded 

naturalistic data by using an instrumented 

traditional bicycle, which was fitted with the 

following equipment, as shown in figure 1:

Figure 1. An instrumented traditional bicycle.

© Marco Dozza

3. Klauer, S. G., Dingus, T. A., Neale, V. L., Sudweeks, J.D., 

Ramsey, D.J. (2006). The Impact of Driver Inattention 

on Near-Crash/Crash Risk: An Analysis Using the 100-

Car Naturalistic Driving Study Data. Washington, DC: 

Department of Transportation, National Highway Traffic 

Safety Administration.

4. Olson, R.L., Hanowski, R.J., Hickman, J.S., & Bocanegra 

J. (2009). Driver Distraction in Commercial Vehicle 

Operations. Washington, DC: Department of 

Transportation, Federal Motor Carrier Safety 

Administration.

5. Victor, T., Bärgman, J., Gellerman, H., Hjälmdahl, M., 

Hurtig, S., Kircher, K., Moeschlin, F., Svanberg, E., (2010). 

Sweden–Michigan Naturalistic Field Operational Test 

Phase 1: Final Report. Gothenburg, Sweden: SAFER 

Vehicle and Traffic Safety Centre at Chalmers. 

6. Sayer, J., LeBlanc, D., Bogard, S., Funkhouser, D., 

Bao, S., Buonarosa, M.L.., Blankespoor, A., (2011). 

Integrated Vehicle-Based Safety Systems Field 

Operational Test Final Program Report. Washington, 

DC: Department of Transportation, Research and 

Innovative Technology Administration.

7. Bike Europe (2013). Europe's E-Bike Imports 

Indicate Market Size. Retrieved July 7, 2014, from 

http://www.bike-eu.com/Sales-Trends/MarketReport/2013/8/Europes-E-Bike-Imports-IndicateMarket-Size-1326022W/.
5

 • High-definition camera (30 frames per 

second, waterproof, lightweight, and efficient).

 • Inertial measurement units (IMU, 100 Hz).

 • Global positioning system (GPS, 10 Hz).

 • Brake force sensors (100 Hz).

 • Cyclist sensor to record starts and stops.

 • Logger.

 • Modem.

 • Simple human–machine interface with a 

push button for time stamping.

Dozza informed participants that the 

data gathered in this cycling study are 

fundamentally very similar to those 

gathered in driving studies. The objective 

data collected includes videos, positions, 

and kinematics (e.g., GPS and IMU), in 

addition to controls (e.g., brakes and pedals). 

Subjective data collected includes interviews, 

diaries, demographics, and cycling behavior 

questionnaires. Other types of data were 

derived, such as glance behavior and use of 

maps. Analyses were performed by using 

Matlab (a high-level language and interactive 

environment for numerical computation, 

visualization, and programming) and 

NatWare (a toolkit developed at the Vehicle 

and Traffic Safety Center at Chalmers).

Cycling Behavior

Figure 2 shows average speeds using the 

naturalistic cycling data collected. This map 

of downtown Gothenburg, Sweden, is a 

representation of cyclist usage of different 

types of roads. In Sweden, 30 km/hr (18.6 

mi/h) is the maximum speed for bicycles 

before riders may be fined. The red areas on 

this map shows that traditional bicycles are 

driving illegally. It is expected that when the 

same map is produced with e-bikes, it will 

show even higher levels of excess speed. 

The researchers for this study went on to 

examine the speed profile of cyclists riding 

traditional bikes, as shown in figure 3. The 

average speed for cyclists was about 14 km/

hr (8.6 mi/h). 

The research team collected data for figure 3 

in 2012 for 16 cyclists using traditional bikes. 

Currently, researchers are performing the 

same study for cyclists using e-bikes and, 

based on preliminary data, the average speed 

for e-bike users is expected to increase by 

almost 10 km/h (6.2 mi/h). This is important 

to note because some bike paths are shared 

with pedestrians in Sweden, and it is well 

known that increased speed increases the 

risk of an accident occurring.

The research team also used the naturalistic 

cycling data it gathered as observational 

data, which is considered one of the many 

benefits of gathering naturalistic data. The 

team examined cyclists’ obedience to cycling 

rules and gathered information on gender, 

helmet use, crossing behavior, and proper 

light usage at night. 

Accident Causation

The research team performed an eventbased safety analysis for its study. The team 

examined 63 critical events (both crash and 

near crash) by using the button presses from 

the cyclists, who were instructed to press the 

button any time they experienced a safety 

Figure 2. Average cycling behavior speeds.

© Marco Dozza, Google Earth, DigitalGlobe, Lantmäteriet, Metria
6

or uncomfortable situation. These 63 events 

were complemented by 126 baseline events 

chosen at random. The team annotated 

factors related to the environment and road 

users’ behavior for all events. The team also 

calculated odds ratios by looking at the 

difference between critical and baseline 

events in the prevalence of different 

factors. For this critical events’ analysis, the 

team developed a map pinpointing where 

the baseline and critical events occurred 

around Gothenburg. 

The team found that daylight was not a risk 

factor between baseline and critical events; 

however, the analysis indicated that cyclists 

are 10 times more likely to get into trouble 

when there are surface issues (e.g., holes) 

and are at even greater risk in proximity of 

intersections with reduced visibility. The 

team learned that risk also increased when 

there were other pedestrians and bicyclists 

on a potential collision path with the cyclist 

participating in the study. 

Because the team only had six crashes to 

work with, it used near crashes in its analysis 

as well (safety–critical situations from the 

button presses); however, the potential 

issue is whether one can assume that near 

crashes are predictive of crashes. To test this 

assumption, the team will combine its data 

with bicycle accident data from STRADA. 

For this analysis, the team will also account 

for exposure into account and the number of 

single-bicycle accidents during each hour of 

the day. The main purpose of this analysis is 

to show researchers that they can combine 

different data to address questions more 

effectively and that safety–critical situations 

are a sound surrogate for crashes. 

Future Trends: Cooperative Systems and 

Wireless Communication

Dozza told participants that there is ongoing 

research focusing on wireless communication 

methods for bicycles. In particular, applications 

are being developed for smartphones that 

address safety for bicyclists. One example 

is BikeCOM, a cooperative application that 

assists drivers and cyclists at intersections. 

This application, developed by a student at 

Chalmers University, communicates with a 

bicycle and a car approaching an intersection 

by transmitting the positions of the two, 

calculating the estimated time to collision, 

and transmitting a warning to the bicyclist and 

the driver in the form of an audible alert. The 

application performs a threat assessment and 

warns both the bicyclist and driver, depending 

on the probability of collision. Although 

this application was originally designed for 

use by bicyclists, other road users, such as 

drivers and pedestrians, could also use it. This 

application is in the developmental stage and 

has been mainly used as a proof of concept 

for cooperative systems that address multiple 

road users, including cyclists. 

In addition, Safety Pilot is a USDOT field 

operational test (FOT) of vehicle-tovehicle (V2V) communication. Dozza 

told participants that the test is currently 

Figure 3. Cyclists' speed profile.

© Marco Dozza
7

gathering data from 2,843 vehicles and from 

one instrumented bicycle from Sweden to 

evaluate traffic patterns and behavior. 

Lessons Learned and Future Directions

In summary, Dozza noted that future research 

can benefit from the use of naturalistic data. 

For example, with naturalistic data, one is 

able to combine video data with other data 

types to address accident causation, roaduser behavior (including obedience to traffic 

rules and distraction), infrastructure design, 

and intelligent applications. In addition, 

there are important research questions 

that can be better answered by combining 

naturalistic data with other road data from 

accident databases. Dozza also noted 

that naturalistic datasets can be reused, 

and told workshop participants that the 

next step for this research is to compare 

behavior across electric and non-electric 

two-wheelers. Existing tools and methods 

from the naturalistic driving study and FOT 

analyses will be reused, and the new data 

will be integrated. Ultimately, wireless 

communication among road users will 

complement naturalistic data, providing 

new information about road users and 

their surroundings. 

Discussion

After the presentation, the group discussed 

various topics, including the following:

 • Combining crash data and near-crash 

data for bicycles—Although this has 

been discussed, the analysis has not 

been performed yet because of a lack 

of resources. 

 • Comparing motorcycle crashes—

Motorcycle crashes are very different 

because they share the road with cars, so 

the team did not perform a comparison 

with motorcycles. 

 • Downloading the study application—

Dozza informed workshop participants 

that the student-developed application 

is not available for download. Its purpose 

is to demonstrate wireless connectivity 

and should be considered more of a 

feasibility study. 

Additional Resources

Dozza made available to participants a 

selection of additional resources. These 

resources are outlined below.

Videos

 • An example of naturalistic cycling data can 

be viewed here: http://www.youtube.com/

results?search_query=prebikesafe&sm=3 

 • The bikeCOM application featured in 

this presentation can be viewed here: 

http://www.youtube.com/watch?v=kK_

G9wShj2g 

Papers

 • Dozza, M, & Fernandez, A. (2014). 

Understanding Bicycle Dynamics and 

Cyclist Behavior from Naturalistic Field 

Data. IEEE Transactions on Intelligent 

Transportation Systems, 15(1) 376–384.

 o The authors of this paper present the 

hardware used for data collection and 

address cycling comfort in terms of bicycle 

dynamics. They also explore possible ideas 

for the development of an ITS for bikes.

 • Dozza, M., & Werneke, J. (2014). 

Introducing naturalistic cycling data: 

What factors influence bicyclists’ 

safety in the real world? Transportation 

Research Part F–Traffic Psychology and 

Behaviour, 24, 83–91.
8

 o The authors of this paper apply some 

of the basic epidemiology tools to 

determine what factors are associated 

with critical events for bicycles.

 • Dozza, M. (2013). What Is the Relation 

Between Bicycle Dynamics and Safety 

in the Real World? Paper presented at 

the Bicycle and Motorcycle Dynamics 

Conference, Narashino, Japan.

 o The authors of this conference paper 

examine the relationship between 

kinematics and critical events for 

bikes by building classifiers models. 

They mainly show that speed and 

vertical acceleration are the best 

predictors for critical events and 

that speed is more related to near 

crashes (perceived safety), whereas 

vertical acceleration is more related 

to crashes (impacts). Implications 

for the development of ITS or simply 

for searching for critical events in 

the database are addressed. 

 • Dozza, M. (2012). What factors influence 

drivers' response time for evasive 

maneuvers in real traffic? Accident 

Analysis and Prevention, 58, 299–308.

 o The authors of this paper show the tools 

developed and used to analyze data. The 

same tools and data format are used for 

the second Strategic Highway Research 

Program (SHRP 2).

 • Gustafsson, P., Muñez, J., Lindgren, L., 

Boda, C., & Dozza, M. (2013, September)

BikeCOM—A Cooperative Safety 

Application Supporting Cyclists and 

Drivers at Intersections. Paper presented 

at the Driver Distraction and Inattention 

Conference, Gothenburg, Sweden.

 o The authors of this conference paper 

present an application to warn two 

road users (a cyclist and a driver in 

the experiment) at risk of collision 

when passing an intersection. The 

application runs on smartphones and 

is based on threat assessment from 

trajectory estimation. The application 

was developed by four students who 

also wrote the paper and presented it 

at the conference.
9

Overview

Dr. Toru Hagiwara and Dr. Hidekatsu Hamaoka 

informed participants that there are many fatal 

accidents involving pedestrians in crosswalks 

and right-turning vehicles every year in 

Japan, where vehicles travel on the left side 

of the road. For example, in 2012 there were 

4,411 motor-vehicle–related fatalities, and 

more than 1,500 of these were pedestrian 

fatalities.8 Pedestrian accidents occur mainly 

at intersections and are frequently known as 

R-type accidents. An R-type accident occurs 

when a pedestrian approaches to cross the 

crosswalk from the same direction as a rightturning vehicle, as shown in figure 4. A driver’s 

inability to detect pedestrians as they cross is 

one of the main reasons why these types of 

accidents occur. For this reason, drivers need 

help to become more aware of pedestrians in 

the crosswalk. 

This presentation focused on three studies 

that investigated driver and pedestrian 

recognition behavior as a basis for developing 

ways to avoid conflict. These studies focused 

on the following:

 • How do drivers recognize pedestrian 

behavior and how do they select 

a strategy to avoid conflict with 

pedestrians who approached from the 

right at intersections? 

 • What is the performance of a 

pedestrian–vehicle dedicated shortrange communications (PV-DSRC) 

system in which dedicated short-range 

communications (DSRC) transmits data 

to drivers and pedestrians about the 

dynamic conditions at the intersection?

 • What is the crossing behavior of 

pedestrians in crosswalks, and do they 

confirm the approaching right- or leftturning vehicle while crossing the crosswalk? 

Driver Recognition Behavior

For the first study, the research team looked 

at how drivers recognized pedestrians in 

intersections. The team assessed driver 

behavior for avoidance of conflict with 

pedestrians who approached from the right 

and how the driver predicted the pedestrian's 

rate of crossing the intersection. The team 

conducted field experiments to measure the 

Driver and Pedestrian Recognition Behavior for 

Avoiding Conflict with Each Other at an Intersection

Dr. Toru Hagiwara Dr. Hidekatsu Hamaoka

Hokkaido University, Japan Akita University, Japan

Figure 4. An R-type accident.

© Toru Hagiwara and Hidekatsu Hamaoka

8. Japanese National Police Agency (2013). Retrieved July 

7, 2014, from http://www.worldhighways.com/sections/

general/news/improved-road-safety-for-japan/.
10

time it takes drivers to recognize a pedestrian 

and also measured drivers’ avoidance 

behavior under various conflict conditions 

as a function of the pedestrian’s visibility. To 

measure the driver’s avoidance behavior, the 

team varied the interval between the time 

at which the pedestrian passed the conflict 

point, or the point of impact, and the time at 

which the right-turning vehicle passed the 

conflict point. 

The team developed a time–space diagram 

and estimation for four types of driveravoidance behavior. One of these avoidance 

behaviors is front passing, which is when 

the right-turning vehicle passes through the 

conflict point in front of the pedestrian, as 

shown in figure 5. Other behaviors include 

stopping, which is when the right-turning 

vehicle stops before the conflict point to 

avoid hitting the pedestrian; avoidance, which 

is when the driver brakes and slows to yield 

to the pedestrian after starting to turn right; 

and passing behind, which is when the rightturning vehicle passes through the conflict 

point after the pedestrian without braking 

and slowing. 

The team used the following formulas to 

calculate the predicted time lag (PTL) and 

observed time lag (OTL): 

 PTL= Time 1 - (Time 2 + running time) (1)

 OTL= Time 1 - Time 3 (2)

Time 1 is the time when the pedestrian 

passes through the conflict point, Time 2

is the time when the first oncoming vehicle 

passes through the conflict point, and 

Time 3 refers to the time when the rightturning vehicle passes through the conflict 

point. The running time from the start to 

the passage through the conflict point is 

5.02 sec, if the driver does not perform any 

avoidance behavior. The team performed 

315 runs in the field. Results showed that:

 • For PTL exceeding 3 sec, the vehicle 

passing in front of the pedestrian 

predominates.

 • For PTL between 2 and 4 sec, the 

vehicle stopping for the pedestrian 

predominates.

 • For PTL of less than 2 sec, the vehicle 

braking and slowing or some other 

avoidance response predominates.

 • For PTL of less than -1 sec, the 

vehicle passing behind the pedestrian 

predominates.

Overall, the authors of this study of drivers’ 

pedestrian-recognition behavior found 

that the driver’s choice of avoidance 

behavior correlated with the PTL to hit the 

pedestrian. The minimum PTL at which 

drivers will yield to the pedestrian at the 

conflict point was approximately 2 sec. 

In addition, drivers tended to choose the 

avoidance behavior of passing behind the 

pedestrian when the drivers focused on the 

pedestrians before starting the right turn. 

Figure 5. Front-passing diagram.

© Toru Hagiwara and Hidekatsu Hamaoka
11

Performance of a PV-DSRC System in 

which DSRC Transmits Data to Drivers and 

Pedestrians at Intersections

The researchers of this second study assessed 

the data transmission capability of a PVDSRC system for situations with right-turning 

vehicles and pedestrians at intersections. 

The goal was to measure the performance 

of the PV-DSRC data transmission between 

the right-turning vehicle and the pedestrian 

under dynamic conditions at the experimental 

intersections. The researchers also evaluated 

the capability of a PV-DSRC data transmission 

system at actual intersections. To evaluate 

this system, the researchers reproduced the 

potential for collision conflicts between rightturning vehicles and pedestrians by using a 

test track in Tomakomai City, Japan, as shown 

in figure 6. For each run, the pedestrian starts 

from one of the four starting points (R1, R2, 

L1, L2) shown in figure 6 and then crosses the 

intersection. The starting point is expected 

to affect the data transmission performance 

because of the positional relationship 

between the pedestrian and the DSRC device. 

The PV-DSRC system used in these 

experiments is the same intervehicle DSRC 

(IV-DSRC) system that met the experimental 

guidelines for IV-DSRC systems that use 

the 5.8 GHz band (ITS FORUM RC-005 ver 

1.0). Figure 7 shows how the pedestrian 

communicates with the vehicle. The 

researchers followed up by conducting 

a field experiment at three intersections 

in Yokosuka City, Japan, to evaluate the 

performance of data transmission between 

the pedestrian and a right-turning vehicle in a 

real-world setting. The purpose of this study 

was to assess the influence of intersection 

Figure 6. Reproduced collision conflict between right-turning vehicle and pedestrian.

© Toru Hagiwara and Hidekatsu Hamaoka
12

size, location of the right-turning vehicles 

in the intersection, and the presence of an 

oncoming vehicle and a leading right-turning 

vehicle on data transmission performance. 

The experiment was performed by using 

multiple passes through the intersections 

at three different sized intersections: small, 

medium, and large. The receiving power at 

the large intersection showed that when 

the distance was between 50 m (164 ft) and 

-50 m (-164 ft), values of received power far 

exceeded the required level, but the received 

power values tended to be lower when 

there was an oncoming vehicle. Results of 

the packet-arrival rate, or throughput, show 

that they achieved the needed 80-percent 

packet-arrival rate required in Advanced 

Safety Vehicle-3 technology. When the rightturning vehicle is between 100 m (328 ft) and 

30 m (98 ft) distance, the packet-arrival rates 

exceeded the 80-percent threshold.

Overall, the study indicated that the data 

transmission capabilities of a PV-DSRC 

system between right-turning vehicles and 

pedestrians at intersections were effective. 

If equipped with the IV-DSRC system, rightturning vehicles could communicate with 

pedestrians in crosswalks who cannot be 

detected by the drivers or local sensors alone. 

Simultaneously, pedestrians could be alerted 

to their associated risk. Ultimately, the DSRC 

data transmission system could provide 

effective support to drivers who do not 

notice their risk of colliding with pedestrians 

both before and while making a right turn. 

Behavior of Pedestrians in Crosswalks and 

Recognition of Approaching Turning Vehicle 

The researchers conducted the third field 

experiment to understand the crossing 

behavior of pedestrians in the crosswalk. The 

researchers investigated how pedestrians 

identified the approach of right- or leftturning vehicles while crossing the crosswalk. 

They analyzed the head-turning behavior 

of pedestrians for right- or left-turning 

vehicles and considered the limitations of 

what a pedestrian can see and hear. The 

purpose of this experiment was to identify 

the point where pedestrians can confirm 

an approaching vehicle. The researchers 

achieved this by comparing the head-turning 

behavior of the subjects to assess whether 

they have an average confirmation or an 

appropriate confirmation. 

Figure 7. Pedestrian–vehicle dedicated short-range communications system.

© Toru Hagiwara and Hidekatsu Hamaoka
13

Figure 8 shows the pedestrian-crossing 

experimental intersection scenario, which 

was implemented on a test track. Each 

pedestrian test subject was asked to proceed 

through the crosswalk as a vehicle makes a 

right or left turn toward the point. Only one 

vehicle can turn into the crosswalk. Each of 

the 44 subjects carried out this experimental 

procedure 16 times. The repetitions of the 

testing had the following variations:

 • Two start positions (right or left).

 • Two approaching vehicles (right or left).

 • Two sight restrictions (day or night).

 • Two hearing restrictions (wearing loose 

headphones or not).

By linking the head-turning angle with the 

location of intersection, the researchers 

showed characteristics of head-turning 

behavior. Subjects were both young and 

elderly, and some wore headphones. The tests 

were conducted during the day and at night. 

Subjects wore a hat with a head camera and 

a six-axis sensor to precisely measure headturning angle (50 Hz, 0.001 deg/s unit). 

This pedestrian study documented the 

importance of designing countermeasures 

for traffic accidents that involved a vehicle 

and a pedestrian from the viewpoint of the 

pedestrian. The field experiment analyzed 

head-turning behaviors of crossing 

pedestrians relative to their starting position 

and the approach of vehicles. 

Summary and Lessons Learned 

By conducting field experiments, the 

researchers were able to study driver 

recognition behavior, pedestrian recognition 

behavior, and the potential effectiveness of 

PV-DSRC systems. In doing so they were able 

to conclude the following: 

 • Drivers tend to choose safety-avoidance 

behavior when they focus on pedestrians 

before starting to make a right turn.

 • Locations where pedestrians confirm 

the imminence of a left- or right-turning 

vehicle depend on whether the vehicle 

was making a left or right turn and 

whether it is day or night.

 • The PV-DSRC data transmission system 

can provide effective support to drivers 

and pedestrians who do not notice the 

potential for collisions. 

Discussion

After the presentation, the presenters noted 

that the goal of the experiment was to test the 

most difficult intersection conditions, which 

is why an intersection without street lights 

was purposefully chosen. The presenters 

also noted that the pedestrians used a 

pair of headphones to block out noise and 

participants suggested that the researchers 

re-run the study with pedestrians listening to 

music with their headphones. One workshop 

participant observed that the interaction 

between pedestrian and vehicle maintains 

a constant speed dictated by the study 

parameters, and suggested that naturalistic 

data be gathered to capture that interaction. 

The presenters and participants noted that 

in the United States, there can exist blind 

spots for drivers turning into an intersection, 

especially when making a right turn.

Figure 8. Pedestrian crossing an 

experimental intersection.

© Toru Hagiwara and Hidekatsu Hamaoka
14

Overview

Dr. Michael Manser examined strategies 

for data collection and analysis of the 

relationship between the driver and 

infrastructure, with a focus on intersections 

and complex interchanges. He discussed four 

ways to conduct research on infrastructure: 

laboratory, simulator, test track, and 

real world. Manser informed workshop 

participants how these tools can be used 

to analyze the relationship among driver, 

infrastructure, and the roadway. He identified 

ways to conduct infrastructure research 

during the presentation. The objectives of this 

presentation were to (1) introduce the range 

of methods and emerging technologies to 

obtain and analyze driver–infrastructure and 

roadway data, (2) discuss the data sources 

that are used in analyses, and (3) consider 

more effective ways to approach this type 

of data. 

Research Environment and Capabilities 

Manser informed workshop participants 

that the relationship between driver and 

infrastructure (e.g., signage and electronic 

billboards above the roadway) and between 

driver and roadway (e.g., roadway geometrics 

or striping) can be examined in four types of 

research environments: laboratory testing, 

driving simulation, test track, or real-world 

field observations. Research conducted in the 

laboratory included tests such as computerbased testing, surveys, and questionnaires. 

Manser noted that there is a continuum of 

fidelity across different simulators, ranging 

from desktop simulators to driving simulators; 

however, with the advances in technology 

they all have increasingly higher fidelity. Test 

tracks are closed-course facilities that can be 

highly controlled, and no other vehicles can 

impinge on the research protocol. Real-world 

research environments use vehicles with 

embedded data collection systems. These 

vehicles operate in largely uncontrolled 

environments on a prescribed course, route, 

or self-selected route, and researchers then 

mine the data.

In the last decade, real-world field research 

environments have matured significantly 

because of the power of computing. Manser 

identified the advantages of two types of 

real-world research environments: on-road 

controlled and semi-controlled. An on-road 

controlled real-world research environment 

uses an artificial scenario and a test vehicle 

with an extensive vehicle data acquisition 

(vehDAQ) system. For example, drivers 

proceed through an intersection multiple 

times under different conditions in a highly 

controlled on-road research environment and 

only proceed to cross when instructed to do 

so (i.e., when there are specific types of traffic 

gaps or streams of gaps). The researchers are 

able to determine which gap a driver would 

normally take and which gap they would 

select in response to alternative intersection 

signs. In semi-controlled research settings, 

drivers operate a test vehicle or their own 

Driver–Infrastructure and Roadway Data 

for Human Factors Research

Dr. Michael Manser

Center for Transportation Safety, Texas A&M Transportation Institute 

Formerly with the University of Minnesota
15

vehicle to exhibit natural driving behaviors, 

but the researchers retain control of where 

and when the driving occurs. A small vehDAQ 

system is installed in either vehicle, and 

drivers are instructed to follow a prescribed 

course at a prescribed time. This differs from 

naturalistic driving in which drivers choose 

their routes, timing, and sequence of driving, 

and therefore there is very little experimental 

control. Figure 9 shows examples of onroad controlled and semi-controlled studies 

conducted at the University of Minnesota.

Validity and Experimental Control 

Different research environments result in 

tradeoffs between validity and experimental 

control. In terms of validity, researchers must 

ask whether they are measuring what they 

intend to measure. In terms of experimental 

control, researchers want to know how 

much control they have over the variables 

involved. To understand the driver and the 

infrastructure, certain variables of interest 

are measured, including the driver, vehicle, 

intersection characteristics, time of day, and 

weather. As illustrated in figure 10, there is a 

positive linear relationship in validity moving 

from left to right or from surveys to in-field 

observations. The opposite is expected 

to be true for experimental control. Here, 

it is assumed that experimental control 

will decrease systematically, with more 

experimental control in a survey environment 

compared with an in-field observation or 

real-world environment.

Although figure 10 shows what is expected, 

in terms of the relationship between validity 

and experimental control by research 

environment, Manser proposed an enhanced 

model. Although not yet scientifically proven, 

the model offers a more realistic way to 

conceptualize the relationship among the 

research environments. The enhanced 

model adds on-road controlled and semicontrolled environments in between test 

Figure 9. Real-world research environments.

© Michael Manser
16

track and naturalistic research environments. 

As figure 11 illustrates, experimental control 

can be high in surveys, laboratory testing, 

and driving simulation where the driving 

scenarios can be controlled. With the advent 

of better technology on the test track and in 

the on-road controlled and semi-controlled 

environments, experimental control has 

become significantly higher. Therefore, 

instead of a linear decrease in experimental 

control through the range of testing 

environments, the level of experimental 

control can be maintained from surveys to 

semi-controlled environments and naturalistic 

studies. Although naturalistic studies can 

have more experimental control, it has not 

yet achieved the same level as the other 

environments. Likewise, although validity 

is lower in survey and laboratory testing 

environments, it can increase substantially in 

the on-road controlled and semi-controlled 

environments. For example, drivers can be 

placed into real-life driving scenarios and 

in traffic while experimenters maintain a 

substantial amount of experimental control 

by deciding where they are driving and when. 

Selecting a Research Environment 

The choice of research environment may 

depend on the product development 

phase; for example, in the case of a safety 

intervention, this may include signs, roadway 

geometrics, and striping. A study of gap 

perception in drivers would be better suited to 

a different research environment than would 

a study of a developed product or products. 

Researchers who studied a product concept 

might consider some laboratory testing to 

understand drivers’ mental models and basic 

level of understanding when studying gap 

perception, in addition to how drivers select 

what gap or streams of gaps to accept. The 

use of a driving simulator can show if one 

product is better relative to another. As the 

product becomes more and more refined, 

it is important for researchers to move into 

naturalistic testing. 

Identifying Data Sources

When looking at infrastructure-based as 

well as roadway-based safety solutions, 

there are four categories of data to consider: 

driver, objects, road or infrastructure, and 

traffic. Figure 12 illustrates the tools and 

environments in relation to these data 

categories. The driver data describes 

how the driver behaves and responds in 

terms of two variables. The first variable 

measures the primary control of the vehicle 

through acceleration, use of pedals and 

brakes, and steering. Physiological data is 

the second variable and measures driver 

reactions. For example, if researchers are 

interested in driver stress, then they can 

look at galvanic skin responses, or if they 

are interested in a faster-changing metric, 

then they can look at heart-rate variability 

or brain-wave activity. 

Figure 10. Expected research environments 

in terms of validity and experimental control.

© Michael Manser© Michael Manser

Figure 11. Adjusted research environments 

in terms of validity and experimental control.
17

The second data category includes objects 

in the environment. This is important for 

infrastructure research because there is a need 

to know where an object is, such as an overhead 

sign or a roadside traffic sign, and how drivers 

respond to these alternative locations. Drivers 

may change their behavior appropriately based 

on the location of that sign, or they might ignore 

or miss it. Researchers who study ITS may need 

to look at the particular state of signs and how 

they influence driver behavior. 

The third category includes road and 

infrastructure because it is important to 

examine how a driver behaves and responds to 

the roadway or relative to the roadway. Some 

of the data elements to capture include lane 

boundaries, centerlines, and road geometrics. 

Researchers can use these data to understand 

driver response to specific roadway elements. 

The traffic category refers to sources of data 

that are difficult to control. Traffic encompasses 

many factors, such as velocity, acceleration 

or deceleration, location, trajectory, and 

lane and traffic density. These factors affect 

driver decisions in traffic, such as speeding, 

maintaining speed, or crossing intersections. It 

is a challenge to control these variables. 

These data categories should be thought 

of macroscopically as well as in terms of 

microscopic driving behaviors, that is, how 

individual drivers react to something in the 

roadway. It is important to understand the 

aggregation of all the behavioral changes 

each driver makes and how they affect 

road transportation’s efficiency and safety. 

Cooperative Intersection Collision 

Avoidance System–Stop Sign Assist Project 

In selecting a research environment to 

evaluate a safety measure, Manser informed 

the workshop participants of a cooperative 

intersection collision avoidance system–stop 

sign assist (CICAS-SSA) project in Minnesota, 

funded by FHWA. This project is one of many 

in the United States in which researchers are 

looking at different intersection technologies. 

The goal of the Minnesota research is to 

study gap-size rejection at particularly 

dangerous rural intersections in Minnesota. 

This intersection exhibits a higher crash rate 

than what would be predicted for this type 

of intersection. This intersection represents 

many intersections across the United States, 

and the researchers of this project are 

focused on determining what is problematic 

about the intersection and what can be done 

about it. 

Manser told the workshop participants that 

the research team found that gap perception, 

that is the ability to determine an acceptable 

gap in traffic, is fairly poor for drivers. Based 

on previous research, the gap perception 

problem was considered to be the “root 

evil” of intersection crashes. This research 

began with laboratory studies in which the 

researchers presented several intersection 

concepts to participants, and the participants 

selected which sign they preferred, as shown 

in figure 13. By using the laboratory testing 

facilities, the researchers were able to narrow 

the field and conduct simulation testing with 

the more promising concepts. After evaluating 

their better sign concepts, the researchers 

© Michael Manser

Figure 12. Tools and environments 

in relation to data sources.
18

moved to product evaluation and tested the 

best-rated sign in on-road controlled studies 

and semi-controlled studies. For the next 

step, the researchers have been performing 

an FOT funded by USDOT, which is underway 

to measure the effectiveness of a best-rated 

sign. This project provides an example of how 

researchers can take particular concepts and 

evaluate them early on, refine those concepts, 

take those concepts into the simulation 

studies, and eventually evaluate the best 

concept in FOTs. 

Infrastructure-Based Driver and Traffic Data

To test whether a sign has an effect on traffic, 

researchers used an instrumented vehicle 

equipped with a GPS on the car roof. The 

instrumented vehicle measured driver behavior 

and location of the vehicle within centimeters. 

The researchers of the CICAS-SSA study also 

set up a fully instrumented intersection, 

equipped with radar sensors on all legs, to 

monitor traffic approaching the intersection 

according to factors that included location, 

speed, and velocity. Test drivers operated the 

vehicle both when the sign was on and off. 

The researchers recorded variables, which 

included right, left, and crossing maneuvers, 

gap-safety margins, movement time and 

rejected gaps, eye-glance behavior toward 

sign and traffic, and subjective measures 

from random gap-simulation studies. 

An interesting aspect that Manser highlighted 

for workshop participants was that, when 

these data are fed into the CICAS-SSA 

system, one can look at the interaction with 

drivers and the traffic. It was also possible 

to look at, dynamically and in real time, the 

gaps in front of or on the side of the driver, in 

addition to which gaps drivers reject in traffic. 

With the richness of these data, researchers 

were able to develop profiles of successful 

gap acceptance and gap rejections. 

Research Challenges

On the basis of this research and knowledge 

of driver infrastructure and roadway 

data, Manser informed participants of the 

significant challenges that researchers 

face in the field. For example, increasing 

validity requires more complex research 

Figure 13. Experimental signs used in laboratory study.

© Michael Manser
19

environments, which generate higher costs. 

Unlike driver simulation studies, on-road 

studies often require an instrumented vehicle 

and an instrumented intersection outfitted 

with sensors. Researchers must coordinate 

multiple streams of data into one, but analyzing 

multiple data streams is a long and intense 

process, which raises the cost. In addition, 

efforts to increase generalizability require 

increases in sample size, which also generates 

higher costs. There are also costs related to 

staff time needed in the field because of the 

number of staff required at one time to monitor 

the instruments, multidirectional traffic flows, 

and cue the test vehicle. Finally, improving the 

validity of research depends on larger samples 

and use of differential GPS, which produces 

more accurate data but at higher costs. The 

bottom line is that to improve validity, costs 

will increase. 

Research Gaps

Manser informed the workshop participants 

that eye trackers have the potential to be a 

strong tool to examine the efficacy of new 

infrastructure and roadway-based systems; 

however, there are research gaps with eye 

trackers that limited their use for the study 

of driver–infrastructure roadway data. There 

are error rates that cumulate with distance, 

and most eye-tracking manufacturers claim 

that their eye trackers have about 3-degree 

accuracy. The result is that when drivers look 

at signs at about 61 m (200 ft), the subtended 

angle becomes 3 m (10 ft), which is a distance 

that is too great to determine accurately if a 

driver is looking at a sign. 

Differential GPS keeps track of head position 

and has an error rate of several centimeters. 

There are also lag times between the eye 

tracker and data collection system that 

produces an error rate. The driver’s head 

movement as he or she proceeds through 

the intersection also produces an error 

rate. All of these errors cumulate, reducing 

the accuracy of eye trackers for studying 

driver behavior in complex intersections. In 

summary, Manser noted that aspects of eye 

behavior that must be examined to study a 

specific infrastructure or roadway element 

include looking at the infrastructure or 

roadway element and the area around an 

infrastructure or roadway element. 

New Tools for Accuracy

Manser also highlighted light detection and 

ranging (LIDAR) data as a promising new 

tool that portends to be useful for driver 

infrastructure and roadway data research. 

LIDAR is a scanning laser-based radar 

type system that is able to pick up objects, 

including cars, trees, pedestrians, and 

buildings, quickly and accurately. It offers 

tools to measure how drivers respond to 

objects in the environment and correlates 

driver behavior with environmental inputs in 

real time. One of the major challenges with 

the LIDAR data tool is that, although people 

can recognize the objects that the system is 

picking up as buildings, trees, or pedestrians, 

the computer sees them as zeros and 

ones. Therefore, the larger challenge is not 

collecting the data but understanding what 

the data means. 

Manser told workshop participants that 

Google is one of the major innovators using 

this new tool and that it is implemented in 

the Google self-driving car. The Google car 

obtains information about its surroundings 

by using LIDAR data, along with other 

sensors. If researchers want to look at 

how drivers respond to objects in the 

environment that may be changed as part 

of an experimental study, researchers can 

begin to correlate how the drivers behave 

with the new objects in the environment in 

real time. Manser noted that there needs to 

be more effort and research to use LIDAR. 
20

Discussion

In summary, participants raised questions 

about the challenges of using eye-tracking 

technologies and noted that use of these 

technologies depends on the research 

question under study. For example, if the 

goal is to measure a driver’s attention 

to large signs that are close by, 3-degree 

accuracy is sufficient. Because analysis 

of eye-tracking data can be noisy, some 

researchers segment eye-tracking data 

into zones for ease of analysis. Although 

it is accepted that eye tracking is useful to 

study driver attention to features that are 

close by, it remains difficult to measure how 

drivers attend to objects in the distance. 

Sampling rate can also be an issue when using 

eye trackers, as can individual differences, 

given that some people limit their scan to 

1.5 degrees. Suggestions for analysis include 

looking at patterns as well as the accuracy 

of the data, which necessitates using good 

software able to pick up patterns. Manser 

noted that eye trackers are most effective 

in daylight but that this may not be a major 

constraint, because 90 percent of driving 

occurs during the day. Manser also noted 

that there are lower tech alternatives to 

eye tracking, such as video data, which can 

answer research questions including head 

position in relation to road activities and 

pedestrian movements. 
21

Introduction

During this presentation, Dr. Susan Chrysler 

informed workshop participants how to 

find the most effective methods to address 

research needs based on target questions. 

Examples of research on traffic control 

devices (TCD) were used to demonstrate how 

a variety of methods could be applied to the 

same research question. The examples were 

drawn from materials prepared to educate 

traffic engineers on how to select effective 

evaluation methods for TCDs. The advantages 

and disadvantages of alternative research 

methods were also outlined, including focus 

groups and open and closed test courses. 

Human Factors Research on Traffic Control 

Devices

TCDs include signs, pavement markings, 

and signals. FHWA’s Manual on Uniform 

Traffic Control Devices (MUTCD) provides 

standards and specifications for the design 

and application of TCDs in the United States.9 

Local and State governments use the MUTCD 

standards and guidelines to produce TCDs 

to manage their local road safety and traffic. 

Localities can modify options permitted in 

the MUTCD according to an FHWA process 

that requires evaluation of new candidate 

TCDs. Chrysler noted that TCD practices are 

selected for inclusion or modification based 

on data collected through experimentation 

and that the MUTCD provides extensive 

guidance about the review process to be 

used by State and local governments. 

Chrysler informed workshop participants 

that, because of the importance of ensuring 

appropriate evaluation for new TCDs, State 

and local governments often conduct 

research to receive approval for their new 

TCD or new application of an existing 

TCD. The human factors research topics 

include visibility, legibility, comprehension, 

compliance, and preference. Chrysler 

highlighted that examining these topic 

areas, in relation to the specific application, 

will determine the effectiveness of 

TCDs to convey directions and warnings 

to drivers. For example, visibility human 

factors research topics include brightness, 

color, and shape aspects of signs, markings, 

and signals. In addition, legibility of signs 

is dependent on adequate font type, size, 

and proper color contrast in different road 

and environmental conditions. In addition, 

beyond visibility and legibility, TCDs need to 

be understood by drivers. Comprehension 

can be tested by using simple test methods 

within more complex methods involving 

traffic observations. Collecting preference 

data for TCDs identifies designs with which 

drivers are more comfortable compared with 

the other allowable options. This preference 

data, however, are not always predictive of 

driver behavior.

Traffic Control Devices and In-Vehicle Systems 

Dr. Susan Chrysler

National Advanced Driving Simulator, University of Iowa

9. Federal Highway Administration. (2009). Manual 

on Uniform Traffic Control Devices. Washington, DC. 

Retrieved June 11, 2014, from http://mutcd.fhwa.dot.gov/

kno_2009r1r2.htm
22

These considerations undergo a thorough 

examination to generate valid methods for 

evaluation. The FHWA publication, Pedestrian 

and Bicycle Traffic Control Device Evaluation 

Methods (Publication No. FHWA-HRT-11-035), 

recommends a variety of methods to 

evaluate TCDs for pedestrian and bicyclist 

TCDs. The methods presented in this report 

are applicable for the evaluation of any TCD. 

Chrysler informed workshop participants 

of several key issues that can affect human 

factors data collection. The issues relate to 

data collection methods used to evaluate 

behavior as follows:

 • Subject sample representativeness—The 

representativeness needs to be assessed 

to account for differences among 

the research sample and the general 

population. Decisions about who to 

test need to be made in the context of 

the problem tested. For example, if the 

problem is that school children are not 

obeying a signal, then school children 

should be tested, rather than adults. 

People who volunteer for experiments 

may be very different from people who 

do not participate. They are more likely 

to have higher socioeconomic status 

and level of education. Efforts should 

be made to recruit people with different 

reading abilities, education levels, and 

visual abilities. 

 • Self-selection bias—Volunteers may 

not be representative of the general 

population. For example, participants 

may be better drivers. People within 

a reasonable distance from a facility 

where a highway simulator or test track is 

located are often more likely to participate 

in the study. Any bias associated with 

location should be assessed. 

 • Human subjects’ protection regulations— 

Researchers must be aware of and 

comply with human subjects’ protection 

principles to perform behavioral research. 

Conformance can be established by an 

institutional review board committee set 

up to monitor human research. 

 • Recruitment methods—Different 

populations for experiments can be 

obtained depending on how researchers 

recruit and compensate subjects.

Validity and Experimental Control in 

Experimental Methods

Researchers must make trade-offs between 

validity (i.e., the ability of the study results 

to predict behavior on the road) and 

experimental control when selecting research 

methods. As figure 14 illustrates, validity is 

inversely proportional to experimental control. 

On one hand, utilizing in-field observation, for 

example, allows a “natural” observation with 

no influence by experimenters on decisions 

made by road users. On the other hand, 

this method does not allow for any control 

of traffic or weather, so not every subject 

is exposed to exactly the same conditions. 

Surveys and laboratory testing are examples 

of controlled experiments in which the validity 

of a participant’s stated behavior is questioned 

because of desirable response bias, that is, in 

Figure 14. Trade-off between 

validity and experimental control.

© Susan Chrysler, University of Iowa
23

which everyone reports that they will comply 

with the device but may not in real life. 

Overview of Research Methods

Chrysler next gave workshop participants 

an overview of several research methods. 

These were categorized as surveys, focus 

groups, controlled experiments, and 

observational experiments. 

Surveys

This category includes experiments in which 

researchers do not control the amount of time 

participants view the question. In addition, 

a survey may also allow for open-ended 

responses. In this context, laboratory tests in 

which researchers can control exposure time 

would not be considered part of the survey 

category. Survey methods may be considered 

either interactive (e.g., telephone surveys 

or intercept on-site) or non-interactive. 

Non-interactive survey methods include 

questionnaires that are conducted via mail 

or email, as well as self-paced questionnaires 

that use computer or paper.

Chrysler told workshop participants that, 

to measure what is intended, researchers 

need to ask the same question in 

different ways to verify the given answer. 

For example, instead of using a direct 

question that results in an open-ended 

response, questions can use illustrations or 

scenarios that provide a venue for subjects 

to demonstrate their understanding by 

indicating what action would be taken. This 

means that if people are asked, “What does 

a yellow line mean?” they may not be able 

to answer; however, when shown a photo 

of a one-way street with a yellow line, they 

can correctly identify the direction of travel, 

often without being able to identify why 

they answered in a particular way. Openended questions are time-consuming to 

code and summarize, so multiple choice or 

true-or-false questions may be preferred.

Focus Groups

In this category, groups of people are selected 

based on specific demographics or other 

characteristics to represent a target population. 

Evaluation that uses focus groups can occur 

during early phases of research and can be 

conducted at multiple locations. Chrysler 

noted that focus groups are particularly useful 

to narrow down the number of TCD alternatives 

that should be tested in subsequent studies 

that used more controlled methods. They are 

also helpful to gain insight into baseline driver 

understanding of a new traffic operation so 

that a new TCD can be designed to match that 

native understanding. Chrysler also cautioned 

that a dominating personality in the group 

may influence opinions. 

Controlled Experiments

 • Laboratory Experiments of Comprehension

This tool is considered useful to analyze 

how well subjects comprehend signs, 

markings, and signals. Researchers can 

measure response time, accuracy, and 

limited viewing time of traffic signs. The 

use of a “button box” is suitable in these 

experiments because it provides a way 

to measure duration of viewing time, is 

easy to use, and is therefore accessible 

to much of the population. Button boxes 

are portable, can be connected to laptops, 

and allow experiments to be performed 

on a large scale.

 • Simulation

TCDs can also be analyzed by using 

driving simulators. For example, the 

influence of changeable message signs on 

a driver’s decisions can be measured by 
24

using verbal questions under the task load 

of driving. It is not necessary to record 

trajectory measurements (i.e., speed 

and acceleration). After driving in the 

simulator, subjects can be asked about the 

instructions displayed on a changeable 

message sign. 

Simulators can also reveal the dynamic 

aspects of behavior. Lane changing over 

time is an example of an observed behavior 

with dynamic aspects. The flexibility of 

using simulators to conduct experiments 

permits different starting lanes and driving 

environments and can reveal speed 

changes and errors that usually precede 

vehicle crashes. Another advantage of 

driving simulators is that they provide a 

detailed, cost-effective way to test multiple 

versions of TCDs or in-vehicle displays and 

warning systems. 

 • Closed Course, Test Track, and Open Road

Closed-course test facilities are paved 

facilities that have availability for testing 

when not in use, such as unused fairgrounds 

or mothballed runways. In closed-course 

experiments, the test stimuli are actual 

roadways, and the closed courses use 

infrastructure (e.g., road intersections) 

as scenarios to perform evaluations. Test 

tracks are paved, dedicated runs without 

access to the outside world and generally 

have adjustable field instrumentation to 

measure vehicle performance parameters. 

Open-road testing refers to testing that 

uses roads that are in use or that may be 

temporarily closed for the test protocol. It 

is thought that drivers are under a more 

realistic attentional load when the study is 

conducted on an actual road. 

Drivers can wear or use eye-tracking 

devices to monitor their visual behavior 

during testing on closed courses or test 

tracks. Eye-tracking methods measure 

the position, duration, and movement 

of the driver’s eyes, which is a proxy for 

where they are looking. Researchers have 

noted that this is useful data to correlate 

with the vehicle inputs and behavior in 

response to cues and prompts outside 

the vehicle. Because there are technical 

limitations with day time use of eyetracking devices, glance behavior needs 

to be hand coded from in-vehicle video 

cameras. Researchers often prefer test 

tracks over closed courses because they 

can accommodate more dangerous and 

higher speed scenarios. 

Observational Experiments

Chrysler told workshop participants that 

observational methods require no direct 

contact with drivers. The measurements 

usually sought in observational experiments 

account for driving behavior—speed can be 

measured by using tubes or radar, and video 

data collection can be used to observe lane 

changing and compliance with signs and 

markings. Using existing cameras and live 

coding of traffic from a traffic management 

center (TMC) provides additional resources 

for researchers to evaluate specific conflicts.

Test devices can be installed, or researchers 

can use existing ones. Regardless of what 

method is used, Chrysler noted that finding 

comparable sites for data collection is a 

challenge. For example, signs might work 

for specific road geometries but may not be 

appropriate for others.

 • Open-Road Drives

During the presentation, Chrysler informed 

workshop participants that some of the 

procedural limitations when performing 

open-road drives are the requirements 

involved when installing new signs or 
25

pavement markings. These experiments 

are preceded by a long process that 

involves obtaining permission to install 

devices, material fabrication, and 

installation. In addition, this method 

has risks associated with insurance 

coverage and liability for participants, 

researchers, and operators. It is more 

efficient to conduct these types of 

tests on a closed course where the 

experimenter has more latitude to make 

changes in the environment.

Another challenge noted when 

performing open-road research is the 

lack of experimental control. Participants 

may be able to identify the purpose of 

the research upon seeing the first test 

device. To compensate for the lack of 

control and to avoid order effects, it is 

necessary to alternate routes so that 

the first device is not the same for every 

subject. Limitations of being able to 

control different external factors, such 

as weather, traffic, and time of day, also 

make it difficult to equate or measure 

driving behavior across participants. 

Another issue to be aware of is 

possible vandalism and theft of signs 

and vehicle equipment. 

Chrysler noted that data collected from 

on-road tests accurately reflect observed 

driving performance under realistic 

conditions (e.g., lighting, workload, 

and traffic). Experiments on the road 

are justified because of their greater 

acceptance and validity for practitioners—

traffic engineers are often more convinced 

of a finding if it has been tested on 

the road. Researchers have to prove 

and demonstrate to the professional 

community that the use of test track, 

driving simulators, focus groups, and 

surveys to analyze behavior is valid. 

On-road experiments can be tied to 

driving simulator experiments, for 

example, by using the same vehicle type in 

experimental simulations and then asking 

participants to drive the vehicle on road 

during the same day. The goal of having 

the same person during the same day 

using the same type of car is to predict 

their driving behavior. One example of 

this is eye-tracking studies during night 

conditions, which can be analyzed by 

using both research methods. 

Comparison of Methods

Some of the factors considered important 

when comparing different sources were 

outlined as cost, time for study, experimental 

control, safety, diversity of sample, face 

validity, and the number of alternatives that 

can be tested. As figure 15 shows, there is 

no perfect method that can account for all 

aspects. It all depends on what the specific 

question is and the measure wanted. 

One of the critical factors when choosing a 

research method is cost. Researchers select 

research methods that are most cost-effective 

and can answer the research question. 

Research methods that are simple to set up at 

a low cost are often the most difficult to score 

and to be used for comparison (e.g., traffic 

surveillance cameras). Research methods 

with automatic scoring are also difficult to set 

up (e.g., a properly constructed survey).
26

Research Needs

Chrysler highlighted several research needs 

for attention at the end of the presentation, 

as follows:

 • Comparing different models—There is 

scarce previous research on how to do this 

and assumptions on validity are grounded 

in what is observed from the surface. 

 • Data mining—This is an alternative strategy 

to consider when evaluating different 

sources. Researchers tend to focus on 

their own experiments and exclude other 

researchers’ process of analyzing data to 

extract useful information.

 • Limited data—The number of experiments 

that use the same vehicle to collect data 

from a driving simulator and for on-road 

testing, for the same participants and on 

the same day, is limited.

Discussion

Following the presentation, the workshop 

participants discussed several topics, as follows:

 • Testing comprehension—This allows 

researchers to measure understanding 

of alternative TCD formats. For 

example, typically a sign would be 

considered acceptable if 75 percent 

of people understand it; however, it 

may be necessary to compare results 

from surveys and driving simulators to 

confirm that drivers can demonstrate 

their understanding of the sign through 

their behaviors.

 • Developing survey techniques—Results 

are often sensitive to the administration 

format and phrasing of the items. 

Different survey techniques, such as 

asking questions in different ways or 

including a scale, can help to factor out 

inconsistent responses. 

 • Ensuring validity—There are various 

views on whether a given technique 

will reveal actual behavior or desired 

responses. It may not be useful to 

analyze aggressive driving by using 

Figure 15. Research methods.

© Susan Chrysler, University of Iowa
27

surveys, because respondents may 

not give truthful responses or may 

not be aware of how to evaluate their 

driving rates. Driving in a driving 

simulator may also fail to evoke or 

capture aggressive driving because of 

its artificiality. Categorizing research 

methods based on validity can be 

complicated when the metrics are not 

clearly calibrated both to the subject 

as well as to the experimenter. 

 • Identifying the research phase—The 

method chosen may depend on what 

phase the research is in and the type of 

research question to be analyzed.

 • Introducing variability—It is useful to have 

variability in the sample both in terms of 

participants and sites to account for a 

range of likely behavior. 

Additional Resources

Chrysler made available a selection of 

additional resources to workshop participants. 

These resources are outlined as follows:

Papers

 • Chrysler, S. T., Fitzpatrick, K., Brewer, 

M. A., & Cynecki, M. (2011). Pedestrian 

and Bicyclist Traffic Control Device 

Evaluation Methods (Report No. FHWAHRT-11-035). Washington, DC: Federal 

Highway Administration.

 • Chrysler, S.T., & Nelson, A. (2011). Design 

and evaluation of signs and pavement 

markings using driving simulators. In 

Fisher, D. L., Rizzo, M., Caird, J. K., & Lee, J. 

D. (Eds). Handbook of Driving Simulation 

for Engineering, Medicine, and Psychology 

(ch.39). Boca Raton, FL: CRC Press.

Examples of Studies

 • Clark, K. L., Hummer, J. E., & Dutt, N. (1996). 

Field evaluation of fluorescent strong 

yellow-green pedestrian warning signs. 

Transportation Research Record, 1538(1), 

39–46. 

 • Gates, T. J., Carlson, P. J., & Hawkins, H. 

G. (2004). Field evaluations of warning 

and regulatory signs with enhanced 

conspicuity properties. Transportation 

Research Record, 1862(1), 64–76.

 • Funkhouser, D., Chrysler, S., Nelson, A., & 

Park, E. S. (2008). Traffic sign legibility for 

different sign background colors: Results 

of an open road study at freeway speeds. 

Proceedings of the Human Factors and 

Ergonomics Society Annual Meeting, 

52(23), 1855–1859. 

 • Golembiewski, G. A., Katz, B. J., Knoblauch, 

R. L., & Rousseau, G. K. (2006). Determining 

colors for traffic control devices at 

transponder-controlled tollbooth lanes 

with a sign simulator. Transportation 

Research Record, 1973(1), 48–54. 

 • Dutta, A., Fisher, D. L., & Noyce, D. A. (2004). 

Use of a driving simulator to evaluate and 

optimize factors affecting understandability 

of variable message signs. Transportation 

Research Part F: Traffic Psychology and 

Behaviour, 7(4), 209–227.

 • Shinar, D., Dewar, R. E., Summala, H., & 

Zakowska, L. (2003). Traffic sign symbol 

comprehension: A cross-cultural study. 

Ergonomics, 46(15), 1549–1565.

 • Dudek, C. L., Schrock, S. D., & Ullman, B. 

R. (2007). License plate and telephone 

numbers in changeable message sign 

amber alert messages. Transportation 

Research Record, 2012(1), 64–71. 
28

Introduction

Dr. John Lee told workshop participants that 

combining different data sources through 

a strategy to triangulate methods provides 

new approaches to addressing research 

needs. Lee suggested that mixing on-road 

data with data collected from simulators, 

or from drivers themselves, provides a 

comprehensive approach that can compensate 

for the deficiencies of a particular method. 

Simulators provide meaningful parameters 

to take into account when comparing onroad driving behavior with simulator data. 

Lee presented a proof of concept study to 

workshop participants, which used existing 

infrastructure and data collected via social 

media, to triangulate multiple sources of data to 

investigate driving behavior.

Triangulating Data Sources to Understand 

Driver–Vehicle Behavior

Rather than picking the best method in an 

absolute sense (i.e., the most valid or selecting 

the best one for a particular problem), multiple 

methods can be combined to triangulate a 

problem. Lee told workshop participants that 

the proposed approach was to collect data 

with one method and combine them with 

other datasets to provide new insights. 

Triangulation is a metaphor used in 

reference to position-fixing in navigation. 

First, a landmark is selected to establish a 

line of reference with which to compare the 

current position. When a second landmark 

is identified, a location is determined by 

using the point where both reference lines 

from the two landmarks intersect. In light 

of this metaphor, a driving simulator and 

on-road test can represent the landmarks 

to determine whether results converge. 

Ideally, a third line from a third landmark will 

also intersect at the same spot; however, in 

geographic applications in the real world, 

the third line typically does not meet the 

other two bearings because of errors, such 

as mapping a region instead of limiting it to 

where a possible position exists. 

Lee mentioned that one of the critical 

considerations when choosing landmarks 

in navigation is to select from orthogonal, 

highly separated choices, located at right 

angles from the current position. Lee stated 

that when translating this concept into the 

research context, it is important to select 

maximally different methods of collecting 

data to address the same question. The 

challenge that comes with triangulation is 

coordinating and synthesizing observations 

that come from using different methods. 

The challenges with triangulating data 

include logistical problems, sampling issues, 

philosophical aspects, costs, and modeling. 

During the presentation, these challenges 

were outlined as follows:

 • Logistical problems—These require that 

data structures be harmonized and that 

variables and units are measured the 

Triangulating Data Sources to Understand Driver–Vehicle Behavior

Dr. John Lee

Department of Industrial and Systems Engineering, University of Wisconsin–Madison
29

same way and have consistent definitions. 

The goal is to make sure researchers use 

the same equations to calculate summary 

measures and, at the very least, use 

consistent units. 

 • Sampling issues—A second concern 

is to acquire a representative sample 

of participants who volunteer for the 

experiment and allow videotaping and 

other intrusions on their daily behavior. It 

is important to recognize potential biases 

in studies that are based on volunteers 

or self-selection. It is also necessary to 

acquire a representative sample of the 

driving context. 

 • Philosophical aspects—Conflicts regarding 

the “best” data or method can be addressed 

by recognizing the complementarity of 

methods, not by ordering them in terms of 

being less or more valid, but by recognizing 

there is no gold standard. All methods 

give an insight into different elements 

considered in the research.

 • Costs—This can be expensive unless the 

research approach is inventive. One way 

to be inventive and enhance the value of 

the study is by making use of free or lowcost data by using existing sources. More 

expensive approaches include collecting 

data in driving simulators and replicating 

the study by using on-road tests. 

 • Modeling—The last challenge is to 

create a model to clarify, combine, and 

accumulate findings. 

Research Approaches

Lee showed workshop participants three 

different approaches towards triangulating 

data. These included linking driving simulators 

to on-road behavior; determining ways to go 

beyond using existing infrastructure, such as 

loop detectors; and using drivers themselves 

as sensors of activity on the road. 

Approach 1: 

Linking Simulators to On-Road Behavior

The first project that Lee presented was 

a large-scale public–private collaboration, 

involving the National Advanced Driving 

Simulator (NADS), Iowa State University, 

Montana State University, and SAIC, which 

received funding from the EAR Program. 

The purpose of this research project was to 

replicate actual road segments in different 

driving simulators. One of the benefits of 

this approach was to improve the usefulness 

of simulator data for understanding driver 

behavior. Different road segments that all 

road users find particularly challenging, such 

as roundabouts and gateways, were sampled 

and recreated in a simulated environment. 

Figure 16 shows the replication in simulation 

and its match to what is seen in the real world.

Researchers analyzed how simulated driving 

data corresponded to data collected on the 

road. The driving speed for each state in 

the different simulators, in terms of mean 

and standard deviation, was compared with 

data from driving through roundabouts 

collected from the real world. Ideally, the 

mean for speed in simulations will follow 

the mean for real-world data. Lee noted 

that this is the same for standard deviation 

in simulated scenarios, with data from actual 

driving in the roundabout. Comparisons across 

different simulators demonstrated good 

Figure 16. Comparison of real and simulated scenarios.
30

correspondence across the different roadway 

situations. One interesting outcome was that 

motion and visual complexity in a simulated 

environment had little effect on driving 

behavior, and similar results were obtained 

when motion was enabled or disabled.

Models for Transformation and Interpretation 

of Simulator Data

Lee examined the data by using two types of 

models, a linear regression and a generative 

process model. This was to understand how 

drivers actually negotiate a curve using a 

closed-loop model with perceptual cues, 

desired speed, and adjustments to speed. 

Regression Model

Lee compared speed values across different 

roundabouts in simulators and in the real 

world. He noted that, ideally, simulators 

will have all data points lined up on the 

standardized diagonal, representing realworld conditions. Overall, results show a good 

correspondence between simulated and realworld data, indicating an absolute validity for 

the patterns observed.

Generative Model

The generative model uses data from a 

simulator, which provides for a deeper 

understanding of driving behavior in 

negotiating the curve in the roundabout. 

Lee noted that this model is a useful way 

to accumulate knowledge on driving 

performance based on a parameterized 

estimation of driving behavior. These 

simulator data were used to develop the 

driver model. 

Lee informed workshop participants that 

harmonizing data resolution was one of the 

challenges of developing and validating this 

driver model. In this case, the data specifying 

the roadway were relatively coarse compared 

to the fine-grained resolution of the 

response process of the driver model. The 

data specifying the road curvature in the 

simulator were different than the information 

about roadway curvature portrayed on 

the simulator screen, because the textures 

used to create the visual scene tend to 

mask discontinuities in the underlying road 

database. This created a database resolution 

that is not compatible for the driver model. 

The discontinuity on road segments, as 

in the case of roundabouts where straight 

segments are followed by curves, introduced 

errors in the driver model that had to be 

addressed by smoothing the segments that 

comprised the roadway database. 

Lessons Learned: Triangulating Data in 

Roadway Design

During the presentation, Lee proposed the 

following recommendations to improve 

roadway scenarios replicated in driving 

simulation:

 • Naturalistic data—This is a useful source 

to identify critical design issues to 

be replicated in the driving simulator. 

Scenarios to include are road geometry, 

traffic-control device placement, and 

traffic situations.

 • Analog approach—Alternate between 

low-fidelity to high-fidelity simulations 

as an analog phase approach for human–

computer interaction.

 • Integrate data—To accomplish a 

comprehensive approach, it is necessary 

to integrate a driver model with 

simulator evaluation and naturalistic 

data (i.e., surveillance data). 

Approach 2: 

Determining Ways to Go Beyond Loop 

Detectors for On-Road Data

Researchers for the second research 

project underway at the University of 
31

Wisconsin–Madison are collecting data 

beyond spot speed from loop detectors. 

They are analyzing intersection approaches 

by using on-road experiments and existing 

infrastructure to collect data at a relatively 

low cost. The researchers use existing 

infrastructure and radar sensors on traffic 

signals to capture trajectory data. Their 

objective is to identify vehicle trajectory 

(i.e., position and speed across time) and to 

compute the real-time safety performance 

measurements for vehicle movements. 

One of the applications of this research 

project could be the validation of data 

from simulators at very low cost by using 

existing and available data.

Approach 3: Using Drivers as Sensors

Drivers can be used as sensors in different 

ways to identify problems. By taking 

advantage of technology, drivers’ surveillance 

may help researchers to identify problems 

and solutions. Two examples of how to do 

this include the safety complaints recorded 

in the National Highway Traffic Safety 

Administration (NHTSA) Vehicle Operator 

Questionnaire (VOQ) database and Twitter.

Safety Complaints

NHTSA compiles complaint data in an NHTSA 

database that contains drivers’ entries on the 

NHTSA VOQs. These data can be downloaded 

at no cost and offer a low-cost tool to analyze 

problems with vehicles.10

Data can be analyzed by using a languagesemantic-analysis, and the content can be 

analyzed by using cluster analysis. The results 

can be analyzed across time and offer a way to 

conduct surveillance by using drivers as sensors. 

Twitter Data

Lee informed workshop participants that 

another way to use drivers as sensors to 

collect data is by looking at Twitter text to 

identify hot trends and compare Twitter 

data across place and time. A University 

of Wisconsin–Madison computer science 

research team used its socio-scope approach 

to extract the spatio-temporal signal from 

Twitter and create spatial-temporal maps 

to target pre-defined target phenomenon. 

Some issues to be aware of when using this 

type of data are population bias, imprecise 

location of phenomena, and low counts of 

events. The research team demonstrated 

this surveillance method through Twitter 

to evaluate the intensity of road-kill events 

across the continental United States.11 

The proof of concept can be extended into 

different areas by using proper filters and text 

classifiers. This type of data can be obtained 

at a low cost. Workshop participants noted 

several interesting possibilities for Twitter data 

for future consideration. These could target the 

following areas:

 • Patterns of distracted driving.

 • Road infrastructure problems.

 • Early warning of vehicle automation issues.

Conclusion

In conclusion, Lee informed workshop 

participants that triangulation of data sources 

provides several research opportunities; 

however, the following challenges still remain:

 • Harmonizing data structures, data 

resolution, and variable definitions.

 • Harmonizing representative sampling and 

synthesis of drivers and contexts.

10. Ghazizadeh, M., & Lee, J. D. (2012). Consumer 

complaints and traffic fatalities: Insights from 

the NHTSA vehicle owner’s complaint database. 

Proceedings of the Human Factors and Ergonomics 

Society 56th Annual Meeting (pp. 2256–2260). 

Boston, MA.

11. Xu, J-M., et al. (2012). Socioscope: Spatio-temporal 

signal recovery from social media. Proceedings of 

the European Conference on Machine Learning and 

Principles and Practice of Knowledge Discovery in 

Databases (pp. 644–659). Bristol, UK.
32

 • Recognizing complementary methods—

there is no gold standard for measuring 

driver behavior.

 • Reducing the expense of triangulation.

 • Performing driver modeling to clarify, 

combine, and accumulate findings.

Discussion

During the discussion following the 

presentation, Lee informed participants 

that the project results will be published 

soon. The discussion also raised several 

points relating to using Twitter data as a 

new surveillance method for driver–vehicle 

research. When asked about the general 

cost of a Twitter data-collection effort, such 

as the effort to look at road kill across the 

United States, Lee noted that the cost is in 

the range of tens of thousands of dollars. 

Lee also told participants that computer 

technology is increasingly more powerful and 

less expensive, and therefore practical for 

research. Lee explained that the example that 

used Twitter data regarding species, time, and 

place of road kill was intended to be a proof 

of concept to illustrate the technique because 

the ground truth is known. In contrast, less is 

known about the characteristics of teenagers 

who tweet, and Twitter analyses may have 

potential to illuminate this issue. In response 

to a question regarding whether a generative 

model of curve negotiation is an example of 

modeling helping with triangulation, Lee noted 

that it explains why people behave differently 

with different simulators. In addition, the 

parameters that are estimated from the model 

can help researchers understand how to take 

into account on-road data. 
33

Introduction

During this presentation, Dr. Linda Ng Boyle 

recommended integrating methodologies 

rather than choosing a single methodology for 

human factors research on driver and vehicle 

data. Boyle informed workshop participants 

that there is immense value in considering 

different research and analytical tools. 

Multiple data sources can offer different but 

complimentary perspectives and can provide 

greater insights for a common research 

topic. Boyle showed workshop participants 

examples of how to complement data 

sources, looking specifically at in-vehicle 

driver support systems. The proposed 

framework integrates different data sources 

to understand driving behavior and safety 

outcomes, using the adaptive cruise control 

(ACC) in-vehicle driver support system as 

an example. 

Adaptive Cruise Control

In-vehicle driver support systems provide 

traffic and other information to users with 

the purpose of improving traffic flow and 

enhancing driver comfort and safety. These 

systems are integrated into vehicles today 

and also exist as cloud-based systems that 

can be used with smart mobile devices.

ACC is an example of an embedded invehicle driver support system, where a driver 

can set the desired speed and distance from 

a lead vehicle. 

Boyle noted, although ACC has been 

available in the United States since 2001, its 

safety benefits have not been fully assessed. 

Depending on the vehicle make and model, 

ACC may not necessarily work in stop-andgo traffic, and has limited ability to recognize 

a vehicle that has stopped directly in front or 

on a sharp, curvy road.

User Survey Data

To understand consumer’s perceptions and 

actual use of ACC, Boyle's research team 

conducted a survey in Washington state 

between 2010 and 2011. In this study, ACC 

owners were asked about ACC's functional 

limitations. Of the surveys sent, 584 surveys 

were returned, and 118 were from actual 

ACC users. Many ACC users reported that 

the ACC system was helpful in stop-and-go 

traffic, on curved roads, and for recognizing 

stopped vehicles. Some users reported that 

they did not know whether or not ACC was 

successful in assisting them under these 

conditions. A similar survey was conducted 

in Iowa between 2008 and 2009, with 514 

surveys returned and 132 of the surveys 

from actual ACC owners. Across both 

surveys, over 50 percent of drivers were 

not aware of the limitations of ACC, as 

indicated by the percent responding “Yes” 

or “Don’t Know.” 

During the presentation, Boyle highlighted 

that survey data can provide some insights 

Driver-Vehicle Data for Human Factors Research

Dr. Linda Ng Boyle

University of Washington
34

on drivers’ motivation, attitudes, and previous 

experiences; however, there are many other 

factors that can impact the overall safety of the 

driver, which need to be gathered from other 

data sources. Mediating factors are based on 

more subjective traits but may actually have 

a greater influence on the driver’s behavior. 

These factors emerge from long-term 

exposure to a system, in conjunction with 

their other driving experiences, motivational 

factors, and driver limitations.

Initiating factors come from the direct 

interaction with the system, and the feedback 

that the driver receives at the moment the 

system is in use for any given road, traffic, and 

environmental situation. Both the mediating 

and initiating factors need to be considered 

to understand the safety implications of 

ACC and the impact this system has on the 

driver. Boyle noted that driver response can 

be observed in a naturalistic environment and 

tested in various conditions in a simulated 

environment. The system is a closed loop, 

as illustrated in figure 17, and changes in the 

initiating and mediating factors will impact 

the drivers’ response for the next response or 

action that is taken by the driver. 

Complementing Survey Data with Field Data

The data from the closed-loop system 

in figure 17 comes from a myriad of data 

sources to assess the overall safety impacts 

of the driver–ACC system interactions. Boyle 

reminded workshop participants that many 

mediating factors can be obtained through 

surveys; however, on-road or field data is 

better for capturing initiating factors. 

Field Data

Boyle told workshop participants that 

the University of Michigan Transportation 

Research Institute (UMTRI) conducted a 

field operational test for novice ACC system © 2012 IEEE. Reprinted, with permission, from “Drivers’ Adaptation to Adaptive Cruise Control: Examination of Automatic and Manual Braking” by Huimin Xiong and Linda Ng Boyle, in IEEE Transactions on Intelligent Transporation Systems, September 2012.

Figure 17. Impacts on driver response when using adaptive cruise control.
35

users. The purpose of this study was to 

observe drivers using ACC in the real world. 

In accordance, investigators looked at three 

closing events with lead vehicle braking. A 

closing event lasts from the moment that 

the ACC’s automatic braking control is 

activated until any braking (or deceleration) 

stops. The three categories included low 

risk, conflict, and near crash. The likelihood 

that a driver will intervene when ACC is on 

differed by age. More specifically, middleaged drivers were more likely to intervene 

when compared with older drivers. Boyle 

noted that user settings were also related 

to the likelihood of intervention. Users 

that preferred long gap settings were less 

likely to intervene compared with those 

drivers that preferred short settings. The 

likelihood of driver interventions was also 

related to the roadway environment—

drivers were less likely to intervene during 

highway driving. 

Complementing Field Data with Simulation 

Data

Although the associations between driving 

behavior and ACC can be observed by using 

field data, Boyle suggested simulation data 

should also be considered to gain insights 

on situations not observed in the real world. 

Simulators can be used to focus on the 

relationships identified from the field data 

that may have the greatest safety impact. 

They make it possible to examine “what if” 

scenarios, as well as to more closely examine 

various driver characteristics in a variety of 

scenarios that may not be encountered by all 

ACC users. Boyle told workshop participants 

that the benefit of establishing a relationship 

between the outcomes from field data and 

simulation lies in the ability to identify factors 

that impact safety while using ACC.

Simulation Data

Driving simulators allow one to examine 

the use of ACC in controlled settings and 

for various driving scenarios that could be 

of safety concern to drivers. It can also be 

used to examine differences in novice and 

experienced users. In a simulation study 

conducted at NADS using cluster analysis, 

researchers tested measurements previously 

identified in the field data in a motion-based 

simulator. The research questions included: 

 • How often do drivers disengage ACC? 

 • How many warnings did drivers get 

from ACC? 

 • What are drivers' ACC gap settings?

 • How fast did the drivers drive?

Figure 18 shows the outcomes of the cluster 

analysis, which grouped drivers into three 

categories: conservative, moderately risky, 

and risky drivers. The simulator data can make 

it possible to identify causal relationships 

among ACC owners and to help identify 

the different types of risk-seeking behavior 

by using the same driving performance 

measures in the same drive scenarios. On one 

hand, those who drove conservatively, for 

example, disengaged the ACC system quite 

often and drove below the speed limit. On 

the other hand, risky drivers received more 

warnings, disengaged the ACC system less 

often, and drove above the speed limit.

Figure 18. Cluster analysis using simulation data. 

Note: ACC = adaptive cruise control.
36

Complementing Simulation Data with 

Survey Data

Boyle told workshop participants that 

simulation data may not be sufficient to 

explain all of the drivers' behavior. Survey 

methods can be used in conjunction with 

the simulator data to understand why some 

drivers had more risk propensity. Surveys can 

capture information on drivers’ perception 

of ACC, willingness to use ACC, and drivers' 

understanding of ACC. Perceived risks, 

motivation, attitude or biases, experience, 

and limitations are all mediating factors that 

can be obtained from survey data. 

Through the analysis of survey data, driving 

behavior can be classified according to risk 

propensity. For example, survey results 

showed that risky drivers tended to feel too 

comfortable trusting ACC and were easily 

distracted. Moderately risky drivers have the 

lowest level of trust in ACC and are confident 

with their driving skills. Finally, conservative 

drivers demonstrated the highest level of 

overall trust in the system and resembled 

cautious driving styles. Boyle noted that 

these findings demonstrate the value of using 

survey data to complement simulation 

data. Although simulation can be used 

to quantify the objective performance 

associated with risky behavior, surveys 

can extend the analysis by categorizing 

driving behavior according to the driver’s 

confidence and trust in the ACC system. 

Integration of Data Sources

During the presentation, Boyle showed 

that driving behavior can be measured by 

using field, simulation, and survey data. 

Field data measures actual behavior on the 

road. Behaviors identified in the field can then 

be manipulated in a simulated environment to 

observe and identify the factors that would 

influence a response. Survey data was used to 

understand drivers’ motivations, perceptions, 

and preferences for ACC use. When the 

different perspectives are considered, the 

complete picture of safety outcomes is 

obtained; however, to understand how this 

information comes together, it is important to 

recognize that exposure to this technology will 

change driving behavior. An FOT methodology 

can provide insights on initial exposure, but 

drivers will behave differently after becoming 

accustomed to an ACC system. Similarly, Boyle 

noted that if an ACC system is embedded in 

a driver’s personal vehicle for several years, a 

different driving behavior profile will result 

because of adaptation. 

Adaptation and Road Safety

Boyle told workshop participants that a 

driver’s behavior may differ given the length 

of time the driver is exposed to an ACC 

system. Using an ACC system for the first 

time is accompanied by a novelty effect 

that results in high performance. Over 

time, drivers’ attitudes, expectations, and 

perceptions of the ACC system may change, 

which can impact the drivers’ longer term use 

of ACC systems. Drivers can also experience 

positive and negative transitions when they 

switch to vehicles without ACC capabilities. 

Boyle suggested that behavioral adaptation 

may explain some of the variation among 

users and the differences in driving behavior 

among conservative and risky drivers. 

Conclusion

Boyle discussed the importance of 

considering multiple sources to conduct 

human factors research in this presentation. 

As technology evolves, different systems are 

studied, and the need to find associations 

and causality persists. Different research 

methods can complement each other 

to understand outcomes. A research 

framework with a driver model delegates 

different information to different data 

sources. This type of framework enhances 

the value of each dataset, without ignoring 

the limitations of each method. When 
37

identifying differences, not only between 

tools but also within methods, it is possible to 

understand complementary aspects of data 

sources. For instance, there may be different 

results using the same tool in different labs. 

Different simulators produced different 

outcomes because of geographic variations; 

however, it is necessary to compare these 

outcome variations to validate findings and 

to identify individual differences. Finally, 

Boyle noted that triangulating data is crucial 

to obtain the complete picture of safety 

outcomes when using ACC systems or other 

in-vehicle systems.

Discussion 

The workshop participants discussed several 

topics following the presentation. These are 

summarized below:

 • Each data source provides causality and 

association of driving behavior according 

to its capabilities and therefore can be used 

collectively. Causality identifies factors 

present in a particular incident under a 

scenario that results in certain behaviors. 

Simulation can repeat and reproduce 

these conditions, however, associations 

in terms of risk can be established when 

using naturalistic data. 

 • Driver models provide an expected 

trajectory when conducting an 

experiment. They embody the causal 

factors and give the distribution of 

parameters in experimental design. 

The formulation of hypotheses can 

help select which method to use. 

The challenge is to fit driver models 

together and to build a comprehensive 

research framework. 

 • The main purpose in performing human 

factors research is crash prevention. There 

are several viewpoints that underpin 

this endeavor. One method analyzes 

the distribution of parameters in terms 

of descriptive statistics (e.g., mean 

and standard deviation) and the other 

considers crashes as rare events. There 

is uncertainty as to how to incorporate 

relationships between safety and crash 

outcomes while putting together the 

basis to compare multiple data sources as 

an approach to identify their strengths. 

 • The characteristics of fatalities in crashes 

are not representative of the rest of the 

driver population, and using crash data 

alone may constrain research.

 • Investigating the implications of in-vehicle 

systems with respect to actual use may 

not provide the intended outcomes. 

For example, in one system examined, 

time to collision did not appear to be 

incorporated in the ACC algorithm. There 

is a benefit when researchers work with 

the technology after it is deployed into the 

market and may have to work backward 

(reverse engineer) to determine how the 

system engages given the preferred gap 

and speed settings. 

 • Researchers find it very beneficial to share 

data, and they can use data repositories 

to identify issues.

 • Surveys conducted at different locations 

may give similar and different results 

because of duration of exposure to the new 

elements. There are a surprising number of 

ACC system users who are not familiar with 

its limitations, although the feature has 

been on the market for a decade.
38

Acknowledgments

The work described in this presentation 

summary was supported by funding from the 

National Science Foundation (NSF, IIS Grant 

No. 1027609), National Highway Traffic Safety 

Administration (Contract No. DTNH 22-06-D00043, Task Order 0005), with data collected 

at UMTRI for an FOT project on intelligent 

cruise control, and the University of Iowa 

National Advanced Driving Simulator (NADS). 

Graduate students David Dickie, Huimin Xiong, 

Jarrett Bato, and Yuqing Wu assisted with data 

collection and analysis. Any opinions, findings, 

and conclusions or recommendations expressed 

in this report are those of the presenter (L. Boyle) 

and do not necessarily reflect the views of NSF, 

USDOT, UMTRI, or the University of Iowa.

Additional Resources

Papers

 • Manser, M., Creaser, J., & Boyle, L.N. (2013). 

Behavioral adaptation issues. In RudinBrown, M., & Jamson, S. (Eds.), Behavioural 

Adaptation and Road Safety: Theory, 

Evidence and Action (pp. 389–358). Boca 

Raton, FL: CRC Press.

 • Xiong, H., & Boyle, L.N. (2012). Driver's 

adaptation to adaptive cruise control: 

Examination of automatic and manual 

breaking. IEEE Transactions on Intelligent 

Transportation Systems, 13(3), 1468–1473.

 • Xiong, H., Boyle, L.N., Moeckli, J., Dow, B., 

& Brown, T. (2012). Use patterns among 

early adopters of adaptive cruise control. 

Human Factors, 54(5), 722–733.
39
40

Day Two:

Discussion and Summary

Following the presentations, the second 

day of the workshop used expert panel and 

small group discussion to identify research 

gaps and recommendations. 
41

Day two began with an expert panel 

discussion session, in which the seven 

presenters served as panelists. The 

discussion focused on three examples of 

where researchers need to take advantage 

of the information provided by multiple 

datasets. The panel moderator stated that 

the goal is to link risk, prevalence, and 

ecology of behaviors to crashes, an effort 

that may require resolving contradictions 

(contradictory datasets), creating linkages 

among the datasets (complementary 

datasets), and generating entirely new 

datasets (comprehensive datasets). These 

datasets are outlined below.

Contradictory Datasets

In some cases, different datasets lead to 

different conclusions, for example, the 

information available on the increase in 

crash risks caused by cell phones. Simulator 

studies of cell phone records lead to one 

conclusion, naturalistic studies to another 

conclusion, and retrospective studies to 

contradictory conclusions. The moderator 

asked the panelists to provide examples of 

contradictory datasets and methods one 

might undertake to resolve the controversies.

Complementary Datasets

In some cases, information is available 

on driver and other road-user behaviors 

in different datasets that appears to be 

complementary but that is not formally 

linked. These complementary datasets can 

radically expand the ability to understand 

increases in risk tied to particular behaviors 

in a given scenario, the likelihood of those 

behaviors in the selected scenarios, and the 

prevalence of the scenarios. For example, 

information can be gathered on advanced 

yield markings at marked midblock 

crosswalks from the glance and yielding 

behaviors of drivers on the simulator. In 

addition, information can be gathered in 

the field using semi-controlled studies and 

naturalistic studies, and also from field 

observational studies. 

Panelists noted simulator studies are wellsuited for providing information on the 

increase in risky behaviors in particular 

scenarios but not the likelihood of such 

behaviors in these scenarios, or the 

prevalence of the scenarios. Naturalistic 

studies can also provide information on 

the prevalence of particular scenarios but 

cannot be so easily used to identify the 

increase in risk in the scenarios that can 

be attributed to particular behaviors. With 

the rapid increase over the last decade 

in multiple complementary datasets, it is 

now possible to provide information on the 

increase in risk that a particular behavior 

creates in a given scenario, the likelihood 

that the driver engages in the behavior, and 

the prevalence of the scenario. 

The moderator asked panelists to identify 

which datasets are best suited to providing 

Expert Panel Discussion

Moderated by Dr. Donald Fisher

University of Massachusetts Amherst
42

information about the risk of particular 

behaviors, the likelihood that drivers engage 

in those behaviors in particular scenarios, 

and the prevalence of the scenarios.

Comprehensive Datasets

Panelists noted an ever-increasing ability to 

predict the incidence of crashes and nearcrashes from knowledge gained about the 

risk and prevalence of driver behaviors. A 

comprehensive dataset contains information 

on behaviors and crashes at a particular 

location; however, although such datasets 

do not yet exist, they could in the near 

future. This could include a dataset at a busy 

intersection that could provide information 

on a range of risky driver behaviors, the 

prevalence of those behaviors, and the 

frequency of crashes. An overarching 

model of driver behavior is required that 

is sensitive to factors, such as driver state 

and the roadway environment. This model 

would not only predict when drivers engage 

in risky behaviors but also predict the 

likelihood of a crash or near-crash when the 

driver is engaging in a particular behavior. 

The datasets will need to include where 

actual crashes are recorded, and panelists 

were asked how they might go about 

creating such datasets. 

In summary, the moderator asked panelists to 

discuss what sorts of issues they were studying 

that required the use of contradictory, 

complementary, or comprehensive datasets. 

Panelists gave several examples of these 

issues encountered in their research, as 

outlined below.

Contradictory Datasets—Examples from 

Panelists

The moderator highlighted the following 

examples of contradictory datasets:

 • Effects of driver interaction with warning 

signs in cooperative intersections in the 

field often contradict data from simulated 

experiments.

 • Roadway departures at night with 

one person in the car are practically 

impossible to replicate in the simulator. 

 • Research that studies driver behavior with 

pavement markings and delineator posts in 

roadway curves can lead to contradictory 

datasets. In particular, brighter road 

markers lead to over compensation in one 

setting but not the other. 

 • Two major NHTSA crash databases, 

the National Automotive Sampling 

System General Estimates System and 

the Fatality Analysis Reporting System, 

often yield apparently contradictory 

data and different results depending on 

the context of the crash. 

The panellists noted several reasons for 

these contradictions:

 • Exposure—Relatively frequent in the 

simulator and relatively infrequent on 

the open road. 

 • Feedback loops—Drivers change their 

behavior based on feedback from the 

environment. 

 • Abstract reality—Scenarios are abstractions 

of reality and do not allow for real-world 

interactions. 

 • Sampling bias—Bias in the sampling of 

laboratory studies (often college-age 

students) can produce different effects 

depending on the driving population.

Complementary Datasets—Examples from 

Panelists

The moderator highlighted the following 

examples of complementary datasets:

 • Different datasets can sometimes be 

combined to produce better estimates 
43

of high-risk periods. For example, the 

number of bicycle crashes reported is 

highest for noon and midnight; however, 

bringing crash reports and exposure 

data together shows that the real risk of 

bicycle crashes is highest very early in 

the morning. 

 • Different datasets may need to be 

kept separate to understand different 

aspects of a problem. For example, 

understanding the cause of a crash can 

frequently come from naturalistic data, 

but finding the pattern of where crashes 

occur may come from crash data. 

 • Contradictory datasets can become 

complementary if the methods used to 

collect the data are changed in ways that 

potentially account for the contradiction. 

For example, roadway departure studies 

conducted in the field complement 

simulator studies in that drivers will 

depart from the simulated roadway if the 

drive is long enough.

 • When the issues are multifaceted, the 

best combination of complementary 

methods and datasets for any given set 

of issues can be identified by using the 

table of methods and issues shown in 

figure 15. 

Comprehensive Datasets—Examples from 

Panelists

The moderator highlighted the following 

examples of comprehensive datasets:

 • Single bicycle crashes—Datasets do not 

capture many of the potential causes 

of theses crashes; thus, comprehensive 

datasets are still a long way off.

 • Pedestrian and vehicle crashes—

Police crash databases in Japan are 

not useful for understanding causation 

but can still be used to generate 

possible countermeasures; however, 

comprehensive datasets are not a 

near-term possibility. 

The panelists noted that researchers need 

to understand when data are corrupted, for 

example, a top-down approach can help 

when the bottom-up approach is in error. 

General Discussion

Following the panelist session, the group 

discussed the following topics:

Current Datasets

 • Need for theory: Researchers require a 

theory of behavior to inform data mining, 

serve as a framework, and make different 

types of data coherent. 

 • Need for careful problem identification: 

Problem identification, by using data to 

identify that there is a problem, can lead 

to the wrong conclusion if the research 

community is not careful in choosing the 

database or methodology used.

 • Need for understanding limitations: 

Data can be easily misinterpreted, not 

only in analysis but also in collection. For 

example, SHRP 2 data were collected for 

a specific purpose, and researchers need 

to recognize that there are limitations to 

these data.

Future Datasets

 • Standardization: There is a strong need 

for standardization. Human factor 

researchers and traffic engineers use 

terminology differently: The former 

define headway as front bumper to front 

bumper, but the latter define headway 

as rear bumper to rear bumper. Another 
44

example is crash reports, which differ 

across different municipalities. Missing 

data can create a gray area for researchers. 

 • Broad goals: To take full advantage of 

the data that are collected in the future, 

researchers should remember that their 

research questions are not the only 

questions that need to be answered in 

regard to the data collected.

In summary, the panelists noted that there 

are two different ways of viewing how best 

to deal with multiple, contradictory datasets 

as follows:

 • Bottom up—It is possible to take various 

known instances in which there are 

contradictions across datasets and 

identify why these inconsistencies arise 

and what can be done to avoid them in 

the future.

 • Top down—A study across multiple sites 

would allow for the collection of various 

different types of data. It would then be 

possible to look for inconsistencies across 

sites in the same dataset and inconsistencies 

within sites across datasets.

Recommendations

In conclusion, the panel members considered 

what type of study would be needed to (1) 

understand how to resolve long-standing 

contradictions among different datasets; 

(2) allow for the use of complementary 

datasets to generate information on the 

risk of different behaviors, their likelihood, 

and the prevalence of the scenarios in 

which they occur; and (3) generate a 

comprehensive dataset that links behaviors 

and crashes. Panelists were unanimous 

in recommending that there should be 

an attempt to understand how to use the 

different types of data in a study, which 

includes the following components: 

 • Multiple sites (e.g., locations, geometries, 

traffic density, and environment).

 • Multiple types of data gathered at each 

site (e.g., survey, simulator, and field).

 • Multiple users (e.g., bicyclists, 

pedestrians, motorists, and drivers).

 • Multiple methods of analysis (e.g., 

descriptive and inferential statistics, and 

quantitative behavioral models).

The panelists suggested intersections, 

road departures, and connected vehicles 

as possible areas of focus for the study. 

They noted that intersections have many 

characteristics suitable to the study, 

are one of the best places to study V2V 

communications, and perhaps are one 

of the only places to study vehicle-topedestrian communications. Moreover, the 

three major issues surrounding multiple 

datasets can be studied at intersections. 

In particular, the data from naturalistic 

and simulator studies, which often lead to 

different estimates of risk, can easily be 

compared at intersections to determine 

why the various contradictions among 

datasets exist. Information on both the 

risk of a particular behavior (e.g., the risk 

that failing to take a secondary glance 

has on crashing, given that a vehicle 

materializes when the driver fails to take 

the glance), the likelihood of a particular 

behavior in a given scenario (e.g., the 

likelihood of taking a secondary glance), 

and the prevalence of the scenario (e.g., 

the prevalence of situations in which the 

driver fails to take a secondary look and a 

vehicle materializes) can be used to create 

complementary datasets. Finally, given 

the high incidence of crashes at selected 

intersections, the behavioral data can be 

combined with the crash and near-crash 

data to generate comprehensive datasets.
45

Overview

For the final session of the workshop, 

participants gathered into groups to discuss 

three key topics relating to data needs for 

human factors research: (1) driver–driver and 

other road user data, (2) driver–vehicle data, 

and (3) driver–infrastructure and roadway 

data. Following extensive group discussion, 

involving multidisciplinary experts from 

government, academia, and industry, the 

participants reconvened, summarized their 

findings, and made recommendations. 

This section presents the overall findings 

of the breakout groups. 

Driver–Driver and Other Road Users’ Data 

for Human Factors Research

Need for Pedestrian and Bicyclist Exposure 

Data

 • Researchers do not have ways to 

measure the exposure of pedestrians 

and bicyclists. For example, although 

there are automatic counting systems 

for vehicles at intersections, there is 

no analogous system to record the 

numbers of cyclists and pedestrians 

passing through. 

 • The lack of exposure data makes it 

impossible to calculate a risk ratio for 

pedestrians or bicyclists, although 

the numbers of fatalities are known. 

For this reason, it is impossible 

to compare the United States’ 

pedestrian and bicyclist fatalities 

with the risks to pedestrians and 

bicyclists in other countries. 

 • Technology is becoming more available 

to gather pedestrian and bicyclist data. 

For example, in Gothenburg, Sweden, 

researchers installed several boxes 

with sensor equipment that can detect 

and record the number of bicyclists 

that pass each of these boxes. This 

current technology is rudimentary, for 

example, it is not able to distinguish 

between a bicycle or a stroller and is 

subject to erroneous input because of 

people intruding in bike lanes where 

they should not be. 

Normative Variations in Pedestrian 

Behavior 

 • Participants noted that pedestrian 

behavior varies depending on culture 

and education. For example, in Australia, 

pedestrians do not have the right of way, 

resulting in more cautious pedestrian 

behavior. In Sweden, pedestrians do not 

jay walk, unlike places such as Boston 

or Washington, DC, where jaywalking 

is a huge problem resulting in many 

pedestrian–vehicle strikes.

 • Participants also discussed who is at 

fault for incidents in which pedestrians 

are struck, injured, or killed and referred 

to the largest study of fault attribution 

conducted to date in the United States. 

This showed that blame fell equally 

on pedestrians and motorists. The 

assignment of fault is different in Japan 

where drivers are found to be at fault 90 

percent of the time, and 10 percent of 

the fault falls to the pedestrians. 

 • Pedestrians in the vision-impaired 

community had specific concerns with 

the practice of permitting right turns 

on red. Although FHWA has looked into 

Group Session Summary
46

this issue, the subset of data is small, and 

police reports detailing these incidents 

lack sufficient detail and consistency 

to permit a fuller understanding of the 

risks of this type of vehicle behavior. 

Other issues affecting pedestrian and 

bicyclist safety include visibility and 

distraction at intersections. 

Hazards and Solutions in Roadway Design 

and Setup

 • Participants noted that many 

intersections lack sufficient visibility 

for pedestrians and bicyclists because 

intersections were designed to give 

vehicles a good line of sight, rather 

than to give all road users that good 

line of sight. For example, the Highintensity Activated crossWalK (HAWK) 

signal provides a protected pedestrian 

crossing as a way to increase safety. It is 

used only for pedestrian crossings and 

does not control traffic on side streets. 

The HAWK signal for pedestrians 

at larger crossings on a multilanedivided highway may pose risks to 

pedestrians—drivers, once they come 

to a stop, may be cleared to go but 

cannot see the pedestrian. 

 • Improved reflectivity for pedestrians 

and bicyclists was another suggestion 

made to improve roadway safety for 

pedestrians and bicyclists. Pedestrians 

and bicyclists need to be more visible 

and could employ ways to emit more 

reflectivity. At present, bicycles must 

have one reflector but pedestrians 

have no obligation to emit reflectivity. 

It was recommended that pedestrians 

be encouraged to wear clothing with 

at least one reflective element to 

increase their visibility for drivers. 

 • Infrastructure could be changed to 

separate cars and bicycles so that cars 

and bicycles do not cross paths. 

 • Participants agreed that if the roadway was 

set up so that speed was controlled, and 

if that speed was slower at intersections, 

then this would reduce many incidents. 

The group showed participants an 

example of a traffic-calming roadway 

setup in Japan, where intersections are 

compact so a driver cannot increase his 

or her speed. The participants suggested 

that future studies investigate ones that 

make vehicles or bicycles speed up or 

slow down, especially in roundabouts, 

which are not considered safe for 

pedestrians or bicyclists. 

Technology for Pedestrian and Bicyclist Safety

There are potential ITS technologies 

that can improve pedestrian and bicycle 

safety. DSRC devices that transmit 

information through mobile devices may 

be an option in the future to improve 

pedestrian and bicyclist safety. At 

present, the ITS Joint Program Office’s 

Connected Vehicle program is exploring 

the feasibility of introducing pedestrian 

DSRC by using a smartphone. In 

addition, Volvo has recently introduced a 

pedestrian detection system, and bicycle 

manufacturers are starting to introduce 

auto brakes for bicycles, in addition to 

external airbags. 

Driver–Vehicle Data for Human Factors 

Research 

Participants in this group illustrated how 

to sequence methodologies to conduct 

research, by using multiple data sources. The 

process to address driver–vehicle research 

is detailed as follows:
47

Naturalistic Scenario Sampling and Problem 

Definition

There are many methods that can be 

used to monitor interactions among road 

users, such as TMC, traffic cameras, and 

social media. Traffic cameras can be used 

because of their availability; they can 

reveal possible navigation issues, traffic 

movements and conflicts; and they can 

recognize trends in traffic. Participants 

also suggested social media offers an 

alternate source of naturalistic data, for 

example, applications such as Twitter 

and Waze can also provide traffic and 

road information. 

Simulation

Simulation is useful to examine an issue 

in detail and to explore conflict situations 

not readily detected from observation. 

With simulation, it is possible to vary the 

frequency of the driver’s exposure to an 

intervention and to analyze the resultant 

driving behavior. For example, simulation 

is useful to study gap acceptance. 

Because of a disproportionate number 

of fatal accidents at rural intersections, 

Wisconsin DOT used simulation as an 

initial tool to test whether different types 

of signage would affect gap rejection 

and to encourage the acceptance of safer 

gaps, prior to using the more expensive 

on-road testing of alternative signage. 

There are other issues in which the utility 

of simulation is constrained because of the 

lack of exposure. For example, roadway 

departure crashes account for about 

half of all vehicular fatalities but these 

scenarios are challenging to replicate in 

simulation.12 It is very difficult to replicate 

the contributing factors, such as fatigue, to 

roadway departure. Simulation gives useful 

null results, but it is not possible to get a 

good understanding of roadway departure 

because it is impossible to replicate the 

scenarios, and the simulation process lacks 

sufficient exposure data. 

Intervention Development and Evaluation

By using response data obtained from 

simulators and information culled from 

naturalistic data, it is possible to design and 

develop interventions to modify behavior and 

to meet safety needs. Although the simulator 

results in Wisconsin on gap acceptance 

were not definitive, they did provide trend 

information, which became the basis for 

subsequent road testing of signage most 

likely to foster acceptance of safer gaps.

Field Operational Test Data: Data from the 

field can provide complementary information 

about the effects of a new or modified 

intervention on drivers and vehicles. 

Model-Based Benefit Estimation: Societal 

benefits can be estimated based on the 

effectiveness of interventions evaluated in 

experiments. Short-term benefits provided 

by treatments that have a significant impact 

(e.g., crash worthiness) can be identified 

and measured after implementation. Longterm benefits will change and evolve as the 

population of users adapt to the intervention. 

Policy Design: Transportation policies are 

outlined based on evaluation of interventions, 

in addition to societal benefits.

Bayesian and Model-Based Surveillance: 

Model-based analysis provides a measurement 

to evaluate scenarios after an intervention 

12. Federal Highway Administration Safety Program. 

Retrieved June 13, 2014, from http://safety.fhwa.dot.

gov/roadway_dept/.
48

has been implemented. Continuous 

surveillance can serve as a method to 

simultaneously collect new naturalistic 

data for a newly identified problem. In 

this case, this step will provide feedback 

to the procedure for a new strategy. 

Driver–Infrastructure and Roadway Data 

for Human Factors Research

Connected Vehicle Technology and Driver 

Behavior

One of the key topics that participants 

raised during discussion was the importance 

of researching how connected-vehicle 

technology will impact driver behavior. For 

example, connected-vehicle technology 

has the ability to warn following vehicles if 

they are not slowing down in response to a 

slowing lead vehicle, even if the lead vehicle 

is several vehicles ahead. It is important to 

understand whether drivers believe that the 

warning is specific to them, because they 

will be more likely to acknowledge it. There 

is also an issue of whether the warning 

should be placed in the infrastructure or in 

the vehicle and how the alternative venues 

might affect compliance. Finally, there is 

also a need to explore connected-vehicle 

signage options, in terms of alternative 

methods of presentation and where the 

signage should reside in the infrastructure.

Operation and Safety

Participants identified several areas 

where operation and safety are linked. 

The operational simulation models, which 

include car following and lane modeling, 

raise human factors issues. It is important 

to calibrate these models correctly in 

the simulator to obtain the surrogate 

measurements to be applied to the realworld behaviors. 

Research Priorities

Participants identified the following areas 

of priority concerning safety: roadway 

departure, urban intersections, vehicle and 

pedestrian to bicyclist interaction, and data 

analysis. Participants suggested a good 

synthesis project for roadway departures 

could ask a basic question such as, “When 

does the driver begin reacting to the 

curve?” There is also a need to evaluate the 

effectiveness of current signage used on 

roadways. Specifically, researchers want to 

know if and how current signs affect driving 

behavior, for example, when approaching a 

curve or in urban intersections, where all 

types of roadway users meet, and in leftturn conflicts. It was noted that there is 

difficulty running scenarios, such as speed 

perception, in simulators because of the 

inability to measure lateral acceleration in 

these settings. 

Data Sources

There are existing data sources that can be 

mined at low cost to provide insight, such 

as TMC data. Researchers need to look into 

how to combine different data sources to 

gain more powerful insights than can any one 

dataset provide. Simulation was suggested 

as a good tool for identifying issues.
49

Participants identified many areas of 

priority for human factors research, which 

could make use of the expanding datasets 

now available and soon to be available. 

These include modeling, safety, roadway 

departure, urban intersections, vehicle, 

pedestrian and bicyclist interaction, and 

data analysis. Several items were suggested 

for further research, as follows:

 • Evaluation of the effectiveness of 

current signage used on roadways.

 • Research on speed perception.

 • Solutions to improve roadway safety 

for pedestrians and bicyclists.

 • Evaluation of current ITS technologies 

for pedestrian and bicycle safety.

 • Developing a methodology to conduct 

research that uses multiple data sources.

 • Methods to measure exposure of 

pedestrians and bicyclists.

To advance understanding and use of 

multiple data types, the participants 

recommended a study, possibly focused at 

intersections, that includes multiple sites, 

multiple data types gathered at each site, 

multiple user types, and multiple methods 

of analysis. This study could provide 

critical information on how to resolve 

contradictions among datasets, how to 

put together complementary datasets 

that describe risky behaviors, and how to 

generate comprehensive datasets that link 

behaviors and crashes.

Workshop Recommendations
50

Appendices
51

Wednesday, November 6, 2013

10–11:15 a.m. Pre-Workshop Event

 Tour of research tools at the Federal Highway Administration’s Human 

Factors Laboratory

11:15 a.m.–1 p.m. Lunch

1–1:15 p.m. Introduction and Welcome 

1:15–1:30 p.m. Overview of the Federal Highway Administration’s Exploratory 

Advanced Research Program 

1:30–1:45 p.m. Workshop Objectives and Expected Outcomes

1:45–2:45 p.m. Presentation Set 1

 “Driver-Driver and Other Road Users’ Data for Human Factors Research”

 Presenters: 

 Dr. Marco Dozza, Chalmers University of Technology, Sweden 

 Dr. Toru Hagiwara, Hokkaido University, Japan 

 Dr. Hidekatsu Hamaoka, Akita University, Japan

2:45–3 p.m. Break

3–4 p.m. Presentation Set 2

 “Driver–Infrastructure and Roadway Data for Human Factors Research” 

 Presenters: 

 Dr. Michael Manser, University of Minnesota 

 Dr. Susan Chrysler, University of Iowa

4–5 p.m. Presentation Set 3

 “Driver–Vehicle Data for Human Factors Research”

 Presenters: 

 Dr. John D. Lee, University of Wisconsin–Madison

Dr. Linda Boyle, University of Washington 

5 p.m. Day 1 Adjournment 

Appendix A: Agenda

UTILIZING VARIOUS DATA SOURCES FOR SURFACE TRANSPORTATION 

HUMAN FACTORS RESEARCH 

Turner-Fairbank Highway Research Center, McLean, VA
52

Thursday, November 7, 2013

8–8:15 a.m. Recap of Day 1

8:15–9:30 a.m. Expert Panel Discussion and Q&A

 Moderator:

 Dr. Donald Fisher, University of Massachusetts Amherst

9:30–9:45 a.m. Break 

9:45–10:45 a.m. Small Group Discussion and Recommendations from Group Discussion 

10:45–11:45 a.m. Conclusions and Recommendations from Group Discussion

11:45 a.m.–12 p.m. Workshop Wrap-Up 

12 p.m. Day 2 Adjournment
53

Marco Dozza is an assistant professor at Chalmers University of Technology 

in the Department of Applied Mechanics. Since 2010, Dozza has been part 

of the Accident Prevention Group at Chalmers. His research interests focus 

mainly on naturalistic field operational test analysis (including bicycle safety) 

and methodology and active safety. Dozza is currently working on several 

national and international projects related to field operational tests, such as 

the second Strategic Highway Research Program (SHRP 2), as well as his own 

grants mainly related to cycling safety. Dozza is skilled in research framing, 

design, analysis, and interpretation and presentation of research results. He is 

the author of more than 40 scientific articles and peer-reviewed contributions 

to conferences. 

Toru Hagiwara is a professor at Hokkaido University, Japan, in the Department 

of Civil Engineering. His research interests include on-road human factors and 

accident analysis in Japan. Hagiwara is currently working on several projects 

related to analyzing conflicts between drivers and pedestrians at intersections. 

This work includes (1) evaluating driver behavior to avoid conflicts with 

pedestrians, (2) evaluating pedestrian behavior to avoid conflicts with right/

left-turning vehicles, and (3) evaluating a dedicated short-range communication 

system to avoid conflicts between drivers and pedestrians. His work is mainly 

conducted on test tracks.

Hidekatsu Hamaoka is an associate professor at Akita University, Japan. 

Hamaoka’s research field is traffic safety analysis, with a focus on human factors. 

His work includes design to decrease the number of right-turn accidents (leftturn accidents in the United States) with pedestrians at major intersections, 

arterial-local intersection design for bicycle safety, and proposals to reprogram 

traffic signals to avoid rear-end collisions. Hamaoka has worked with Hagiwara 

to conduct test track experiments for nearly a decade. 

Michael Manser’s work focuses on designing and evaluating novel technologybased transportation systems intended to support driver performance and 

safety. Manser’s work includes (1) designing and evaluating an infrastructurebased collision avoidance system intended to facilitate driver decisionmaking 

at high-risk rural intersections; (2) evaluating driver performance, workload, 

and usability associated with the use of a novel vehicle-based haptic collision 

avoidance system; and (3) evaluating driver mental and behavioral adaptation 

to the introduction and continued use of infrastructure-based driver support 

systems. His work is conducted in simulation and on-road testing environments. 

Appendix B: About the Presenters
54

Susan Chrysler joined the University of Iowa as the Director of Research at the National 

Advanced Driving Simulator in 2011. Chrysler’s areas of expertise include human factors, 

driving simulation, driver behavior, visual attention, traffic operations, visibility, and 

photometry. Chrysler has served as principal investigator (PI), or task leader, for projects 

on traffic-sign design and comprehension, pavement-marking effectiveness, visibility, and 

traffic operations. She recently completed the FHWA guide to evaluation methods for 

traffic control devices intended to aid practitioners who request experimentation in the 

MUTCD process. 

John D. Lee is the Emerson Electric professor in the Department of Industrial and Systems 

Engineering at the University of Wisconsin–Madison and director of the Cognitive Systems 

Laboratory. Lee’s research seeks to better integrate people and technology in complex 

systems, such as cars, semi-autonomous systems, and telemedicine. He has served on 

the National Academy of Sciences committees on human system integration, electronic 

vehicle controls and unintended acceleration, and autonomy in civil aviation. He has also 

served as an editor for many publications related to this field. 

Linda Ng Boyle is an associate professor in the College of Engineering at the University 

of Washington. She received her Ph.D. in Civil and Environmental Engineering from the 

University of Washington in 1998. Boyle is an associate editor for the journal Accident 

Analysis Prevention and chairs the Transportation Research Board committee on Statistical 

Methods (ABJ80). She has been the PI, or co-PI, on several SHRP 2 and USDOT projects 

that involve on-road and simulator data for examining driver distraction, crash risk, and 

road-user safety. Boyle is the recipient of the National Science Foundation Career Award 

and is a member of the Human Factors and Ergonomics Society, American Statistical 

Association, and Institute of Industrial Engineers.

Donald Fisher heads the Department of Mechanical and Industrial Engineering at the 

University of Massachusetts Amherst. He is also the director of the Arbella Insurance 

Human Performance Laboratory and serves as a Faculty Fellow at the Volpe National 

Transportation Systems Center in Cambridge, MA. Fisher’s research focuses on efforts to 

understand the behaviors of novice, older, and distracted drivers that increase their risk of 

crashes and developing and evaluating training programs to reduce their risk. Fisher has 

served as an investigator on projects designed to identify the characteristics of warning 

systems inside the vehicle, along with signs, signals, and pavement markings outside the 

vehicle that are most likely to lead to behaviors that decrease the risk of injuries and crashes.
55

Federal legislation establishes an Exploratory Advanced Research (EAR) 

Program for transportation to address longer term, higher risk, breakthrough 

research with the potential for dramatic long-term improvements to 

transportation systems, improvements in planning, building, renewing, and 

operating safe, congestion-free, and environmentally sound transportation 

facilities. The Federal Highway Administration’s (FHWA) EAR Program secures 

broad scientific participation and extensive coverage of advanced ideas and 

new technologies through stakeholder engagement, topic identification, and 

sponsored research. 

The uncertainties in the research approach and outcomes challenge 

organizations and researchers to be innovative problem-solvers, which can lead 

to new research techniques, instruments, and processes that can be applied to 

future high-risk and applied research projects.

For more information, please visit the program Web site at 

http://www.fhwa.dot.gov/advancedresearch/. 

About the EAR Program
56
57
Office of Safety Research and Development

Turner-Fairbank Highway Research Center

6300 Georgetown Pike

McLean, VA 22101-2296

Publication No. FHWA-HRT-14-077

HRTM-30/07/14(1M)E