The Operational Domain and the 

Human Driver as the 

Baseline for AV Metrics

David Agnew

VP AV Business Development

dagnew@dataspeedinc.com

NIST

Standards and Performance Metrics 

for On-Road Automated Vehicles 

Workshop

Sept 5, 2023
0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150 Periods of Violent Deaths in the U.S. 

(deaths/day)

WWI

WWII

Korean 

War

Vietnam 

War

Civil War

MexicanWar of 1812 American War

Spanish & 

Philippine Wars

American 

Revolutionary 

War
0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150

Major Causes of Violent Deaths in the U.S. 

(deaths/day)

WWI

WWII

Korean 

War

Vietnam 

War

Civil War

MexicanWar of 1812 American War

Spanish & 

Philippine Wars

American 

Revolutionary 

War

Commercial 

Aviation x 10
0

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150

Major Causes of Violent Deaths in the U.S. 

(deaths/day)

Car Crashes

WWI

WWII

Korean 

War

Vietnam 

War

Civil War

MexicanWar of 1812 American War

Spanish & 

Philippine Wars

American 

Revolutionary 

War

Commercial 

Aviation
The Operational Domain and the 

Human Driver as the 

Baseline for AV Safety Metrics

• There is a need for effectively measuring AV performance

• A highly significant aspect of driving is crashes and fatalities

• Benchmarking AV to human safety performance provides starting 

point

Autonomous Vehicle Performance Metrics

Not addressing here:

• Travel performance (speed)

• Comfort performance
One Year in U.S. (2022)*: 

-Humans drove 3.12 trillion miles

-42,795 fatalities occurred

What metric is used for measuring performance?

*NHTSA

73 million miles / fatality

The Human Driver Causes 94% of All Crashes

(NHTSA)
The Human Driver Causes 94% of All Crashes

(NHTSA)

One Year in U.S. (2022)*: 

-Humans drove 3.12 trillion miles

-42,795 fatalities occurred

What metric is used for measuring performance?

*NHTSA

73 million miles / fatality

million-miles/fatality or fatalities/million-miles
an Emerging Opportunity The Opportunity

73Mm/f

Human Safety 

Performance

This Metric Changes from Year to Year

From NHTSA data
an Emerging Opportunity The Opportunity

What is the Target for AV’s? 

AV Capability?

equal or better than 

human?

73Mm/f
73 million miles between fatal mistakes

to eliminate 90% of accidents, AV must be better ~ 10x

so AV target: 730 million miles AV

AV Safety Performance Expectations

One Approach

Current 

State

10 x 

Better
73 million miles between fatal mistakes

to eliminate 90% of accidents, AV must be better ~ 10x

so AV target: 730 million miles AV

AV Safety Performance Expectations

One Approach

Current 

State

10 x 

Better

But the conditions and the scope of the driving matters ...
73 mm/f

Operational Domain for the Driving Situation

The Conditions and the Scope of Performance Metric

• All U.S. roadways

• All driver ages

• All Times of Day

• All driver behaviors

-Aggressive

-Drunk

-Drowsy

What if you are a 47yr-old driver only drive in Wisconsin? 
Operational Domain: Region or State where Driving

73 million miles / fatality

IIHS 2021 Data

105 million miles / fatality

U.S.

Wisc.

For one from Wisconsin, a 

US based target falls short 

by 44%
1

Operational Domain: Age of Driver

Analysis by author

Data from Texas Transportation Institute study

Normalized Fatality Rate

vs 

Age Group
AVG 1

You

Operational Domain: Age of Driver

Analysis by author

Data from Texas Transportation Institute study

Significantly better 

than average (40%)

Normalized Fatality Rate

vs 

Age Group
1

You

Analysis by author

Data from Texas Transportation Institute study

For a 47yo driving from MI to FL, 

would having an 18yo driving be ok?

Normalized Fatality Rate

vs 

Age Group

680% more 

fatalities

Operational Domain: Age of Driver
Operational Domain

Something to Think About

Is the AV safe enough?
Measuring AV Performance

In Summary

• A highly significant aspect of driving is crashes and fatalities

• Benchmarking AV to human safety performance can and will be 

done

• Human performance can be measured as miles-between-fatalities

• The conditions (OD) of human to AV comparison matter
The Operational Domain and the 

Human Driver as the 

Baseline for AV Metrics

David Agnew

VP AV Business Development

dagnew@dataspeedinc.com

NIST

Standards and Performance Metrics 

for On-Road Automated Vehicles 

Workshop

Sept 5, 2023