Most automobile manufacturers and several technology companies are testing automated vehicles on public roads. While automation of the driving task is expected to reduce crashes, there is no consensus regarding how safe an automated vehicle must be before it can be deployed. An automated vehicle should be at least as safe as the average driver, but national crash r ates include drunk and distracted driv ing, meaning that an automated vehicle that crashes at the average rate is somewhere between drunk and sober. In this paper, automated vehicle safety benchmark s are explored from three perspectives. First , crash rates from naturalistic driving studies are used to determine the crash risk of the model ( i.e., sober, rested, attentive, cautious) driver. Second , stated preference surveys in the literature are reviewed to estimate the public’s acceptable automated vehicle ri sk. Third, crash, injury, and fatality rates from other transportation modes are compared as baseline safety levels . A range of potential safety target s is presented as a guide for policymakers , regulators , and automated vehicle developers to assist in evaluating the safety of automated driving technologies for public use. Disclaimer: The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policies or positions of any agency of the Commonwealth of Virginia. 2 INTRODUCTION The task of driving has become increasingly automated i n recent decades, particularly since the start of large -scale public road testing in 2010 (1). Vehicles with computer -control of combined lateral and longitudinal movement over a range of environments are referred to as autonomous or automated vehicles. Most vehicle manufacturers and several technology companies are developing automated vehicles, and these developers have conducted 6.5 million miles of supervised autonomous testing on public roads in California as of November 2019 . The safety of these vehicles remains an open question . As advanced automated vehicles are not yet sold to the public , safety assessments have relied mostly on mand ated reporting of autonomous testing mileage and crashes to the California Department of Motor Vehicles (2). These studies have found that when equivalent crash definitions are compared , automated vehicle s crash at lower rates than the general public (3, 4). Automated vehicle t esting in California , however, requires the use of a trained safety driver to monitor the vehicle and take control as needed . These drivers are trained to anticipate collisions and unexpected vehicle behavior, and to take control back from the automated driving system. As these safety drivers made 159,840 unplanned takeovers over 3,669,472 miles since between September 2014 and November 2018 (5), it is probable that crash rates from testing with trained safety drivers would not translate to general public drivers who may be distracted, tired, and un familiar with the technology . There are as yet no defined safety requirements that automated vehicles must meet prior to being tested or sold to the public. Applications to test new automation technologies are generally handled on a case -by-case basis at the state level. Some automation features would likely be regulated at the national level , similar to how conventional vehicles in the United States are regulated by the National Highway Traffic Safety Administration (NHTSA) , which is obligated to order a manufacturers to remedy defects that present an “unreasonable risk ” (6) of crashes, death, or injur y. In these cases, m anufacturers would also be responsible for remedying defects, and dealers would be prohibited from selling vehicles with known defects (7). Vehicles can be defective even if they comply with NHTSA ’s performance standards if they present “unreasonable risk of accidents ” (8). In California, the state ’s Department of Motor Vehicles (DMV ) regulates automated vehicle testing within state boundaries , and requires that vehicles be insured, bonded, and in most cases monitored by a licensed driver directly employed by the AV developer (2). Although n o regulations in the U.S. at the state or federal level prohibit AVs that exceed predetermined crash, injury, or fatality rates , a known safe ty benchmark could be used to begin to define “unreasonable ” risks that specific AVs should not present. Private sector AV developers may also find crash rate targets useful when evaluating and communicating the safety of their automated driving systems. Were AV safety benchmark s to be expressed as crash, injury, and fatality rates, what might those benchmark s be? Several benchmark s have been proposed in the literature and by commentators. One answer is that AV must at least be as safe as the average driver (9). While this seems reasonable, it fails to take into account the average driver ’s crashes that involve 3 unacceptable and illegal driving behavior such as drunk driving and speeding. An automated vehicle that is safe as the average driver is, statistically speaking, somewhere between drunk and sober driving in terms of safety. Society does seem to have accepted these risks to some extent — speed governors could eliminate speeding (10) and compulsory alcohol detection systems would restrict drunk driving (11), yet neither is mandatory for new vehicles. But there is a clear difference between p rohibition and aggressive enforcement, and l aws against s peeding a nd drunk driving are still enforced . Introducing an automated vehicle with a crash rate equivalent to an occasional law -breaking human driver seems unlikely and irresponsible. Others have proposed higher standards for automated vehicle safety. Sparrow an d Howard argue that AVs that are safer than average with respect to other road users should be made mandatory (12). Kalra and Pad dock , in calculating the number of miles traveled to establish statistical significance of safety improvements , used a 20% reduction in average fatal crash rate as an example (13). Commentators have recommended safety improvements by factors varying from 2 to 100 , while others use the baseline of a mature, skilled driver (14). A widely referenced numb er is NHTSA’s estimate that driver error is a contributing factor in 94% of crashes (15), implying that automated vehicles free from human error might reduce the crash rate by a similar amount. To achieve a 94% crash reduction, however, human driving would have to be eliminated, as automated vehicles would still be struck by at -fault human dr ivers. Beyond the impracticality of banning human driving in the near future, others have pointed out how human error rate and automated vehicle crash reductions are not directly comparable (16). Responsibility for es tablishing the safety of automated vehicles is shared among manufacturers and regulators. These entities m ay choose to define what constitutes a safe autonomous driving system and, more importantly, what does not. Specific safety targets allow AV developer s to communicate their safety record in context and with transparency . Clear definitions of acceptable crash rates tied to achievable benchmarks with wide public acceptance allows transparent regulation and validation of automated vehicles. The purpose of this paper is to present a range of safety benchmarks for automated vehicles. This is accomplished through a calculation of crash rates based on various driver risk factors from naturalistic driving , analysis of stated preference surveys in the literature , and a comparison of crash , injury, and fatality crash rates of other modes of transport ation . CRASH RATES CONSIDERING DRIVER RISK FACTORS Crash rates are simply the rate at which vehicles crash, often expressed in terms of number of crashes per distance traveled. Rates can be broken down by collision angle, crash severity, speed, and a range of other factors recorded in a police report. A significant portion of crashes are due to driver error or impairment. These factors, such as excess ive speeding, fatigue, and distraction should not be present in an AV, nor should they be exhibited by for -hire vehicles such as taxis and buses. Without these impairments, the baseline crash rate for an AV might be closer to an error and impairment -free m odel driver than the average driver. 4 To isolate the effect of these causal factors on crash rates, the number of crashes and distance traveled must be recorded for driving when both exposed to the factor and not exposed to the factor. While some of the fac tors are captured in NHTSA crash records, e.g. blood alcohol content, others are either captured with low confidence (fatigue (17) and distraction (18)) or not captured at all (emotional driving). No factors are collected for non -crash driving, making any claims about the relative safety of driving with risk factors impossible —one cannot claim that distracte d driving is less safe without knowing the total number of distracted miles driven both with and without crashing. There are other shortcomings with conventional crash databases. While crash reporting is detailed and complete for fatal crashes in many coun tries, injury and property damage crashes may have limited data and might never be recorded in databases. In the United States, the National Highway Traffic Safety Administration estimated that 15% of injury crashe s and 24 % of property damage -only crashes are never reported to police (19), while an additional 9% of injury crashes and 24% of property damage -only crashes are reported but not entered into databases (20). These estimates of mis sing crash records are based on telephone surveys and rely on respondents recalling crashes they were involved in over the prior year (19). Respondents might not recall minor crashes, fu rther weakening the completeness of crash records. To accurately capture crash rates, both crashes and mileages would ne ed to be recorded. Naturalistic Driving Data Naturalistic driving studies, where drivers are passively recorded over a period of time, allow the collection of a range of risk factors over both crash and non -crash driving. Naturalistic driving studies have the added benefit that crashes can be automat ically recorded via the vehicles kinematic sensors without relying on police -reporting. Further, vehicle mileages can be recorded with much greater precision and accuracy tha n FHWA miles -driven estimates. The Second Strategic Highway Research Program Naturalistic Driving Study (SHRP 2 NDS) completed data collection in 2015 and remains the largest NDS of its kind, collecting video, kinematic, and audio data from over 3,500 drivers over three years and 30 million miles (21). Crashes, near -crashe s, other safety critical events, and baseline non -crash events were analyzed by data reductionists and classified based on observed driver error and impairment. Because this dataset captured risk factors for both crashes and baseline non -crash driving, it can be used to calculate the effect of individual risk factors on crash rate. Calculation of Driver Risk Odds Ratios Dingus et al. (22) presents the odd s ratios and baseline prevalence for various risk factors based on model driving. The odds ratio is defined as crash rate when exposed to a risk factor divided by the crash risk when not exposed. O.R. = crash rate under risk factor crash rate under model driving (1) 5 To determine the crash rate for each risk factor, the NDS needed to collect not only crashes but also the prevalence of each risk factor under non -crash driving. Due to limits in in - vehicle data storage, drivers were not recorded continuously. Prevalence of driver behaviors under non -crash dr iving was instead estimated from data recorded for short time intervals while the vehicle was moving. Over the course of the study, 19,732 non -crash intervals were selected based on a two -staged stratified random sampling method and subjected to the same d ata reduction by trained analysts as crash scenarios. From these samples, a baseline of exposure data can be developed. To determine the odds ratio from baseline data, the crashes per baseline intervals when the risk factor was present is the numerator, wh ile the crashes per baseline intervals when under model driving is the denominator. O.R. = 𝑐𝑟𝑖𝑠𝑘 𝑏𝑟𝑖𝑠𝑘 𝑐𝑚𝑜𝑑𝑒𝑙 𝑏𝑚𝑜𝑑𝑒𝑙=𝑐𝑟𝑖𝑠𝑘𝑏𝑚𝑜𝑑𝑒𝑙 𝑏𝑟𝑖𝑠𝑘𝑐𝑚𝑜𝑑𝑒𝑙 (2) In this equation, c is the number of crashes and b is the baseline mileage under the risk and model driving behaviors, respectively. Dingus et al. (22) provides values for odd s ratios and percentage of baseline events exhibiting each risk factor. To conduct any comparative analysis, the crashes and baseline events for model driving are needed. Crashes are provided (12.3% of 905 crashes occurred under mo del driving), and baseline prevalence can be calculated by using a risk factor with a known odds ratio, crash percentage, and baseline percentage. As an example, driver distraction is used below, which represented 68.3% of injury and property damage crashe s and 51.93% of baseline samples. O.R. = 𝑐𝑟𝑖𝑠𝑘 𝑏𝑟𝑖𝑠𝑘 𝑐𝑚𝑜𝑑𝑒𝑙 𝑏𝑚𝑜𝑑𝑒𝑙 (3) 2= 0.683 0.5193⁄ 0.123 𝑏𝑚𝑜𝑑𝑒𝑙 (4) 𝑏𝑚𝑜𝑑𝑒𝑙 =0.187 (5) The odds ratio for all driving compared to model driving can then be calculated as: 6 O.R. =𝑐𝑟𝑖𝑠𝑘 𝑏𝑟𝑖𝑠𝑘 𝑐𝑚𝑜𝑑𝑒𝑙 𝑏𝑚𝑜𝑑𝑒𝑙= 1 1 0.123 0.187≈1.5 (6) The all -driving injury and property damage crash rate for the naturalistic data is provided by Blanco et al. (4) as 20.2 crashes per million vehicle -miles traveled. (Although t he SHRP 2 NDS purposely over -sampled drivers under age 25 and over age 65 (22), the all -driving crash rate of 20.2 crashes per million vehicle -miles used when calculating risk factor crash rates is age - weighted to represent the U.S. national driving population (4)). The crash rate for model driving can be obtained using the general driving odds ratio and reformulating equation (1). crash rate under model driving = crash rate under risk factor O.R.=20.2 1.5=13.5 crashes/MVMT (7) In this equation, MVMT refers to million vehicle -miles traveled. For the purposes of this paper, the crash rate under a risk factor is less useful than the crash rate when not under a risk factor, i.e., when the driver is not distracted, fatigued, etc. Crashes and baseline driving often have multiple risk factors occurr ing simultaneously. To calculate the odds ratio, the crashes and baseline samples where the risk factor does not occur must be isolated. O.R. 𝑖𝑛𝑣𝑒𝑟𝑠𝑒 =1−𝑐𝑟𝑖𝑠𝑘 1−𝑏𝑟𝑖𝑠𝑘 𝑐𝑚𝑜𝑑𝑒𝑙 𝑏𝑚𝑜𝑑𝑒𝑙 (8) For some risk factors in Dingus et al. (22), crisk is not available. It can be obtained and substituted into equation 8 using: 𝑐𝑟𝑖𝑠𝑘 =𝑐𝑚𝑜𝑑𝑒𝑙 𝑏𝑟𝑖𝑠𝑘O.R. 𝑏𝑚𝑜𝑑𝑒𝑙 (9) The risk factors and associated odds ratios and crash rates are shown in Table 1. These crash rates are significantly higher than 208 per MVMT reported by NHTSA for 2018 (23). This difference is because SHRP 2 NDS includes low -speed no -damage crashes, while NHTSA records includes only report police -reportable damage, injury, and fatal crashes . While the crash rate for all drivers is 202 0 per hundred million miles, the crash rate is reduced sl ightly for sober and res ted driving and significantly for cautious and attentive driving. A driver exhibiting all of these qualities would be considered a model driver with a crash rate of 1347 per 100 million miles, a reduction of 33% compared to all driv ing. 7 Table 1 Crash Rates and Odds Ratios from SHRP 2 Naturalistic Driving Study Driver Status Crashes per 100 Million Vehicle -Miles Odds Ratio to Model Driving All Driving 2020 1.5 Sober 2010 1.5 Rested 2007 1.5 Non-impaired 1954 1.5 Cautious 1479 1.1 Attentive 1350 1.0 Model Driving 1347 1.0 Child in rear seat 673 0.5 STATED PREFERENCE SURVEYS Another way to determine potential safety benchmark s is to ask users what crash or fatality rates they would find acceptable. This is referred to as stated preference , a widely used method to establish acceptable risk levels (24). Liu et al. (25) surveyed 499 residents of Tianjin, China about acceptable fatality rates in terms of miles and population for either AVs or HDVs. Even though no respondent was asked about both AVs and HDVs, answers showed a clear preference for lower fatality rates for AVs. Based on the developed logarithmic model, half of respondents felt that a fatality rate of 0.20 per 100 million miles (3.2e -9 km) was acceptable for HDVs, but for AVs, the rate should be 0.028 per 100 million miles (4.5e -10 km), a reduction of 86%. The authors compared these rates to actual fatality rates per population globally but, t o maintain consistency, the U.S. 2018 figure of 1.13 fatalities per 100 million miles is used here for comparison (23). The AV fatal crash r ate that 50 percent of respondents found acceptable represents a 98% reduction in crashes from current rates. At the same time, respondents preferred that human drivers crash 85% less than today ’s drivers. This suggests that survey participants might accept an AV that is only 87% safer than today’s drivers. Liu et al. defines three risk levels based on survey findings: unacceptable risk where crash rates are higher than those of the average human driver, tolerable risk when AVs are 75 –80% safer than human drivers, and broadly acceptable when AVs reduce crash rates by 99% compared to human drivers. No definition is provided for AVs that are between 0% an d 75% safer than human drivers, but for now this is defin ed as barely tolerable risk. In the survey instructions (25), respondents were informed that US rates are 321 tr affic crashes, 124 injuries, and 1.7 deaths per billion vehicle -kilometers , but the authors do not provide a source for these figures. In fact, t hese rates are approximately 25% of NHTSA ’s 2018 U.S. rates when converted to the same base units (23). Using Liu et al. ’s rates as actual rates, 50 percent of respondents require d AVs to reduce current crash rates by 90%. The same percentage of respondents preferred a human -driven crash rate that is 27% lower tha n current rates. This may have introduced some bias in Liu et al. ’s results, as respondents were prompted with human -driven fatality rates significantly lower than actual rates. Additionally, different cultural 8 attitudes towards acceptable risk (26) may limit the applicability of these findings to the United States. Nees (27) performed a similar survey on AV risk acceptance . Nees noted that respondents may have had difficulty answering questions about acceptable crash given that these values are quite small and people have limitations when interpreting small numbers . Nees attempted to solve this problem by asking drivers to provide safety criteria as a function not of fatalities per vehicle -miles traveled but as a percentile of all drivers ’ fatality rates . In a survey of 504 participants via Amazon Mechanical Turk, respondents first ranked their own driv ing. Eighty -one percent of participants rated their driving as safer than the media n driver. This was in agreement with previous studies such as Svenson (28) where 88% of U.S. drivers rated themselves as safer than the average driver. The median driver considered t hemselves at the 74th (73–76, 95% CI) percentile of safe drivers. When asked at what percentile an AV should be before being allowed on public streets, the median participant preferred an AV that was 8. 7 (6.9– 11.1, 95% CI) percentile points higher than the mselves, with 66% preferring an AV safer than themselves. The results are difficult to interpret for two reasons. First, most participants desired an AV in the 95th to 99th percentile. This may suggest not that they want an AV that was 12–16 percentile po ints safer than themselves necessarily, but rather an AV that is as safe as the best human driver regardless of the participant ’s own perceived safety. Second, there are few examples in the literature of percentile crash rates of drivers, and no public data sets from which to develop a risk profile. Most population crash studies use department of motor vehicle crash records (29). These studies do not control fo r exposure through mileage, so the “safest ” drivers with the fewest crashes may not be driving at all. Other studies have estimat e exposure from driver surveys, but these may suffer from respondents’ failure to accurately recall past mileages (29–32). The SHRP 2 NDS provides individual records of crashes and mileages, but the dataset contains only 1,148 crashes for 3,54 6 participants, again leaving a large percentile range with perfect safety records (17). Near crashes, where vehicles made sudden movements or hard decelerations, could be used as safety surrogates (26) , but t his might weaken the accuracy of the findings. Many insurance companies now use GPS sensors on vehicles to track mileage. These data could be merged with crash records to create individual crash rate records, but these datasets are generally proprietary and nonpublic. If a percentile crash rate of a ge neral population of drivers could be created, either from insurance data, NDS data, or some other source, then a desired safety percentile could be translated into a specific crash rate. This is beyond the scope of this article but may be an area for fu rther research. Even if individual risk profiles could be created, crashes are still rare, and many drivers will never crash in their lifetimes . Any crash risk distribution from individuals ’ data would have a significant percentile range with zero crashes. A n AV attempting to achieve a 99th percentile safety rating might never be permitted to crash, an impossible task. Finally, p articipants generally overestimated their safety in relation to the other drivers. Previous studies have shown that more than half of drivers believe themselves safer than both the 9 average driver (28) and drivers within their peer group (33). It is not clear whethe r this overestimation has been factored into their stated risk acceptance. Consider driver s at the 40th percentile, who think they drive at the 70th percentile, and who want AV s to be at the 90th percentile. Do these driver s want vehicle s that are at the 9 0th percentile for safety, or merely vehicle s that are 20 percentile points above themselves , i.e., at the 60th percentile ? This is a difficult question to answer from surveys and may instead require a revealed preference approach. CRASH RATES FROM OTHER TRANSPORT ATION MODES When defining thresholds for acceptable AV crash rates, the safety of other modes of transportation may serve a s useful benchmarks. Liu et al. , for example, mentions rail and commercial aviation as two examples of transportation modes with broadly acceptable safety rates (25). In this section, crash, injury, and fatality of various modes of transportation in the United States between the years of 2008 –2017 are compared. Counts of crashes, injuries, fatalities, and mileages were mostly obtained from the U.S. Department of Transpo rtation ’s Bureau of Transportation Statistics (34). Safety rates were compared across several exposure data. The standard crash risk measure is events per distance, with distance often expressed in terms of vehicle -miles traveled (VMT) . This metric can bias against low -capacity modes such as passenger vehicles which have more crashes per mile but with fewer passengers are exposed to each crash. An alternative metric is explored in this effort, person -miles traveled (PMT) , defined as the number of people traveling a given distance or vehicle -miles multiplied by average vehicle occupancy . When person -miles data was not available, it was calculated from average vehicle occupancy , often obtained from the U.S. National Househo ld Transportation Survey (35). While exposure is often expressed as a unit of ti me, this may bias against slower moving vehicles. Buses , for example, has higher fatality rates than commercial air travel per mile, but lower rates per hour due to their slower speeds. Alternative metrics, vehicle -hours traveled (VHT) and person -hours traveled (PHT) are used here. Vehicle -based metrics are used for crash rate, as the number of occupants is not affected. Person -based metrics are used for occupant injury and occupant fatality rates, where the number of passengers in a vehicle is inversely correlated to individual risk. Table 2 shows crash, injury, and fatality rates for several transportation modes. Commercial air travel has the best safety rate across several metrics, with the exception of occupant fatalities per person -hour traveled where bus rates are lower. Bus and passenger rail have low injury and fatality rates, lower than walking and cycling a nd within range of elevators and escalators. Motorcycles have the worst safety record across all metrics. 10 Table 2 Crash, Injury, and Fatality Rates by Transportation Mode Mode1 Per Unit Distance Per Unit Time Crashes / 100 M VMT Occupant Injuries / 100 M PMT Occupant Fatalities / 100 M PMT Crashes / 100 M VHT Occupant Injuries / 100 M PHT Occupant Fatalities / 100 M PHT Passenger cars2 213.9 29.9 0.28 5,946 831.7 7.9 Motorcycles2 546.3 375.9 20.27 13,385 9,210.5 496.6 Large trucks3 130.8 9.4 0.24 5,885 421.2 10.5 Buses4 408.4 5.1 0.015 4,655 57.8 0.18 Commercial air2 0.37 0.003 0.001 161 1.5 0.52 Pedestrians6 - 208.6 14.7 - 688.4 48.5 Cycli sts6 - 599 8.65 - 3,893 56.2 Passenger rail7 - 6.7 0.034 - 155 0.8 Elevators8 - 993 0.74 - 1,523 1.1 Escalator9 - 1944 0.50 - 1957 0.51 1 All data 2008 -2017 from (34) unless otherwise noted. Spee d and o ccupancy were used to convert miles to hours and vehicles to persons. 2 Speeds and occupancy from (35). 3 Trucks are defined as >10,000 lbs. Speeds average of 36 mi/hr for recreational vehicles (35) and 55 mi/hr Interstate Highway System speeds (36). Occu pancy from (VM -1). 4 Speeds from (35). Occupancy from (37). 5 Distance and speeds from (35). 6 Spee ds and occupancy calculated from (38). 7 1997 –2010. Speed calculated as 4.5 floors (45 feet) in 20 seconds from (39). All other data from (40, 41). 8 1997 –2010. Speed calculated as 0.45 m/s (42) and assumes average elevator distance of 10 vertical feet at a 30 degree incline for 20 feet of slope distance. All other data from (40, 41). Of the modes in Table 2, buses appear to be the most applicable benchmark for automated vehicle safety. Buses used trained drivers with specialized licensing , and buses are inspected more regularly than passenger vehicles. Although bus es crash at higher rates per mile than passenger vehicles, this may be due to their lower average speeds of 11.4 mi/hr to passenger vehicles ’ 27.8 mi/hr. Buses also travel in dense urban areas which have higher crash rates than comparative rural/suburban roads and freeways. Buses experience fewer crashes per hour, and fewer occupant injur ies and fatalities across all metrics. In evaluating AV safety benchmark s, industry and regulators must be careful to consider not only the safety rate with respect to passengers, but also the safety rate with respect to those outside the vehicle. Table 3 shows the injury and fatal crash rate for those outside the vehicle, either in other vehicle types or bicyclists and pedestrians. Non-occupants are injured in bus crashes at nearly double the rate per mile as for passenger cars. Other metrics show buses and 11 passenger cars with similar safety metrics . Although bus occupants are at substantially less risk of death and injury than car passengers, this difference is erased for non -occupants. Table 3 Non-occupant Injury and Crash Rates Mode Non-occupant injuries per 100M VMT Non-occupant injuries per 100M VHT Non-occupant fatalities per 100M VMT Non-occupant fatalities per 100M VHT Passenger cars 36.4 1013.0 0.8 20.9 Large trucks 27.2 1225.6 1.2 52.9 Buses 70.4 803.1 1.6 17.8 CONCLUSIONS AND DISCUSSION While there is little consensus on crash rate targets for automated vehicles, some example benchmarks can be collected from other sources. Several potential benchmark s were identified in this study. Data from the SHRP 2 Naturalistic Driving Study suggests that model drivers —i.e., sober, rested, attentive, and cautious —crash 33% less than the average driver. An AV that seeks to emulate not a highly skilled human but merely one driving legally should crash at no more than this lower model rate. Surveys from Liu et al. suggest that drivers prefer an automated vehicle to crash 80 % less (tolerable risk) or 99% less (broadly acceptable risk) than the average human driver (25). These figures are supported by findings in Nees (27). Others in the literature have used 20% lower crash rates as an example of acceptable AV crash risk (13). Crash, injury, and fatality rates from other modes provide examples of accepted risks for public transportation with stricter regulations of driver training and vehicle maintenance and inspections. Of all modes, buses provide the best comparison with AVs given their shar ed travel environment and similar levels of regulatory oversight. Table 4 lists a range of potential safety benchmark s for automated vehicles identified in this study. Benchmark s are expressed in terms of crashes, police -reportable ( i.e., property damage or more severe) crashes, occupant and non -occupant injuries, and occupant and non -occupant fatalities. Total counts are divided by the most relevant exposure data for the metric, either vehicle -miles traveled, vehicle -hours traveled, perso n-miles traveled, person -hours, or more than one exposure metric. 12 Table 4 Potential Crash, Injury, and Fatality Rate Safety Benchmark s for Automated Vehicles Exposure Metric Safety Metric Passenger cars 20% safer (13) Model driving, 33% safer Tolerable, 80% safer (25) Broadly acceptable, 99% safer (25) Buses for comparison 100 M VMT All Crashes 2020 1616 1347 404 20 - PR Crashes 214 171 137 43 2.1 408 Non-occupant injuries 36 29 24 7.3 0.36 70 Non-occupant fatalities 0.8 0.6 0.5 0.2 0.008 1.6 100 M VHT PR Crashes 5946 4757 3964 1189 59 4656 Non-occupant injuries 1013 810 675 203 10 803 Non-occupant fatalities 21 17 14 4.2 0.21 18 100 M PMT Occupant Injuries 30 24 20 6.0 0.3 4.9 Occupant Fatalities 0.28 0.23 0.19 0.057 0.003 0.015 100 M PHT Occupant Injuries 832 665 554 166 8.3 56 Occupant Fatalities 7.9 6.3 5.3 1.6 0.079 0.18 Metric Selection AV developers and regulators should focus on crash metrics initially. As Ka lra and Paddock have shown, fatal crashes are exceptionally rare, and any AV developer that wishes to establish with statistical significance that their vehicles are safer via on -road testing would need to accumulate an impractical number of fatality -free mi les (13). Demonstrating that an AV has a failur e rate equivalent to the national average with 95% confidence would require a fleet of 100 vehicles driving continuously for 12.5 years. The amount of required testing is substantially reduced when establishing performance for the all -crash rate. One could establish that an AV crashes at the same rate as model driving with 95% confidence with only 222,400 miles of crash - free driving. Waymo vehicles travel this distance in autonomous mode every two months in California alone (5). Limitations of a Single Metric This study investigated crash risk as a safety target for automated vehicles, defined as crashes of all severity levels per distance traveled. This metric is a useful starting point for industry and regulators who may be beginning to define safety for AVs using transparent benchmarks , but by no means should it be the end. Relying on a single benchmark is ill-advised . Automated vehicle safety improvements sh ould be equitable, and benefits to one road user class should not come at the expense of another. An AV that kills 1 ,000 fewer car occupants but 100 additi onal pedestrians may not be acceptable, even though 900 net lives are saved. AVs should improve the safety of everyone who interacts with them, and these benefits should be distributed fairly. Crash rates at different severity levels should also be evaluat ed: fewer low -risk crashes do not excuse more severe crashes, and likewise fewer fatalities may not justify a two -fold increase in severe 13 debilitating injuries . The ethics of risk is a mature field that can address some of these issues (43, 44). Similarly, AVs should focus on vehicle -miles and v ehicle -hours as exposure data to avoid the temptation t o put riders in the back seat and artificially inflate person -based exposure. As far as between vehicle -miles and vehicle -hours, both metrics should be used in combination. Using vehicle -hours avoids a bias towards desolate freeway driving which accumulates miles with little risk, while vehicle -miles avoids the unlikely but conceivable scenario where vehicles idle in parking lot s, accumulating hours. A metric such as vehicle -moving -hours or vehicle -miles- hours, might serve as alternative exposure metrics. Finally, different vehicle types may require their own metrics and benchmarks . In one example, s everal manufacturers operate low -speed automated shuttles the drive at speeds no greater that 15 mi/hr on streets with speed limits no greater than 25 mi/hr (45). At these speeds, vehicles will experience few fatal crashes per mile, and even fewer per hour when compared to passenger vehicles . Because of their low weight , however, shuttles may have fewer non - occu pant fatalities per hour when compared to buses . Automated shuttles may require safety targets based on bus crash rates but customized for shuttles’ unique operating environments and characteristics . Automated trucks operating at (relatively dangerous) high speeds on (relatively safe) controlled access highways may also require customized safety benchmarks. Future research should investigate alternative metrics to measure safety. While this study presented metrics in terms of crash, injury, and fatality rates, others might focus on predictability , lateral and longitudinal accelerations, near-misses (46), inverse time -to-collision (47), or other crash surrogate measures. These alternative metrics m ight provide a more robust picture of driving risk with out requiring crashes as samples. Researchers should also apply findings from risk ethics to automated vehicle crash risk. A few q uestions that require answers include under what circumstances an incre ase in crash risk to one road user group can be permitted at the expense of another group, how might AVs distribute risk among occupants and non-occupants, and how non -crash driving situations such as lateral positioning within a lane can affect permissibl e crash risk to others (48, 49). ACKNOWLEDGEMENTS This work was spons ored by the Virginia Department of Transportation. The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policies or positions of any agency of the Commonwealth of Virginia. AUTHOR CONTRIBUTION STATEMENT N.J. Goodall was sole author and is responsible for the entirety of this paper. 14