HIGHWAY SAFETY

More Robust DOT 

Oversight of 

Guardrails and Other 

Roadside Hardware 

Could Further 

Enhance Safety

Report to Congressional Requesters

June 2016

GAO-16-575

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-16-575, a report to 

congressional requesters

June 2016

HIGHWAY SAFETY

More Robust DOT Oversight of Guardrails and Other 

Roadside Hardware Could Further Enhance Safety

Why GAO Did This Study

In 2014, 54 percent of traffic fatalities 

in the United States occurred as a 

result of a vehicle’s leaving the 

roadway, according to U.S. 

Department of Transportation’s (DOT) 

data. Roadside safety hardware, such 

as guardrails, is meant to reduce the 

risk of a serious crash when leaving 

the roadway. But in the last several 

years, a number of serious injuries and 

deaths resulted from crashes into 

roadside safety hardware. GAO was 

asked to review FHWA’s oversight 

framework for roadside safety 

hardware. 

This report assesses: (1) how FHWA 

performs oversight of state policies and 

practices related to roadside safety 

hardware; (2) the laboratory crashtesting process and FHWA’s oversight 

of this process; and (3) the extent to 

which information is available on 

roadside safety hardware’s

performance once installed. GAO 

reviewed federal and state policies, 

surveyed state DOTs and received 44 

responses, and reviewed 

documentation from nine U.S. crash 

test labs.

What GAO Recommends

GAO is making recommendations,

including that DOT monitor and 

periodically report on the transition to 

the MASH crash test standards; 

develop a process for third party 

verification of crash test results; and 

support additional research on 

roadside safety hardware’s in-service 

performance. DOT concurred with the 

recommendations and provided 

technical comments, which were 

incorporated in the report, as 

appropriate.

What GAO Found

The Federal Highway Administration (FHWA) oversees and promotes states’ 

installation of crash-tested roadside safety hardware through guidance and policy 

directives to states and by issuing letters to roadside safety hardware developers 

that provide states with information on roadside safety hardware that has been 

crash tested. States that responded to our survey generally stated they require 

crash testing. However, some inconsistencies across state practices exist, and 

states’ movement to require installation of devices successfully tested to 

updated, improved crash test standards—in the Manual for Assessing Safety 

Hardware (MASH)—has been slow. FHWA, in partnership with the American 

Association of State Highway and Transportation Officials (AASHTO), recently 

established transition dates to the MASH standards for states, but some 

challenges exist in developing and approving a sufficient quantity of roadside 

safety hardware tested to MASH standards. FHWA currently does not have a 

monitoring plan to report on progress to meeting the established dates; 

monitoring and reporting would allow FHWA to keep decision makers aware of 

progress and position FHWA to take corrective actions as needed.

In general, laboratory crash testing appears to be well documented and 

thorough; however, FHWA’s oversight of the process does not address potential 

threats to independence. GAO found that six of the nine accredited U.S. crash 

test laboratories evaluate products that were developed by employees of the 

parent organization—a potential threat to lab independence. FHWA reviews 

crash tests’ results and related documentation, if they are submitted for review, 

but FHWA relies heavily on the labs to make a pass/fail determination. We found 

that some other federal agencies in oversight of similar labs’ testing settings

require third party verification of test results or independent entities to make 

pass/fail determinations. FHWA does not have a process for formally verifying 

the testing outcomes and making its own or providing for an independent

pass/fail determination. Developing a process for third party verification of 

roadside safety hardware’s lab test results could provide greater assurance that

potential threats to independence are fully addressed.

Little is known about the in-service performance of roadside safety hardware 

because few evaluations of this performance have been done. FHWA and 

AASHTO recommend that states and others perform in-service performance 

evaluations (ISPE) of installed roadside safety hardware because crash testing 

cannot fully capture real-world crash conditions. However, few ISPEs have been 

done, in part, because of a lack of inventory and crash data. In the summer of 

2015 in four states, FHWA began a pilot study that could provide useful 

information, but according to FWHA officials, the purpose of this phase of the 

pilot is to determine best practices on data collection rather than assess 

performance of roadside safety hardware. FHWA officials told us they currently

have no plans to include performance findings as part of future phases of this 

study or in their broader research portfolio. Continuing this study or planning to 

make ISPEs part of future research could add to the limited ISPE body of 

knowledge.

View GAO-16-575. For more information, 

contact Susan Fleming at (202) 512-2834 or 

flemings@gao.gov.
Page i GAO-16-575 Roadside Safety Hardware

Letter 1

Background 3

FHWA Encourages States to Install Crash-Tested Roadside 

Safety Hardware, but the Movement to Adopt Improved 

Standards Has Been Slow 8

Crash Testing Is Well Documented and Thorough, but FHWA’s 

Oversight Process Does Not Address Potential Threats to 

Independence 17

Few In-Service Performance Evaluations of Roadside Safety 

Hardware Have Been Conducted, in Part, Due to a Lack of 

Data 27

Conclusions 31

Recommendations 32

Agency Comments 33

Appendix I Objectives, Scope, and Methodology 35

Appendix II Review of FHWA Eligibility Letter Files 40

Appendix III Comments from the Department of Transportation 41

Appendix IV GAO Contacts and Staff Acknowledgments 43

Tables

Table 1: Planned Time Frames for States to Ensure That Newly 

Installed Roadside Safety Hardware Meets the Manual for 

Assessing of Safety Hardware (MASH) Testing Standards 14

Table 2: Number of States That Reported Having Inventory Data 

Collection Efforts by Type of Roadside Safety Hardware 28

Figure

Figure 1: Examples of Types of Roadside Safety Hardware 4

Contents
Page ii GAO-16-575 Roadside Safety Hardware

Abbreviations

AASHTO American Association of State Highway and Transportation

 Officials 

COSO Committee of Sponsoring Organizations of the Treadway 

 Commission COSO

DOT U.S. Department of Transportation

EPA Environmental Protection Agency 

FHWA Federal Highway Administration 

FMVSS federal motor vehicle safety standards 

Green Book Policy on Geometric Design of Highways and Streets 

ISO International Organization for Standardization 

ISPE in-service performance evaluation 

ISTEA Intermodal Surface Transportation Efficiency Act of 1991 

MASH Manual for Assessing Safety Hardware 

NCHRP National Cooperative Highway Research Program 

NHS National Highway System 

NHTSA National Highway Traffic Safety Administration

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-16-575 Roadside Safety Hardware

441 G St. N.W.

Washington, DC 20548

June 8, 2016

Congressional Requesters:

Crashes in which a vehicle leaves the roadway are frequently severe and 

account for the majority of highway fatalities. In 2014, 17,791 fatalities—

54 percent of traffic fatalities in the United States—occurred as a result of 

roadway departures, according to the U.S. Department of Transportation 

(DOT)’s data.1 Roadside safety hardware, such as guardrails and median 

barriers, is meant to reduce the risk of a serious crash. Because of the 

frequency and severity of these incidents, the standards and procedures 

by which roadside safety hardware is tested, installed, and evaluated 

after installation are integral to promoting highway safety. In the last 

several years, there have been a number of serious injuries and deaths 

resulting from crashes into a type of roadside safety hardware, known as 

the ET-Plus guardrail end terminal, which is installed at the end of a 

guardrail and intended to prevent serious injury if it is struck. These cases 

raise questions about the thoroughness of crash testing and oversight of 

roadside safety hardware in general. This crash testing and oversight 

involves multiple parties including those that develop roadside safety 

hardware,2 laboratories that conduct roadside safety hardware crash 

testing, the American Association of State Highway and Transportation 

Officials (AASHTO), the Federal Highway Administration (FHWA), and 

state departments of transportation.

You asked us to examine the framework for overseeing roadside safety 

hardware, including the roles played by FHWA, states, and other parties. 

This report addresses: (1) how FHWA performs oversight of state policies 

and practices related to roadside safety hardware, (2) the thoroughness 

of the laboratory crash-testing process and FHWA’s oversight of this 

process; and (3) the extent to which information is available on roadside 

safety hardware performance once installed.

 1 National Highway Traffic Safety Administration, Traffic Safety Facts Research Note. 

2014 Motor Vehicle Crashes: Overview. Report No. DOT HS 812 246 (Washington DC: 

National Highway Traffic Safety Administration, March 2016).

2 Roadside safety hardware is developed by multiple entities including: commercial 

manufacturers, universities, and state departments of transportation.

Letter
Page 2 GAO-16-575 Roadside Safety Hardware

To address these topics, we reviewed relevant regulations, policy 

documents, and guidance memos from FHWA and selected states on 

roadside safety and design. To determine how FHWA performs oversight 

of roadside safety hardware, we reviewed roadside safety hardware 

standards published by AASHTO and related FHWA guidance, and 

evaluated FHWA’s policies and practices for conducting oversight of 

roadside safety hardware in comparison with federal internal control 

standards for monitoring, designing control activities, and communication 

with external stakeholders.3 To gain perspective on what policies and 

procedures states use to oversee roadside safety hardware’s testing, 

installation, and performance, we distributed a survey to all 50 states, 

plus the District of Columbia and Puerto Rico, and received 44 

responses. Our survey findings represent only the states that responded 

to our survey.

To assess the thoroughness and sufficiency of the crash-testing process 

we requested and reviewed documentation from the nine crash test labs 

in the United States that are accredited to international standards and 

conduct crash testing for the purposes of FHWA review. We also 

reviewed crash-test lab guidance and policy documents. To better 

understand the standards and processes labs are expected to have in 

place and how international laboratory and roadside safety hardware 

testing standards are implemented, we conducted structured interviews 

with officials from each of the nine labs and from three international 

accrediting bodies that accredit test labs. We also reviewed 10 FHWA 

files selected to represent different types of roadside safety hardware for 

years between 1993 and 2014 to better understand the processes by 

which FHWA oversees the testing process.4 To determine the state of 

knowledge of roadside safety hardware performance, we conducted a 

literature review of studies published by state, federal, and academic 

sources between 1993 and 2015. We interviewed officials at FHWA’s 

Office of Safety, and Office of Safety Research and Development, as well 

as state departments of transportation and FHWA’s Division Offices in 

five states (Virginia, Ohio, Texas, Maryland, and California), to obtain 

non-generalizable examples of how federal and state roadside safety 

 3 GAO, Standards for Internal Control in the Federal Government, GAO-14-704G

(Washington, D.C.: September 2014).

4 The year 1993 was selected because it is the year that crash-testing standards that are 

in use today were first developed.
Page 3 GAO-16-575 Roadside Safety Hardware

hardware policies intersect. We selected these states based on the 

presence of an accredited crash-testing facility in the state and 

recommendations from outside experts regarding the quality of 

performance data collection efforts in those states. We visited three of 

these states (Virginia, Ohio, and Texas) to visit crash testing labs and 

spoke with lab personnel, state DOT officials, FHWA division office 

representatives, and other roadside safety hardware stakeholders. 

Because of ongoing litigation, we did not review any issues related to the 

accidents involving the ET-plus guardrail end terminal. See appendix I for 

a full description of our objectives, scope, and methodology.

We conducted this performance audit from April 2015 to June 2016 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe the 

evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

If a vehicle leaves the roadway, ideally, the roadside would be clear of all 

obstructions and be traversable. However, because there are numerous 

roadside areas that cannot be practically cleared of all fixed objects or 

that have sharp declines, roadside safety hardware can be used to 

reduce the consequences of a departure from the roadway.5 The goal of 

roadside safety hardware is met when the hardware contains, redirects, 

or decelerates the vehicle to a safe stop without causing serious injury to 

the vehicle’s occupants or other people. General categories for roadside 

safety hardware are: 1) longitudinal barriers, which include items such as 

guardrails and cable barriers and are intended to reduce the probability of 

a vehicle’s striking an object or terrain feature off the roadway that is less 

forgiving than the barrier; 2) bridge barriers, which function as longitudinal 

barriers but are specific to bridge design; 3) barrier terminals/crash 

cushions, which include items like guardrail end terminals that are 

intended to absorb or divert the energy of a crash into the end of a 

longitudinal barrier; 4) support structures, such as sign supports, which 

are designed to break or yield when struck by a vehicle; and 5) work zone

 5 The roadside is defined, in the AASHTO Roadside Design Guide, 4th Ed. 2011, as the 

area between the outside shoulder edge and the right-of-way limits.

Background
Page 4 GAO-16-575 Roadside Safety Hardware

devices, which include a variety of items used in a work zone that are 

temporary in nature. See figure 1 below for a depiction of these types of 

hardware.

Figure 1: Examples of Types of Roadside Safety Hardware

DOT’s primary mission is to ensure the safety of the traveling public. The 

strategic goals of the FHWA, within DOT, are to provide safe, reliable, 

effective, and sustainable mobility for users of the nation’s highway 

system. FHWA distributes about $40 billion to the states each year 

through the federal-aid highway program (generally providing 80 to 90 

percent of projects’ costs on designated federal-aid highways) for 

highway and bridge infrastructure, a portion of which is spent on safety 

improvements including roadside safety hardware. FHWA issues 

regulations and guidelines, and can perform direct oversight for projects 

that use federal funds, including those on the National Highway System 

(NHS). The NHS consists of approximately 220,000 miles of the nearly 1-
Page 5 GAO-16-575 Roadside Safety Hardware

million miles of roadways eligible for federal aid. The NHS includes the 

47,000-mile Interstate Highway System as well as other roadways, 

connectors important to U.S. strategic defense policy, and connectors to 

major intermodal facilities, such as airports or transit hubs.

FHWA administers and oversees the federal-aid highway program 

through FHWA’s division offices located in all 50 states, the District of 

Columbia and Puerto Rico. As part of FHWA’s risk-based oversight, 

division offices and state DOTs have “Stewardship and Oversight 

Agreements” that specify the terms under which states assume oversight 

responsibility for federally funded projects. Under FHWA’s risk-based 

stewardship and oversight program, FHWA is responsible for determining 

projects that have an elevated risk or projects where FHWA involvement 

can enhance meeting program or project objectives. This involvement 

may include conducting oversight of the entire project or a specific phase 

or element of the project. For all projects that FHWA does not categorize 

as having an elevated risk, responsibility for oversight of design and 

construction of projects is generally assumed by the states. For each 

federally funded project, FHWA enters into project agreements with the 

state in which the state agrees to adhere to all applicable federal laws 

and regulations.

Section 109 of Title 23 of the United States Code directs DOT to work in 

partnership with the state DOTs to develop standards for the NHS and 

other roadway systems.6 To fulfill this responsibility, FHWA works in 

partnership with AASHTO to advance many of its mission areas. 

AASHTO is an association representing highway and transportation 

departments in the 50 states, the District of Columbia, and Puerto Rico. 

AASHTO serves as a liaison between state departments of transportation 

and the federal government and develops and maintains design 

standards for roadways, bridges, and highway materials. FHWA 

incorporates some AASHTO standards into federal regulation,7 for 

example, the Policy on Geometric Design of Highways and Streets 

(Green Book), which lists design criteria across a range of roadway types, 

 6 23 U.S.C. § 109.

7 23 C.F.R. § 625.
Page 6 GAO-16-575 Roadside Safety Hardware

from rural roads to freeways.8 FHWA has an ex-officio, non-voting role on 

AASHTO committees. In cooperation, AASHTO and FHWA sponsor 

research on common transportation issues through the Transportation 

Research Board’s National Cooperative Highway Research Program 

(NCHRP), including research on roadside safety hardware. NCHRP 

studies are funded by the states from federal-aid highway program funds 

apportioned to them.

Roadside safety hardware is developed by manufacturers, states, and 

universities and can be crash tested to assess its safety performance. 

The nine U.S. crash-testing labs that are accredited and recognized by 

FHWA can conduct full-scale crash testing where roadside safety 

hardware is hit by a vehicle to determine whether it meets AASHTOaccepted standards for roadside safety hardware. Of the nine labs, three 

labs are independently operated; two are owned by companies that also 

develop roadside safety hardware; three labs are affiliated with 

universities; and the final lab is operated by a state department of 

transportation. Representatives from roadside safety hardware 

developers, crash test labs, academia, and state and federal 

transportation departments participate in Task Force 13, a committee 

whose mission is to develop specifications for new materials and 

technologies identified for use in highway construction projects.9 As part 

of this mission, Task Force 13 develops, recommends, and promotes 

standards and specifications for roadside safety hardware.

The two ways of assessing this performance are lab crash testing and inservice performance evaluations. Crash tests can quantify performance 

for specific conditions that represent the “worst practical conditions” in 

terms of the speed and angle of the vehicle hitting the hardware. The 

performance of hardware is evaluated in terms of risk to the vehicle 

occupants and structural adequacy of the hardware, among other items. 

AASHTO currently has two sets of crash-testing standards that it 

 8 Other examples of AASHTO standards incorporated by FHWA into federal regulation 

include: Bridge Design Specifications—which govern the design, evaluation, and 

rehabilitation of elements like railings, steel structures, and foundations of bridges—and 

the Policy on Design Standards for the Interstate System, which includes design criteria 

for interchanges, right-of-way, and access controls for the interstate system.

9 Task Force 13 is a committee created in partnership between AASHTO, the Associated 

General Contractors of America, and the American Road and Transportation Builders 

Association. 
Page 7 GAO-16-575 Roadside Safety Hardware

endorses for installing roadside safety hardware: NCHRP Report 350 

standards adopted in 1993,10 which are being phased out, and the 

Manual for Assessing Safety Hardware (MASH) adopted in 2009.11

AASHTO developed the MASH standards as an update to NCHRP 

Report 350, and these standards contain revised criteria for crash tests of 

roadside safety hardware. Updates in MASH include: increases in the 

size and weight of several test vehicles to better match the current vehicle 

fleet, changes to the number of tests and impact conditions, and more 

objective evaluation criteria.

AASHTO has also sponsored research on how to assess the 

performance of roadside safety hardware once it has been installed, an 

assessment that is referred to as in-service performance evaluation 

(ISPE). In 2003, NCHRP Report 490: In-Service Performance of Traffic 

Barriers,

12 published findings of research and suggested practical 

procedures for conducting ISPEs. In-service performance evaluations are 

a way of assessing roadside safety hardware’s performance in “realworld” scenarios not captured in a crash-test setting. For example, 

performance may be affected by installation factors, such as slope and 

grade of roadway and soil type, and maintenance conditions, including 

whether the hardware has degraded over time from weather or accidents, 

none of which are captured in crash-testing.

 10 Transportation Research Board, the National Cooperative Highway Research Program, 

NCHRP Report 350: Recommended Procedures for the Safety Performance Evaluation of 

Highway Features (Washington, D.C.:1993).

11 American Association of State Highway and Transportation Officials, Manual For 

Assessing Safety Hardware, (Washington, D.C.: 2009).

12 Transportation Research Board, NCHRP, NCHRP Report 490: In-Service Performance 

of Traffic Barriers (Washington, D.C.: March 2003).
Page 8 GAO-16-575 Roadside Safety Hardware

FHWA oversees and promotes the installation of crash-tested roadside 

safety hardware through guidance and policy directives to the states and 

by issuing letters to roadside safety hardware developers that submit 

crash-test results for review by FHWA. We found that states generally 

require crash testing; however, some inconsistencies across state 

policies and practices exist, and a movement to adopt the improved 

MASH standards has been slow. In 2016, FHWA and AASHTO released 

a new Joint Implementation Plan stating that states should transition to 

installing only MASH-standard-tested roadside safety hardware in phases 

by 2019. However, some concerns have been raised, and FHWA has not 

developed a plan to track progress of the states and industry in meeting 

the new dates. FHWA has contracted for a full examination of its roadside 

safety hardware oversight processes and expects a report with 

recommendations for potential changes to these processes in the 

summer of 2016.

In line with its overall safety mission as well as that of DOT, FHWA 

encourages states to install appropriately crash-tested roadside safety 

hardware. By law, FHWA is required to ensure that highway projects 

designed and constructed with federal funds are safe.13 FHWA’s Office of 

Safety’s specific mission includes advancing the use of scientific methods 

and data-driven decisions. Also, according to FHWA’s Office of Safety 

website, roadway departure is one of its focus areas. FHWA has issued 

policy that roadside safety hardware should demonstrate acceptable 

crashworthy performance in order to be used on the NHS and receive 

federal-aid reimbursement. To encourage this outcome, FHWA issues 

guidance and policy directives to the states and industry. For example, in 

2015 FHWA issued a memo that encouraged state agencies to upgrade 

their existing installations of guardrail end terminals that had been tested 

to standards issued prior to the NCHRP 350 standards, which were 

adopted in 1993. Congress directs DOT to work in partnership with the 

state DOTs to develop standards for the NHS and other systems.14

FHWA works in cooperation with AASHTO to promote state adherence to 

 13 23 U.S.C. § 109(a) states that the Secretary of the Department of Transportation shall 

ensure that each proposed highway project “is conducive to safety, durability, and 

economy of maintenance...and is designed and constructed in accordance with criteria 

best suited to accomplish these objectives.”

14 23 U.S.C. § 109

FHWA Encourages 

States to Install 

Crash-Tested 

Roadside Safety 

Hardware, but the 

Movement to Adopt 

Improved Standards 

Has Been Slow

FHWA Encourages States 

to Install Appropriately 

Crash-Tested Roadside 

Safety Hardware
Page 9 GAO-16-575 Roadside Safety Hardware

crash-testing standards through joint implementation plans. These plans 

are voted on and must be approved by a majority of AASHTO’s member 

states. FHWA and AASHTO issued joint implementation plans in 2009 

and 2016 that provided guidance for states to follow in transitioning to 

updated crash test standards.

In addition to providing guidance to states, FHWA also issues federal-aid 

reimbursement eligibility letters to roadside safety hardware developers 

that submit their product information, crash test results and other 

supporting documentation for review. Although it is called a federal-aid 

reimbursement eligibility letter, FHWA’s eligibility letter is not required, 

and federal-aid reimbursement is not contingent upon receipt of an 

eligibility letter. FHWA issues these letters as a service to the states to 

provide states with information on the crashworthiness of roadside safety 

hardware. FHWA posts the letters on its website creating a central 

repository of information for states to know which roadside safety 

hardware has been tested. FHWA officials stated that when they receive 

a request from a developer for an eligibility letter, the request includes 

information on the design of the roadside safety hardware device, the 

crash testing report, pictures and videos of the crash testing, and other 

information. FHWA officials told us that they follow up with the developer 

or test lab if they have questions about any of the data or video evidence. 

FHWA also advises developers that if modifications are made to a 

roadside safety hardware device that has received an eligibility letter, the 

developer must resubmit information to FHWA for review.

Though it is FHWA’s policy that all roadside safety hardware installed on 

the NHS should be crash tested, crash testing is not a requirement for 

states to receive federal-aid highway program funds because this policy 

was never incorporated into regulation or other formal agreements with 

the states (such as FHWA’s project agreements with states). According to 

FHWA officials, in the absence of a federal statutory or regulatory 

requirement for crash testing, FHWA cannot withhold federal funding for 

federal-aid highway-program projects’ approvals to a state should the 

state choose to install roadside safety hardware that had not been tested 

to meet appropriate crash test standards. During our review, we found a 

widespread misperception among state DOT and FHWA division office 

officials we spoke with that crash testing of roadside safety hardware to 

applicable standards and obtaining an FHWA eligibility letter was required 

in order to receive federal reimbursement. In 1991, Congress instructed 
Page 10 GAO-16-575 Roadside Safety Hardware

DOT to issue a final rule regarding revised standards for acceptable 

roadside safety hardware.15 In 1993, FHWA issued a rule that 

incorporated crash test standards into regulation by reference as 

guidance.16 FHWA stated at the time that it lacked sufficient knowledge to 

be more prescriptive about roadside safety hardware in general and 

chose not to make crash testing mandatory through regulation.17 FHWA 

has not issued a proposed rulemaking to require crash test standards 

since. FHWA officials told us that they believe encouraging state 

compliance is more effective than requiring it through a rulemaking 

because the current partnership with AASHTO garners support from 

states, and a federal rulemaking can take many years to complete.

Most states that responded to our survey told us that roadside safety 

hardware installed on the NHS is required to be crash tested, and many 

of those states said they had processes in place to limit installation of 

roadside safety hardware to those that have obtained FHWA eligibility 

letters. Nearly all, 43 of the 44 states that responded to our survey, told 

us that crash testing to MASH or NCHRP Report 350 standards is 

required in their state for major categories of roadside safety hardware.18

In addition, 38 of 44 states also responded that they maintain lists of 

“approved” or “qualified” products from which contractors can choose 

roadside safety hardware for installation. Furthermore, 32 of the 38 states 

 15 Intermodal Surface Transportation Efficiency Act of 1991 (ISTEA). Pub. L. No. 102-240, 

§ 1073, 105 Stat. 1914, 2012 (1991).

16 58 Fed. Reg. 38293 (July 16, 1993).

17 This reference to crash test standards as guidance was removed in 1997. 62 Fed. Reg. 

15392 (Apr. 1, 1997).

18 Specifically, these 43 states that responded to our survey told us that crash testing to 

either NCHRP Report 350 or MASH standards prior to installing longitudinal barriers, 

bridge barriers, barrier terminals, and work zone equipment on the NHS is required in their 

state. Forty-one of 44 states indicated that crash testing is required for support structures. 

One state said that crash testing was not required for any category of roadside safety 

hardware.

Most States Said They 

Require Crash Testing, 

though Some 

Inconsistencies Exist, and 

Transition to Improved 

Standards Has Been Slow
Page 11 GAO-16-575 Roadside Safety Hardware

with these lists responded that all roadside safety hardware on their 

qualified or approved product lists have an FHWA eligibility letter.19

While our survey results indicate that FHWA’s guidance has been widely 

implemented at the state level, they also indicate some inconsistencies in 

state policies and some misperception about FHWA policy. First, 10 

states responded that that they do not have a specific law, regulation, or 

policy document that establishes crash-testing requirements. In follow-up 

responses, four of these states told us that they do not have documented 

requirements because they believe FHWA requires crash-testing of 

roadside safety hardware and that the FHWA requirement governs 

roadside safety hardware in their state.20 If a state’s policy is only to refer 

to a federal requirement that does not exist, then effectively no 

requirements govern crash testing in that state. Second, while most 

states approve installation of only roadside safety hardware that has 

received an FHWA eligibility letter, not all states do so. For example, 11 

states reported that they have conducted their own crash testing in the 

past 10 years, and 6 of the 11 responded that they do not always submit 

those roadside safety hardware devices for FHWA review prior to 

approving devices for installation. Officials from one state told us they 

only submit results for eligibility letter review when they believe the device 

is likely to be used by other states.

Federal standards for internal control highlight the need for agencies to 

design control activities—policies, procedures, techniques, and 

mechanisms—to achieve objectives and address related risks.21 In June 

2012, FHWA issued a memo indicating that division offices should 

encourage states to have written policies that incorporate AASHTO’s

guidance on current roadside safety information and operating practices.

However, FHWA has not directed its division offices to help ensure that 

 19 Remaining states identified circumstances under which not all approved products would 

have FHWA eligibility letters. These circumstances include: (1) that a device has been 

crash tested, but an eligibility letter has not been applied for; (2) that there is no 

appropriate crash-tested device for some road conditions in their state; and (3) that the 

device has not been crash tested but has otherwise been shown to be effective. 

20 Five of these 10 states told us that crash-testing requirements for at least some types of 

roadside safety hardware are governed by state-specific policies, whether through design 

standards or the project approval process. One state declined to supply a response, citing 

concerns about ongoing litigation.

21GAO-14-704G.
Page 12 GAO-16-575 Roadside Safety Hardware

states have policies governing crash testing of roadside safety hardware 

installed on their roadways as part of this procedural review. Officials in 

the five FHWA division offices we interviewed told us they have a 

procedure for reviewing states’ standards and design specifications, 

which could include states’ standards and requirements for roadside 

safety hardware. However, officials in FHWA division offices we 

interviewed said that they generally do not examine roadside safety 

hardware practices on individual projects as part of FHWA’s risk-based 

oversight. Officials in one division office noted that topics like pedestrian 

safety would be a higher priority for the division office because officials 

said there are more pedestrian deaths than there are deaths from 

roadside safety hardware. Division office officials stated that they rely on 

the state to ensure that what is incorporated in the project meets state 

standards, and officials from four out of five division offices stated that 

they do not verify that states are installing state-approved products. 

Furthermore, FHWA’s Office of Safety officials told us that they do not 

monitor and collect information on state policies with respect to roadside 

safety hardware. The absence of written requirements at the state level 

and inconsistencies in state practices could, in some cases, result in the 

risk of reduced assurance that states are fully implementing appropriate 

crash-testing standards.

According to FHWA and AASHTO, MASH crash test standards are an 

improved set of standards because they better reflect the current vehicle 

fleet, which has become heavier and taller over the past 25 years. Two 

studies compared the NCHRP Report 350 standard testing to the MASH

test standards. The results of these studies indicated that in some cases 

MASH test standards provide a more rigorous evaluation for crash testing 

roadside safety hardware. First, in 2010, NCHRP conducted an 

evaluation of existing roadside safety hardware devices approved under 

NCHRP Report 350. Re-testing these devices and evaluating 

performance using the criteria in MASH revealed that 6 of the 21 tests 

performed on NCHRP Report 350-compliant roadside safety hardware 

devices did not pass.22 Second, in September 2015, a joint 

AASHTO/FHWA review of guardrail end terminals concluded that the 

MASH crash test standards incorporate tests relevant for guardrail end 

 22 Transportation Research Board, NCHRP, Research Results Digest 349: Evaluation of 

Existing Roadside Safety Hardware Using Manual for Assessing Safety Hardware (MASH) 

Criteria (Washington, D.C.: March 2010).

Transition to Improved 

Standards Has Been Slow
Page 13 GAO-16-575 Roadside Safety Hardware

terminals that are not included in NCHRP Report 350 test standards.23

Specifically, the study found that NCHRP Report 350 standards do not 

fully address performance issues in the areas of side and shallow-angle 

impacts. The study recommended fully implementing MASH for new 

installations of guardrail end terminals.

States have been slow in transitioning to implement the MASH crash-test 

standards. In 2009 AASHTO and FHWA issued a Joint Implementation 

Plan adopting MASH as the updated crash-test standards necessary for 

an applicant to receive an FHWA eligibility letter for a new roadside safety 

hardware device. However, this plan said that states could continue to 

install roadside safety hardware tested to the previous NCHRP Report 

350 standards. Therefore, manufacturers could continue to produce, and 

states could continue to install roadside safety hardware that had already 

received an eligibility letter without retesting to MASH crash test 

standards. In January 2016, FHWA and AASHTO released a new Joint 

Implementation Plan stating that states should transition to installing only 

MASH-standard-tested roadside safety hardware. According to the plan, 

FHWA will no longer issue eligibility letters for new or modified roadside 

safety hardware tested to standards other than the MASH crash-test 

standards. The 2016 Joint Implementation Plan calls for states to 

complete the transition to the MASH crash-test standards between 

December 2017 and December 2019, depending on the type of 

hardware. (See table 1 below.)24 If states comply with the 2016 Joint 

Implementation Plan’s dates for transitioning roadside safety hardware 

installations to meet the MASH crash-test standards, this transition will be 

8 to 10 years after the 2009 Joint Implementation Plan, and states may 

continue to install non-MASH-tested hardware on the NHS until 

December 2017 at the earliest. FHWA officials noted that roadside safety 

hardware often remains on the roads for at least 20 years before being 

replaced due to aging, so hardware tested to the older NCHRP Report 

350 standards could be on the roads for years to come.

 23 Joint FHWAA-AASHTO Task Force on Guardrail Terminal Crash Analysis, Safety 

Analysis of Extruding W-Beam Guardrail Terminal Crashes (Washington, D.C.: 

September, 2015).

24 FHWA officials said that the dates reflected discussions between AASHTO/FHWA and 

industry regarding the likely time frames for making MASH-compliant hardware 

commercially available.
Page 14 GAO-16-575 Roadside Safety Hardware

Table 1: Planned Time Frames for States to Ensure That Newly Installed Roadside Safety Hardware Meets the Manual for 

Assessing of Safety Hardware (MASH) Testing Standards

Date Type of roadside safety hardware

December 31, 2017 Guardrails and concrete barriers

June 30, 2018 Guardrail terminals

December 31, 2018 Cable barriers, cable barrier terminals, and crash cushions

December 31, 2019 Bridge rails, transitions, all other longitudinal barriers (including portable barriers installed 

permanently), all other terminals, sign supports, and all other breakaway hardware

Temporary work zone devices, including portable barriers 

Source: Association of State Highway and Transportation Officials/Federal Highway Administration Joint Implementation Plan, January 7, 2016. | GAO-16-575

However, at this point it is not clear that states will be able to comply with 

the dates set in the plan. In order to meet the transition dates, industry will 

have to develop and test products to the MASH standards that have not 

previously been tested to these standards, and FHWA will have to review 

applications-for-eligibility letters for developers that request them. States 

will then have to make changes to either their design and specification 

policies or approved lists of products to incorporate only MASH-tested 

roadside safety hardware. Industry, to this point, has been slow to move 

to develop and test products to the MASH standards. Using eligibility 

letters as an indicator, as of March 2016, there are currently only two 

guardrail end terminals with eligibility letters that have been tested using 

the MASH standards, compared to the13 guardrail end terminals tested to 

NCHRP Report 350 with eligibility letters. In the category of longitudinal 

barriers, there were only 17 MASH-compliant eligibility letters among the 

348 active eligibility letters. In an open letter to AASHTO, the American 

Traffic Safety Services Association, an association representing highway 

safety industries, expressed concern with the ability for industry to have 

enough hardware that meets the MASH crash-test standards by the 

transitions dates, as well as the ability of states to approve new hardware 

and for FHWA to post new eligibility letters in a timely manner.

FHWA officials told us that states and manufacturers have responded 

positively to the new deadlines. However, FHWA officials did express 

some concern as to whether states will be able to fully implement MASH

standards by the dates in the 2016 Joint Implementation Plan. Their 

concerns included the need for the market to react in a timely manner and 

have enough products available to support competition, and to invest in 

testing categories of roadside safety hardware that have had little testing 

to MASH standards to this point. FHWA officials also told us that as 

industry reacts to the dates, FHWA will likely have an influx of requests to 

review eligibility letters; FHWA officials told us that they already have a 
Page 15 GAO-16-575 Roadside Safety Hardware

backlog of eligibility letter applications since FHWA stopped issuing 

eligibility letters for modifications to hardware tested to non-MASH

standards at the end of 2015.

Federal standards for internal control highlight the need for agencies to 

obtain information needed to achieve their objectives from external 

parties, including significant matters related to risks.25 FHWA officials 

stated they will be in a better position in a year to say whether states are 

likely to be able to successfully transition to MASH crash-test standards 

by the dates specified in the January 2016 Joint Implementation Plan. 

However, FHWA has not developed a plan to track progress of the states 

and industry in meeting the new dates. Moreover, we found that FHWA 

and states currently do not collect information that would assist in 

monitoring the transition to MASH standards. For example, as discussed 

in the following section, FHWA can interact with developers and crash 

test labs during the test process, but FHWA does not collect information 

from developers and labs to be informed when hardware that was 

previously tested to older standards is re-tested to MASH and fails. 

Without this information on test failures, FHWA and states may be 

unaware of setbacks to the transition. Also, if states do not have this 

information, it may result in the states unknowingly installing failed 

hardware during the transition period. In addition, 12 states responded to 

our survey that they currently do not require developers to notify them of 

modifications made to an approved device. While FHWA requests such 

notification, 3 of the 12 states do not have eligibility letters for all 

approved devices and could be unaware of design changes. Federal 

standards for internal control also highlight the need for agencies to 

provide quality information to external parties, including the general public 

to help achieve agency objectives.26 Monitoring and reporting industry 

and state progress to the goal dates set in the 2016 Joint Implementation 

Plan would allow FHWA to keep decision makers in both DOT and 

Congress aware of progress. Such monitoring and reporting of progress 

would also position FHWA to take corrective actions as needed to better 

assure that states and industry are successfully moving to meeting 

improved standards.

 25 GAO-14-704G.

26 GAO-14-704G.
Page 16 GAO-16-575 Roadside Safety Hardware

FHWA contracted in May 2015 with DOT’s Volpe National Transportation 

Systems Center to conduct a full review of its roadside safety hardware 

oversight process and expects a report with recommendations for 

potential changes to its oversight program in summer 2016. Officials 

stated the review will include a full examination of the process by which 

roadside safety hardware is developed, evaluated, funded, and assessed, 

as well as recommendations for any improvements needed. Specifically, 

the report will include:

• documentation of existing laws, regulations, policies, standards, and 

guidelines associated with the roadside safety hardware process;27

• documentation and review of all the steps in FHWA’s current crashtesting evaluation process; and

• findings and recommendations to FHWA to improve its oversight.

FHWA officials told us that there may be ways to improve the agency’s 

oversight of roadside safety hardware and that everything in the process, 

from the partnership relationship with AASHTO to the eligibility letter 

process, will be included in the review. During the course of our review, 

FHWA implemented some changes to its program, such as clarifying the 

need for any modifications to hardware with eligibility letters to be 

reevaluated, but FHWA officials stated they were holding off on major 

changes to the current oversight program until the Volpe National 

Transportation Systems Center’s review is complete.

 27 Included in this review will be documentation of existing processes and procedures at 

the federal, state, association, testing facility, and any other relevant levels. 

FHWA Is Currently 

Reviewing Its Oversight 

Program
Page 17 GAO-16-575 Roadside Safety Hardware

At all nine U.S. labs accredited to conduct crash testing of roadside safety 

hardware for FHWA review, laboratory crash testing was well 

documented and thorough in terms of consistency in documentation and 

test procedures across labs. As part of the crash-testing process, labs 

and test sponsors have discretion in making testing decisions in several 

important areas. In addition, there is an inherent potential threat to 

independence in the testing process because employees in some labs 

can test devices that were developed within their parent organization. The 

independence requirement in the standards used to accredit labs is 

general, and we found varying interpretations and differences in 

approaches for mitigating threats to independence across the labs. 

FHWA does not require third party verification of crash testing and does 

not make its own pass/fail determinations or provide for independent 

pass/fail determinations for test results. FHWA also does not provide 

additional guidance to labs and accrediting bodies on independence 

mitigation measures for crash testing roadside safety hardware. We found 

that some other federal agencies with similar testing programs have more 

measures than FHWA has to mitigate potential risks to independence.

FHWA requires that crash test labs conducting testing for the purposes of 

FHWA eligibility letters be accredited to International Organization for 

Standardization (ISO) 17025 standards,28 which contain management 

and technical requirements for labs to be deemed competent to run 

laboratory testing. There are nine crash test labs in the United States that 

are accredited to these standards and conduct crash testing for the 

purposes of FHWA eligibility letters. Our review of the nine accredited 

U.S. labs found that individual crash tests were well documented and 

thorough because test reports contained documentation that would allow 

a third party to understand how the lab conducted the test and how the 

test results were interpreted. To evaluate the thoroughness and 

documentation for labs’ crash testing, we created both interview 

questions29 and a document request list30 for all the labs based on 

 28 23 C.F.R. § 637.209(a)(5).

29 Our questions addressed how labs select the test equipment and testing environment, 

how they interpret test results, how they document each test, how they comply with 

conflict of interest requirements in the ISO, and about their communication protocols with 

FHWA.

Crash Testing Is Well 

Documented and 

Thorough, but 

FHWA’s Oversight 

Process Does Not 

Address Potential 

Threats to 

Independence

Lab Crash Tests Are Well 

Documented and 

Thorough
Page 18 GAO-16-575 Roadside Safety Hardware

international accreditation requirements as well as the crash-testing 

guidelines in MASH. All nine example test reports31 we reviewed clearly 

identified the test standard32 and the test level33 the lab used to test the 

roadside safety hardware device. The pass/fail criteria being used to 

evaluate the roadside safety hardware device was clearly identified, and 

all reports described the test results against each of the evaluation 

criteria. In addition, all reports described the setup of the device and pretest procedures, which could include verifying the integrity of the soil, 

when applicable, and structural integrity of the test vehicle. Each report 

also included between 20 and approximately 100 pictures of the testing 

process, along with a description of the results. For more information on 

the documents we requested and reviewed, see appendix I.

Labs generally described using requirements specified in test or 

accreditation standards as the basis for their procedures. Labs described 

sending equipment out to a qualified calibration laboratory, or obtaining 

additional expertise and certification to calibrate their own equipment. 

Several labs also stated that they keep the test objects on site for a 

period of time, in case follow-ups were needed. Specifically, five labs told 

us that they kept test documentation on file for at least 2 years, and in 

three of these cases, kept records indefinitely. Labs also described going 

beyond what standards require in certain instances. For example, five of 

the nine labs described using additional cameras or data recording 

devices to better capture data that would be useful to industry research or 

to the customer.

Accrediting bodies are expected to use the ISO 17025 standards, along 

with test standards specific to the industry, such as the MASH crash-test 

standards in this case, as the basis for accrediting crash test labs.

Officials from three labs said they had developed documentation 

practices specifically in reference to the accreditation process. Each 

accreditation body said that it conducts routine audits each year as part of 

 30 We asked for corresponding test plans and test reports, as well as the quality manual, 

which documents the policies and procedures, and the accreditation report. The test 

reports requested were representative of a report that would be submitted to FHWA as 

part of an eligibility letter request.

31 Each lab provided one example test report.

32 Such as NCHRP Report 350 or the MASH crash-test standards.

33 Such as “Test Level 3,” which means the roadside safety hardware can be used on high 

speed roadways.
Page 19 GAO-16-575 Roadside Safety Hardware

its accreditation cycle,34 where accreditation bodies told us they evaluate 

such aspects of the testing process as setup, equipment calibration, 

competence of personnel, documentation, and record keeping. One lab 

reported that its accrediting body assisted it with improved lab procedures 

by setting up calibration procedures; two labs reported that accreditation 

requirements guide their policies on document retention. In addition, 

accreditation standards require labs to collaborate and compare results in 

inter-laboratory collaborations, a procedure that labs do via Task Force 

13, in order to work toward greater consistency in test procedures and 

results interpretation.35

Although individual crash tests are well documented, full-scale crash 

testing to evaluate the performance of an individual piece of hardware is a 

complex process that requires labs to use professional judgment when 

deciding which tests need to be run, and how to interpret the results. Both 

NCHRP Report 350 and MASH have a suite of tests in order to cover a 

range of crash speeds, angles, and size and weight of vehicles to assess 

the performance of the roadside safety hardware device. A majority of 

labs (five of nine) reported that they usually recommend to test sponsors 

that they run the full suite of tests outlined in the test standard.36

However, for modified devices that have previously been tested, there is 

some discretion on which tests to run. Because an individual full-scale 

crash test can cost about $55,000 (according to a crash test lab we spoke 

with) it is advantageous to only run what test sponsors think are the most 

critical tests for a given device. For example, in one of the testing 

scenarios we reviewed, the lab engineers determined based on prior 

testing with a larger vehicle that the MASH test for small cars would not 

be necessary for the tested device. Although this decision is documented 

 34 According to the accrediting bodies we spoke with, full accreditation audits may occur 

every 2 or 3 years, depending on the accrediting body, but all three accrediting bodies we 

spoke to conduct at least yearly monitoring audits. 

35 At the October 2015 Task Force 13 meeting, members discussed the results of an interlaboratory comparison that analyzed how different crash-testing labs measured the hood 

heights of test vehicles. Participants voted to standardize hood height measurements 

across labs, though this proposal will need to be approved by AASHTO. Participants said 

that since hood height affects the angle of the test, this is important to standardize across 

the labs.

36 In this report we refer to “test sponsors” as including manufacturers, state DOTs, and 

universities. 

Labs Use Professional 

Judgment in Some CrashTest Selection and 

Interpretation of Test 

Results
Page 20 GAO-16-575 Roadside Safety Hardware

in the test report, the reasoning is not detailed, and so it is hard for a 

reviewer to evaluate this decision. Of the nine labs we interviewed, four 

told us that they frequently consult with FHWA in the test-planning 

process, and that these labs generally run the tests in agreement with 

FHWA. The other labs told us they either rarely or never consult with 

FHWA; these labs encourage test sponsors to communicate directly with 

FHWA if they plan to seek an eligibility letter, and in these cases the lab 

runs the tests the sponsor requests. As part of the eligibility letter 

process, labs can, but are not required to, consult with FHWA for advice 

on which tests to run.

Labs have some discretion in interpreting the test results against the 

pass/fail criteria of the crash test standard. According to MASH crash test 

standards, some interpretation will be necessary for the criteria due to the 

“very complex nature of vehicular collisions and the dynamic responses of 

an occupant to the collision, as well as human tolerances to impact.”37

Eight of the nine labs reported that engineering judgment was necessary 

to make a pass/fail decision in at least a small minority of tests, although 

one lab reported that up to 30 percent of all NCHRP Report 350 or MASH

crash tests require professional judgement. One lab noted that MASH has 

more specific criteria than NCHRP Report 350, but leaves room for 

interpretation when it comes to defining failure limits for penetration,38

occupant intrusion,39 and deformation limits,40 which are all part of the 

occupant risk criteria. For example, six of the labs reported that occupant 

intrusion standards were the main subjective parameter, because 

characteristics such as the amount, type, and location of the intrusion 

were important in determining whether the occupant could be harmed. 

Lab officials told us that MASH standards, which specify maximum 

allowable levels of occupant intrusion, do not always address applied 

testing scenarios. For example, officials in one lab described a test on a 

post that sliced and made holes in the floor pan of the test vehicle. The 

 37 Manual for Assessing Safety Hardware.

38 Penetration is when a component of the test article penetrates into the occupant 

compartment. 

39 Occupant intrusion occurs when the occupant compartment is deformed and reduced in 

size, but no actual penetration is observed.

40 The potential for serious injury varies by the area of the vehicle damaged. For example, 

deformation in the roof area may potentially be more serious than the wheel/foot well area 

due to the proximity of the head to the roof. 
Page 21 GAO-16-575 Roadside Safety Hardware

lab officials said they interpreted this to be a failure, although lab officials 

suggested that MASH crash test standards do not specify whether holes 

in the floor of the vehicle mean the test fails.

In the roadside safety hardware-testing community there is an inherent 

potential threat to lab independence because there is often not a formal 

separation between design and testing roles within a lab’s parent 

organization. Specifically, six of the nine crash test labs we reviewed can 

test products that were developed by employees of the same parent 

organization. Two manufacturer-owned labs can test products created by 

another division of the same company; three university-run labs can test 

university employees’ designed products; and a state-based facility tests 

products designed by the state department of transportation. In order to 

be an accredited lab, the ISO requires labs to identify any conflicts of 

interest and have policies to ensure labs are free from undue pressure. 

The three accrediting bodies we interviewed told us that the ISO 

requirements are usually met by documented conflict-of-interest policies 

and by having an organizational structure in which lab employees do not 

have conflicting lines of reporting to their parent organization. However, 

documentation we obtained and interviews we conducted with labs and 

accreditation bodies revealed varying interpretations of the level of 

involvement of the device designer in the crash-testing process, and of 

the level of involvement of the lab in providing design feedback based on 

crash test results, that is appropriate to ensure independence. For 

instance, while four labs told us they would offer advice on how to redesign the device if it failed a crash test, the other five labs said they did 

not make such recommendations, and one specifically said it interprets 

ISO standards to mean that labs should not be involved in making design 

recommendations after testing. The ISO standards are intended to 

broadly cover testing and calibration laboratories across many industries 

to ensure technical competence. Varying interpretations suggest a lack of 

specificity in ISO requirements to ensure independence in the testing of 

roadside safety hardware.

Of the six labs that test devices developed within their parent 

organization, two labs told us they have policies that formally separate the 

role of the designer and the tester, although only one had this policy 

documented. One of the two labs designates an independent approving 

authority to make the final determination of whether the hardware passed 

or failed each test and specifies that this person could not have been part 

of the design or development of the hardware. The other told us that if a 

member of the lab was involved in the design of a device, that person 

Potential Threats to Lab 

Independence
Page 22 GAO-16-575 Roadside Safety Hardware

would not be allowed to make the pass/fail determination.41 However, the 

other four labs do not have a separation that is this clear. Two of these 

labs provided us with the general conflict of interest policies of their 

parent organizations, and two labs pointed us to conflict-of-interest 

policies in their quality manuals, which did not have information about 

separating design and testing. The Committee of Sponsoring 

Organizations of the Treadway Commission (COSO), a joint initiative of 

multiple private-sector-accounting organizations, publishes the Internal 

Control-Integrated Framework to help organizations design internal 

controls to achieve their objectives.42 This framework highlights the 

importance of the separation of duties within an organization, to reduce 

the risk of inappropriate conduct in the pursuit of objectives. The standard 

states that when selecting and developing control activities, management 

should consider whether duties are divided or segregated among different 

people to reduce the risk of error or inappropriate or fraudulent actions. 

Labs that do not have this formal separation between design and testing 

functions could have threats to the independence of their test analyses.

One of the three accrediting bodies told us that independence can be 

difficult to assess because it is not clear what labs that are affiliated with 

manufacturers, for instance, must do to mitigate any conflicts of interest. 

Officials from two accrediting bodies told us that other federal agencies 

provide them with additional guidance on independence and technical 

expertise, respectively, and one accrediting body told us that it is 

preferable when an agency provides guidance so that the accrediting 

body can better apply standards when accrediting labs in a specific 

industry. For example, officials from this accrediting body provided an 

example of a federal agency that has developed more specific ethics and 

integrity requirements than the ISO. Accrediting body officials told us that 

this agency requires the accrediting body to assess the labs to these 

more specific requirements.

 41 The lab told us it is planning to revise its quality manual to document this policy.

42 Committee of Sponsoring Organizations of the Treadway Commission, Internal ControlIntegrated Framework (2013).
Page 23 GAO-16-575 Roadside Safety Hardware

Federal standards for internal controls state that agencies should 

establish policies and procedures to respond to risks as part of their 

internal control system.43 However, apart from the accreditation 

requirement, FHWA does not have other mitigation measures in place 

with regard to lab independence. FHWA does not provide guidance to 

crash test labs or accrediting bodies on mitigating the risks posed by 

threats to independence. Providing such guidance could provide greater 

assurance that crash-testing is being performed in an independent, 

unbiased fashion.

FHWA reviews crash test results as part of its eligibility letter process; 

however, FHWA does not have a process for formally verifying the testing 

outcomes and making or providing for an independent pass/fail 

determination. FHWA relies heavily on the labs to determine whether the 

crash test outcome results in a pass or fail determination for roadside 

safety hardware. According to ISO standards for accreditation, when a lab 

states whether a product complies with requirements, it is offering an 

opinion, and it must be marked as such. Officials from one accrediting 

body said it would be preferable for labs to provide only the crash-test 

result data and have a third party apply criteria in MASH crash test 

standards and make the pass/fail determinations. Officials at one lab we 

spoke to added that they prefer not to make pass/fail determinations, but 

they do so for each test. In the eligibility letter process, FHWA requires 

that lab personnel apply the results to relevant crash-test standards and 

make a pass/fail determination of the test results. FHWA officials 

explained that as part of their eligibility letter-review process, they 

examine the crash test lab report, including pictures, videos, and the test 

data summary sheets. If FHWA officials have questions, they will contact 

the lab or developer. However, eligibility letters state that FHWA is relying 

on the assessment of the lab. Moreover, we reviewed 10 case files for 

eligibility letters issued between 2005 and 2015 and found that 

documentation was not sufficient to determine the rationale behind 

FHWA’s decision to issue these letters. For more information on our 

review of FHWA’s eligibility letter process, see appendix II.

FHWA officials acknowledged that lab employees’ testing devices that 

were developed within their greater parent organization poses the 

appearance of an independence threat. In May 2015, FHWA issued a 

 43 GAO-14-704G.

FHWA Does Not Mitigate 

Potential Threats to 

Independence
Page 24 GAO-16-575 Roadside Safety Hardware

memo directing the developer and test labs to submit financial conflict-ofinterest information in order for a developer to receive an eligibility letter. 

FHWA officials told us that this information will not influence a device’s 

ability to receive an eligibility letter, but that the information could be 

published along with the final eligibility letter for the public to review, in an 

effort to increase transparency. FHWA officials also told us that this was 

an immediate change they could make but that they are awaiting the 

results of the Volpe National Transportation Systems Center’s review 

before deciding whether to take additional steps in this area.

According to federal internal control standards, agencies should ensure 

that they communicate quality information to external parties so they can 

help the agency achieve its objectives and address related risks.44

However, as explained above, there is a potential threat to independence 

in the lab crash-testing environment for roadside safety hardware. In 

other test settings we found that federal agencies require third party 

verification of test results or independent entities to make pass/fail 

determinations.

We found that both the Environmental Protection Agency (EPA)—in its 

ENERGY STAR Program—and the National Highway Traffic Safety 

Administration (NHTSA)—in its testing for Federal Motor Vehicle Safety 

Standards and the New Car Assessment Program—have stricter 

oversight over the lab-testing process and require third party certification 

and/or verification testing. EPA’s ENERGY STAR Program is a voluntary 

program to identify and promote energy-efficient products and buildings. 

Lab testing of products is conducted to determine whether a product 

meets program specifications for efficiency. As we’ve previously found, 

the testing requirements for EPA’s ENERGY STAR program have 

evolved in response to weaknesses identified in the program by us in 

2007 and EPA’s Office of Inspector General in 2008, including a lack of 

assurance that tested products met the qualification criteria.45 In response 

to these findings, EPA and the Department of Energy signed a 

memorandum of understanding in 2009 to propose several program 

enhancements. As part of a review of this program, before these changes 

 44 GAO-14-704G.

45 GAO, Energy Star: Providing Opportunities for Additional Review of EPA’s Decisions 

Could Strengthen the Program, GAO-11-888 (Washington, D.C.: Sept. 30, 2011).

Other Agencies Have 

More Oversight of Lab 

Testing
Page 25 GAO-16-575 Roadside Safety Hardware

had been implemented, GAO submitted fictitious products for certification 

and found that the program was vulnerable to fraud and abuse because 

manufacturers could self-certify that their devices met energy standards 

without third-party verification.46 In 2011, we found that EPA had made 

considerable progress in addressing these issues by including verification 

testing and third party certification in the approval process.

Currently, in order to earn an ENERGY STAR label, products must be 

tested by EPA-recognized laboratories, and a subset of products is 

verified annually by third-party certification entities. EPA standards 

require labs, their accrediting bodies, and third-party certification body 

laboratories that verify test results to abide by respective sets of 

conditions and criteria in order to be recognized by the ENERGY STAR 

Program. EPA also has an application process for all three types of 

entities to receive ENERGY STAR Program recognition. Under this 

process, EPA requires that labs test products for review by third-party 

certification bodies, which determine if the devices meet the program 

standards for a product to carry an ENERGY STAR label. EPA’s 

standards for ENERGY STAR recognition also require certification bodies 

to verify lab test data and make a pass/fail determination, and EPA 

officials added that the labs themselves are not supposed to make this 

decision. In addition, certification bodies are to conduct verification testing 

for a sampling of devices, including off-the-shelf devices, across multiple 

categories each year. EPA officials told us they closely oversee the 

certification bodies through frequent communication and periodic audits.

We also interviewed officials at NHTSA regarding two forms of vehicle 

crash testing that they oversee. First, NHTSA issues federal motor 

vehicle safety standards (FMVSSs),47 with which vehicle manufacturers 

must comply, and manufacturers must self-certify that their products meet 

these standards.48 NHTSA then sponsors verification testing of some 

vehicle models, where they purchase vehicles from dealer lots and 

subject them to crash testing to confirm the manufacturer’s certification. 

The verification testing is conducted by labs selected by NHTSA. NHTSA 

 46 GAO, Energy Star Program: Covert Testing Shows the Energy Star Program 

Certification Is Vulnerable to Fraud and Abuse, GAO-10-470 (Washington, D.C.: Mar. 5, 

2010).

47 49 C.F.R. Part 571.

48 49 C.F.R. Part 567.
Page 26 GAO-16-575 Roadside Safety Hardware

officials told us that they would not select a lab that has the potential 

conflict of interest of being a part of a manufacturer business. Once the 

testing is conducted, the raw data is sent to NHTSA and NHTSA officials 

make the determination as to whether the vehicle has met the FMVSSs.

Testing for NHTSA’s New Car Assessment Program is similar to that of 

the FMVSSs in design, but rather than checking whether vehicles meet 

minimum safety standards, NHTSA awards vehicle models with up to 5 

stars for their safety performance in crash testing to standards that, 

according to NHTSA officials, typically exceed those in the FMVSSs. 

Officials stated that because these standards are not federal 

requirements, vehicle manufacturers do not have to comply or self-certify 

compliance. Officials noted, however, that because the New Car 

Assessment Program provides safety ratings information to consumers

the manufacturers have an incentive to receive the highest safety rating 

possible. Similar to testing under the FMVSSs, officials said that NHTSA 

purchases vehicles from dealer lots and then tests them at selected labs. 

The crash-test data is then sent to NHTSA where NHTSA officials 

determine the star rating for each test vehicle.

In contrast to these programs, FHWA does not require either third party 

certification or verification of crash testing; nor does FHWA provide 

additional guidance on independence mitigation measures for crash 

testing roadside safety hardware. Establishing a process for third-party 

verification of crash test results could provide greater assurance that 

threats to independence are fully addressed. FHWA officials told us that 

they would favor considering some form of third party review over crash 

test results. These officials added that having FHWA conduct the third 

party review could be challenging and that FHWA would need to assess 

the resources, technical capacity, and legal capacity to perform that role.
Page 27 GAO-16-575 Roadside Safety Hardware

According to FHWA and AASHTO-sponsored research, in-service 

performance evaluations (ISPE) are recommended for effective roadside 

safety hardware oversight because real-world crash conditions, such as 

vehicle characteristics, as well as the terrain of the roadway, may vary 

widely from those experienced in crash testing.49 Moreover, crash testing 

cannot fully replicate the effects of installation conditions over time on 

roadside safety hardware’s performance. In establishing a methodology 

for conducting ISPEs, NCHRP Report 490 states that collecting crash 

data over multiple years and examining crash sites in real-time can 

enable researchers to report more information on roadside safety 

hardware’s installation and maintenance issues, the costs associated with 

making repairs to damaged hardware, and the severity of injuries 

resulting from crashes that involve roadside safety hardware.50 This can 

better equip states to make cost-benefit determinations regarding 

roadside safety hardware replacement or new product development. 

ISPEs can also inform whether crash-testing standards are appropriately 

suited to assessing the effectiveness of roadside safety hardware.

Based on our review of studies published since 1993,51 when FHWA 

recognized NCHRP Report 350 testing standards, few formal ISPEs of 

roadside safety hardware have been conducted to fully assess the 

performance of roadside safety hardware in actual conditions. After 

reviewing government, industry, and academic sources, we found 14 

formal ISPEs that were published since 1993. While other studies 

included elements of an in-service performance evaluation, 14 studies in 

our review combined crash data analysis with real-time visits to crash 

sites to document and assess the damage, which is a key characteristic 

of a formal ISPE as defined by NCHRP Report 490. Additionally, these 

ISPEs tended to focus on longitudinal barriers, such as guardrails and 

cable barriers, and barrier terminals, such as guardrail end terminals, 

while other types of roadside safety hardware were generally not the 

subject of ISPEs.

 49 NCHRP Report 490.

50 NCHRP Report 490.

51 As we describe in appendix I, we reviewed studies using NCHRP Report 490’s

guidelines for conducting ISPEs, specifically with regard to the analysis of crash data and 

use of real-time crash site visits. We also assessed whether the studies addressed a 

specific type of roadside safety hardware. If a study contained all of these elements, we 

considered it a formal ISPE.

Few In-Service 

Performance 

Evaluations of 

Roadside Safety 

Hardware Have Been 

Conducted, in Part, 

Due to a Lack of Data
Page 28 GAO-16-575 Roadside Safety Hardware

A key challenge to states conducting ISPEs appears to be the lack of 

fully-developed data. As NCHRP Report 490 indicates, having inventory 

data on the number of roadside safety hardware devices being studied 

and their location is critical to calculating rates of collision with roadside 

safety hardware within the study area. In our survey of state DOTs, we 

asked officials to describe their data and inventory efforts, and states 

reported a general lack of established inventory data. As table 2 shows, a 

majority of states indicated that they have inventory data-collection efforts 

for barrier terminals/crash cushions, for example, but many of these 

efforts are new or are ongoing and therefore are not fully established. For 

instance, of the 29 states that reported in response to our survey that they 

have inventory data-collection efforts for barrier terminals/crash cushions, 

18 said that their efforts are ongoing, and 12 of these said that they had 

only been collecting data since 2014. State DOT officials we interviewed 

in four states also told us that inventory data they collect may not include 

information on condition or location of roadside safety hardware, which as 

NCHRP Report 490 notes is necessary for a full understanding of 

performance. The other key piece of data is crash data. The current state 

of crash data reporting may not facilitate conducting ISPEs of roadside 

safety hardware. According to NCHRP Report 490, police will likely not 

comment as part of their crash records on factors like soil conditions, 

which could influence how guardrail posts, for instance, function in a 

crash. Police crash records also do not capture any unreported collisions 

and may not consistently document the type of roadside safety hardware 

involved in an accident.

Table 2: Number of States That Reported Having Inventory Data Collection Efforts by Type of Roadside Safety Hardware

Type of roadside safety hardware

Number of states with data collection efforts

(of those 44 states that provided survey responses)a

Longitudinal barriers (e.g. guardrails, cable barriers) 25

Bridge barriers 20

Barrier terminals/crash cushions (e.g. guardrail end terminals) 29

support structures (e.g. sign supports) 18

Work zone equipment 3

Source: GAO survey of state Departments of Transportation. | GAO-16-575

a

One state did not answer the question with regard to “Barrier terminals/crash cushions.”
Page 29 GAO-16-575 Roadside Safety Hardware

According to our survey, only 6 of 44 states that responded said that they 

had conducted any formal ISPEs in the last 10 years.52 State officials we 

interviewed also described less formal efforts to evaluate roadside safety 

hardware’s performance. For instance, officials in one state told us that 

they perform a trial run for any new proprietary roadside safety hardware 

device in a sampling of locations and monitor on-site the in-service 

performance for 12–18 months prior to approving the device to be used 

by contractors across the state. However, state officials told us this effort 

is not published in a report. Without published results that document a 

methodology that others can repeat, however, such results do not 

ultimately add to the broader knowledge base of ISPEs.

Officials in four of the five states we interviewed indicated that they have 

cost and/or data constraints related to collecting the necessary data to 

conduct formal ISPEs. Officials from the fifth state we interviewed 

described a software application they developed to inventory all of the 

guardrail end terminals in their state. According to state officials, local 

maintenance crews in the state use a custom web application on a mobile 

device to record the total number, along with data on the type and 

location, of guardrail end terminals in their state. This data is then 

uploaded to a central database. State officials said they were planning to 

make this a long-term project and apply it to other types of roadside 

safety hardware. Officials noted that they are still in the process of adding 

the capability for keeping the data up to date. These officials also told us 

that the application was relatively inexpensive to develop, and FHWA 

officials noted that at least one state was interested in learning more 

about the application. State officials told us that as of yet, however, this 

technology has not been shared across states.

 52 We asked states whether they had conducted a formal ISPE in the last 10 years for 

longitudinal barriers, bridge barriers, barrier terminals/crash cushions, support structures, 

and/or work zone equipment. 
Page 30 GAO-16-575 Roadside Safety Hardware

FHWA has ongoing research to identify best practices for the collection of 

data on roadside safety hardware. However, this research is limited to 

guardrail end terminals, and the planned scope of work may not be 

sufficient to fill the gaps created by the lack of ISPE literature at the state 

level. FHWA officials told us that in the summer of 2015, FHWA began a 

pilot study on the collection of data on guardrail end terminals’

performance. According to FHWA officials, the first phase of this pilot 

study is expected to last through the end of 2016. Officials plan to identify 

current challenges to conducting ISPEs as well as recommend best 

practices for: 1) the collection of real-time data on crashes involving 

roadside safety hardware; 2) interagency communication at the state level 

regarding crash reporting; and 3) data management regarding hardware 

maintenance and location. FHWA is currently collecting inventory and 

crash data in four states (Missouri, Pennsylvania, Massachusetts, and 

California) that have agreed to participate in this pilot. FHWA officials 

stated that within a selected area of each state, data will be collected by 

examining crash sites for six different models of guardrail end terminals. 

This data could produce information needed to assess the performance of 

the devices with respect to the risk of severe occupant injury if the study 

were to be continued. According to FHWA officials, crash specialists from 

NHTSA, the agency that collects and reports data on fatal crashes for 

DOT, will conduct detailed on-site investigations for fatal and serious 

injury crashes, generally within 24 hours of receiving notification of the 

crash. Data on crashes resulting in property damage only and other minor 

crashes will also be collected. Officials told us that they plan to continue 

collecting data through 2016 for this phase of the project.

According to FHWA officials, however, publishing findings on the 

effectiveness of guardrail end terminals’ performance is not part of their 

current efforts because they first want to provide guidance to states on 

best practices for performance data collection. Officials noted that the 

time frame for the current phase of the pilot would be insufficient to collect 

enough data for statistically significant findings. FHWA officials told us 

that they will not determine whether to include performance findings as 

part of future phases of the pilot study until this phase is complete at the 

end of 2016.

As noted previously, FHWA’s Office of Safety includes in its mission the 

need to advance the use of scientific methods and data-driven decisions 

in highway policy. The current lack of in-service performance findings and 

established inventory data for roadside safety hardware poses challenges 

to states making data-driven decisions about highway maintenance. 

FHWA officials told us they currently have no plans to include additional 

Current FHWA Efforts Aim 

to Improve Data Collection 

Practices but Do not Fully 

Address In-Service 

Performance Evaluation 

Information Shortfalls
Page 31 GAO-16-575 Roadside Safety Hardware

ISPEs for other types of roadside safety hardware as part of their broader 

highway-safety research portfolio. Officials cited cost concerns with 

gathering data and explained that ISPEs would take on greater relevance 

in the future as more MASH-compliant devices are installed on roadways. 

However, continuing this study and reporting on the performance of 

guardrail end terminals, or planning to make ISPEs part of other future 

research, could add to the limited body of knowledge regarding the inservice performance of roadside safety hardware. FHWA officials also 

noted that hardware that was installed could still be on the roadways for 

20–30 years. ISPEs on current devices can therefore still provide states 

with critical information regarding how they might prioritize maintenance

tasks—such as replacing older devices—to best ensure safety for their 

motorists. Without robust, ongoing in-service performance evaluations, 

less safe hardware may remain in use longer than is necessary.

FHWA’s cooperation with AASHTO and state DOTs has resulted in states 

having policies to install crash-tested roadside safety hardware on the 

NHS. However, challenges exist for states, industry and FHWA as the 

improved MASH crash-testing standards are phased in over the next few 

years. These changes will require cooperation and action from industry, 

the states, and FHWA. FHWA has the opportunity to exercise more 

robust oversight to ensure greater consistency in the implementation of 

improved crash test standards. First, FHWA, through its division offices’

oversight of states’ standards and design specifications, can help ensure 

that states have written policies in place that fully reflect the terms of the 

2016 state-approved Joint Implementation Plan to address inconsistent 

practices across states. Second, monitoring and reporting the states’ and 

industry’s progress transitioning to the MASH crash test standards, as 

federal standards for internal controls suggest, and making this 

information available to Congress and the public would facilitate 

transparency and position FHWA to consider midcourse corrections if 

required.53

FHWA can also take steps to strengthen its role in the assessment of 

roadside safety hardware performance—both in the test lab and once 

installed on the roadways. Because FHWA’s current oversight process 

does not include verification of lab crash-test results and no specific 

 53 GAO-14-704G

Conclusions
Page 32 GAO-16-575 Roadside Safety Hardware

mitigation measures are in place to address potential threats to 

independence, the risks to ensuring the integrity of the crash-testing 

process remain unaddressed. Other agencies have introduced policies or 

processes into the testing process that mitigate these types of issues; 

FHWA could take similar actions. In addition, other agency practices 

provide a model for FHWA of closer cooperation with the labs and 

accreditation bodies to address the independence issues unique to 

roadside safety hardware’s testing. FHWA also has the opportunity to 

advance its mission in the scientific evaluation of roadside safety 

hardware. FHWA has a pilot project underway that is examining data 

collection practices for in-service performance evaluations but currently 

has no plans to report on performance findings from either this study or 

other research in its portfolio. Continuing this study or planning to make 

ISPEs part of future research could add to what is currently a limited body 

of knowledge regarding the in-service performance of roadside safety 

hardware. FHWA is poised to consider changes to its approach to 

roadside safety hardware through a full programmatic review to be 

completed in the summer of 2016. Opportunities exist to address all these 

issues and to provide states, industry, and the traveling public greater 

assurance that FHWA is fulfilling its safety mission and advancing 

roadside safety.

To promote the transition to improved crash test standards, to strengthen 

FHWA’s oversight of the roadside safety hardware’s crash-testing 

process, and to make more information available to states and industry 

on how roadside safety hardware performs in actual conditions, we 

recommend that the Secretary of Transportation direct the Administrator 

of FHWA to take the following five actions:

1. Direct FHWA’s division offices to help ensure, through their oversight 

of states’ standards and design specifications, that states have written 

policies in place to require the installation of appropriately crashtested roadside safety hardware on the NHS to address inconsistent 

practices across states.

2. Monitor and periodically report to Congress (or report through the 

agency’s publicly available website) progress states and the industry 

are making in transitioning to the MASH crash-testing standards for 

roadside safety hardware.

3. Provide additional guidance to crash test labs and accreditation 

bodies to ensure that labs have a clear separation between device 

Recommendations
Page 33 GAO-16-575 Roadside Safety Hardware

development and testing in cases where lab employees test devices 

that were developed within their parent organization.

4. Develop a process for third-party verification of results from crash-test 

labs.

5. Support additional research and disseminate results on roadside 

safety hardware’s in-service performance, either as part of future 

phases of FHWA’s current pilot study on guardrail end terminals’

performance or as part of FHWA’s broader research portfolio.

We provided a copy of a draft of this report to the Department of 

Transportation for review and comment. In written comments, 

reproduced in appendix III, DOT concurred with GAO’s 

recommendations. FHWA also provided technical comments which we 

incorporated, as appropriate. 

As agreed with your offices, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies to interested congressional 

committees and the Secretary of Transportation. In addition, the report 

will be available at no charge on the GAO website at http://www.gao.gov. 

If you or your staff have any questions about this report, please contact 

me at (202) 512-2834 or Flemings@gao.gov. Contact points for our 

Offices of Congressional Relations and Public Affairs may be found on 

the last page of this report. GAO staff who made key contributions to this 

report are listed in appendix IV.

Sincerely yours,

Susan Fleming

Director, Physical Infrastructure Issues

Agency Comments
Page 34 GAO-16-575 Roadside Safety Hardware

List of Requesters

The Honorable Richard Blumenthal

Ranking Member

Subcommittee on Consumer Protection, Product Safety,

Insurance, and Data Security

Committee on Commerce, Science, and Transportation

United States Senate

The Honorable Cory Booker

Ranking Member

Subcommittee on Surface Transportation and Merchant Marine

Infrastructure, Safety, and Security

Committee on Commerce, Science, and Transportation

United States Senate

The Honorable Tim Kaine

United States Senate

The Honorable Edward Markey

United States Senate

The Honorable Mark Warner

United States Senate

The Honorable Sheldon Whitehouse

United States Senate

The Honorable Mark DeSaulnier

House of Representatives
Appendix I: Objectives, Scope, and 

Methodology

Page 35 GAO-16-575 Roadside Safety Hardware

This report addresses: (1) how FHWA performs oversight of state policies 

and practices related to roadside safety hardware; (2) the thoroughness 

of the crash-testing process and FHWA’s oversight of this process; and 

(3) the extent to which information is available on roadside safety 

hardware performance once installed.

To assess FHWA’s role in the oversight of roadside safety hardware and 

related state policies, we reviewed FHWA documentation including: 

internal memos on roadside safety and implementation of the current 

eligibility letter process; guidance memos to the broader roadside safety 

hardware community; and policy documents such as the template for 

stewardship and oversight agreements agreed to with states. We also 

reviewed relevant laws and regulations governing FHWA’s oversight of 

roadside safety hardware. We interviewed FHWA officials in headquarters 

to understand how policies and practices are carried out. We also applied 

federal internal control standards for monitoring, designing control 

activities and communication with external stakeholders when reporting 

on the agency’s policies and practices for conducting oversight of 

roadside safety hardware.1

To better understand FHWA’s process in reviewing crash test information 

and issuing eligibility letters, we requested and received the FHWA files 

for 10 eligibility letters. Two case files were selected by FHWA as 

example files; we accepted these files and then selected eight more files 

from the roughly 1,000 available. To make our selection, we first limited 

the pool to only applications that came to FHWA since 2005. Then we 

wanted variation in terms of the following variables: age of application 

(variation across those 10 years); new device versus modification to 

existing device; type of device; type of standard tested to (MASH or 

NCHRP Report 350); and proprietary versus generic. Once the files were 

received we reviewed each file to determine whether it had the 

information we would expect in order for a third party to understand how 

FHWA officials came to the conclusion to issue an eligibility letter.

To better understand states’ roles in the oversight of roadside safety 

hardware, we developed and distributed a survey to all 50 states, plus the 

District of Columbia and Puerto Rico. Survey questions addressed topics 

 1 GAO, Standards for Internal Control in the Federal Government, GAO-14-704G

(Washington, D.C.: September 2014).

Appendix I: Objectives, Scope, and 

Methodology
Appendix I: Objectives, Scope, and 

Methodology

Page 36 GAO-16-575 Roadside Safety Hardware

including policies on crash testing of roadside safety hardware, 

procedures for ensuring that only crash-tested hardware is installed on 

the national highway system, efforts to collect inventory data on roadside 

safety hardware, and what, if any, research states had conducted in the 

last 10 years to evaluate the in-service performance of roadside safety 

hardware after it is installed. After developing the survey, we conducted

four pre-test interviews with selected states to ensure that the questions 

were clear and appropriate for our research objectives. We adjusted the 

survey questions as needed in response to feedback prior to survey 

distribution.

In October 2015 we distributed the survey to state departments of 

transportation representatives from all 50 states, plus the District of 

Columbia and Puerto Rico. We followed up and collected responses until 

January 2016, at which point we had received responses from 44 out of 

52 states and territories.2 In instances where states did not supply 

complete responses to individual questions the answers to those 

questions were not included in the survey results, and those states were 

removed from the denominator for purposes of summary analysis. For 

selected questions, we conducted brief follow-up interviews and solicited 

written responses, when appropriate, in order to seek clarification or 

elaboration of states’ responses.

To get more information on how states oversee roadside safety hardware, 

we selected five states—Maryland, Virginia, Ohio, Texas, and 

California—with which to conduct interviews with state departments of 

transportation officials and the FHWA’s division offices that oversee the 

state departments of transportation. We selected these states based on 

the presence of an accredited crash-testing facility in the state and 

recommendations from stakeholders regarding the quality of 

performance-data collection efforts in those states. In the cases of 

Virginia, Ohio, and Texas, interviews with state officials were conducted 

on site, and we also conducted interviews with crash-testing lab 

personnel and roadside safety hardware developers in the cases of Ohio 

and Texas.

 2 States and territories that did not respond to our survey were: Arkansas, Illinois, Kansas, 

New Hampshire, North Carolina, Oregon, Puerto Rico, and South Carolina. 
Appendix I: Objectives, Scope, and 

Methodology

Page 37 GAO-16-575 Roadside Safety Hardware

To gain information on the thoroughness and independence of the crashtesting process and the extent to which FHWA oversight helps ensure 

this, we interviewed the nine domestic crash labs that are accredited, as 

required by FHWA, to international crash test lab standards to test 

roadside safety hardware for FHWA eligibility letters.3 To describe how 

labs are evaluated against international-testing standards, we reviewed 

the international-test lab accreditation standards in ISO 17025 and 

interviewed the three accrediting bodies that accredit the nine domestic 

crash-testing labs.4 To evaluate the thoroughness and documentation for 

lab crash testing, we reviewed the accreditation requirements in ISO 

17025 as well as the crash-testing guidelines in MASH, and analyzed 

these documents to create both interview questions and a document 

request list for all the labs, in consultation with our technologist. The 

questions addressed how labs ensure the quality of the testing 

environment, how they interpret test results, how they document each 

test, how they comply with conflict of interest requirements in the ISO, 

and how communicate with FHWA. We also asked labs to submit their 

accreditation reports, quality manual, any relevant conflict-of-interest or 

ethics policies, and a sample test report for us to review. We then asked 

the nine labs to walk us through a recent example of a product tested for 

FHWA compliance, in order to describe how policies and requirements 

are implemented in practice. We reviewed the conflict-of-interest policies 

to determine the extent to which there was variation in policies across the 

labs, and to evaluate whether there were mitigation measures for 

potential threats to independence. We also visited four crash test labs 

and witnessed two full scale crash tests to gain a better understanding of 

the crash testing process. To collect information on how other agencies 

oversee lab testing, we reviewed documentation and interviewed officials 

from the Environmental Protection Agency’s ENERGY STAR program, as 

well as the National Highway Traffic Safety Administration regarding their 

 3 The nine domestic labs accredited to conduct roadside safety hardware crash-testing for 

FHWA review are: E-Tech Testing Services, Inc. (CA); KARCO Engineering, LLC (CA); 

Midwest Roadside Safety Facility (NE); Roadside Safety Research Group (CA); Safe 

Technologies, Inc. (CA); Southwest Research Institute (TX); Texas A&M Transportation 

Institute (TX); The Thomas D. Larson Pennsylvania Transportation Institute (PA); and 

Transportation Research Center, Inc. (OH). We interviewed officials from the Federal 

Outdoor Impact Laboratory, which is accredited, but did not include them in our lab 

evaluation because they do not do FHWA compliance testing.

4 The three accrediting bodies that accredit the aforementioned labs are: The American 

Association for Laboratory Accreditation (A2LA); the International Accreditation Service 

(IAS); and the Laboratory Accreditation Bureau (LAB).
Appendix I: Objectives, Scope, and 

Methodology

Page 38 GAO-16-575 Roadside Safety Hardware

vehicle crash testing. Both agencies were referenced in our discussions 

with accrediting bodies as examples of other agencies that oversee 

accredited testing programs.

To assess the extent of information available on roadside safety hardware 

performance once devices are installed, we conducted a literature search 

for in-service performance evaluations (ISPE) using government, 

academic, and trade publication sources. We also reviewed studies 

submitted to us by a highway design and roadside safety hardware 

engineering expert. For both sources of studies, we used the National 

Cooperative Highway Research Program (NCHRP) Report 490’s

definition of an ISPE to define criteria for determining whether the studies 

we reviewed constituted ISPEs for the purposes of our report.5

Specifically, we looked for studies that combined analysis of crash data 

with real-time site visits. According to NCHRP Report 490, studies that 

retroactively or contemporaneously examine crash data are known as 

historical studies and collision studies, respectively, whereas an ISPE 

adds the element of real-time crash site analysis. NCHRP Report 490

notes that this technique allows researchers to better determine what type 

of hardware was struck, whether there were installation techniques or 

other site-specific characteristics that contributed to the crash, and 

whether the exact device is something a state DOT still uses. We also 

stipulated that the study in question involve a specific type of roadside 

safety hardware, which we defined according to FHWA’s categories of 

hardware for purposes of federal-aid eligibility letters.6 Moreover, we 

restricted our ISPE classification to studies published between 1993 and 

2015, when NCHRP Report 350 crash-testing standards were published 

and when FHWA first recognized them. As part of our literature search, 

we used online search terms that tailored the searches to specific types 

 5 NCHRP is a research program administered by the National Academies of Science’s 

Transportation Research Board and is jointly funded by FHWA and AASHTO. In 2003, 

NCHRP Report 490 was published, and it identified multiple characteristics of an ISPE 

that differentiate it from other types of studies. Namely, an ISPE should combine analysis 

of crash data with real-time crash site visits so as to better understand the site 

characteristics that might contribute to crashes. 

6 Throughout our report, we categorized the types of roadside safety hardware into either 

(1) longitudinal barriers, such as cable barriers and guardrails; (2) bridge barriers; (3) 

barrier terminals/crash cushions, such as crash cushions and guardrail end terminals; (4) 

support structures, such as sign supports, luminaires, and utility poles; and (5) work zone 

equipment.
Appendix I: Objectives, Scope, and 

Methodology

Page 39 GAO-16-575 Roadside Safety Hardware

of roadside safety hardware as well as key methodological components, 

such as site visits.

To inform all of the research questions we also reviewed documentation 

and interviewed relevant officials from interested stakeholders. We 

reviewed standards and relevant guidance from the American Association 

of State Highway and Transportation Officials (AASHTO). To collect 

information on how crash test standards are developed and are updated, 

we interviewed the AASHTO Technical Committee on Roadside Safety, 

as well as officials at the Transportation Research Board’s National 

Cooperative Highway Research program. To get more detailed 

perspective on how industry, states, and crash-testing facilities 

collaborate, we attended a semiannual meeting of Task Force 13, a joint 

committee of AASHTO, the Associated General Contractors of America, 

and the American Road and Transportation Builders Association, which 

develops standards and specifications for bridges and roadside safety 

hardware. We also interviewed two roadside safety hardware developers.

We conducted this performance audit from April 2015 to June 2016 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe the 

evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.
Appendix II: Review of FHWA Eligibility Letter 

Files

Page 40 GAO-16-575 Roadside Safety Hardware

We reviewed 10 case files for eligibility letters issued between 2005 and 

2015 and found that documentation was not sufficient to determine the 

rationale behind FHWA’s decision to issue these letters. FHWA officials 

explained that as part of their eligibility letter review process, they 

examine the crash test lab report, including pictures, videos, and the testdata summary sheets. If FHWA officials have questions they will contact 

the lab or developer. However, there is no structured protocol for 

documenting the steps FHWA reviewers took or the rationale behind the 

decision to issue an eligibility letter. For example, in two cases where 

FHWA officials asked questions and received answers from the lab or test 

sponsor, it was not possible to trace how FHWA made its determination 

to issue an eligibility letter. We also found three instances in which the full 

suite of testing was not performed, but no documentation was present 

explaining why the lack of testing was acceptable.

We provided this information to FHWA officials who acknowledged that 

the basis for those decisions was not documented but stated that in each 

case FHWA found the information and reasoning provided by the lab or 

test sponsor satisfactory. FHWA officials also told us they made changes 

to the eligibility letter review process in 2015 including documenting 

communications with developers seeking an eligibility letter, a checklist 

for documenting reviews of eligibility letter requests, and updates to the 

eligibility letter request form to identify tests that are not critical or not 

relevant and the reasons why or why not. FHWA officials told us that this 

checklist provides documentation from the submitter concerning why 

certain tests were not conducted or why modifications are considered 

non-significant to better document this information. We did not evaluate 

the impact of these changes since they were made during the course of 

our audit work. Officials also characterized these changes as “interim”

because the eligibility letter review process is part of the ongoing 

independent Volpe National Transportation Systems Center review of the 

program.

Appendix II: Review of FHWA Eligibility 

Letter Files
Appendix III: Comments from the Department 

of Transportation

Page 41 GAO-16-575 Roadside Safety Hardware

Appendix III: Comments from the 

Department of Transportation
Appendix III: Comments from the Department 

of Transportation

Page 42 GAO-16-575 Roadside Safety Hardware
Appendix IV: GAO Contacts and Staff 

Acknowledgments

Page 43 GAO-16-575 Roadside Safety Hardware

Susan Fleming at (202) 512-2834 or flemings@gao.gov

In addition to the contact named above, Steve Cohen (Assistant Director), 

Melissa Bodeau, Devin Braun, William Egar, Sarah Farkas, Sarah 

Gilliland, Judy Guilliams-Tapia, David Hooper, Leslie Locke, Madhav 

Panwar, Malika Rice, Alexandra Squitieri, Jade Winfree, and Elizabeth 

Wood made key contributions to this report.

Appendix IV: GAO Contacts and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments

(541131)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. 

Listen to our Podcasts and read The Watchblog. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.