HSIP Noteworthy Practice Series

HSIP Project 

Evaluation

About the HSIP Noteworthy Practice Series

The Highway Safety Improvement Program (HSIP) is a core Federal-aid highway program with the 

primary purpose of achieving a significant reduction in fatalities and serious injuries on all public 

roads. Many states and local agencies are successfully implementing innovative approaches to HSIP 

planning, implementation, and evaluation. The HSIP Noteworthy Practices Series presents case 

studies of these successful practices organized by specific HSIP topics. The individual case studies 

provide summaries of each practice, key accomplishments, results, and contact information for those 

interested in learning more.

HSIP Project Evaluation

Evaluation is critical to determine if a project or 

group of projects is achieving the desired results 

and to ensure investments are cost-effective. 

Evaluation provides a quantitative estimate of 

the effects on safety, which is valuable information for future planning. Evaluation results 

enable a state to determine if appropriate countermeasures were used at particular locations, 

whether any adverse impacts occurred, if corrective action is necessary, and how effective those 

countermeasures would be for similar sites in 

the future. 

Various methods exist for evaluating projects, but 

any evaluation should consider a minimum of 

three to five years of before and after crash data, 

the target crash type of the improvement, and 

crash severity (a countermeasure may increase 

the total number of crashes, but reduce the crash 

severity). Ideally, project evaluation should incorporate more advanced techniques (e.g., safety 

performance functions (SPFs), Empirical Bayes 

(EB) method) to account for natural fluctuations 

in crashes from year to year and other changes 

potentially impacting evaluation results. 

The majority of states are conducting project 

evaluations based on a simple before-after analysis, and a few are using evaluation results to 

develop state-specific crash modification factors 

(CMFs) for various countermeasures. While 

simple before-after evaluations are rather easy 

to perform and may provide a basic understanding of safety changes, they assume any change 

was due solely to the safety improvement at the 

site and may misrepresent the true effectiveness 

of a project due to the effects of regression-tothe-mean.1

 The EB method can be incorporated 

into project evaluations to reduce the effects of 

regression-to-the-mean. However, very few 

states have been able to use the EB method since 

it requires calibrated SPFs. Many states do not 

have the training, resources, tools, manpower, or 

necessary data to calibrate SPFs.

Another challenge is that individual states may 

not have enough installations of a particular 

countermeasure to develop quality CMFs. The 

Evaluations of Low Cost Safety Improvements 

Pooled Fund Study (ELCSI PFS) combines the 

implementation efforts of multiple states to 

develop reliable estimates of countermeasure 

effectiveness. States can independently initiate 

similar efforts.

1 Regression-to-the-mean bias describes a situation in 

which crash rates are artificially high (or low) during the before period and would have decreased (or 

increased) even without an improvement to the site. 

 FHWA-SA-11-02
HSIP Noteworthy Practice Series HSIP Project Evaluation

Noteworthy Practices

The following cases demonstrate noteworthy practices four states are using in HSIP project evaluations:

• The Colorado DOT developed SPFs for all roadway facility and intersection types in the State, which enabled the DOT to 

institutionalize the EB method into all safety project evaluations and reduce the effects of regression-to-the-mean. (read more)

• The Florida DOT developed an on-line database application of safety improvement projects that automates the processes for conducting benefit-cost analysis to compare different countermeasures and for conducting safety project 

evaluations to develop crash reduction factors (CRF). The application has also enabled Florida to develop and continue 

to refine state-specific CRFs for several countermeasures based on the project evaluation results. (read more)

• The North Carolina DOT created a safety project evaluation group to conduct evaluations on all spot safety projects in the 

State. The project evaluations provided field engineers with valuable feedback on the effectiveness of safety projects and 

countermeasures. (read more)

• The University of Wisconsin Traffic Operations and Safety (TOPS) Laboratory, under contract to Wisconsin DOT, developed a project evaluation process incorporating EB analysis into all HSIP project evaluations. The TOPS Laboratory 

compared benefit-cost analysis using simple before-after analysis results and EB to demonstrate the importance of using 

statistical evaluations to reduce the overestimation of safety benefits due to regression-to-the-mean. (read more)

To access these full case studies, click on the individual links above or visit the FHWA Office of Safety on-line at: 

http://safety.fhwa.dot.gov/hsip.

Highway Safety Improvement Program

Data Driven Decisions

FHWA-SA-11-02

The Focus is Results
HSIP Noteworthy Practice Series HSIP Project Evaluation

Project Evaluations 

Using Empirical Bayes

Colorado

Incorporating the Empirical Bayes (EB) method into 

project evaluations reduces the potential overestimation 

of safety benefits due to regression-to-the-mean. While 

the EB method is not difficult in itself, it requires safety 

performance functions (SPF) for the type of facilities on 

which projects are being evaluated.

SPFs were originally developed by Colorado Department 

of Transportation (CDOT) for use in the network 

screening process. While crash rates are commonly used 

to measure safety, the crash rate implies a linear relationship between safety and exposure, which can often 

be misleading since rates change with Annual Average 

Daily Traffic (AADT). To capture how this rate change 

takes place, design engineers at the CDOT started to 

calibrate SPFs in the late 1990s, as part of the development of the Level of Service of Safety (LOSS) concept. 

LOSS is used to identify locations with potential for 

safety improvement and reflects how a roadway segment 

is performing in regard to its expected crash frequency 

and severity at a specific level of AADT, based on the 

SPF. By 2001, CDOT had calibrated SPFs for all public 

roadways (state and local) in Colorado, stratified by the 

number of lanes, terrain, environment, and functional 

classification. In 2009, CDOT in collaboration with 

consultants developed SPFs for all intersection types. 

The development of SPFs has not only advanced CDOT’s 

network screening process, it also has enabled CDOT to 

institutionalize the use of the EB method as a standard 

procedure for safety evaluation analysis. Colorado has 

traditionally used a simple spreadsheet with three to five 

years of before and after data to conduct project evaluations. CDOT is currently working on applying an EB 

correction to evaluate sites on an SPF graph as shown. 

The use of the EB method is particularly effective when 

it takes a long time for a few crashes to occur, as is often 

the case on Colorado rural roads.

Key Accomplishments

•	 Developed SPFs for all roadway facility and

intersection types in the state.

•	 Institutionalized the use of the Empirical

Bayes method as a standard procedure for

safety evaluation analysis to reduce effects of

regression-to-the-mean.
HSIP Noteworthy Practice Series HSIP Project Evaluation

Results

CDOT developed SPFs for all state and local roadway facilities and intersection types. The development of the 

SPFs has enabled CDOT to fully institutionalize the EB method for all safety analysis at CDOT and reduce the 

effects of regression-to-the-mean.

Contact

Brian Allery

Colorado Department of Transportation

303-757-9967

brian.allery@dot.state.co.us

Jake Kononov

Colorado Department of Transportation

303-757-9973

jake.kononov@dot.state.co.us

Highway Safety Improvement Program

Data Driven Decisions

 FHWA-SA-11-02 The Focus is Results
HSIP Noteworthy Practice Series HSIP Project Evaluation

Crash Reduction 

Analysis System Hub

Florida

Crash reduction factors (CRF) provide agencies with an 

estimate of the expected crash reduction and/or benefits 

associated with various countermeasures. However, since 

local conditions (e.g., roadway, driver, traffic, weather, 

crash investigation techniques) may vary from agency to 

agency, state specific CRFs provide a more accurate indication of the effectiveness of various countermeasures. 

Prior to the development of the Crash Reduction Analysis 

System Hub (CRASH), the Florida Department of 

Transportation (FDOT) did not have a central database 

that combined crash data and safety project data to determine CRFs, or a mechanism in place to provide FHWA 

with a report on the effectiveness of safety projects in 

reducing crashes. The individual districts maintained 

the historical data for their safety improvement projects, 

which were in various formats and were not easily accessible for developing CRFs. In an effort to systematically 

maintain statewide safety improvement project data and 

facilitate a continual process of developing and updating 

state specific CRFs, the FDOT funded a research project 

with the Lehman Center for Transportation Research to 

develop the CRASH application. 

The CRASH application is a web database of safety 

improvement projects on the FDOT intranet. The 

District Safety Engineers (DSE) input all Highway Safety 

Improvement Program (HSIP) funded projects into 

CRASH, which stores individual safety improvement 

projects and crash data. The application uses before and 

after crash counts to evaluate the CRFs for the countermeasures implemented in the safety improvement projects 

entered in the system. The State Safety Engineer (system 

administrator) updates the CRFs annually; although they 

can be updated at any time. The CRFs are typically calculated based on five years of before and after data, but 

the system administrator may specify a time period for 

the calculation.

The CRASH application enables the DSEs to easily evaluate different countermeasures and conduct 

a benefit-cost analysis by inputting project limits and 

selecting crash data years. CRASH currently includes 

135 different improvements types. When the user 

selects a proposed countermeasure, the application 

provides a range of CRFs for crashes in various categories based on historical crash reductions or increases 

associated with past projects. The user can select the 

standard CRF or input a user defined value in cases 

where no sufficient studies in Florida exist to develop 

a state specific CRF. 

The CRASH system enables easy performance tracking 

of safety efforts. It provides various functions for data 

retrieval and exportation for other analysis and reporting 

purposes, including the annual HSIP report.

Key Accomplishments

•	 Developed an online database of safety

improvement projects and state-specific CRFs.

•	 Automated processes for benefit-cost analysis

and safety project evaluations.
HSIP Noteworthy Practice Series HSIP Project Evaluation

Results

The CRASH application has enabled Florida to develop state-specific CRFs for several countermeasures based on 

the evaluation results of implemented HSIP projects. The system has also reduced the level of effort required to 

conduct benefit-cost analyses and project evaluations by automating the processes.

Contact

Joseph Santos

Transportation Safety Engineer

Florida Department of Transportation

850-245-1502

joseph.santos@dot.state.fl.us

Highway Safety Improvement Program

Data Driven Decisions

 FHWA-SA-11-02 The Focus is Results
HSIP Noteworthy Practice Series HSIP Project Evaluation

Safety 

Evaluation Group

North Carolina

In 1999, the North Carolina Department of Transportation 

(NCDOT) created a permanent group of employees to 

focus on safety project evaluation. The purpose for the 

group was to establish a formal project evaluation process 

to verify the success of the state’s efforts in safety.

In the first four years following its formation, the group 

established a process for conducting project evaluation and 

identified what results would be most useful to the field 

engineers. The group’s initial efforts were more research 

and technically oriented but, to better serve the needs of 

the field engineers, the results of the evaluation studies 

were simplified (the field engineers were most interested 

in the before and after crash diagrams and changes in 

crash patterns). Originally, the group conducted about 

50 project evaluations a year with one supervisor and six 

engineers, but now the group completes approximately 

200 evaluations a year with reduced staff (one supervisor, 

four engineers, and one technician). 

The safety evaluation group conducts simple before-after 

studies on all spot safety projects once a minimum of three 

years of before and after data are available (the same time 

periods are used for both the before and after periods). 

For each project, the group prepares an evaluation report 

including before and after crash type and severity data, 

collision diagrams, photos, and discussion of the study 

results. After an evaluation report is completed, it is submitted to the field engineer who originally developed the 

project to provide feedback on whether the project successfully mitigated the previously identified safety issue. 

For projects unsuccessfully mitigating the safety issue or 

resulting in a different crash pattern, the evaluation report 

provides the field engineer with an opportunity to reassess 

the conditions and identify a different countermeasure. 

NCDOT is currently working on developing a process to 

track projects not successfully mitigating the safety issue 

they were intended to address.

The evaluation group compiles a spreadsheet of all the 

completed project evaluation studies. The spreadsheet 

provides the category of improvement, before and 

after traffic volumes, location, traffic control, geometry, etc., and provides a link to the detailed evaluation 

report. The spreadsheet is updated regularly and posted 

on the NCDOT web site (http://www.ncdot.org/doh/

preconstruct/traffic/safety/Reports/completed.html). It 

can be used by engineers to determine which treatments 

have worked in the past. 

The group also develops crash modification factors (CMF) 

using the Empirical Bayes (EB) method when enough 

sample sites and data are available. North Carolina specifically focuses on developing CMFs for countermeasures 

not already extensively researched. 

Key Accomplishments

•	 Established a group focused on safety project

evaluation.

•	 Promoted the use of effective countermeasures.

•	 Provided feedback to engineers on the effectiveness of their individual safety projects and various

countermeasures.
HSIP Noteworthy Practice Series HSIP Project Evaluation

In efforts to promote a particular countermeasure to the field engineers, NCDOT’s evaluation group has also conducted studies on well documented countermeasures to provide evidence of crash reduction effectiveness. One 

example is the use of four-way stop control. Although several studies document the effectiveness of this countermeasure, many field engineers in North Carolina did not consider it a viable countermeasure. The group evaluated 

over 50 intersections throughout the State and demonstrated four-way stop controls were effective. Study results 

were presented to field engineers in an effort to change their perspective. While field engineers were very receptive to 

the study results, it is too early to determine if it has increased the use of four-way stop control in the State.

Results

Since the establishment of the safety evaluation group, North Carolina has evaluated and documented the results of 

more than 600 projects. The evaluation reports provide field engineers with valuable feedback on the effectiveness 

of their safety projects, as well as various countermeasures, and promote the use of effective countermeasures.

Contact

Shawn Troy

North Carolina Department of Transportation

919-773-2897

stroy@ncdot.gov

Highway Safety Improvement Program

Data Driven Decisions

 FHWA-SA-11-02 The Focus is Results
HSIP Noteworthy Practice Series HSIP Project Evaluation

University Conducting HSIP 

Project Evaluations Using 

Empirical Bayes Wisconsin

Wisconsin DOT contracted with the University of 

Wisconsin Traffic Operations and Safety (TOPS) 

Laboratory to investigate multiple project evaluation 

methods through a research grant for HSIP evaluation 

support. Initial research efforts included project evaluations based on before and after collision maps using the 

software Intersection Magic and before-after evaluations 

using benefit-cost analysis. From the beginning of the 

research, the intent was to use Empirical Bayes (EB) analysis in the project evaluations, but Wisconsin did not have 

safety performance functions (SPFs), which are required 

for the EB method. However, once the State acquired the 

SafetyAnalyst software, the TOPS Laboratory was able to 

incorporate the EB method into the project evaluations by 

using the SPFs contained in SafetyAnalyst. The SPFs in 

SafetyAnalyst were developed using national data and are 

intended to be calibrated to local conditions. While it was 

not possible to calibrate the SPFs to Wisconsin conditions 

due to lack of data, the TOPS Laboratory uses the SPFs 

to provide a comparison of performance in Wisconsin to 

that of the nation.

The TOPS Laboratory developed a process to extract the 

appropriate crashes (by location, type, and year) from the 

Wisconsin crash database based on the project location 

and scheduled start and completion dates for evaluation 

purposes. HSIP projects are evaluated based on five years 

of before data and three years of after data. Fatal and 

injury crashes are the focus of the evaluation, but the analysis also considers target crash types based on the nature 

of the improvement. 

The TOPS Laboratory conducts a benefit-cost analysis 

based on results of both a simple before-after evaluation 

and an EB analysis to evaluate the projects from an economic perspective. This provides a simple comparison of 

the results of the two evaluation methods (as shown1

) and 

demonstrates how a simple before-after evaluation can 

overestimate the safety benefits.

1 In the table shown, “S. No.” refers to the site number for the project 

evaluated, and the “FOS (financial operating system) ID” is used by 

Wisconsin DOT as the specific project identifier.

Key Accomplishments

•	 Developed a project evaluation process incorporating Empirical Bayes analysis into all HSIP

project evaluations.

•	 Demonstrated the importance of using statistical

evaluations to reduce the overestimation of safety

benefits due to regression-to-the-mean bias.
HSIP Noteworthy Practice Series HSIP Project Evaluation

Results

Originally, engineers in Wisconsin were reluctant to use EB. However, with the assistance of the TOPS Laboratory, 

the Wisconsin DOT was able to successfully implement a project evaluation process incorporating EB analysis and to 

receive buy-in at the regional level. The TOPS Laboratory demonstrated the importance of statistical EB techniques 

in project evaluations through a comparison benefit-cost analysis using simple before and after results to before 

and after using EB. The results demonstrate the EB analysis reduces the overestimation of safety benefits due to 

regression-to-the-mean bias.

Contact

Andrea Bill

Traffic Safety Engineering Research Program Manager

Traffic Operations and Safety (TOPS) Laboratory

University of Wisconsin 

608-890-3425

bill@wisc.edu

Highway Safety Improvement Program

Data Driven Decisions

 FHWA-SA-11-02 The Focus is Results