USA Banner
  U.S. flag
              An official website of the United States government Here's how you knowOfficial US Government Icon
              **Official websites use .gov**A **.gov** website belongs to an official government organization in the United States.
            Secure Site Icon
              **Secure .gov websites use HTTPS**
                  A **lock** ( A locked padlock ) or **https://** means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites. 
 U.S. Department of Transportation Icon United States Department of Transportation
 Search DOT
            Search
                    About FHWAFHWA Organization
                Headquarters Offices
                Field Offices
                Strategic Plan
                Business Opportunities
                Staff Directories
                Careers
                Highway History
                Contact UsProgramsAcquisition Management
                Civil Rights
                Federal Lands Highway
                Infrastructure
                Innovative Program Delivery
                Operations
                Planning, Environment, and Realty
                Policy
                Research, Development, and Technology
                Safety
                Browse by TopicsResourcesCore Highway Topics
                Federal Aid Essentials
                Automation
                Complete Streets
                Laws and Regulations
                Policy & Guidance Center
                Section 508
                Publications and Statistics
                Professional Development
                National Highway Institute
                Resource Center
                Accessibility Resource Library
                FHWA Research LibraryNewsroomPress Releases
                Speeches & Testimony
                Photos
                Videos
                Media ContactsMenu FHWA Highway Safety Programs Home
                        HSIP National Scan Tour


            
            In This Section
        Highway Safety Improvement Program (HSIP)
                                Overview
                About
                Planning
                Implementing
                Evaluating
                Reporting
                Noteworthy Practices
                Legislation and Guidance
                Resources
                HSIP Special Rules
                Strategic Highway Safety Plan (SHSP)
                                Overview
                About
                Develop
                Implement
                Evaluate
                Promote
                Legislation and Guidance
                Resources
                Noteworthy Practices
                Safety Performance Measure
                                Overview
                State Safety Performance Targets
                Policy & Guidance
                Fact Sheets & Informational Webinars
                Target-Setting Resources
                Safety Data Resources
                State Safety Reports
                Safety Contacts
                Technical Assistance & Training
                Railway Highway Crossing Program
                                Overview
                Policy and Guidance
                Resources
                
                Chapter 10–Evaluating the Success of the Program
By regularly evaluating their programs, States can continuously improve their strategies for achieving their SHSP goals and targets, as well as their own efficiency and effectiveness. Tracking and evaluating the effectiveness of completed HSIP projects is also a means to determine if those improvements are providing their predicted benefits. States with highly successful programs also "benchmark" themselves against other States, adopting new practices they find promising.

All States had developed some form of before-and-after project analysis. Alaska, Utah, Illinois, and North Carolina have strong evaluation programs that are being used to directly inform the development of safety performance functions (SPF), CRFs, and other values used in the screening and analysis portion of HSIP project development.

**SPOTLIGHT ON SAFETY:**

**Safety Evaluations in North Carolina**

The NCDOT Traffic Safety Unit conducts a Safety Evaluation for all completed safety projects. This evaluation includes project information, collision diagrams, and analysis of before and after crash frequency and severity. The analysis period is typically three years, but it can be extended for sites with a low pre-project crash rate. The Safety Unit executes roughly 120 safety project and program type evaluations each year and has trained consultant staff to perform evaluations in addition to in-house staff. Advanced evaluations are generally performed by NCDOT staff, such as the Empirical Bayes analysis of multiple sites.

There are three aspects to the NCDOT evaluation process. The first is NCDOT’s evaluation of every HSIP, Spot Safety, and hazard elimination project, a site-specific, simple before/after review of crashes. Headquarters provides the regions with the reports generated from these project-level reviews. *(20)*

The second is NCDOT’s Empirical Bayes evaluation of multiple site locations with similar countermeasure installations, which facilitates distinguishing between different treatments and the development of CMFs. The evaluations can be site-specific or taken across multiple similar sites. This was done for a variety of Intersection Cross-Road Warning System (ICWS) treatments and can be used to compare the effectiveness of varying treatment implementations.

The third aspect is a program-wide evaluation, undertaken program-wide for a Spot Safety or HSIP project. In the case of the Spot Safety program, NCDOT calculated a 14:1 b/c ratio for all improvements in the program across 600 projects.

One of the significant outcomes of NCDOT’s evaluation programs is the validation of CRF. NCDOT data collection efforts for projects, both prior to and subsequent to the construction of improvements, enables staff to conduct network-wide and site-specific evaluations of CRFs. This work, particularly with deployment of ICWS, has been crucial in preserving effective treatments for future implementations while permitting culling of less-effective treatments on the basis of location characteristics and countermeasure performance.

To that end, the NCDOT has developed an internal CRF list, covering roughly 260 countermeasures, and regularly correlates internal data with the FHWA CMF Clearinghouse. An in-house committee of technical experts performs the estimations of the CRFs, and the internal CRF lists are supported by the evaluations, all of which are included on the NCDOT web site.

The institutional establishment of this evaluation program and the various facets it encompasses has assisted NCDOT staff in retaining safety project funding and even obtaining increases in funding. The evaluation of project performance has also impacted other areas of NCDOT operations, leading to increased confidence in NCDOT investments.

HSIP PROJECT AND PROGRAM EVALUATION AND MONITORING

IDOT has not yet established a formal evaluation process. IDOT currently undertakes the evaluation of safety projects with three individual approaches.

Evaluation occurs at a programmatic level. For example, District 1 has a significant number of intersection HSIP projects. By looking at performance on State routes, where funding is directed, and identifying if there has been a significant reduction in fatalities and serious injuries, Districts can determine whether or not the strategies they are applying are working at the program level.
	Districts evaluate projects using before and after data, although this is an informal process and there is a significant delay in obtaining crash data.
	The Illinois Center for Transportation and the Safety Engineering Technical Advisory Group, which is chaired by the State Safety Engineer, conducted three research projects, funded by SP&R, to evaluate system-wide safety improvements.Although they do not yet have a formal evaluation process, IDOT has informally evaluated safety projects at the programmatic, project, and systemic levels.

The Illinois Center for Transportation is conducting the three research projects, which focus on evaluation of the effectiveness of flashing yellow arrows, right-turn islands, and interchange improvements to prevent wrong-way movements. The Safety Technical Advisory Group within the Illinois Center for Transportation selected the research projects. The Technical Advisory Group solicits research proposals, and can also submit their own proposals. Research projects are selected based on what the Group perceives as priority research needs. For example, the SPFs used for statewide network screening and the basis for the SRI were selected by that group in 2007. It is important to note that if a researcher on the TAG submits a research topic proposal, they are not included in the review and approval of the project.

IDOT reviews the number of sites with implementation of countermeasures in order to determine which countermeasures to advance to evaluation in the research projects. Having a large sample size is important to obtaining system-wide before and after data. The research is reviewing the application of the strategies to over 100 intersections and over 400 interchanges. The results of the research projects will be used to develop State-specific CMFs and SPFs.

IDOT also conducts evaluations of experimental features when implementing new countermeasures. Doing so allows them to identify challenges, successes, and lessons learned, as well as develop policy based on what worked best. For example, when implementing cable rail, IDOT used different manufacturers on specific corridors so they could evaluate each type of system on its installation, maintenance, and by looking at crashes. They were then able to determine which types of cable rail to add to their approved products list. While the overall analysis of the cable rail was done as a corridor due to the fact that some segments may have not had any fatalities or injuries, but the corridor-wide data would show a different picture, the HSIP projects that came out of the analysis were done by funding segments.

The Alaska DOT&PF developed a methodology for demonstrating the effectiveness of HSIP projects. The Region Traffic Engineer prepares a Project Evaluation Worksheet for each project, evaluating the project based on 3-year post-construction data. The data is annualized for computation, and the cost of the project includes both the construction cost and the maintenance costs over the analysis period.

The Alaska DOT&PF evaluates projects based on 3-year post-construction data. They also use a program-wide b/c ratio to track performance of the overall HSIP.

The DOT&PF regions also use a project tracking report for HSIP that is turned into the Headquarters. The project report, which is separate from the HSIP Project Tracking Spreadsheet discussed in Chapter 9, evaluates targeted crashes for the before/after analysis and provides an overview of project performance. Incapacitating injuries are evaluated with fatalities to account for infrequency and isolation of crashes.

The DOT&PF has anticipated reductions in b/c ratios on account of the systemic approach that broadens applicability and breadth of a project, and these reductions are accepted as a result of the program’s initial effectiveness at addressing high-benefit locations. The b/c ratio for each project is calculated, when possible, and then aggregated to develop a program-wide b/c ratio, which is also an important metric of performance for DOT&PF.

CMF DEVELOPMENT

UDOT uses 3-year before/after comparisons of completed projects to analyze the performance of every HSIP project. Utah has developed a State specific CMF registry, which provides discrete CMFs and serves as a resource for project planning and screening. UDOT applies only one CMF to the targeted crashes identified for each project as part of the b/c analysis. This practice eliminates the additive effects of CMFs and ensures reliable b/c projections.

Enforcement presence can have an effect on traffic safety and the Alaska DOT&PF is conducting a research project to develop a CMF for enforcement presence (another means of quantifying the benefit of enforcement expenditures), particularly in the Alaska Transportation Safety Corridors. The research effort would be primarily focused on time spent on enforcement, providing the basis for a cost associated with potential benefits.

SUMMARY

All Host States have developed some form of before-and-after project analysis. Utah, Illinois, and North Carolina have strong evaluation programs that are being used to directly inform the development of SPF modifications, CRFs, and other values used in the screening and analysis portion of HSIP project development.

In both Alaska and North Carolina, each project is evaluated based on a standard evaluation form and methodology, using up-to-date crash data and measures of effectiveness appropriate for the countermeasures selected for use in the project. IDOT has an informal evaluation process that occurs at both the project and programmatic levels. In addition, IDOT is conducting research projects to evaluate system-wide safety improvements.

North Carolina DOT. *Safety Evaluation Group – Safety Evaluation Completed Projects.*Available online at https://connect.ncdot.gov/resources/safety/Pages/Safety-Evaluation.aspx. Accessed February 24, 2016.
      
  Book traversal links for Chapter 10–Evaluating the Success of the Program
      ‹ Chapter 6–Using Advanced Safety Analysis Methods and Tools
                    Up
                    Chapter 11–Observations and Conclusions ›
                
                Book:HSIP National Scan Tour
              Chapter 1–Introduction
                Chapter 2–Documentation of HSIP Processes
                Chapter 4–Understanding the Relationship between the SHSP and HSIP
                Chapter 6–Using Advanced Safety Analysis Methods and Tools
                Chapter 10–Evaluating the Success of the Program
                Chapter 11–Observations and Conclusions
                Appendix D: HSIP Resources
                References U.S. DEPARTMENT OF TRANSPORTATION

**Federal Highway Administration**

1200 NEW JERSEY AVENUE, SE

WASHINGTON, DC 20590

202-366-4000Subscribe to email updates
                    About
  

        
              About FHWA
                Careers
                Org Chart
                Staff Directories
                Work with UsPrograms
  

        
              Acquisition Management
                Civil Rights
                Federal Lands Highway
                Infrastructure
                Innovative Program Delivery
                Operations
                Planning, Environment, and Realty
                Policy
                Research, Development, and Technology
                SafetyPolicies, Rights, Legal
  

        
              About DOT
                Accessibility
                Budget and Performance
                Civil Rights
                FOIA
                Information Quality
                No FEAR Act
                Office of Inspector General
                Privacy Policy
                USA.gov
                Web Policies and Notices
                Web StandardsNews & Events
  

        
              Newsroom
                Press Releases
                Speeches & Testimony
                Media Contacts
                Connect with Us