Notice

This document is disseminated under the sponsorship of the U.S. Department of Transportation in 

the interest of information exchange. The U.S. Government assumes no liability for the use of the 

information contained in this document. This report does not constitute a standard, specification, 

or regulation.

The U.S. Government does not endorse products of manufacturers. Trademarks or manufacturers’

names appear in this report only because they are considered essential to the objective of 

the document.

Quality Assurance Statement

The Federal Highway Administration (FHWA) provides high-quality information to serve Government, 

industry, and the public in a manner that promotes public understanding. Standards and policies are 

used to ensure and maximize the quality, objectivity, utility, and integrity of its information. FHWA 

periodically reviews quality issues and adjusts its programs and processes to ensure continuous 

quality improvement.

Cover and Banner Image Sources:

Left – Female Driver: Comstock Images.

Middle Left – Roundabout: Cambridge Systematics, Inc.

Middle Right – Police: Cambridge Systematics, Inc.

Right – Road: PhotoDisc Inc.

Evaluation Process Model Logo: Federal Highway Administration.
Technical Report Documentation Page

1. Report No.

FHWA-SA-12-035

2. Government Accession No. 3. Recipient’s Catalog No.

4. Title and Subtitle

Strategic Highway Safety Plan – Evaluation Process Model

5. Report Date

March 2013

6. Performing Organization Code

7. Author(s) 8. Performing Organization Report No.

9. Performing Organization Name and Address

Cambridge Systematics, Inc.

100 CambridgePark Drive, Suite 400

Cambridge, MA 02140

10. Work Unit No.

11. Contract or Grant No.

DTFH61-05-D-00026

12. Sponsoring Agency Name and Address

U.S. Department of Transportation

Federal Highway Administration

Office of Safety, 

1200 New Jersey Avenue, SE

Washington DC 20590

13. Type of Report and Period Covered

Final Report

May 2012-October 2012

14. Sponsoring Agency Code

15. Supplementary Notes

FHWA: Jennifer Warren, Office of Safety

16. Abstract

This document provides an evaluation process model (EPM) that States can use to support a program evaluation of their 

Strategic Highway Safety Plan (SHSP). Program evaluation looks at the overall SHSP and helps States assess their SHSP’s 

process and performance. 

Safety improvements depend on a program of data-driven priorities and proven effective strategies. Evaluation helps States 

achieve such a program by determining whether current SHSP activities deserve enhancement, revision, or replacement. In 

addition, the Highway Safety Improvement Program (HSIP) requires regularly recurring SHSP evaluation to ensure the accuracy 

of data and proposed strategies; the EPM can help States address this evaluation requirement as well. 

The EPM features discussions on four basic evaluation components (planning for evaluation, process evaluation, performance 

evaluation, and using evaluation results). For each component, the document provides background information and recommended actions, as well as checklists (Chapters 1 and 4) and self assessment questions (Chapters 2 and 3) for practitioners to 

use to evaluate their SHSP. SHSP leaders and managers can benefit from considering the information and recommendations in 

this document. Regions, localities, tribes, and others may also find the EPM useful for evaluating safety plans.

17. Key Words 

Transportation safety, SHSP evaluation, SHSP 

performance measures, transportation 

planning, SHSP emphasis areas, safety goals 

and objectives.

18. Distribution Statement

No restrictions. This document is available to the public through the National 

Technical Information Service, Springfield, VA 22161.

19. Security Classif. (of this report) 

Unclassified

20. Security Classif. (of this page) 

Unclassified

21. No of Pages 

64

22. Price

Form DOT F 1700.7 (8-72) Reproduction of completed pages authorized

Foreword

The Strategic Highway Safety Plan Evaluation Process Model (EPM) could not have arrived at a better 

time. Now more than ever transportation safety focuses on the need for performance-based planning 

and management, which can only be accomplished through evaluation. Evaluation is critical to any 

safety planning process and the Strategic Highway Safety Plan (SHSP) is no exception. Through 

evaluation States determine the impact of their SHSP, enabling them to recognize their successes, 

uncover challenges in implementing strategies or meeting goals and objectives, and identify opportunities for improvement.

The EPM contains background information, recommended actions, checklists, and self assessment 

questions for addressing four basic evaluation components: 

•	 Planning for evaluation;

•	 Assessing the SHSP management effort;

•	 Assessing SHSP implementation and performance; and

•	 Interpreting and applying the results. 

The EPM contains examples from States successfully using evaluation methods to assess the SHSP; 

demonstrating evaluation is not only possible, but preferable. A series of worksheets are provided, 

which are designed to encourage a deeper review of each of the self assessment questions. After 

thoughtfully responding to the questions in these worksheets, States can develop an action plan that 

builds on existing strengths and addresses areas of weakness. Armed with this information, SHSP 

managers can improve the SHSP process, focus resources on the largest problems, and apply the 

most effective countermeasures. 

Thorough SHSP evaluation is an important investment in the State transportation safety program. The 

EPM was developed with this in mind and aims to provide States with a helpful and effective process 

to further their evaluation efforts. 

Sincerely yours, 

Anthony Furst

Associate Administrator for Safety

Federal Highway Administration

i

Table of Contents

The Strategic Highway Safety Plan 

Evaluation Process Model Overview .......................................................1

History and Background...................................................................... 2

EPM Purpose...................................................................................... 2

EPM Components............................................................................... 4

Chapter 1 – Planning for Evaluation: Getting Started............................5

Introduction ....................................................................................... 5

Purpose of Evaluation Planning............................................................ 6

Methods............................................................................................ 6

Evaluation Planning Checklist ........................................................... 14

Chapter 2 – Process Evaluation: Getting on the Right Track ...............15

Introduction ..................................................................................... 15

Purpose ........................................................................................... 15

Methods.......................................................................................... 15

Self Assessment Questions ................................................................ 26

Chapter 3 – Performance Evaluation: Outputs and Outcomes ...........28

Introduction ..................................................................................... 28

Purpose ........................................................................................... 28

Self Assessment Questions ................................................................ 34

Chapter 4 – Using Evaluation: The Focus is Results .............................35

Introduction ..................................................................................... 35

Interpret the Data ............................................................................. 35

Apply the Results ............................................................................. 36

Using Evaluation Checklist ................................................................ 38

Conclusion .............................................................................................39

Appendix – EPM Worksheets .................................................................40

1

The Strategic Highway 

Safety Plan Evaluation 

Process Model Overview

Transportation safety reached an important milestone in October 2007 when 

all 50 States and the District of Columbia completed development of 

Strategic Highway Safety Plans (SHSP). Since then, dramatic road safety 

improvements have occurred across the nation, including a decrease in 

fatalities and serious injuries. These improvements may in part be related to 

changes resulting from the implementation of data driven, multidisciplinary, 

collaborative, results-oriented SHSPs. 

Today all States are implementing SHSPs, and some are asking questions 

about SHSP effectiveness, e.g., which elements work well and which do not 

meet expectations. An organized approach to evaluation or an Evaluation 

Process Model (EPM) helps answer these important questions.

Elements of SHSP evaluation already are in place in many States; however, 

additional benefits can be realized by organizing and institutionalizing these 

elements into a comprehensive program evaluation. Evaluation strengthens 

SHSP efforts by:

•	 Demonstrating the SHSP’s contribution to transportation safety;

•	 Uncovering challenges in prioritizing or implementing programs 

and strategies; 

•	 Determining progress in meeting transportation safety goals and 

objectives;

•	 Validating emphasis areas and strategies, or revealing the need to 

revise them;

•	 Identifying opportunities for greater efficiencies and improvements to 

the SHSP; 

•	 Confirming the need for a comprehensive, data driven approach to 

safety; and

•	 Underscoring the need to prioritize transportation safety resources 

and funding. 

Evaluation is...

The systematic collection 

of information about the 

activities, characteristics, 

and outcomes of a 

program to make judgments about it, improve its 

effectiveness, and/or 

inform decisions about 

future programming.
2

History and Background

To address the devastating human and economic consequences of traffic 

crashes, the American Association of State Highway and Transportation 

Officials (AASHTO) published an SHSP in 1997 and encouraged States to 

develop data-driven, evidence-based SHSPs addressing the emphasis areas in 

the AASHTO plan. Some States subsequently developed SHSPs using the 

AASHTO plan as a model. In 2005, Federal legislation1 required States to 

develop SHSPs. 

The Federal Highway Administration (FHWA), together with other safety 

partners, provided SHSP development guidance in A Champion’s Guidebook 

to Saving Lives (Champion’s Guidebook) and implementation guidance in the 

SHSP Implementation Process Model (IPM). The next logical step after development and implementation is to evaluate SHSP process and performance to 

determine if limited safety resources are deployed efficiently and effectively. 

The information in the EPM builds upon and is supported by current evaluation 

methodology, model practices, and processes identified through in-depth 

interviews in six States, technical review and comment by FHWA safety experts, 

the knowledge and experience of the EPM development team, and lessons 

learned from a multi-State pilot test.

EPM Purpose

Why Should States Use the 

Safety improvements depend on a program of data driven priorities and 

proven effective strategies. Evaluation helps States achieve such a program by 

analyzing SHSP process and performance and determining whether current 

activities deserve enhancement, revision, or replacement. 

Evaluation is intended to take the place of trial and error, guesswork based on 

anecdotal evidence, and intuition. The Highway Safety Improvement Program 

(HSIP) requires regularly recurring SHSP evaluation to ensure the accuracy of 

data and proposed strategies. The EPM can help States address this evaluation 

requirement as well. 

The EPM provides guidance on evaluation methods, including when, where, 

and how to use them. It will help States answer basic questions of evaluation, 

such as:

1. What are we trying to do?

2. How well are we doing it?

3. How can we improve?

1 23 U.S.C. §148.

The definition of insanity is 

doing the same thing over 

and over and expecting 

different results.

Albert Einstein

(attributed)
3

Not every element described in the EPM will apply to every SHSP, but all SHSP 

leaders and managers can benefit from considering the information and 

recommendations. Regions, localities, tribes, and others also may find the 

EPM useful for evaluating safety plans.

When Should a State Use the E

The EPM’s primary purpose is to support a “point in time” program evaluation 

of the SHSP process and performance. The specific timing of SHSP evaluation is 

determined by the State, but must take into consideration Federal requirements 

as well as the needs and circumstances of the State.

The EPM is applied more successfully if certain elements of evaluation are 

ongoing and are instituted early in the SHSP process. For example, tracking 

the implementation status of strategies and setting performance goals and 

objectives are evaluation practices typically initiated early in the SHSP process 

and then monitored regularly to gauge progress. Practices such as these 

support the needs of SHSP evaluation and the EPM. 

Who Should Use the E

The EPM’s intended audiences are SHSP leaders, such as safety program 

managers/coordinators, transportation safety planners, and other traffic 

safety professionals.

How is Program Evaluation Differe

Project Evaluation

Program evaluation2 looks at the overall SHSP. It can identify where SHSP and 

emphasis area goals are met or unmet and point toward likely strengths or 

shortcomings, i.e., the failure to implement certain strategies or the identification of strategies not having the expected effect. A project-level evaluation 

examines the effectiveness of specific projects, such as replacing a four-way 

intersection with a roundabout, improving a pedestrian crossing, etc. Please 

refer to the Highway Safety Improvement Program (HSIP) Manual and/or the 

National Highway Traffic Safety Administration’s (NHTSA) The Art of 

Appropriate Evaluation: A Guide for Highway Safety Program Managers for 

additional information on project-level evaluation.

Is Program Evaluation Worth the Ef

SHSP program evaluation takes time and effort, but it is essential for continued 

improvements. The EPM is designed to help simplify and provide direction for 

achieving the most useful results. Sometimes program managers are reluctant 

to conduct evaluations because they fear potential fallout from negative 

2 In this document, the terms program evaluation and evaluation are used 

interchangeably. Project evaluation will be specified as such. 
4

results; however, both positive and negative results can lead to positive 

change. Successful SHSP implementation relies on being open to continual 

feedback and adjusting the SHSP as needed. Evaluation provides this feedback and helps identify successful, as well as ineffective strategies, make 

warranted course corrections, and enhance the return on safety investments.

EPM Components

The EPM contains background information, recommended actions, checklists, 

and self assessment questions for addressing four basic evaluation components: planning, process evaluation, performance evaluation, and the use of 

evaluation results. The recommended actions are steps designed to help 

States prepare for and conduct an SHSP evaluation, and the self assessment 

questions help identify successes, challenges, and opportunities in the SHSP 

itself or in the implementation process.

The appendix contains worksheets for Chapters 2 and 3. The worksheets are 

intended to provide a deeper review of the self assessment questions, document evaluation results, and record follow up action items to maintain or 

improve SHSP process and performance.

Planning for Evaluation – Getting Start

Chapter 1 addresses how to prepare for an evaluation and describes various 

planning methods and steps required to organize the evaluation. If an SHSP 

evaluation plan already is in place, Chapter 1 can be reviewed for ideas to 

strengthen it.

Process Evaluation – Getting on the Right

Chapter 2 is intended to help States evaluate SHSP management methods, 

including the strength of the leadership structure; whether the development 

process was sufficiently data driven, evidence-based, multidisciplinary, multimodal, and collaborative; and the degree of alignment among agency 

priorities. This examination will help leaders and managers identify 

opportunities to improve the overall SHSP process.

Performance Evaluation – Measuring Outpu

Outco

Chapter 3 provides methods for evaluating SHSP outputs and outcomes. Outputs 

measure the degree to which SHSP strategies and action plans are implemented. 

Outcomes measure the degree to which SHSP goals and objectives are met.

Using Evaluation – The Focus 

Chapter 4 offers methods for interpreting, applying, and sharing the evaluation results to improve the SHSP process and performance. 

Source: Federal Highway

Administration.
5

Chapter 1 – Planning for 

Evaluation: Getting Started

Introduction

It is never too early to institute good evaluation practices; in 

fact, planning for evaluation should begin when the SHSP is 

developed. During the early stages of SHSP development 

attention should be given to how progress will be measured 

and success determined. Some essential elements to have in 

place prior to conducting the SHSP evaluation include:

•	 SHSP emphasis areas with performance measures, goals, 

and measurable objectives;

•	 Implementation or action plans for each SHSP emphasis 

area with action steps, countermeasures, assigned roles 

and responsibilities, etc.; 

•	 Ongoing data collection and analysis; and

•	 Mechanisms for tracking SHSP implementation, and 

monitoring progress toward reaching goals and objectives.

Chances are most of these practices already are in place, 

providing a strong base for conducting an SHSP evaluation. 

The Champion’s Guidebook and IPM provide more detail on 

SHSP development and implementation. The steps and 

recommendations in these guides help create an SHSP ready 

for evaluation.

The EPM supports a comprehensive, high-level evaluation of 

the SHSP. This can occur at any point in time, but should take 

into consideration Federal requirements for regularly recurring 

evaluation, as well as the State’s needs and circumstances. 

Some likely times are:

•	 Prior to or as part of an SHSP update;

•	 After active SHSP implementation; or

•	 When an SHSP leader or other official wants to know whether the SHSP is 

making a difference in transportation safety.

Evaluation in Actio

California Regularly Tracks SHSP Implementation 

California’s comprehensive Strategic Highway 

Safety Plan (SHSP) includes 17 Challenge Areas 

and currently 175 Actions. The SHSP Steering 

Committee meets bimonthly with Challenge Area 

Leaders, Action Leads, and other SHSP safety 

stakeholders to discuss implementation issues, 

present new actions, address challenges, and 

celebrate success. The State uses an electronic 

system (OnTrack) to track the implementation and 

progress of SHSP Actions, which allows Action 

Leads to update the status of the Actions for which 

they are responsible and provide updated comments on-line. A unique feature of the OnTrack 

system is the ability to produce a variety of tailored 

reports. For example, users can sort data by 

agency to see a summary of the status of implementation of Actions by Lead Agency. Monthly 

SHSP status reports are generated from OnTrack 

and posted, as required, on the SHSP web site. 

The tracking tool keeps safety stakeholders 

involved and provides them with a report and 

agenda to discuss at SHSP Steering Committee 

meetings and Challenge Area Team meetings. It 

also establishes a level of accountability and gives 

safety stakeholders ownership in the SHSP 

implementation process. 
6

States should always be thinking about how to gauge success and what 

indicators will be used to determine progress. Typically these indicators fall 

into two categories: process evaluation and performance evaluation.

•	 Process evaluation addresses the SHSP procedural, administrative, 

and managerial aspects and assesses progress in these areas. 

•	 Performance evaluation addresses the outputs and outcomes resulting from SHSP implementation. It assesses the progress of SHSP 

implementation and the degree to which it is meeting goals and 

objectives.

Keeping these indicators in mind early in SHSP development and implementation will ensure important questions about the SHSP can be answered at 

any time.

Purpose of Evaluation Planning

The purpose of evaluation planning is to place managers and stakeholders in 

the best position to conduct the evaluation. It provides a formalized process and 

direction for evaluation activities, and helps answer the following questions:

•	 Objectives: What SHSP issues or questions need to be addressed? 

•	 Data: What data are needed to address the objectives?

•	 Resources: What resources are available?

•	 Roles and Responsibilities: Who is responsible? Who is involved?

•	 Results: How and when will the results be reported and used?

Methods

This section provides more detail on planning methods.

Identify Evaluation Object

Program managers should determine which SHSP process and performance 

areas to evaluate and develop a set of objectives to focus the evaluation effort 

and identify the resources needed to conduct the evaluation. Once objectives 

are drafted and resources are identified, they should be circulated to decisionmakers with authority over the program to enlist their support for the evaluation 

objectives and the resources needed, such as staff and funding support.

SHSP program managers will most likely develop three levels of evaluation 

objectives that address both process and performance issues.

SHSP Evaluation Planning Proc

Source: Cambridge Systematics, Inc.
7

•	 Level 1: Process. Process-level evaluation objectives relate to the 

overall SHSP process, such as the organizational structure by which the 

program is developed, managed, implemented, and evaluated. These 

objectives typically focus on elements related to leadership and 

management structures, collaboration, communication, etc.

•	 Level 2: Outputs. Output-level evaluation objectives relate to how the 

SHSP is being implemented and how well the actual implementation 

matches up to the plan.

Neither process- nor output-level evaluations require elaborate data collection efforts or a research design, but they do require an understanding of 

what should have happened in terms of leadership and program implementation. They also require a systematic approach for tracking program/

strategy implementation.

•	 Level 3: Outcomes. Outcome-level evaluation objectives focus on the 

impact the SHSP is having on transportation safety with respect to fatalities, injuries, and crashes. Changes in awareness, attitudes, and 

behaviors also can be addressed through outcome-level evaluation.

All three evaluation levels require program managers and leaders to develop 

evaluation objectives they believe will provide the most effective feedback 

regarding the progress of the SHSP. While an evaluation specialist can provide 

suggestions, the persons responsible for program effectiveness must ultimately 

decide what measures best indicate the overall success of the SHSP.

Table 1 illustrates evaluation objectives for the three levels. In this example, 

SHSP leadership determined assessing SHSP representation was a priority and 

established objectives to assess partner participation (level one objective). 

Impaired driving, occupant protection, and head on crashes were emphasis 

areas, so objectives were established to measure outputs and outcomes in 

these areas (level two and three objectives). To answer the level two and 

three evaluation objectives, the data must be available and tracked to 

measure progress.

Outputs: The extent to 

which SHSP strategies and 

actions are implemented.

Outcomes: The degree 

to which SHSP strategies 

and activities contribute to 

reducing fatalities and 

serious injuries, improve 

road user safety attitudes 

and behaviors, etc.
8

Table 1. Example Objectives by Evaluation Level

EVALUATION OBJECTIVES

Level 1: 

SHSP Process

Level 2: 

SHSP Outputs

Level 3: 

SHSP Outcomes

•	 Determine if toplevel management 

is involved in the 

SHSP and if it is 

sufficient to support 

the process.

•	 Determine if the 

emphasis area team 

members represent 

the 4 E’s of safety.a

•	 How many occupant protection and 

impaired driving 

high-visibility 

enforcement 

campaigns were 

conducted?

•	 How many citations 

were issued during 

the occupant 

protection impaired 

driving high-visibility 

enforcement 

campaigns? 

•	 How many miles of 

centerline rumble 

strips were installed 

on two-lane 

rural roads?

•	 What is the safety 

belt use rate statewide and in 

high-risk areas and 

populations?

•	 What is the 

unbelted fatality 

and serious injury 

rate in the State?

•	 What is the 

impaired driving 

fatality and serious 

injury rate in 

the State?

•	 How many head-on 

crashes on rural 

roads occurred in 

counties where 

centerline rumble 

strips were 

installed?

•	 What is the headon crash fatality 

and serious injury 

rate in counties 

where centerline 

rumble strips 

were installed?

a 4 E’s: engineering, enforcement, education, and emergency medical 

services/response.
9

Recommended Actions

1. Determine the SHSP process and performance areas to evaluate.

2. Identify the resources needed to conduct the evaluation.

3. Develop evaluation objectives for assessing the SHSP process.

4. Develop evaluation objectives for assessing SHSP performance 

(outputs and outcomes).

5. Circulate the objectives and resource requirements among SHSP 

decision-makers to obtain their support.

Review Methods for Data Collection and Mana

Quality data is necessary for conducting any evaluation. Fortunately, data 

quality and timeliness are improving in all States due to various programs that 

support data improvements, such as FHWA’s Crash Data Improvement 

Program (CDIP) and NHTSA’s traffic records assessment. If sufficient data 

are not available, program managers may need to revise the evaluation 

objectives or implement a data collection effort. Reviewing data collection 

and management methods will reveal whether:

•	 Data are available and of sufficient quality to track and assess progress 

and answer the evaluation objectives;

•	 Resources are available to collect data currently not available; and

•	 Objectives for which no evaluation data are available are reconsidered 

or modified.

Most States have some type of tracking mechanism in place. Tracking the 

following elements supports process and performance evaluations:

•	 The implementation status of SHSP strategies and related action steps;

•	 Outputs from implementing the SHSP strategies;

•	 Number and rates of fatalities and serious injuries; and

•	 Changes in road user attitudes and behaviors.

Recommended Actions

1. Review the existing data collection and management methods to 

determine the following:

a. Quality data are available to track progress and answer the 

evaluation questions;

Crash Data Qualit

Characteristi

•	 Timeliness.

•	 Accuracy.

•	 Completeness.

•	 Consistency/Uniformity.

•	 Integration.

•	 Accessibility.

For more information refer to 

the FHWA Crash Data 

Improvement Program Guide.
10

b. Resources are available to collect data currently not available; and 

c. Objectives for which no evaluation data are available are reconsidered or modified.

2. Review the tracking mechanism(s) in place to determine: 

a. Where the information can be obtained and what is captured (e.g., 

implementation status of SHSP strategies and actions, number and 

rates of fatalities and serious injuries, etc.). 

b. If information needed for the evaluation is not formally tracked, can 

a mechanism be put in place or information gathered using an 

alternative method?

Determine How to Measure P

Sufficient data are not always available to measure everything, but establishing 

specific evaluation objectives will help identify which data are a priority. Lack 

of data should not preclude the attempt to evaluate the SHSP. If quantitative 

data are not yet available, a qualitative evaluation can be performed. The 

important point is to focus on what can be evaluated. 

Measuring success becomes more difficult as the measurement moves from 

outputs (e.g., what activities occurred, such as the number of traffic citations) 

towards outcomes. However, outcomes, such as a decrease in traffic crashes 

or crash-related injuries and deaths hold greater importance. The following 

list suggests performance measures that could be used in descending order 

of importance. 

•	 Primary Outcome Measures. Reductions in the number and rate of 

fatal and serious injury crashes statewide and by Emphasis Area.

•	 Secondary Outcome Measures (also called Proxy Measures). 

Changes in observed behavior, such as safety belt and helmet use, 

speeding, red light running, pedestrians who jaywalk, changes in average 

speeds due to traffic calming countermeasures, and other observed 

behavior changes.

•	 Self Report Measures (what people say). Have you ever driven after 

having too much to drink? How often do you wear your safety belt? 

•	 Attitudinal Data Measures (what people believe). Support for 

legislative initiatives; knowledge of safety belt laws; teen attitudes about 

drinking and driving; and attitudes toward roundabouts, rumble strips, and 

other infrastructure safety improvements.

•	 Awareness Data Measures (what messages people have heard). 

Awareness of high-visibility enforcement, perceived risk of getting a 

traffic ticket. 

Source: Cambridge Systematics, Inc.
11

•	 Activity Levels (program implementation). Miles of rumble strips, 

pavement markings, signage, and other infrastructure improvement; 

citations issued by the police, special police patrols, and check points; 

presentations; training programs; media coverage; legislation;3 etc.

If the highest levels of measurement are not available, other more obtainable 

measures might be considered.

Recommended Actions

1. Review the existing SHSP performance measures and categorize them as 

(in descending order of importance): 

a. Primary outcome;

b. Secondary outcome;

c. Self reporting;

d. Attitudinal; 

e. Awareness; or

f. Activity level. 

2. Include the higher level performance measures (e.g., primary outcomes) in 

the evaluation if data are available (to answer the evaluation objectives) 

and consider other, more attainable measures if they are not available.

Identify and Secure Resou

Evaluation often requires resources, including funding, staff time, and evaluation expertise. It is important to identify and secure these resources so they are 

available for the program evaluation and throughout the life of the SHSP. 

Evaluation does not require high-level scientific or technical expertise; however, 

it does require people capable of objectively collecting, tracking, and analyzing 

data and other information to produce and interpret results. SHSP leadership 

should dedicate staff and financial resources to support this function.

Evaluation of other safety plans or programs, such as the HSIP, Commercial 

Vehicle Safety Plan (CVSP), or the Highway Safety Plan (HSP), should be 

underway and could provide additional resources or expertise for the SHSP 

evaluation effort. Universities (e.g., professors and graduate students who 

may be looking for evaluation projects) provide another good source of 

evaluation expertise. 

Establish a timeframe for evaluation or a target date for completion. While 

most of the information and data needed for the evaluation already are 

collected and accessible, it will take time to gather and analyze all of the 

pieces and synthesize them into meaningful results. 

3 The National Highway Traffic Safety Administration (2008). The Art of Evaluation, 

page 32.
12

Recommended Actions

1. Identify available resources to support SHSP evaluation.

2. Identify individuals or agencies with the skills to analyze data and other 

information and provide evaluation support.

3. Explore the availability of universities, professors/graduate students, and 

others if extra help or expertise is needed.

4. Collect information on current evaluation efforts among the partners, e.g., 

CVSP, HSP, etc.

5. Determine a timeframe for conducting SHSP evaluation.

Assign Evaluation Roles and Respo

Evaluation typically includes a management function to drive the effort and 

an analysis function for collecting, analyzing, and reporting data and results. 

These functions could be performed by a single person or by several people. 

Responsibilities should be clearly defined in either case and include a 

description of the data to be collected, the level and type of analyses to be 

conducted, and a reporting schedule.

Recommended Actions

1. Determine the agency responsible for coordinating the overall program 

evaluation effort.

2. Identify the individual(s) who will manage the evaluation effort, collect and 

analyze data, and report the evaluation results.

Report and Use Evaluation

Define the reporting process and describe how the results will be used to 

improve SHSP process and performance. The reporting process should 

identify the parties responsible for generating and distributing the results, the 

audiences who will receive the information, when will they receive it, and how 

it will be presented (e.g., spreadsheets, formal written documents, verbal 

updates, e-mail notices, etc.). Assign a lead person responsible for generating 

and distributing evaluation results.

States should document how they intend to use the results, e.g., to identify 

successes, gaps, challenges, and opportunities. At a minimum use the results 

to review and, where necessary, improve SHSP process and performance. 

Other uses of evaluation results include informing elected officials, engaging 

safety stakeholders, and reaching out to a broader audience, e.g., the public. 

For more information on how to use evaluation results, see Chapter 4.
13

Recommended Actions

1. Identify the parties responsible for generating the evaluation results.

2. Assign a lead person to pull together the results from the various parties 

and distribute them to the appropriate committees, agencies, etc. 

3. Determine who will receive the results of the evaluation, i.e., agency 

executives, Steering Committee members, all stakeholders, etc.

4. Decide how the results will be formatted, i.e., formal reports, 

spreadsheets, PowerPoint presentations, etc.

5. Identify how the evaluation results will be used.

Document Evaluation Ap

States should document the evaluation approach. This need not be a long or 

complicated plan. It could be as simple as a one-page description of the 

evaluation objectives, data needs, resources, roles and responsibilities, and 

methods for applying the results.

Documenting these elements of the evaluation will formalize the process, keep 

the evaluation focused on the original objectives, record details of the evaluation, such as the specific tasks and roles and responsibilities, establish a level 

of accountability, and serve as a reference for new leadership and staff.

Recommended Actions

1. Document the evaluation elements (objectives, data needs, resources, roles 

and responsibilities, application methods, etc.) to formalize the process.

Source: Federal Highway

Administration.
14

Evaluation Planning Checklist 

The following checklist is designed to support evaluation planning. If most or 

all of these activities are completed, the State is prepared for SHSP evaluation.

1. Identify evaluation objectives.

2. Identify the data needed to address the objectives and perform 

the evaluation.

3. Determine if existing data collection strategies are sufficient for evaluation.

4. Identify resources needed to collect data or adjust evaluation objectives if 

available data are insufficient for evaluation purposes.

5. Assign responsibility for generating and distributing evaluation results.

6. Document a reporting process to update agencies, partners, and 

decision-makers on SHSP evaluation results.

7. Determine how evaluation results will be applied.

8. Document the approach or plan for the evaluation.
15

Chapter 2 – Process 

Evaluation: Getting on the 

Right Track

Introduction

Process evaluation is the examination of SHSP management processes. The 

results identify successful practices; alert SHSP leaders, managers, and stakeholders to potential needs, weaknesses, and threats; and provide insights for 

overcoming those challenges and improving the process.

Purpose

Evaluation requires attention to SHSP outputs and outcomes, but also to the 

management of the SHSP. Conducting a process evaluation provides insight 

into a variety of SHSP program management elements, such as organizational 

structure; coordination; the use of data in determining emphasis areas, goals, 

objectives, strategies and actions; and the alignment of agency priorities.

Methods

Methods to assess SHSP processes will vary, but this section offers some 

potential methods used effectively by States.

Identify Process Evaluation E

The following elements should be assessed for a comprehensive process 

evaluation. These have been identified in the SHSP Implementation Process 

Model (IPM) as essential for successful SHSP implementation, but States can 

consider additional elements as well.

•	 SHSP organizational structure;

•	 Multidisciplinary, multimodal collaboration;

•	 SHSP goal and objective setting methods; 

Source: Cambridge Systematics, Inc.
16

•	 Data driven and evidence-based emphasis areas, strategies, and 

actions; and

•	 Aligned agency priorities.

SHSP Organizational Structure

Assessing the organizational structure for SHSP implementation provides 

information about how well it is performing. The organizational structure 

varies from State to State, but regardless of the form it takes, it should function 

to manage the entire SHSP process; from development and implementation to 

evaluation and measuring performance. Assessing the SHSP organizational 

structure helps determine if it is providing the support needed to manage the 

SHSP process. An example of an SHSP organizational structure is provided in 

Figure 1 below.

Figure 1. Example SHSP Organizational Structure

Key features of this structure include:

•	 An executive committee or leadership council composed of members who 

are leaders of departments and agencies, such as the department of 

transportation, the State Highway Safety Office (SHSO), public safety, 

statewide law enforcement organizations, licensing agencies, departments 

of health and education, and others.

•	 A steering committee or working group designed to represent not only the 

executive committee, but also the majority of partners.

•	 Multidisciplinary emphasis area teams composed of stakeholders from the 

safety community, as well as other interested partners, experts, and citizens. 

Some States also encourage local, tribal, regional, and district representation 

to provide additional focus for certain geographical regions.

Executive Committee

State and Community Elected and Appointed Officials

Steering Committees/Working Groups

Agency Managers Community Coalitions

Emphasis Area Teams

Agency Managers Safety Stakeholders Technical Experts

SHSP Program 

Coordinator 

or Manager

Source: PhotoDisc Inc.

Source: Cambridge Systematics, Inc.
17

•	 An SHSP program coordinator who is responsible for the overall 

day-to-day management of SHSP activities.

A review of the organizational structure should identify and document:

•	 How representatives are recruited. A State should review how they 

recruit representatives for the various SHSP committees and teams to 

determine if they have an active and successful process for reaching out to 

partners. For example, in some States, leaders of one or more of the key 

agencies recruit their counterparts from other agencies. In other States, 

the SHSP manager may hold one-on-one meetings with top-level executives to discuss their participation and identify incentives for their 

involvement. States also should determine if potential partners can easily 

contact and reach out to those already involved in the SHSP to express 

their interest in participating. 

•	 Representation on various committees or working groups. A 

State should examine the representation on SHSP committees and working 

groups to determine if they are multidisciplinary and multimodal. 

Representation should include a broad range of stakeholders from key 

agencies and organizations. 

•	 Access to leadership, resources, etc. States should assess the level 

of access the various SHSP committees and working groups have to top 

management and leadership. They also should consider if they have 

ample opportunities to keep leadership informed of SHSP progress, and 

gain their support in addressing challenges. Emphasis area teams should 

have access to steering or executive committee members as well as 

managers in their respective agencies.

•	 Meeting frequency and level of participation on SHSP committees, teams, and working groups. States should review the 

degree to which frequency and participation levels have met or changed 

from original expectations. 

•	 The role and function of the SHSP committees, teams, and/or 

groups. Use the following questions to assess the roles and functions: 

− Does a committee (e.g., executive committee) oversee the SHSP effort? 

How often does it meet to assess progress, determine priorities, 

recommend course corrections, and address challenges? Does the 

committee appoint representatives to a steering committee or to 

working groups? Do these representatives have access to top-level 

executives and the ability to make decisions?

− Is a committee (e.g., steering committee) responsible for day-to-day 

SHSP implementation?

− Is a committee responsible for providing support to local and 

regional partners?
18

− Do the Emphasis Area Teams develop the performance measures, 

strategies, and action steps necessary for translating the goals and 

objectives of the SHSP into detailed action plans?

− Is there an SHSP program manager or coordinator who oversees the 

overall day-to-day management of SHSP activities (e.g., convenes, 

facilitates, and documents committees, working groups, emphasis area 

teams, etc.)?

− How do the existing roles and functions of the various SHSP committees 

and groups compare to original expectations?

A candid discussion among the committee and/or working group members 

can often identify the strengths and weaknesses of the organizational structure 

(e.g., multidisciplinary representation, leadership or access to leadership, etc.) 

and identify strategies for improvement.4

Recommended Actions

1. Review the SHSP organizational structure to identify and document its 

format and functions(s).

2. Examine the positions of persons serving on SHSP committees (e.g., 

steering and executive committees), as well as emphasis area and local/

regional/district teams to determine their contribution to the SHSP process 

and access to leadership and resources. 

3. Review the schedule of SHSP leadership and committee meetings to 

determine if they meet as frequently as planned or needed.

4. Review the SHSP organizational structure to determine the level of support 

provided to partners in local and regional coalitions. 

5. Review the role and function of SHSP committees, teams, and/or groups. 

Compare these current roles and functions with the expectations set at the 

beginning of the SHSP process.

Multidisciplinary, Multimodal Collaboration

Traffic fatalities and serious injuries involve multiple contributing factors which 

affect the mission and work of many disciplines and agencies. This highlights 

the need for multidisciplinary and multiagency approaches and is why the 

SHSP is required to involve the input of disciplines representing the 4 E’s of 

safety (engineering, enforcement, education, and Emergency Medical Services 

(EMS)). SHSP partners typically include the State Department of Transportation 

(DOT); the State Highway Safety Office (SHSO); departments of public safety 

(State police or patrol); health and education; Motor Carrier Safety Assistance 

Program (MCSAP) managers; Federal partners (FHWA, FMCSA, and NHTSA); 

4 Review the SHSP Implementation Process Model (IPM) for information about establishing effective executive, steering, and emphasis area teams.
19

metropolitan planning organizations (MPO); local agencies; tribal governments; special interest groups; and others.

The various agencies and organizations involved in the SHSP bring unique and 

valuable perspectives to bear on the roadway safety problem. Their differing 

philosophies and problem solving approaches, however, can sometimes make 

collaboration challenging. Measuring the effectiveness of communication and 

collaboration among the disciplines and agencies can be difficult; however, it 

is important to know if the right people are at the table and if they are working 

together effectively.

Safety improvement also requires attention to the different roadway users and 

the interaction among them, e.g., passenger vehicles, commercial vehicles, 

trains, pedestrians, bicyclists, motorcyclists, transit users, and others. 

Communication, collaboration, and interaction among the disciplines and 

modes improve the potential for identifying and addressing issues, implementing 

programs, and assessing effectiveness. Each State’s SHSP leadership should 

assess the decision-making environment and encourage key agencies and 

individuals to be involved. The Champion’s Guidebook and IPM provide further 

detail for identifying the collaboration partners and processes.

The representatives that make up emphasis area action teams, task forces, 

etc., can be interviewed or surveyed to determine the degree to which multiple 

disciplines and modes are represented and collaborative arrangements are in 

place. These individuals can be contacted during regular monthly or quarterly 

meetings; however, input from less active stakeholders also is valuable. Often 

working group members are program managers rather than practitioners or 

field personnel. Interviews with field personnel will strengthen the evaluation’s 

validity and lead to more robust results. States should identify the interviewees 

depending on the SHSP emphasis areas, strategies, and action plans. Typical 

interview or survey candidates include: 

•	 Project and program directors;

•	 State and local law enforcement participating in SHSP-related campaigns;

•	 HSIP, SHSO, and MCSAP staff;

•	 Transportation planners at the State, regional, and local levels;

•	 County, regional, and local traffic and safety engineers;

•	 Other State, regional, and local safety professionals (from Departments of 

Health, Departments of Education, etc.); and

•	 State and local elected and appointed officials.
20

When reviewing collaboration processes, States should ask:

•	 Does a basic foundation for effective collaboration and a 

process to support collaborative efforts exist? For example, a 

memorandum of understanding (MOU) can be a useful tool for institutionalizing the collaborative process. MOUs also help with the 

implementation of strategies and build sustainability and accountability. 

As stakeholders change and new partners come on board, commitment 

on the part of all agencies can be reaffirmed by updating the MOU. 

What mechanisms are in place to support collaboration?

•	 Does collaboration result in multidisciplinary safety decisions 

that are reflected in other plans and programs? Other safety 

and transportation plans, such as the HSP, HSIP, the CVSP, the long-range 

transportation plan (LRTP), statewide and regional transportation improvement programs (S/TIP), and other planning and programming documents 

should be consistent with the SHSP goals, objectives, and strategies as 

appropriate to their missions, regulations, requirements, and stakeholders. 

Conversely, SHSP managers and stakeholders should consider the relevant 

goals, objectives, and strategies in the other plans. Has collaboration 

resulted in this level of coordination and alignment?

•	 Are the vision, mission, and goals of the SHSP clearly and 

continually communicated to all partners and stakeholders? 

Identify formal and informal communication mechanisms for ensuring 

frequent and continuous information regarding the SHSP, such as regularly 

scheduled steering committee meetings, newsletters, retreats, etc. How 

effective are they?

Reviewing the status of SHSP collaboration should reveal if the SHSP process is 

truly multidisciplinary and multimodal. Collaboration should be evident not 

only in the SHSP development process, but also in the alignment of State 

safety priorities and plans and the implementation of SHSP strategies. 

Recommended Actions

1. Review the membership of the various committees, emphasis area action 

teams, task forces, etc., to assess the degree to which multiple disciplines 

and modes are represented and actively involved.

2. Determine if mechanisms are in place that facilitate an active, efficient 

collaborative process, such as MOUs.

3. Review the planning documents of the various agencies and safety partners to discern how well they reflect elements of the SHSP, such as the 

goals, objectives, and strategies.

4. Determine if the SHSP vision, mission, and goals are clearly and continually communicated to all partners and stakeholders.
21

Goal and Objective Setting Methods

Establishing performance measures and setting transportation safety goals and 

objectives are becoming widely advocated practices in the U.S. and internationally. Evidence shows reductions in fatalities and fatality rates are positively 

correlated with setting measurable objectives.5

 Objectives improve road safety by:

•	 Providing a way to measure program effectiveness;

•	 Steering programs toward implementation of proven effective road safety 

countermeasures;

•	 Motivating stakeholders to act; and

•	 Establishing a method for assessing accountability.6

The process evaluation should identify and assess the methods used to set 

SHSP goals and objectives. For example:

•	 Are data driven objective setting methods used, as opposed to 

aspirational goals?

•	 Are goals aggressive yet achievable?

•	 Are objectives specific, measurable, time bound, and realistic?

SHSP goals and objectives typically have a foundation in data analysis and 

broad political support. They should be aggressive yet achievable. Numerous 

methods are used to identify and set objectives, such as using historical trends, 

modeling, benchmarking, relying on expert judgment, collaborating with 

stakeholders, applying legislative mandates, adopting a national strategy or 

recommendation, or some combination of the above. One of the most commonly used methods is trend analysis, which is discussed in more detail below.

Using Trend Analysis to Set Objectives

Often some form of trend analysis is employed to determine where the State will 

be if current trends continue. The anticipated effects of the State’s efforts are 

then applied to the trend to estimate the impact and set SHSP goals and 

objectives.

To conduct a trend analysis, evaluators should chart a rolling average over 

time using three to five years of data. A rolling average chart provides a 

smoother line than a chart of individual data points, making trends more 

evident. These trend lines show what has occurred in the past and can then 

5 Gargett, S., Connelly, L.G., and Nghiem, S. (2011). Are we there yet? Australian road 

safety targets and road traffic crash fatalities, BMC Public Health, 11:270, 2011. 6 OECD and International Transport Forum. (2008). Towards Zero: Ambitious road 

safety targets and the safe system approach, Paris, OECD.

Evaluation in Actio

Louisiana Uses Data to Set 

Annual SHSP Objectives 

Louisiana uses data to set 

annual SHSP objectives for 

the reduction of trafficrelated fatalities and serious 

injuries. The State adopted 

a goal to halve fatalities by 

2030 and uses a baseline 

average of 2006-2008 

fatality and serious injury 

data to calculate the rate of 

change necessary to achieve 

the 50 percent reduction. 

Louisiana regularly evaluates the plan’s effectiveness 

by using the annual number 

of motor vehicle-related 

fatalities and serious injuries 

as performance measures. 

The same metric is also 

used to track performance 

for each emphasis area and 

to indicate the number of 

countermeasures underway, 

completed, or not started. 

Louisiana bases the selection of SHSP emphasis areas 

on baseline data, which 

clearly defines the problem, 

the contributing crash 

factors, programs and 

projects with the greatest 

potential for improvement, 

and whether sufficient 

resources to implement 

proven countermeasures 

are available.
22

be extrapolated into the future to provide a sense of what might be expected if 

the trends continue. This extrapolated trend line becomes the baseline against 

which goals and objectives are set.

Figure 2 shows historical fatality data from Missouri. The bars show the actual 

number of fatalities from 2000 to 2009 while the line shows the fatality rate 

per 100 million vehicle miles of travel over the same time period. To conduct 

a trend analysis, this data would be projected into the future assuming no 

changes in current safety efforts. From this baseline, future goals and objectives are set.

Figure 2. Missouri Fatalities and Fatality Rates

Trend analysis also can be conducted by emphasis area to show more precisely where safety improvements and degradations are occurring.

Figure 3 demonstrates another method for displaying historical data to set 

future objectives. These data show general trends for Washington State’s 

emphasis area and can help SHSP leadership identify which emphasis areas 

need greater attention. The horizontal bars show three-year total fatalities by 

emphasis area. The red bars show earlier years (2003-2005) and the blue 

bars demonstrate the most recent years (2006-2008). For example, the blue 

bar associated with Alcohol and/or Drug Involved Fatalities extends further 

than the red bar which means the number of fatalities is increasing rather than 

staying the same or improving. To conduct trend analysis for each of the 

emphasis areas, a State would track historical fatalities by emphasis area, set 

a baseline by projecting the resulting trends into the future, and set goals and 

objectives against the baseline. 

Source: Cambridge Systematics, Inc. based on data from the Missouri Department 

of Transportation.
23

Figure 3. Emphasis Area Chart (Dervied from Washington 

State SHSP)

Recommended Actions

1. Review the analysis methods used to set objectives. Are they data driven?

2. Determine if objectives are specific, measurable, time bound, and realistic.

Data-Driven and Evidence-Based Emphasis Areas, Strategies, 

and Actions

Process evaluation helps States assess whether their efforts are data driven and 

focused in areas where documented safety problems exist, for both infrastructure and behavioral programs. Process evaluation also assesses whether the 

appropriate strategies and actions have been selected. 

A data-driven approach to problem identification is critical because it provides 

the information and statistics needed to determine the most significant safety 

problems (emphasis areas) and select the appropriate performance measures. 

Using crash and other safety data to guide emphasis area selection helps 

direct resources to the areas of greatest need.

Process evaluation also should include a review of how strategies are selected. 

It is best to select strategies and countermeasures based on evidence from the 

research. These can be found in a variety of sources, including the NCHRP 

500 Series,

7 the FHWA Crash Modification Factors Clearinghouse, the 

7 NCHRP: National Cooperative Highway Research Program.

Evaluation in Actio

Virginia Uses Data 

to Identify SHSP 

Emphasis Areas 

Virginia uses a data-driven 

process to identify emphasis 

areas for the SHSP. 

Stakeholders review the 

percentage of total traffic 

deaths and severe injuries 

attributed to each of the 

potential emphasis areas in 

the AASHTO SHSP. While 

crash factors are often 

interrelated and reflected in 

more than one emphasis 

area, the significant areas 

that Virginia’s crash data 

highlights are roadway 

departure, speeding, 

intersection crashes, and 

young drivers. These 

became the 

Commonwealth’s SHSP 

emphasis areas.

Source: Cambridge Systematics, Inc. based on data from Washington State’s Strategic 

Highway Safety Plan.
24

Highway Safety Manual, Countermeasures that Work, and NCHRP 622: 

Effectiveness of Behavioral Highway Safety Countermeasures. If a new, 

promising, or innovative strategy with less research to support its effectiveness 

is identified, the strategy should be accompanied by an evaluation to validate 

its impact and justify future use.

Recommended Actions

1. Review methods used for identifying safety problems and selecting emphasis areas. Determine whether safety data analysis was the primary input to 

problem identification and emphasis area selection. 

2. Determine whether the latest safety data and research was used to identify 

evidence-based strategies and actions. 

3. Identify strategies in the SHSP that may lack a preponderance of evidence 

of effectiveness. Determine if there was enough information or evidence 

(e.g., success in another State) to justify its use, or if an evaluation was 

conducted, or is planned, to validate the use of the strategy.

Aligning Agency Priorities

Another type of process evaluation measures shifts in agency priorities. 

Effective SHSP management leverages the resources of other transportation 

planning and programming activities. SHSPs are designed to be the umbrella 

or overarching safety plan for a State; hence, it is expected the partner agencies will begin to shift their priorities where appropriate to align with the SHSP 

goals, objectives, performance measures, emphasis areas, etc. Evaluation of 

these often subtle changes could be accomplished by examining the individual 

agency programs before and after SHSP implementation to identify programmatic and budgetary shifts in agency priorities, as well as changes to the safety 

decision-making culture. These would indicate a shift of institutional processes 

and practices to align with SHSP goals, objectives, and performance measures 

at agencies responsible for transportation safety.

Alignment should occur in the State’s: Highway Safety Improvement Program 

(HSIP); Highway Safety Plan (HSP); Commercial Vehicle Safety Plan (CVSP); 

Long Range Transportation Plan (LRTP); and the Statewide and Metropolitan 

Transportation Improvement Program (S/TIP). Alignment also should be 

evident in other transportation planning documents, such as pedestrian/bicycle 

plans, corridor plans, and freight plans among others.

Recommended Actions

1. Review the HSP, CVSP, HSIP, LRTP, and S/TIPs to determine the degree to 

which they align with the SHSP.

2. Review other plans, e.g., pedestrian/bicycle plans, corridor plans, local 

road plans, etc., to determine the degree to which their goals and strategies align with the SHSP. 

Evaluation in Actio

Maine Uses Data to Select 

SHSP Strategies 

Maine reviews crash, 

complaint, citation, injury, 

and emergency medical 

services (EMS) data, as well 

as observational and 

attitudinal survey data to 

inform the selection of SHSP 

strategies. The State 

collects crash and EMS data 

electronically and strong 

collaboration among State 

agencies facilitates effective 

data sharing and access, 

which results in excellent 

data quality and enables 

the stakeholders to uncover 

detailed information about 

traffic safety challenges. 

For instance, a review of the 

crash data revealed the 

need for further analysis of 

operating after suspension 

(OAS) data to filter out 

suspensions unrelated to 

driving behavior, such as 

failure to pay child support; 

hence it is difficult to link 

OAS strategies to crash 

outcomes. Therefore, 

Maine adopted a set of 

strategies in the SHSP to 

address the issue, such as 

develop a mechanism in the 

Maine Crash Reporting 

System to identify the reason 

an operator involved in a 

crash is suspended.
25

Process Evaluation Activ

States can use a number of methods to collect process evaluation information. 

Examples include:

1. SHSP Retreat or Group Exercise. The working group or steering 

committee holds a retreat or group exercise and devotes themselves to a 

candid discussion of the SHSP process. An experienced facilitator should 

be considered to keep the participants on topic and help the group work 

through areas of uncertainty or potential disagreement. The discussion 

should focus on accomplishments and identify opportunities for improvement. A retreat report should document the findings, follow up actions, 

and persons responsible for implementing the actions. Refer to the HSIP 

Self Assessment Toolbox for more ideas on how to format and facilitate 

this type of activity.

2. Internal Evaluation. The SHSP Program Coordinator, with staff 

support, conducts an internal evaluation of the SHSP process. This 

approach depends on support from the State’s SHSP leadership.

3. Objective Observer. An objective observer collects information 

through a survey and/or a series of interviews with working group members, safety stakeholders, top management, and others. The observer 

documents the findings and follows up with the interviewees to validate or 

clarify the findings and gather any additional information. The observer 

prepares a report for review by SHSP leadership, the working group, and 

other stakeholders.

4. Peer Review or Exchange. A State hosts a peer review or exchange 

with one or more other States to present its SHSP process elements and 

methods. Together they identify strengths and weaknesses and the peer 

State representatives offer alternative methods to improve SHSP process 

effectiveness. Peer exchanges provide a good model for improving 

practice, as demonstrated by the State DOT research divisions which have 

been implementing this practice for many years. An important element is 

the involvement of an experienced facilitator with knowledge of the 

process and the subject areas who keeps the discussion on track and 

documents and reports the findings.

Recommended Action

1. Review the possible methods to collect process evaluation information. 

Identify the pros and cons of each method.

2. Determine and document the most appropriate method and the rationale 

for the selection.
26

Self Assessment Questions

The following self assessment questions are designed to inform process 

evaluation. Answering “yes” to a question indicates the State has a well 

functioning SHSP process in that area of review. Answering “no” indicates 

improvements can be made.

1. Is the SHSP process supported by an actively engaged 

organizational structure?

2. Are top-level managers represented in executive committees or leadership 

structures/groups established for the SHSP?

3. Are members of the executive or leadership group, the steering committee, 

the emphasis area teams, and other groups multidisciplinary and 

multimodal?

4. Do members of the executive committee or leadership group have the 

decision-making authority needed to effectively support the SHSP process?

5. Do members of the executive committee or leadership group assign 

persons with decision-making authority to the steering committee or 

working group?

6. Are multiple transportation modes represented, and do they actively 

participate on the steering committee/working group and emphasis 

area teams?

7. Has an SHSP program coordinator or manager been assigned? What 

percentage of this person’s time is dedicated to the SHSP?

8. Do the leadership and working groups/committees meet as frequently 

as expected?

9. Are emphasis areas supported by teams with engaged leaders?

10. Are local/regional/district coalitions supported by the SHSP organizational 

structure?

11. Are the necessary disciplines, modes, and agencies (representing the 

4 E’s) engaged in SHSP decision-making and implementation?

12. Do the stakeholders regularly collaborate on decisions that affect SHSP 

updates and implementation?

13. Do the necessary stakeholders collaborate and jointly decide on SHSP 

goal and objective setting methods?

14. Are data-driven methods, such as trend analysis, used to establish goals 

and set aggressive, yet achievable, objectives?

Evaluation in Actio

Rhode Island Peer Exchange 

Provides Help for SHSP 

Update

At the beginning of the SHSP 

update process Rhode Island 

hosted a peer exchange to 

learn from the experiences of 

Georgia and Maine. A focus 

of the discussion was 

tracking SHSP implementation and measuring SHSP 

effectiveness. Rhode Island 

would like to invest in data 

collection and analysis 

systems to accurately identify 

emphasis areas and track 

performance. Because data 

are central to the evaluation 

process, the State was 

interested to learn how peer 

States are addressing the 

issue of incomplete data. 

Georgia identified training 

for law enforcement personnel and the implementation 

of an electronic crash 

reporting system as critical 

for improving crash data 

accuracy and timeliness. 

Maine reported the benefits 

of their simple query format 

to perform crash data 

analysis on attributes 

recorded in crash reports, as 

well as the development of 

standard reports on topics 

such as motorcycle crashes 

and high crash locations to 

communicate the latest 

crash trends. 
27

15. Are objectives specific, measurable, time bound, and realistic?

16. Is data analysis used to select the emphasis areas?

17. Are the emphasis area strategies selected through an evidence-based process?

18. Are promising and innovative strategies with less evidence of effectiveness 

accompanied by an evaluation?

19. Have the various agencies and safety partners incorporated elements of 

the SHSP into their planning documents? (HSPs, HSIPs, CVSPs, LRTPs, 

S/TIPs, etc.)
28

Chapter 3 – Performance 

Evaluation: Outputs and 

Outcomes 

Introduction

Over the past 15 years, State DOTs, SHSOs, Metropolitan Planning 

Organizations (MPOs), and other government agencies have increased the 

use of performance management principles to plan, prioritize, track, and 

improve the effectiveness of their programs. Performance management and 

the use of performance measures assess the outputs and outcomes resulting 

from SHSP implementation.

Purpose

The purpose of performance evaluation is to determine how effective the SHSP 

has been in meeting its goals and objectives. Performance evaluation compares the actual degree of SHSP implementation (output evaluation) and the 

degree to which the implemented strategies have contributed to (or are correlated with) measurable change (outcome evaluation).

Output Evaluat

Output evaluation is defined as determining the extent to which SHSP strategies and actions are implemented and outputs are produced; in other words, 

it measures progress and productivity. Identifying the degree to which the 

SHSP is implemented is not only a necessary step to determine SHSP outputs, it 

also will help determine if implementation is impacting SHSP outcomes. 

It is common for an SHSP to generate emphasis area action plans that 

identify strategies and actions to be accomplished, responsibility for completing each action, and expected completion dates. States that regularly track 

SHSP implementation progress are well positioned to perform output evaluation. The expected or planned timeline for completing SHSP actions and the 

level of planned activity forms a baseline. Performance evaluation based on 

outputs compares the actual degree of SHSP implementation to 

this baseline.
29

Examples of output performance measures include:

•	 The number of high-visibility enforcement campaigns;

•	 The number of public service announcements aired (earned media);

•	 The number of intersections with improved pavement markings; and

•	 The number of center line miles with cable median barrier, rumble strips, etc.

Output Evaluation Met

Output evaluation helps determine the status of or the 

extent to which SHSP strategies and actions are implemented (e.g., not started, partially implemented, fully 

implemented, etc.). The idea is to track performance 

measures that relate to progress made in implementing the 

SHSP. For example, if a State has determined lane departure crashes are responsible for a significantly high 

proportion of fatal crashes, it may decide to implement a 

variety of countermeasures to keep vehicles in their lanes. 

One of these might be the installation of shoulder and 

centerline rumble strips. The performance measure is the 

number of miles of rumble strip installed. Some States 

also may decide to track spending on rumble strip installation as a separate performance measure. In both cases it 

is important to collect and report the data.

Many infrastructure-related countermeasures require 

multiple years for completion. In those cases, progress 

milestones can be developed and tracked, such as identifying promising sites for installing the countermeasure. 

These milestones can, at a minimum, show progress 

towards a performance measure.

Outcome Evaluat

Outcome evaluation measures the degree to which SHSP 

goals and objectives are being met and whether there is a 

reduction in fatalities and serious injuries, improvement in 

road user safety attitudes and behaviors, etc. In other 

words, it can help answer the question, “Are we doing the 

right things?”

Typical outcome performance measures relate to the number and rate of 

crashes, fatalities and serious injuries, observed behavior, emergency response 

times, public perceptions of safety for the various transportation modes, etc. 

Performance Mea

Legislation requires the U.S. Department of 

Transportation to establish performance measures for the 

Federal-aid program. In the area of safety these 

measures are the number and rate of serious injuries and 

fatalities (23 U.S.C. 150). States will be required to set 

targets for and report on these performance measures, 

so they should be considered when developing SHSP 

performance measures as well.

NHTSA and the Governor’s Highway Safety 

Association (GHSA) have also developed a set of core 

performance measures, including:

•	 Number of traffic fatalities (three-year or five-year 

moving averages);

•	 Number of serious injuries in traffic crashes;

•	 Number of speeding-related fatalities; and

•	 Number of pedestrian fatalities.

For a list of all NHTSA/GHSA performance measures, 

see: Traffic Safety Performance Measures for States 

and Federal Agencies. 

States can also review A Primer on Safety Performance 

Measures for the Transportation Planning Process

developed by FHWA for help in creating safety 

performance measures.
30

Safety issues vary across the country; therefore, no single set of safety performance measures is applicable to all States.8

Outcome evaluation is challenging because the link between SHSP implementation and crash reduction is indirect. Causality is difficult to establish because 

scientific evaluation conditions and controlled studies are simply not possible 

in the transportation safety field. However, while it is not possible to conclusively demonstrate that action A “caused” condition B, correlations between 

conditions A and B can be identified and used in outcome evaluation.

Even though establishing causality is difficult, statistical analysis to measure 

program outcomes is feasible and can provide important information for 

identifying which efforts are likely contributing to positive safety outcomes. 

However, it is clear the SHSP process has resulted in multiagency, multidisciplinary, multimodal programs nonexistent before SHSPs were legislatively 

required. Interactive, resource sharing, and multidisciplinary programs are 

complex and difficult to implement, manage, and sustain, but recent experience 

in the U.S. proves it is possible and can be very successful. 

Performance evaluation based on outcomes determines whether the strategies 

and actions contribute to (or are correlated with) measurable change. The 

ultimate evaluation question is whether the SHSP has achieved the intended 

results in terms of reducing fatalities and serious injuries on the roadways. 

Outcomes also may measure improvements in road user safety attitudes, 

awareness, and behaviors. If countermeasure implementation costs are 

available, a benefit cost analysis can be performed to compare implementation costs to the economic results of injury and fatality reduction. Performance 

evaluation based on outcomes consists of reviewing performance measures 

and comparing them to baseline data. In this case, baseline data on selected 

performance measures is gathered prior to SHSP implementation (e.g., fatality 

and injury numbers and or rates, awareness measures, observational survey 

results, etc.).

Performance measures for outcome evaluation typically include the following:

•	 Overall fatalities and serious injuries;

•	 Fatalities and serious injuries by emphasis area;

•	 Observed behavior, e.g., annual safety belt observations; and

•	 Knowledge and awareness.

SHSPs encourage multidisciplinary approaches resulting in multiple strategies, 

which often makes it difficult to separate the effect of a single strategy. 

8 FHWA. (2010). A Primer on Safety Performance Measures and the Transportation 

Planning Process.

Source: Cambridge Systematics, Inc.
31

Nevertheless, the impact of a combination of strategies can be assessed at the 

program level. If spending also is tracked, benefit cost ratios can be calculated improving future resource allocation and focusing limited resources on 

the programs with the greatest potential impact.

Outcome Evaluation Met

Outcome performance measures use trend analysis, benefit/cost analysis, 

survey, and other data to shed light on performance outcomes, e.g., fatalities, 

serious injuries, etc. These measures reveal the extent to which actions are 

affecting safety outcomes. Performance evaluation compares outcome data 

on SHSP objectives to baseline data to determine the degree to which objectives are met.

Trend Analysis

As noted previously, trend analysis can be used to set objectives, but it also 

can be used to track progress over time. Most States measure and track safety 

by monitoring multiple years of fatality and serious injury data. 

To measure safety outcomes, most States focus on serious injury crashes. 

Agencies typically calculate the number and rate of fatalities and serious 

injuries as a general statistical measure. Some States also calculate these 

statistics by emphasis area to identify areas where progress is being achieved, 

as well as areas where the numbers do not appear to be moving or are 

moving in the wrong direction. Performance evaluation in this case consists of 

comparing crash outcomes with previously set objectives at both the overall 

and emphasis area levels.

Figure 4 shows the progress in one of Pennsylvania’s emphasis areas. When 

Pennsylvania developed an SHSP in 2006, one of the goals in the infrastructure improvements emphasis area was to reduce head-on crashes, the most 

severe type of crash. On average, head-on crashes accounted for 17 percent 

of total fatalities, but only 4 percent of all reportable crashes in Pennsylvania. 

The State’s objective was to reduce head-on fatalities to 200 in 2008. The 

fatality data show Pennsylvania met the objective; in fact they reached it early.
32

Figure 4. Head-On Fatalities: Historical Fatality Data and 

Future Goals

Crash trends can provide an indication of overall safety performance, but may 

not provide enough data to attribute crash reductions to a specific SHSP 

strategy because other programs or environmental factors, such as an increase 

in population, a decrease in exposure, etc., also are contributing to the 

reduction. For example, implementing the recommendations from a corridor 

safety study may result in not only engineering improvements, such as rumble 

strips, but also public education on the importance of buckling up and highvisibility speed enforcement. In this case, tracking performance based on 

rumble strip installation most likely would over estimate the amount of change 

associated with the countermeasure because it would not take into account 

increased safety belt use and speed reductions. 

This mingling of impacts from multiple strategies and countermeasures sometimes has the unintended consequence of keeping ineffective countermeasures 

in place. For example, consider the combination of an effective and an 

ineffective countermeasure on crash reduction. If both were equally funded, a 

reduction in crashes could be achieved, yet because it is difficult to ascribe the 

reduction to the correct countermeasure, it may appear both countermeasures 

were effective. For this reason, it is important to use proven effective countermeasures and conduct evaluations of those countermeasures with less 

evidence of effectiveness.

Attitude and Behavior Analysis

Many SHSPs contain emphasis areas, strategies, and actions designed to 

change attitudes and improve safety behavior, such as high-visibility enforcement programs, public- and school-based education programs, etc. Typically, 

Source: Cambridge Systematics, Inc. based on data from Pennsylvania’s Strategic Highway 

Safety Plan.
33

these programs attempt to increase the use of occupant protection, reduce 

impaired, aggressive, or distracted driving, etc. Attitude and behavior change 

can be measured by random observational and telephone surveys.9 Survey 

data can be used not only to measure performance, but also to construct 

programs. For example, if survey data show the public strongly supports 

tougher impaired driving laws, the results could be used to educate and 

inform elected officials.

Benefit/Cost Analysis

Conducting a program-level benefit/cost analysis of SHSP-related efforts is 

difficult to accomplish because of the complexity of SHSPs and the difficulty of 

measuring the costs and benefits of many behavioral programs. However, if 

benefit/cost analysis is conducted on a large number of related projects, such 

as a statewide program of projects to improve or restore the super elevation of 

large numbers of horizontal curves, it could provide some level of overall 

SHSP assessment (e.g., the benefits of the program have outweighed the 

costs), and it provides some indication the program has shown success in 

improving safety. Typically, program benefit/cost analysis is conducted using 

three years of data for both before and after implementation of improvements.

The primary use of benefit/cost analysis is to direct resources to program areas 

with the greatest potential to improve safety. Additional information on 

benefit/cost evaluation methods at the individual project level can be found in 

FHWA’s Highway Safety Improvement Program Manual.

Recommended Actions

1. Assemble data for assessing output and outcome performance measures.

2. Identify missing data which may prevent assessment of performance measures. 

3. Determine if other performance measures can be used to determine progress.

4. Document baseline data.

5. Determine the output and outcome measures for each SHSP emphasis area.

6. Compare output performance measures with baseline data.

7. Compare outcome performance measures with baseline data.

8. Compare observation and/or telephone survey results to measure changes 

in awareness, attitudes, and behaviors.

9. Collect and review the data available for benefit/cost analyses.

10. Conduct program-level benefit/cost analyses where feasible.

9 One commonly used observational survey technique is the annual statewide safety 

belt use survey managed by NHTSA.

Evaluation in Actio

Pennsylvania Uses 

Benefit/Cost Analysis to 

Determine SHSP Priorities 

Pennsylvania compiled a 

list of countermeasures 

where the State had 

invested significant 

resources, such as safety 

belts. A review of the data 

revealed the initial infusion 

of resources increased the 

use rate from 76 percent in 

2002 to 86 percent in 

2010; however, continued 

spending had not pushed 

the rate much above the 

86 percent mark. This 

prompted the State to 

review current countermeasures and determine 

whether other promising 

strategies should receive 

support from the State’s 

limited resources. The 

review resulted in a 

realignment of funding 

among the SHSP emphasis 

areas and a grantee 

requirement to justify 

requests in terms of 

effectiveness in reducing 

traffic-related fatalities and 

serious injuries.
34

Self-Assessment Questions

The following self-assessment questions are designed to inform performance 

evaluation. Answering “yes” to a question indicates the State’s SHSP has been 

effective or successful in this area of performance evaluation. Answering “no”

indicates improvements can be made.

1. Has the current status of all output and outcome performance measures 

been gathered and reviewed?

2. Are the performance measures clearly related to SHSP goals and 

objectives?

3. Are the numbers and rates of fatalities and serious injuries used as general 

statistical measures?

4. Are the numbers and rates of fatalities and serious injuries tracked and 

reported by emphasis area and compared to previously set objectives?

5. Have fatality and serious injury objectives been met?

6. Are observation and/or telephone survey data collected and analyzed to 

track changes in awareness, attitudes, and behaviors? 

7. Have awareness, attitude, and behavior objectives been met?

8. Are program-level benefit/cost analyses conducted on certain 

SHSP programs?

•	 If so, have the benefits of the program(s) outweighed the costs?
35

Chapter 4 – 

Using Evaluation: 

The Focus is Results

The preceding chapters introduced the benefits of evaluation, tips on evaluation 

planning, and methods for evaluating both SHSP process and performance. 

The purpose of this chapter is to provide information and encouragement for 

using evaluation results to improve the SHSP process and performance.

Introduction

The EPM introduction explained the basic purpose of evaluation as keeping 

the SHSP process open to feedback, change, and improvement. Assume for a 

moment that recommendations from the previous chapters have been implemented and accomplished. The next steps are to interpret the data and use 

them to improve the SHSP. 

Interpret the Data

It is essential to interpret, document, and share SHSP evaluation results with 

managers and stakeholders. It helps them: understand organizational 

strengths and challenges; gauge progress in meeting goals and objectives; 

and make informed decisions. The interpretation depends on the data collected and the questions rasied. No singular list of questions is applicable to 

all situations. Rather, example questions based on possible process and 

performance results are provided below. 

SHSP Proc

Suppose the analysis of process evaluation data reveals that, over time, the 

high-level decision-makers who were originally involved no longer attend the 

steering committee meetings and have delegated the task to staff. Answers to 

the following questions would help interpret the data: 

•	 Do the staff members have access to leadership?

•	 Can they make decisions on behalf of leadership? 

•	 Are the staff members able to secure resources when needed?

•	 Is enthusiasm for SHSP implementation and evaluation dwindling?

Source: Cambridge Systematics, Inc.
36

Depending on the answers to these and other questions, the current steering 

committee could determine that delegating the steering committee participation to the staff level does not hinder SHSP process and performance. On the 

contrary, it is improved because the current members have fewer competing 

priorities and duties and are able to devote more time and attention to the 

work. On the other hand, if lack of leadership presence has resulted in fewer 

resources and less decision-making authority, methods would need to be 

devised to reengage the leaders. 

SHSP Performan

Analysis of performance evaluation data may reveal the number of fatalities 

and serious injuries in some emphasis areas are not decreasing as planned. 

Again, the solutions will vary, but any of the following challenges could be 

underlying the failure to make progress:

•	 Data deficiencies, e.g., inability to identify problem locations; 

•	 Implementing ineffective (unproven) strategies;

•	 Inadequate resources to implement the countermeasures; 

•	 Lack of sufficient authority among those charged with implementation; 

•	 Inadequate expertise or understanding of how to implement the strategies; or

•	 Lack of the political support necessary to change public attitudes.

Identifying and implementing solutions begins with correctly defining the 

problem. Solutions may then become obvious, but first the data need to be 

interpreted sufficiently.

For example, if an SHSP goal or objective is not met, the results may suggest a 

strategy is ineffective; however, in some cases the strategies may not have 

been implemented as intended. In this case, a State would need to determine 

if new strategies are needed or if implementation issues need to be addressed.

Apply the Results 

Evaluation results must be reviewed to identify and document ways to improve 

SHSP process and performance. These “lessons learned” should be widely 

shared along with the evaluation results. Of course, once the data are interpreted and shared, a commitment to use the results is necessary.

Evaluation in Actio

Alaska Uses Evaluation to 

Educate the Public on 

Traffic Safety 

Alaska used evaluation 

results to announce the 

update of their Strategic 

Traffic Safety Plan (STSP) at 

a media event attended by 

the State’s top traffic safety 

officials. Hosted by the 

Deputy Commissioner of 

the Department of 

Transportation and Public 

Facilities, the event included 

presentations from a U.S. 

Congressman; 

Commissioners for the 

Departments of Public 

Safety, Health and Social 

Services, and 

Administration; and a 

representative from the 

Department of Corrections. 

Evaluation data were used 

to highlight the State’s 

traffic safety issues and to 

celebrate successes such as 

declines in unrestrained, 

alcohol-related, and 

speeding fatalities. The 

officials applauded the 

declining numbers but said 

one death on Alaska’s 

highways is too many. They 

urged everyone to embrace 

the STSP’s goal of “Toward 

Zero Deaths.”
37

Operationalize the Find

Evaluation results should be used to enhance the SHSP process, 

improve performance, inform managers and stakeholders, and 

facilitate decision-making. 

States can use evaluation results to accomplish the following 

purposes, among others. These will be particularly important 

when updating the SHSP:

•	 Verify SHSP strategies and action plans are being implemented as intended;

•	 Validate the expected effectiveness of SHSP processes, 

strategies, and programs;

•	 Identify effective processes, strategies, and programs for 

replication;

•	 Identify weaknesses in the SHSP strategies and actions, such 

as failure to implement strategies, as well as strategies not 

achieving the intended results;

•	 Improve SHSP implementation;

•	 Direct resources to areas with the highest probability of 

improving safety;

•	 Inform elected officials, the media, and the public about the 

SHSP’s impact; 

•	 Identify potential leaders and partners across disciplines and 

modes; and

•	 Generate public understanding of the SHSP and recruit 

participation and support.

Share the Informati

States should establish a feedback loop to ensure evaluation results are studied, used, and incorporated in the SHSP process. Evaluation results should be 

shared with decision-makers, SHSP committees and teams at all levels, safety 

stakeholders, and as appropriate with the public and the media. Information 

sharing informs partners and stakeholders about the SHSP process and performance. SHSP stakeholders should be aware of successful, as well as deficient 

programs as soon as the information is available, reviewed, and validated. 

This enables leaders and managers to address challenges; implement course 

corrections; educate stakeholders about the SHSP process, programs, and 

activities; inform the public to increase understanding and support for the 

SHSP; and inform elected and appointed officials so they consider additional 

funding and support for safety efforts. 

Evaluation in Actio

Missouri Uses Evaluation Results to Inform 

Stakeholders and Examine Emphasis Areas 

Data are critical to the success of the Missouri 

SHSP evaluation process, which is why the State 

determined safety stakeholders should regularly 

access up-to-date data. The State compiles and 

distributes statewide, regional, and county 

fatality numbers to all safety coalition members 

weekly, which allows comparison to previous 

time periods. To compile the information, the 

State uses a standardized tracking methodology, 

which enables consistent longitudinal analysis. 

The data include information on both behavior 

and infrastructure, e.g., areas of the State with 

low safety belt use or infrastructure improvements by location and installation date to enable 

before-and-after evaluations. Distributing the 

data keeps stakeholders involved in the SHSP 

process and regularly reminds them of the 

overall goal to save lives and prevent injuries. 

The data also helps regions and localities 

examine their data closely to identify the most 

pressing highway safety problems.
38

Sharing results helps sustain partnerships and momentum and can help 

persuade decision-makers to support evaluation as an integral part of the 

SHSP process. As awareness about transportation safety issues and programs 

increases, further support for the SHSP is generated. Results are typically 

shared through press releases, news media events, brochures, formal reports, 

newsletters, presentations, and other methods.

Many States hold Safety Summits or conferences to share information and 

generate enthusiasm for SHSP implementation. Sessions could highlight 

evaluation results, especially noteworthy or exemplary practices that encourage others to adopt similar strategies in their communities or organizations. 

Sharing less successful results is also beneficial, especially if this results in 

brainstorming, consensus building, and solutions. 

Using Evaluation Checklist

The following checklist is designed to support the use of evaluation results. If 

an action on the checklist is in progress or completed, the State is well on the 

way to using evaluation results to improve the SHSP process and performance.

•	 Evaluation results have been interpreted and documented.

•	 Evaluation results were reviewed to identify lessons learned.

•	 Lessons learned have been used to improve SHSP process and performance.

•	 SHSP stakeholders are made aware of both successful and unsuccessful 

programs and strategies as soon as sufficient information is available.

•	 Evaluation results are being used to increase public understanding of 

SHSP programs and strategies.

•	 Evaluation results are being used to help inform elected and appointed 

officials so they might support increased funding and resources for safety 

programs and strategies.

•	 Evaluation results are being used to identify additional safety leaders 

and partners.

•	 Evaluation results that identify gaps and weaknesses in SHSP process or 

performance are being addressed through follow-up actions.

•	 Evaluation results are used to direct resources to areas with the highest 

probability of improving safety.
39

Conclusion

Evaluation does not need to be complicated or difficult. In fact, it should be 

as simple and straightforward as possible to ease interpretation and use by 

leaders, managers, and safety stakeholders. Once the data are collected and 

analyzed, the results can be used to assess the level of SHSP implementation 

and SHSP performance in terms of outputs and outcomes. The results can be 

shared and used for continuous SHSP process improvement. Armed with this 

information, SHSP managers can focus resources and efforts on the most 

critical problems and the most effective countermeasures. It is an essential 

investment in the State transportation safety program. 

An important message is, “don’t get stuck!” If the data, expertise, or 

resources are not available in some areas, move on to other areas where 

evaluation is feasible. Also, “don’t let evaluation results gather dust!” 

Develop plans and obtain commitments to use the results to improve SHSP 

efficiency and effectiveness. Good luck!

Source: Cambridge Systematics, Inc.
40

Appendix – 

EPM Worksheets

The sample worksheets provided in this appendix are based on the information 

provided in the EPM and are intended to help States document the evaluation 

results and record follow-up action items. They are based on the self assessment questions posed throughout the EPM. By carefully reviewing and 

answering the questions in these worksheets, States will gain an understanding 

of their strengths and weaknesses and begin to develop an action plan for 

strengthening their SHSP. 

The worksheets are organized by EPM chapter, with one worksheet for 

Chapter 2 and another for Chapter 3.10 Each worksheet is composed of two 

parts. The first part is designed to encourage a deeper review of each of the 

self assessment questions with the goal of identifying action items for improving 

the SHSP. The second part is designed to take the action items and identify 

those who are responsible for completing them, and when they should be 

done. The end result is an action plan that includes tasks to be accomplished, 

persons responsible for accomplishing them, and deadlines. 

While the self-assessment questions can be simply answered with a yes or no, 

greater benefit is derived if they are considered and responded to in a deeper 

fashion. Filling in the fields next to each question fleshes out issues associated 

with them and helps identify next steps. The following example helps describe 

this process.

Exampl:  Chapt

Consider a self-assessment question from Chapter 3, “Are the number 

and rate of fatalities and serious injuries tracked and reported 

by emphasis area and compared to previously set objectives?”

The worksheet provides a place to answer yes or no to this question. If each 

emphasis area has performance measures for fatality and serious injury 

magnitudes and/or rates, the answer to the question is Yes, we track the 

number of fatalities and serious injuries on an annual basis as 

data become available. 

The next space in the worksheet is to provide additional information (“as evidenced by”). A possible answer to this question may be: Annual data reports 

by emphasis area and for the SHSP as a whole are provided to the 

leadership showing progress toward SHSP goals/objectives. 

10 Chapter 1: Preparing for Evaluation and Chapter 4: Using Evaluation are best 

addressed by working through the checklists at the end of each of those chapters.
41

The third space in the worksheet is “what is the impact (positive or negative).” 

You are tracking and reporting on the number of fatalities and serious injuries 

on an annual basis and providing the information to the SHSP leadership so 

they can see progress. What is the positive impact of this activity; is there any 

negative impact? Possible answers include the following:

•	 Positive Impact – Emphasis area teams and SHSP leadership 

are regularly informed and can make course corrections when 

needed, not just at the end of the SHSP cycle/period of 

performance.

•	 Negative Impact – Because fatalities and serious injuries are 

only tracked by actual numbers and not by geographic area, 

it may be difficult to determine where the problem is occurring, which prevents hot spot and systemic analysis of the 

State’s traffic safety problems. 

The fourth space in the worksheet is “opportunities” or how the State can 

improve its process with respect to this question. The response might consist 

of the following items:

•	 Collect and track overall fatalities by rates to allow for a 

comparison with other neighboring States and the national 

fatality rate.

•	 Collect and track fatalities and serious injuries by geographic 

area to identify problem locations.

•	 Provide information on SHSP progress on achieving fatality 

and serious injury goals/objectives to the public.

The last space in the worksheet is for action items. At this point, opportunities 

identified in the previous step can be transformed into tasks such as the following: 

•	 Develop a method to collect and analyze fatalities and serious injuries by geographic area. 

•	 Determine who should receive the data and how often. 

•	 Decide whether the information can be used to form regional 

teams to assist with SHSP implementation. 

This process should be repeated for each question.11 When completed, it will 

result in a list of action items. These action items should be transferred to the 

second part of the worksheet under “Action Item Summary.” Each action item 

can then be assigned to a responsible person and given a deadline. The 

resulting collection of action items, responsible persons, and deadlines creates 

a plan to enhance the SHSP process and efforts.

11 Questions should be modified, added, or removed based on each State’s 

particular circumstances.
42

Strategic Highway Safety Plan

Evaluation Process Model

Worksheet – Chapter 2

The elements examined in process evaluation are: Process Evaluation – Getting on the Right Track

SHSP organizagoal and objective setting methods; data driven and evi tional structure; multidisciplinary, multimodal collaboration; -

aligning agency priorities. dence-based emphasis areas, strategies, and actions; and 

 For each of these elements the primary 

Wquestion driving the evaluation is:

hat will it take to convince us our prog

strategies/actions/investments are on the right their particular situation. nature. States should feel free to add to or modify these questions to fit based on noteworthy practices and current research and is general in process evaluation efforts and identify areas for improvement. The EPM is The questions below will help determine the effectiveness of current 

Column DescriAs evidenced by: What is the evidence for the yes (or the What is the Impact:no) answer?

 What impact, positive or negative, is 

Opportunities: this having on our SHSP process?

 How can we improve our efforts with 

Action Items:respect to this question?

 What steps will we take to improve our 

responsible for completing them? process, when should the steps be completed, and who is 

Structure QuestionsSHSP Organizational 

Yes/No

As evidenced by 

(observations)

What is the 

impact (positive 

or negative)? Opportunities Action Items

1.

by an actively engaged orgaIs the SHSP process supported 

-

nizational structure?
43

SHSP Organizational Structure Questions Yes/No

As evidenced by 

(observations)

What is the 

impact (positive 

or negative)? Opportunities Action Items

2. Are top-level managers 

represented in executive committees or leadership structures/groups established for the SHSP?

3. Do members of the executive committee or leadership 

group have the decisionmaking authority needed to effectively support the SHSP process?

4. Do members of the executive committee or leadership 

group assign persons with decision-making authority to the steering committee or working group?

5. Are multiple transportation 

modes represented, and do they actively participate on the steering committee/working groups and emphasis area teams?
44

Structure QuestionsSHSP Organizational 

Yes/No

As evidenced by 

(observations)

What is the 

impact (positive 

or negative)? Opportunities Action Items

6. Has a SHSP program coorassigned? What percent dinator or manager been -

age of this person’s time is the SHSP?dedicated to 

7. Do the leadership and workas frequently as expected? ing groups/committees meet 8. Are emphasis areas supportleaders? ed by teams with engaged 

QuestionsMultimodal Collaboration Multidisciplinary and 

Yes/No

As evidenced by 

(observations)

What is the 

impact (positive 

or negative)? Opportunities Action Items

1. Are members of the execusteering committee, the em tive or leadership group, the -

multimodal?groups multidisciplinary and phasis area teams, and other 
45

Multidisciplinary and Multimodal Collaboration Questions Yes/No

As evidenced by 

(observations)

What is the 

impact (positive 

or negative)? Opportunities Action Items

2. Are local/regional/district 

coalitions supported by the SHSP organizational structure? 

3. Are the necessary disciplines, 

modes, and agencies (representing the 4 E’s) engaged in SHSP decision-making and implementation? 

4. Do the stakeholders regularly 

collaborate on decisions that affect SHSP updates and implementation?

5. Do the necessary stakeholders 

collaborate and jointly decide on SHSP goal and objectivesetting methods?
46

QuestionsSetting MethodsGoal- and ObjectiveYes/No As evidenced by (observations) What is the impact (positive or negative)? Opportunities Action Items

1.

objectives?aggressive, yet achievable to establish goals and set such as trend analysis, used Are data-driven methods, 

2. Are objectives specific, mearealistic? surable, time bound, and 

Data-Driven and EviAreas, Strategies, and Ac dence-Based Emphasis -tions Questions Yes/No

As evidenced by 

(observations)

What is the 

impact (positive 

or negative)? Opportunities Action Items

1.

select the emphasis areas?Is data analysis used to 

2. Are the emphasis area stratevidence based process? egies selected through an 3.

of effectiveness accompastrategies with less evidence Are promising and innovative 

-

nied by an evaluation? 
47

Aligning Agency Priorities Questions Yes/No

As evidenced by 

(observations)

What is the 

impact (positive 

or negative)? Opportunities Action Items

1. Have the various agencies 

and safety partners incorporated elements of the SHSP into their planning documents? (HSPs, HSIPs, CVSPs, LRTPs, S/TIPs, etc.)
48

Action Item Summary

SHSP Organizational Structure – Action Items Deadline Responsible PersonMultidisciplinary and Multimodal Collaboration

6.5.4.3.2.1.

– Action Items Deadline Responsible Person

6.5.4.3.2.1.
49

Goal- and Objective-Setting Methods – Action Items Deadline Responsible Person

1.2.3.4.5.6.

Data-Driven and Evidence-Based Emphasis Areas, Strategies, and Actions – Action Items Deadline Responsible Person

1.2.3.4.5.6.
50

Aligning Agency Priorities – Action Items Deadline Responsible Person

6.5.4.3.2.1.
51

Strategic Highway Safety Plan

Evaluation Process Model

Worksheet – Chapter 3

Performance Evaluation – Outputs and OutcomesPerformance evaluation is comprised of: output evaluation and outcome evaluation. For each of these areas the primary question driving the evaluation is:What will it take to convince us our programsstrategies/actions/investments are getting uswhere we need to bThe questions below will help determine the effectiveness of current performance evaluation efforts and identify areas for improvement. The EPM is based on noteworthy practices and current research and is general in nature. States should feel free to add to and/or modify these questions to fit their particular situation. 

Column DescriAs evidenced by: What is the evidence for the yes (or the no) answer?What is the Impact: What impact, positive or negative, is this having on our SHSP process?Opportunities: How can we improve our efforts with respect to this question?Action Items: What steps will we take to improve our process, when should the steps be completed, and who is responsible for completing them? 

Performance Evaluation Questions Yes/No

As evidenced by 

(observations)

What is the 

impact (positive 

or negative)? Opportunities Action Items

1. Has the current status of all 

output and outcome performance measures been gathered and reviewed?
52

Evaluation QuestionsPerformance 

Yes/No

As evidenced by 

(observations)

What is the 

impact (positive 

or negative)? Opportunities Action Items

2.

SHSP goals and objectives?measures clearly related to Are the performance 

3.

statistical measures?injuries used as general of fatalities and serious Are the numbers and rates 

4.

of fatalities and serious injuAre the numbers and rates 

-

emphasis area and comries tracked and reported by 

-

set objectives?pared to previously 

5.

injury objectives been met?Have fatality and serious 

6.

track changes in awarecollected and analyzed to or telephone survey data Are observation and/

-

behaviors?ness, attitudes, and 

7.

been met?and behavior objectives Have awareness, attitude, 
53

Performance Evaluation Questions Yes/No

As evidenced by 

(observations)

What is the 

impact (positive 

or negative)? Opportunities Action Items

8. Are program-level benefit/ 

cost analyses conducted on certain SHSP programs?

9. Have the benefits of the 

program(s) outweighed the costs?
54

Action Item Summary

SHSP Organizational Structure – Action Items Deadline Responsible Person

10.9.8.7.6.5.4.3.2.1.