Number 377 May 2009

Traffic Safety Facts

Traffic Tech – Technology Transfer Series

U.S. Department of Transportation 1200 New Jersey Avenue SE., Washington, DC 20590

National Highway Traffic Safety Administration

First released in 1999, The Art of Appropriate Evaluation has 

been one of NHTSA’s most popular publications. NHTSA 

has prepared a revised and updated version of the guide. 

The document is intended for project managers who oversee evaluations of traffic safety programs at the State or local 

levels, but who do not have an evaluation background. The 

guide gives an overview of the steps involved in producing 

an evaluation appropriate to the project, whether conducted 

in-house or by an independent evaluator. 

In approximately 60 pages, the guide explains the importance 

of evaluation without being overly simplistic or too detailed 

from an experimental or statistical point of view. The updated 

version describes how to develop an evaluation mentality to 

assure sound program evaluations. It includes a glossary of 

evaluation terms, new case studies of well-evaluated traffic 

safety programs, and a list of sources for program evaluation 

information in the appendices. 

Traffic safety evaluation is an applied science that works 

within the constraints of state and local program implementation. Most local communities simply do not have the 

volume of traffic deaths and injuries to conduct countermeasure effectiveness evaluations measuring changes in deaths 

and injuries. There are many proven traffic safety countermeasures that improve traffic safety that do not need a comprehensive outcomes evaluation to show, once again, that 

they work. 

Local communities still need to report on their successes 

to their funding sources within their resources. This can be 

accomplished by focusing their evaluation efforts on determining if the particular program they implemented achieved 

its specific objectives and whether behaviors (like seat belt 

use) changed. These are intermediate measures. The guide 

will help a program manager determine what level of evaluation they need by fostering an evaluation mentality.

Evaluation at the Start of a Program

Evaluation should be an integral part of program implementation and needs to be included at the start. Here are the 

seven primary steps involved in evaluation.

1. Identify the problem

2. Develop reasonable objectives

3. Develop a plan for measuring results

4. Gather baseline data

5. Implement your program

6. Gather data and analyze results

7. Report results

Problem identification is often overlooked in evaluations. It 

provides the information necessary for selecting an appropriate countermeasure and target audience for your program. 

You will be looking for information on the magnitude of the 

problem, the underlying causes, and the target groups most 

affected. Problem identification enables you to select the most 

effective countermeasure and provides some of the baseline 

data to determine if the program meets its objectives.

Developing reasonable objectives is the next step to define 

what you expect to accomplish. Many would argue that this 

is the most critical step in the evaluation process because it 

determines what success will be and how it will be measured. 

One approach to writing objectives is to use the SMART 

approach.

■ Specific

■ Measurable

■ Action-oriented

■ Reasonable

■ Time-specific

Avoid using generalities like “improving traffic safety” or 

“increasing awareness” in favor of more specific statements 

like “increase seat belt use to 82%” or “increase citations 15% 

over the baseline.” If you identify exactly what you want to 

happen, then you can document your success. For an objective to be measurable there must be something to quantify, 

like DWI arrests, over time. Action is something you can see 

and count. 

Reasonable objectives flow from the first three steps. For 

example, a small community implemented a public information campaign on the value of traffic safety enforcement. 

The published objective of the campaign was to reduce traffic deaths community-wide. While this would be a desirable 

The Art of Appropriate Evaluation
outcome, it is not reasonable to expect that an advertising 

campaign alone would change behaviors and ultimately 

reduce traffic crashes. Public information and education 

alone are rarely effective in traffic safety unless linked to a 

highly visible enforcement campaign. And finally, deadlines 

make it clear to everyone when the program will end and 

when results can be expected. 

What Will You Measure and When?

Once you have decided what you will measure to determine 

if your program achieved its objectives, you need to decide 

how to gather the information needed to make the measurement. These decisions revolve around deciding where and 

when to collect the data, how much data is needed, and what 

procedures will be followed. There are five basic ways that 

you can measure program effects.

■■

■■

■■

■■

■■

Observational surveys

Knowledge, attitude, and awareness surveys

Activity records

Data records

Media coverage

Table 1 lists some measures that can be used to evaluate traffic safety programs. They are listed in descending order of 

importance for measuring outcomes, but in ascending order 

of convenience and ease of data collection. As you read down 

the list, the data are easier to obtain but have less of a direct 

link to measuring reductions in crashes, injury, and death. 

The guide illustrates these steps by walking through a community level evaluation of a selective traffic enforcement program (STEP) to increase seat belt use in Chemung County, 

NY. There are sample data collection forms as a new feature 

of the updated guide—one for observational data, and a second for a driver licensing office survey to collect information 

about knowledge and attitudes. 

Because some complex evaluations may require a professional 

evaluator, there is a chapter summarizing the core knowledge, skills, and abilities that program managers should look 

for in the evaluators they hire. There are tips about sources to 

tap to locate the right evaluator for the job.

Program managers who want to learn more about evaluation concepts should consider attending NHTSA’s two-anda-half-day course at the Transportation Safety Institute in 

Oklahoma City. Data Analysis and Evaluation is specifically 

for highway safety program managers who do not have formal training in research methods or statistics but have the 

responsibility of evaluating their programs. More Important Less Convenient Primary Outcomes

Changes in Crashes (the number or severity)

Reductions in fatalities and the severity of injuries

Secondary Outcomes 

(also called Proxy Measures) Examples

Less ImportantMore Convenient

Changes in observed behavior • Observing the use of seat belts and child

safety seats

• Observing the use of bicycle and

 motorcycle helmets

•	 Measuring the speed of vehicles

•	 Observing red-light running

• Counting the number of pedestrians

who jaywalk

Changes in reported behavior

(what people say they do

when asked)

•	 How often do you wear your seat belt?

• Have you ever driven after having too

much to drink?

Changes in attitude 

(what people believe)

•	 Support for legislative initiatives

•	 Knowledge of a seat belt law

•	 Teen attitudes about drinking and driving

Changes in awareness 

(what messages people

have heard)

• Awareness of high visibility enforcement

campaigns

•	 Perceived risk of getting a traffic ticket

Changes in activities 

 conducted 

(new program implemented)

•	 Citations issued by the police

•	 Special police patrols and check points

•	 Presentations

•	 Training Programs

•	 Media coverage

•	 Legislation changes

Table 1. Important vs. Convenient Possible Evaluation 

Measures1

1

Strategy for Devon.	 Accessed	 at:	 http://www.devon.gov.uk/index/	

transport/roads/road_safety/safer_speeds.htm.

How to Order

Download a copy of The Art of Appropriate Evaluation 

(56 pages plus appendices) from www.nhtsa.gov or write to 

the Office of Behavioral Safety Research, NHTSA, NTI-130, 

1200 New Jersey Avenue SE., Washington, DC 20590, fax 202-

366-7394. Ian Reagan was the Contracting Officer’s Technical 

 Representative for this project.

 Adapted	from	Devon	County	Council.	(No	Date).	Speed Management 

TRAFFIC TECH is a publication to disseminate information about 

traffic safety programs, including evaluations, innovative programs, and new publications. Feel free to copy it as you wish. If you 

would like to receive a copy, contact Angela H. Eichelberger, Ph.D., 

Editor, fax 202-366-7394, e-mail: angela.eichelberger@dot.gov.

1200 New Jersey Avenue SE., NTI-132

Washington, DC 20590