Focus the 

Evaluation
STEP 1: ENGAGE STAKEHOLDERS

1.1 Determine how and to what extent to involve stakeholders in

program evaluation

STEP 2: DESCRIBE THE PROGRAM

2.1 Understand your program focus and priority areas

2.2 Develop your program goals and measurable (SMART) objectives

2.3 Identify the elements of your program and get familiar with 

logic models

2.4 Develop logic models to link program activities with outcomes

u STEP 3: FOCUS THE EVALUATION

3.1 Tailor the evaluation to your program and stakeholders’ needs

3.2 Determine resources and personnel available for your evaluation

3.3 Develop and prioritize evaluation questions

STEP 4: GATHER CREDIBLE EVIDENCE

4.1 Choose appropriate and reliable indicators to answer your

evaluation questions

4.2 Determine the data sources and methods to measure indicators

4.3 Establish a clear procedure to collect evaluation information

4.4 Complete an evaluation plan based on program description 

and evaluation design

STEP 5: JUSTIFY CONCLUSIONS

5.1 Analyze the evaluation data

5.2 Determine what the evaluation findings “say” about your program

STEP 6: ENSURE USE OF EVALUATION FINDINGS AND SHARE

LESSONS LEARNED

6.1 Share with stakeholders the results and lessons learned from 

the evaluation

6.2 Use evaluation findings to modify, strengthen, and improve 

your program

Practical Use of Program Evaluation among

Sexually Transmitted Disease (STD) Programs

SUGGESTED CITATION: 

Salabarría-Peña, Y, Apt, B.S., Walsh, C.M. Practical Use of Program

Evaluation among Sexually Transmitted Disease (STD) Programs, Atlanta (GA):

Centers for Disease Control and Prevention; 2007.
F

ocusing your evaluation is the third step in CDC’s framework

for program evaluation. A focused evaluation enables you to

answer the questions of greatest concern to program staff and

other stakeholders about the STD program. This is based on

the assumption that the entire program does not need to be

evaluated at any point in time. Rather, the evaluation(s) you

conduct will most likely be of a program component or activity, and

will focus on what question is being asked, by whom, and what will

be done with the information. Step 3 will help you determine the

resources and personnel that are needed to implement the evaluation,

the most important questions for the evaluation, and the type of

evaluation(s) that you will conduct.

Step 3 is broken down into three evaluation tools:

• Tool 3.1 provides information on types of evaluation and when

to conduct them.

• Tool 3.2 provides guidance on determining what resources and

personnel are available for your evaluation. 

• Tool 3.3 describes how to develop and prioritize your evaluation

questions based on program and stakeholders’ needs and

resources.

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS 119

Focus the 

Evaluation
TOOL 3.1: TAILOR YOUR EVALUATION TO YOUR 

PROGRAM AND STAKEHOLDERS’ NEEDS 

_____________________________________

INTRODUCTION

In Steps 1 and 2 you learned how to engage stakeholders to fully

describe your STD program.You developed a logic model that

includes activities, outputs, and outcomes. Now it is time to

determine which program activities in your logic model should be

evaluated based on your stakeholders’ needs and available resources.

Once you make that determination, you are in a better position to

identify the types of evaluation you can conduct, the purpose(s) of

the evaluation, who will use the evaluation results and for what. The

flowchart below provides a description of where these activities fit

with the program planning and program evaluation process. 

120

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

ENGAGE

STAKEHOLDERS

THROUGHOUT

EVALUATION

EVALUATION PURPOSE, USERS, USES

UNDERSTANDING OF PROGRAM FOCUS AND PRIORITY AREAS

PROGRAM GOALS

LOGIC MODEL

SMART

PROCESS OBJECTIVES

SMART

OUTCOME OBJECTIVES

INPUTS ACTIVITIES OUTPUTS

PROCESS

SHORT-TERM

OUTCOMES

INTERMEDIATE

OUTCOMES

LONG-TERM

OUTCOMES

OUTCOME
LEARNING OBJECTIVES

Upon completion of this tool, you will be able to:

1. Define process evaluation and outcome evaluation. 

2. Describe reasons for conducting process and outcome

evaluations. 

WHAT ARE THE MOST COMMON 

TYPES OF EVALUATION?

There are several types of evaluations that can be conducted. Some

of them include the following: 

• Formative evaluation ensures that a program or program activity

is feasible, appropriate, and acceptable before it is fully

implemented. It is usually conducted when a new program or

activity is being developed or when an existing one is being

adapted or modified. 

• Process/implementation evaluation determines whether program

activities have been implemented as intended. 

• Outcome/effectiveness evaluation measures program effects in the

target population by assessing the progress in the outcomes or

outcome objectives that the program is to achieve. 

• Impact evaluation assesses program effectiveness in achieving its

ultimate goals. 

This tool will focus on process and outcome evaluations because they

are the most common types of evaluations you will probably conduct.

WHAT ARE PROCESS AND OUTCOME EVALUATIONS? 

Process evaluation determines whether program activities have been

implemented as intended and resulted in certain outputs. You

conduct process evaluation periodically throughout the life of your

program and start by reviewing the activities and output components

of the logic model (i.e., the left side). 

Results of a process evaluation will strengthen your ability to report

on your program and use information to improve future activities. It

allows you to track program information related to Who, What,

When, and Where questions: 

• To whom did you direct program efforts? 

Example: What types and how many target population members

received STD services?

• What has your program done? 

121

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
Examples: 

– Did the program staff distribute the STD screening protocols

to clinics? 

– Did medical staff counsel, screen, and appropriately treat 

clinic patients for STDs?

– How many professional development workshops were 

provided for disease intervention specialists (DIS) on 

protocols for interviewing clients and conducting case 

management? 

– Did the program staff collaborate with the stakeholders or 

other partners in designing a screening program? 

• When did your program activities take place? 

Example: How many days after interviewing index cases were

contacts treated prophylactically?

• Where did your program activities take place? 

Example: Where was outreach conducted to reach the 

target population(s)? 

Suppose you were implementing an initiative to address a syphilis

outbreak among men who have sex with men (MSM). Here are

some of the questions you could answer with a process evaluation: 

• How did DIS and STD program management collaborate with

community-based organizations (CBOs) or other partners to

reach MSM who engage in high-risk behaviors?

• How many CBO outreach workers received STD training? 

• What activities did these partners implement to address the

problem? 

• When were these activities conducted? 

• Where were these activities conducted?

• Was the target population reached? 

• What were the problems encountered in reaching the target

population?

Outcome evaluation measures program effects in the target

population by assessing the progress in the outcomes that the

program is to address. To design an outcome evaluation, begin with

a review of the outcome components of your logic model (i.e., the

right side of the model), which reflects the intended changes in your

target population (knowledge, awareness, attitudes, skills, and

behaviors) as well as potential changes in program policies that you

hope to achieve.

122

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
Some questions you may address with an outcome evaluation include:

• Were medical providers who received intensive STD training

more likely to effectively counsel, screen and treat patients than

those who did not? 

• Did the implementation of STD counseling in CBOs result in

changes in knowledge, attitudes, and skills among the members

of the target population?

• Did the program have any unintended (beneficial or adverse)

effects on the target population(s)?

• Do the benefits of the STD activity justify a continued allocation

of resources?

If you were implementing the initiative noted above (i.e., addressing

a syphilis outbreak in the MSM community), the following are some

of the questions you could address with an outcome evaluation: 

• As a result of the syphilis initiative, was there any change in the

awareness of the syphilis outbreak among MSM who engage in

high-risk behaviors?

• Was there any change in attitudes toward condom use among

MSM who engage in high-risk behaviors?

• Was there any change in intention to use condoms among MSM

who engage in high-risk behaviors?

• Was there any change in syphilis incidence rates among MSM

who engage in high-risk behaviors?

• Did the benefits of the activity justify continued allocation of

resources?

It is important to note the usefulness of conducting process

evaluation while you are implementing outcome evaluation. If the

outcome evaluation shows that the program did not produce the

expected results, it may be due to program implementation issues

(e.g., inadequate resources). Therefore, it is recommended that if

you conduct outcome evaluation you also implement process

evaluation.

123

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
HOW DO YOU CHOOSE THE FOCUS 

OF YOUR EVALUATION?

Use the following steps to help program staff and stakeholders focus

the evaluation.

1. Decide on the purpose of your evaluation. 

The purpose of your evaluation is what you intend to get from

the evaluation activities. This serves as the basis for the

evaluation design, questions, and methods. By thoughtfully

conducting this first step, you will prevent premature decisionmaking regarding how to carry out the evaluation. Example of

purposes include improve program operations, determine the

effects of the program in the target population(s), and gain

knowledge about program activities.

2. Identify the users of your evaluation results.

The users of your evaluation are the specific persons who receive

and use evaluation findings (e.g., stakeholders). Support from

the users increases the likelihood that the evaluation results will

be used to improve the program or program activity under

evaluation. 

3. Identify the uses of your evaluation results.

The uses of your evaluation are the specific ways that program

staff and stakeholders plan to utilize the evaluation findings. To

identify the uses of your evaluation, program staff and the

various stakeholders should discuss the different expectations and

needs they have for the evaluation (e.g., What aspect of the

program am I most interested in? How can I use the evaluation

results?). A clinical supervisor, for example, might expect that an

evaluation of the implementation of an activity (process

evaluation) would provide feedback on whether staff is

conducting quality STD screening. Therefore, the use of the

evaluation might be to improve the quality of screening. Other

examples of uses of evaluation findings include: deciding how to

allocate resources, deciding whether to expand the locations

where a program activity is carried out, identifying program areas

that need improvement, documenting the level of success in

achieving objectives, and soliciting more funds. 

124

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
Once program staff and stakeholders have identified their

expectations for conducting an evaluation, you will need to prioritize

the various uses of the evaluation. 

The following is an example to guide this process based on decisions

made by program staff and stakeholders. They decided that the

overall purpose of the evaluation should be to determine if

Chlamydia screening (activity that has been carried out for six

months) has been implemented as planned. Table 1 summarizes

stakeholder expectations, evaluation users, and evaluation uses for the

proposed process evaluation. 

125

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

CDC, local STD program director

and manager expect the

evaluation to document the scope

and quality of activity

implementation and demonstrate

that the activity is being

implemented as designed.

Clinical supervisors and staff

expect the evaluation to provide

feedback on whether staff

members are conducting quality

screening.

CBO partners expect the

evaluation to inform them about

whether their outreach efforts are

reaching the target population for

the activity.

Representatives from the target

population expect the evaluation

to inform them if the activity is

culturally/language appropriate

and whether their rights are being

protected.

CDC

STD program director,

STD manager and staff

Clinical supervisors,

STD program director

and staff

CBO partners,

STD program director,

manager and staff (DIS,

field staff supervisors),

community leaders

Target population

members, STD program

director, manager and

staff

STAKEHOLDER NEEDS/

EXPECTATIONS

EVALUATION 

USERS

• Justify that STD program resources are

being appropriately used.

• Identify the extent to which activity

plans were implemented.

• Make midcourse adjustments to improve

the activity and refine the activity plan.

• Identify the extent to which Chlamydia

screening is being implemented as

planned.

• Improve the quality of screening.

• Improve outreach strategies to the

community.

• Increase Chlamydia screening in the

community.

• Enhance the activity’s cultural

competence.

• Verify that participants’ rights are

protected.

EVALUATION 

AUDIENCES

Table 1: Stakeholder Expectations, Evaluation Users, and Evaluation Uses for

Process Evaluation
4. Identify the stage of development of your program to

determine if the focus of your evaluation is realistic.

With the implementation of a new STD program activity, consider

conducting a process evaluation to determine if the program is

being delivered as planned and if improvements are needed. An

evaluation that included outcomes would make little sense at this

stage. With a more mature program, you may want to conduct

an outcome evaluation to assess your program’s effectiveness and

to demonstrate that it is making productive use of resources. For

instance, if your STD program has been implemented for 2-3

years, you may want to focus on assessing whether short-term

outcomes (e.g., changes in knowledge, attitudes, and skills of the

target population) are achieved. If (1) the program has been

implemented for several years, (2) program delivery is going

smoothly, and (3) you have evaluated the implementation of its

activities, you may want your evaluation to focus on

intermediate- and/or long-term outcomes (e.g., changes in STD

risk behaviors and sexual health status of the target population).

126

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

c Review your logic model, particularly the activities, outputs, and outcomes.

c Identify the difference between process and outcome evaluation, and how they

can benefit your program.

c Involve stakeholders in determining the purpose, users, and uses of the

evaluation.

c Identify the purpose of the evaluation based on your program’s stage of

development.

c Determine who will use the evaluation results (i.e., users of the evaluation).

c Determine how evaluation results will be used (i.e., uses of the evaluation).

c Review your activities and outputs to focus your process evaluation activities.

c Review your outcomes to focus your outcome evaluation activities.

SUMMARY CHECKLIST: Focusing the Evaluation
CONCLUSION AND NEXT STEPS

With this evaluation tool you have learned about process and

outcome evaluation and the essentials of focusing your evaluation.

You have also learned the importance of having an evaluation that is

realistic and feasible. 

The next tool, Tool 3.2 (Determine resources and personnel available

for the evaluation), details what you need to know about

determining the resources and personnel available for your evaluation. 

ACRONYMS USED IN THIS TOOL

CBO - Community-based organization

DIS - Disease Intervention Specialist

MSM - Men who have sex with men

STD - Sexually transmitted disease

KEY TERMS

Activities: Actual events that take place as part of your program

(e.g., developing pamphlets, testing patients).

Effectiveness: This relates to outcome evaluation, and it refers to

the contribution a program makes to produce changes in the target

population/organization. 

Fidelity: When your STD program or intervention is implemented

as intended.

Inputs: Program’s resources (e.g., money, staff, materials).

Intermediate outcomes: Intended effects of your program in the

target population/organization that takes longer than short-term

outcomes to occur (e.g., changes in STD-related policy or in

behavior of the target population).

Logic model: A picture of how a program/component/activity is

supposed to work.

127

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
Long-term outcomes: Intended effects of your program in the

target population/organization that may take several years to achieve,

such as reduced disease transmission and incidence. 

Outcome evaluation: A type of evaluation that determines the

effects of your program activities in the target population (e.g.,

changes in: knowledge, attitudes, beliefs, skills) or organization. The

outcome components of the logic model (the right side) are used to

plan an outcome evaluation. 

Outputs: The direct products of your program activities or services

delivered (e.g., pamphlets developed, patients tested).

Process evaluation: Also referred to as implementation evaluation,

is a type of evaluation that determines whether your program and its

activities are implemented as intended and why?/why not? 

Information gathered is used for refining or modifying these

activities and related procedures. The inputs, activities, and outputs

of a logic model (the left side) are used to plan a process evaluation. 

Purpose of evaluation: General intent of evaluation (e.g., to finetune program operations). Short-term outcomes: Immediate effects

of a program in the target population/organization (e.g., changes in

knowledge, attitudes, skills, awareness, or beliefs).

Stage of development: The level of maturity of your program,

which influences the type of evaluation to conduct (e.g., planning,

implementation and maintenance stages).

Users of an evaluation: The specific persons/organizations that will

employ the evaluation findings in some way (e.g., STD Director,

CBO, funder). 

Uses of an evaluation: The specific ways that program staff and

other stakeholders will apply what is learned from the evaluation

(e.g., change STD clinical practice, inform STD prevention policy).

128

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
CASE SCENARIO

The STD Program of the city of Chancri-La had implemented a

syphilis elimination media campaign for three years to address a

syphilis outbreak in the MSM community. Program staff and

stakeholders used the results of a process evaluation to track their

implementation efforts and improve the scope and quality of the

campaign delivery to the target population. The next step was for

program staff and stakeholders to come together to discuss the

possibility of conducting an outcome evaluation. Along with the

syphilis elimination team from the health department and other

program staff, the stakeholders present at the meeting included

representatives of the gay media, business owners of bathhouses and

other businesses frequented by MSM, CBOs that serve high-risk

MSM, and MSM advocating for their community. Discussion topics

addressed the following questions: 

• What will be the purpose of our evaluation? 

• How realistic is what we plan to do?

• What expectations do individuals have from an outcome

evaluation?

• Who will be the users of the evaluation information?

• What will be the uses of the evaluation?

Because the syphilis elimination media campaign had been widely

profiled in the mass media, including the gay media, several

community stakeholders wanted to measure the effectiveness of the

intervention in attaining its objectives. Their process evaluation

efforts led to several improvements in the activity, and they felt it was

now “mature” enough to undergo outcome evaluation. Also, some

stakeholders needed to know if the syphilis outbreak had been

effectively addressed via the activity so they could divert scarce

resources to other program activities. The stakeholders had differing

expectations about the outcome evaluation and views about its users

and uses of the evaluation information. Table 2 summarizes this

information.

129

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
130

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

1. STD program manager expects the

evaluation to compare activity costs and

activity benefits, helping him/her make

decisions with the STD program director

about where to allocate resources.

2. Representatives of gay media expect that

the evaluation will produce data that can

be used in media events about the success

of the program, the types of community

members who benefited, and how they 

benefited.

3. Representatives of businesses frequented

by MSM expect the evaluation will inform

them about the improved health of their

patrons and how the business community

contributed to the success of the initiative.

4. CBOs that serve high-risk MSM expect the

evaluation will determine how effective

the activity was in changing MSM

knowledge, attitudes, and behaviors

related to STD transmission and

prevention. They want to know how CBOs

contributed to the success of the initiative.

5. MSM advocating for their community

expect the evaluation to demonstrate that

members of their community responded

well to the activity. They want to know if

the health of the community has

improved.

CDC,State legislature,

STD program director,

and STD manager.

Various media outlets,

STD program director

and STD manager.

Businesses frequented

by MSM, STD program

director and STD

manager.

CBOs; institutes and

agencies that focus on

health of MSM,

STD program director

and STD manager.

MSM advocates;

leaders in the MSM

community;

community members at

large. STD program

director, and STD

manager.

STAKEHOLDER NEEDS/

EXPECTATIONS

EVALUATION 

USERS

• Decide whether to allocate

resources to different activities.

• Justify that STD program

resources were appropriately

used.

• Document the level of success

in accomplishing objectives.

• Identify how partnership

between the community and

health department (HD)

contributed to success.

• Gather success stories.

• Assess improved health of the

MSM community and reduction

of syphilis.

• Compare change in MSM health

over time.

• Compare change in MSM STDrelated knowledge, attitudes,

and behaviors over time.

• Gather success stories.

• Assess improved health of the

MSM community and reduction

of syphilis.

• Gather success stories.

EVALUATION 

AUDIENCES

Table 2: Stakeholders’ Needs/Expectations, Uses, andUsers for the Outcome Evaluation
REFERENCES

Centers for Disease Control and Prevention, National Center for

Chronic Disease Prevention and Health Promotion, Division of

Adolescent and School Health. (2004). Evaluation steps tools.

Unpublished manuscript.

Centers for Disease Control and Prevention. (1999). Framework for

program evaluation in public health. MMWR Recommendations

and Reports, 48(RR-11). Retrieved February 22, 2005 from

http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm

Fitzpatrick, J. L., Sanders, J.R., Worthen, B.R. (2004). Program

Evaluation:Alternative Approaches and Practical Guidelines (3rd

ed.). New York: Allyn and Bacon.

MacDonald, G., Starr, G., Schooley, M., Yee, S. L., Klimowski, K.,

& Turner, K. (2001, November). Introduction to program

evaluation for comprehensive tobacco control programs. Atlanta, GA:

Centers for Disease Control and Prevention. Retrieved October

17, 2004, from

http://www.cdc.gov/tobacco/evaluation_manual/Evaluation.pdf 

Rossi, P. H., Freeman, H. E., & Lipsey, M. W. (1999). Evaluation: A

Systematic Approach (6th ed.). Thousand Oaks, CA: Sage.

Stecher, B. M., & Davis, W. A. (1987). How to focus an evaluation.

Newbury Park, CA: Sage.

Thompson, N.J., & McClintock, H.O. (1998). Demonstrating your

program’s worth: a primer for evaluation on programs to prevent

unintentional injury. Atlanta: Centers for Disease Control and

Prevention, National Center for Injury Prevention and Control.

U.S. Department of Health and Human Services. (2002). Physical

Activity Evaluation Handbook. Atlanta, GA: US Department of

Health and Human Services, Centers for Disease Control and

Prevention.

131

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
W.K. Kellogg Foundation. (1998, January). W.K. Kellogg Foundation

Evaluation Handbook. Retrieved October 17, 2005, from

http://www.wkkf.org/Programming/ResourceOverview.aspx?CID=

281&ID=770

W.K. Kellogg Foundation (2001, December). Logic Model

Development Guide. Retrieved January 8, 2005, from

http://www.wkkf.org/Programming/ResourceOverview.aspx?CID=

281&ID=3669

132

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
TOOL 3.2: DETERMINE THE RESOURCES AND

PERSONNEL AVAILABLE FOR YOUR EVALUATION

_____________________________________

INTRODUCTION 

In Steps 1 and 2, you learned how to engage stakeholders and to

fully describe your STD program. You also developed a logic model

that included inputs, activities, outputs, and outcomes. Using Tool

3.1, you learned how to tailor your evaluation to your program/

stakeholder needs and to identify the types of evaluations you could

conduct. In Tool 3.2 you will learn about the importance of

considering the time, the skills, and the fiscal resources needed to

conduct an evaluation.

LEARNING OBJECTIVE

Upon completion of this activity, you will be able to:

Determine the amount of financial resources, time, and personnel

available/needed to plan a feasible evaluation.

WHEN DO YOU NEED THE RESULTS 

OF YOUR EVALUATION?

Part of planning an evaluation involves determining when

results are needed. You can then work backward by projecting startup and completion dates of the evaluation. 

For instance, if you plan to conduct a process evaluation of a

relatively new STD activity/intervention, you will want to generate

evaluation results in an early stage of implementation in order to

improve the activity. Keep in mind that stakeholders may want to

use evaluation results to make policy or programmatic decisions

concerning this STD activity/intervention. If so, you will need to

plan your evaluation activities so that these results are available

according to the policy decision-making schedule. 

Some evaluation activities occur occasionally (e.g., collecting

baseline measures of clients’ STD knowledge on symptom

recognition), while others are ongoing (e.g., monitoring

implementation of your syphilis elimination activities over time).

You need to create a timeline for every evaluation you conduct of

your STD program. Please refer to Appendix A to see a sample

evaluation timeline and an accompanying case scenario. 

133

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
WHO WILL CONDUCT THE EVALUATION? 

You will need individuals with evaluation expertise to conduct the

evaluation. You should consider recruiting this expertise at the

beginning of the evaluation process so that these individuals can

provide input early on. Here is some advice to consider in

determining how to staff your evaluation.

1. Assess the evaluation skills and needs of the STD 

program staff.

A variety of evaluation skills are needed at different points in an

evaluation. These skills range from the ability to communicate well,

to the ability to analyze data. Assess which evaluation skills the STD

program staff has. You may want to use Appendix B as guidance to

determine which evaluation skills you have within the STD staff and

skills that are still needed to undertake the evaluation.

2. Consider staff professional development or CDC technical

assistance.

If individuals are available to do the required evaluation work,

but lack the necessary skills, consider professional development

in these skill areas, which in the long run can be very beneficial

for the STD program. Training your staff to use these evaluation

tools developed by CDC’s Division of STD Prevention

(DSTDP) is a good place to start. There are other resources STD

programs can contact for training on program evaluation. The

National Coalition of STD Directors, the Prevention Training

Centers, or local universities may know of resources. You may

also want to contact your program consultant and request

evaluation technical assistance from DSTDP. See Appendices C

and D for program evaluation technical assistance services offered

by DSTDP. 

3. Explore local evaluation resources and peer-to-peer

assistance.

Evaluation resources and expertise may exist in several places in

your locale. One place to look is in other health department

(HD) programs or divisions. You may also locate peer-to-peer

assistance by exploring evaluation resources connected with your

partners and evaluation stakeholders. Another place to locate

evaluation expertise is in colleges and universities and specifically

in educational, social science and public health programs. A

relatively easy way of garnering evaluation expertise is to work

with a skilled graduate student who is interested in your project. 

134

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
4. Consider hiring an evaluation consultant, 

if funding is available.

If there are evaluation activities that require specific expertise

outside the skills of your program staff (e.g., qualitative data

analysis), you may want to consider hiring an external evaluation

consultant if funds are available. Faculty at local colleges and

universities may be interested in serving as a consultant. See

Appendix E for how to recruit an evaluation consultant.

Keep in mind that your STD program is ultimately responsible for

any STD program evaluation activities conducted in your project

area. Therefore, you need to become familiar with the program

evaluation process, be an integral part of the decision process, and

monitor the progress of the evaluation activities implemented, even if

you are using an external consultant.

WHAT FINANCIAL RESOURCES ARE AVAILABLE 

FOR YOUR EVALUATION?

Along with time and skill, you need financial resources to

conduct your evaluation. Funding an evaluation may seem a bit

daunting for your STD program when you are working with limited

year-to-year funding and pressing program priorities. However, you

can conduct meaningful evaluation activities with limited resources.

Part of your decision-making process for using funds for evaluation

will be based on the extent that you, your program staff and other

stakeholders value and prioritize evaluation. STD programs and

stakeholders who value the benefits of evaluation, and are creative,

find ways to begin, maintain, and even expand effective program

evaluation activities in the midst of severe fiscal constraints. 

You may want to consider building an evaluation line item

directly into your program budget to be used for evaluating priority

program activities or interventions. In this regard, based on internal

program decision-making, you may want to consider dedicating a

proportion or set dollar amount of the annual budget to program

evaluation. Another common funding pattern is to take advantage of

opportunities for unexpected funding through such avenues as

supplemental applications (i.e., from government, private

foundation, non-profit entity) or rollover funds. 

Another issue you should consider in identifying financial

resources for an evaluation is cost-sharing with other entities that can

benefit from the evaluation. If you are already working with other

135

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
agencies on the program activity or component being evaluated, they

may be willing to cost-share on the evaluation (e.g., sharing

personnel, providing space or equipment for evaluation activities).

You may also want to consider calling on local businesses and asking

for their support in supplying incentives (e.g., food coupons, tickets

to entertainment events) for those who participate in the evaluation

as a token of appreciation for their time. However, this is not a

requirement. 

The amount of fiscal resources you will need for an evaluation

will vary considerably depending on the design of the evaluation you

choose and the scope of activities you plan. See Appendix F for how

to develop an evaluation budget. Such guidance was adapted from

and can be found at the CDC website http://www.cdc.gov/od/pgo/

funding/budgetguide.htm. 

136

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

c Develop a timeline for your evaluation that takes into

account the needs of your program and its stakeholders.

c Determine how to staff your evaluation.

c Assess the evaluation skills and needs of the STD

program staff.

c Consider staff professional development (training) or

CDC technical assistance.

c Explore local evaluation resources and peer-to-peer

assistance.

c Consider hiring an evaluation consultant, if funds are

available.

c Determine what financial resources you have available

to conduct evaluation activities.

c Develop a realistic budget for the evaluation.

SUMMARY CHECKLIST: Identifying the Resources

And Personnel Available For Your Evaluation 
CONCLUSION AND NEXT STEPS

By using this program evaluation tool, you learned about the factors

(i.e., time, staff, and fiscal resources) that you need to consider when

planning an evaluation. 

Next, Tool 3.3 will allow you to further focus your evaluation. It

will help you select what you want to evaluate and will assist you in

developing and prioritizing your evaluation questions. 

ACRONYMS USED IN THIS TOOL

AEA – American Evaluation Association

CDC – Centers for Disease Control and Prevention

DSTDP – Division of STD Prevention

HD – Health Department

HSREB – Health Services Research and Evaluation Branch

MSM – Men who have sex with men

SOW – Statement of work

STD – Sexually transmitted disease

TIG – Topical Interest Group

KEY TERMS

Activities: Actual events that take place as part of your program

(e.g., developing pamphlets, testing patients).

Effectiveness: This relates to outcome evaluation, and it refers to

the contribution a program makes to produce change in the target

population/organizations. 

Focus group: A qualitative method used to collect data from a

group of people (about 6 - 11) who meet for 1-2 hours to discuss

their insights, ideas, and observations about a particular topic with a

trained moderator. Participants are selected because they share

certain characteristics (e.g., individuals who have been tested for

syphilis, women in detention facilities) relevant to the evaluation. 

137

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
Qualitative data: Detailed/narrative information that provide an

in-depth understanding of a topic/issue/population. An example of

qualitative data is the answers representatives of a CBO would

provide when asked for their thoughts on how to reach high-risk

adolescents.

Quantitative data: Numerical information. An example is data

that identify the number of times (e.g., 1, 2, 3, 10) each client has

visited your clinic within the last year. 

Stakeholders: Individuals or organizations directly or indirectly

affected by your program and/or the evaluation results (e.g., STD

program staff, family planning staff, representatives of target

populations). 

REFERENCES

Fitz-Gibbon, C.T., & Morris, L.L. (1987). How to design a program

evaluation. Newbury Park, CA: Sage publications.

Horn, J. (2001). A checklist for developing and evaluating evaluation

budgets. Retrieved December 28, 2004, from:

http://www.wmich.edu/evalctr/checklists/evaluationbudgets.htm

Joint Committee on Standards for Educational Evaluation (1994).

The program evaluation standards (2nd ed.). Thousand Oaks, CA:

Sage. (Ed.). Thousand Oaks, CA: Sage Publications.

King, J.A., Morris, L.L., & Fitz-Gibbon, C.T. (1987). How to assess

program implementation. Newbury Park, CA: Sage. 

Langford, L., DeJong, W. (2002). How to select a program evaluator.

Retrieved December 28, 2004, from:

http://www.edc.org/hec/pubs/prev-updates/prog-eval.html

Roe, K.M. (1997, January). Evaluation methodologies with limited

resources. (Concept paper prepared for the Academy for Educational

Development).

138

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
Rossi, P. H., Lipsey, M. W., Freeman, H. E. (2004). Evaluation: A

systematic approach (7th ed.). Thousand Oaks, CA: Sage.

Sarvela, P.D., & McDermott, R.J. (1993). Health education

evaluation and measurement: A Practitioner’s perspective. Madison,

WI: Brown and Benchmark.

Windsor, R., Baranowski, T., Clark, N., & Cutter, G. (1999).

Evaluation of health promotion, health education, and disease

prevention programs (2nd edition). Mountain View, CA: Mayfield.

Witkin, B.R., & Altschuld, J.W. (1995). Planning and conducting

needs assessments. Thousand Oaks, CA: Sage.

Wholey, J.S., Hatry, H.P., Newcomer, K.E. (Eds.) (1994) Handbook

of Practical Program Evaluation. San Francisco: Jossey-Bass.

139

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
APPENDIX A

CASE SCENARIO

The STD program in State X has been implementing a Chlamydia

screening initiative in family planning clinics for 3 years to address a

continued high prevalence of Chlamydia among females ages 14 to

24. The STD Director is evaluating the program to determine its

effectiveness and to use the results to solicit future funding for this

program, since the present funding will end this fiscal year. Finally,

all stakeholders have agreed that the results obtained from the

evaluation would be used to improve the program. 

The director and other stakeholders concluded at a meeting that an

evaluation (process and outcome) is feasible at this time because the

program goals and objectives are well-defined and can be used to

measure performance. The director is also confident that the clinic’s

internal database will provide the information that is needed because

evaluation was part of planning the initiative, thus allowing the

program to collect the appropriate data 

First the director and program staff identified when evaluation results

are needed for planning and decision making. They determined that

process information is needed at key time intervals to improve the

program. They also decided that data collection on clinic patients

needs to occur throughout the program and be analyzed at strategic

times in order to demonstrate effectiveness to program stakeholders.

The timeline for the evaluation is provided below.

140

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
The STD director and staff, as well as other stakeholders, also

identified the evaluation skills that are necessary to conduct the

evaluation activities. They felt that as part of the process evaluation, a

sample of the female clients who have visited the clinic should be

interviewed individually and as a group. In order to obtain the most

reliable and valid information from clients, the director and staff felt

the evaluator should be an experienced interviewer who has

conducted both individual and focus groups interviews. In

addition, statistical skills are needed to analyze the clinic database on

all female clients aged 14 through 24 for the outcome evaluation

(effects of initiative on target population). Since many of the clients

are young females, the evaluator should feel comfortable with this

target population. The director developed a budget for the

141

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

Table 1: Projected 1ST Year Timeline for Evaluation Activities

For State X Chlamydia Screening Initiative 

1 2 3 4 5 6 7 8 9 10 11 12

MONTHS EVALUATION ACTIVITIES

Develop evaluation data collection

instruments.

Collect baseline measures on female

family planning clinic patients ages

14-24.

Monitor implementation of the

Chlamydia screening initiative.

Conduct focus groups with female

STD clinic patients.

Collect same measures from 

patients as was collected at 

baseline (i.e. follow-up).

Analyze evaluation data

Report findings.

Revise program, if needed.
evaluation using the guidance provided in Appendix E and following

the fiscal processes dictated at the local level. Along with

stakeholders, the director developed a scope of work to recruit an

evaluation consultant and distributed it through different channels

(e.g., internet, bulleting boards at universities, word of mouth). The

program hired an evaluator from a local university. 

142

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

Note. Keep in mind that you and other stakeholders develop a

final timeline and budget when you have agreed on what to

evaluate, on staffing, and on the design and methods that are

to be applied in the evaluation. The last two topics are

discussed in Tools 4.1, 4.2, 4.3 & 4.4.
APPENDIX B

SKILLS OF A QUALIFIED EVALUATOR

Core Evaluation Skills

• Knowledge about program evaluation theory and methodology.

In order to conduct any evaluation, your evaluator should be

trained in the most current evaluation theories and evaluation

methodologies (e.g., participatory evaluation, process and

outcome evaluation).

• Ability to differentiate between research and program evaluation

procedures. Program evaluation addresses practical problems,

whereas research addresses theoretical issues first, with less focus

on practice. In addition, the intent of program evaluation is

program improvement, while the intent of research is primarily

on testing hypotheses. It is helpful if your evaluator has the

ability to differentiate between the two and has primary

experience with program evaluation principles and standards.

• Experience and comfort working with diverse populations. Your

evaluator should have experience, and feel comfortable, working

with the target populations your STD program works with, and

with policy makers. 

• Experience with designing and conducting an evaluation. It is very

important that your evaluator is experienced with designing and

conducting evaluations. You will be able to assess this skill by

reading previous evaluation reports that he/she has written. The

reports should clearly communicate evaluation concepts and

findings to diverse audiences. The reports should address

evaluation questions and discuss both the strengths and

limitations of the evaluation design, methods and data.

• Experience with qualitative and/or quantitative data sources and

analyses. If you have a database that contains quantitative

information (e.g., demographic information, test results, number

of sexual partners) or need to develop one, your evaluator should

have an understanding of how that information was collected

and needs to be qualified in designing, collecting and analyzing

these types of data. In addition, your evaluation may involve the

collection of qualitative data via in-depth interviews, focus

143

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
groups, or observations, among other methods. Again, your

evaluator should have experience collecting and analyzing these

types of data as well as knowledge of the strengths and

weaknesses of either kind of data and when to apply them.

Desirable Evaluation Skills

• Knowledge about STD content areas. Your evaluator should be

familiar with the best practices of STD prevention and control,

how STD programs operate, the communities affected by STDs

and how these diseases can impact the community.

• Ability to train staff in evaluation concepts and skills. To make

your evaluation cost effective, it is helpful if a qualified evaluator

can train program staff in different evaluation skills (e.g., use of

data collection methods, how to use qualitative software). 

144

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
APPENDIX C

PROGRAM EVALUATION TECHNICAL ASSISTANCE (TA)

SERVICES AT CDC

Program Evaluation Technical Assistance is available from evaluation

specialists located in the Health Services Research and Evaluation

Branch (HSREB) at CDC’s Division of STD Prevention (DSTDP).

We provide evaluation TA to help project areas build evaluation

capacity so they can evaluate their own STD program components. 

What services do we provide? 

• Building program evaluation skills by assisting individuals in:

– Developing goals and SMART objectives

– Developing a logic model

– Formulating evaluation questions

– Determining data collection methods and sources

– Developing an evaluation plan

– Developing indicators 

– Developing evaluation instruments

– Developing a timeline for evaluation activities

– Providing recommendations about qualities to look for when 

hiring or contracting evaluators

– Developing dissemination/reporting strategies for evaluation 

results

• Evaluating a component/activity of a program

• Evaluating implementation of a program or its activities

• Evaluating effects of a program in the target population

(outcomes)

• Conducting empowerment evaluation (coaching others on how

to evaluate their programs)

• Conducting evaluation workshops

• Identifying/providing evaluation materials (e.g., tools, resources)

Who is on the Evaluation TA Team?

The team consists of two evaluation specialists with expertise in the

practical and theoretical aspects of program evaluation. Depending

on the time and complexity of the request, we will either provide

direct evaluation TA or refer project areas to other DSTDP branch

staff, other state/local STD program staff with expertise in specific

evaluation-related subjects, or local evaluation resources.

145

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
How can you request evaluation TA?

• Contact your program consultant to let him/her know about the

need for evaluation TA.

• Fill out the TA Evaluation Form (see Appendix D).

• Submit your completed form and fax it at (404) 639-8607.

• TA requests will be processed within 10 working days from the

time received.

Where to call for questions?

For more information, call HSREB at (404) 639-8276 or email at

eval@cdc.gov.

146

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
147

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

APPENDIX D

TA EVALUATION FORM - PROGRAM EVALUATION 

TECHNICAL ASSISTANCE SERVICES AT DSTDP

Attention:__________________________________ Date: __________________

Name of Project Area:

Name of Person Requesting Evaluation TA:

(e.g., individual from project area, program consultant, other staff member at PDSB/DSTDP)

Address of Contact Person at Project Area (street name/number, city, zip code):

Phone (xxx-xxx-xxxx):_____________________ Email:________________________

c Developing goals and SMART objectives

c Developing a Logic Model

c Formulating evaluation questions

c Determining data collection methods and sources

c Developing a timeline for evaluation activities

c Hiring evaluator

c Developing an evaluation plan

c Developing indicators

c Developing dissemination/reporting strategies for evaluation results

c Identifying/providing feedback (evaluation plans, evaluation instrument, etc.)

c Program Evaluation skill building

Type of Evaluation TA Requested (please check all that apply):
Please respond to the following as completely and clearly as

possible. If you need assistance in clarifying your TA request, 

please contact your Program Consultant.

• Briefly describe the issue or problem that prompted this TA request.

• Specify your timeline to resolve the issue for which you are

requesting TA (e.g., 3 weeks, 1 month, 4 months, 1 year)

• Describe the goal(s) or objective(s) from your STD Program to

be addressed through this request.

• Describe how the evaluation TA being requested will help

improve your program.

You can either download this form and complete by computer, or

print it out and complete by hand. Please either fax or email the

completed form to the Health Services Research and Evaluation

Branch at (404) 639-8607 or eval@cdc.gov respectively.

148

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

For CDC use:

Date received: ______________________________________

By: _________________________________________________

Date of Disposition:_________________________________

Disposition: ________________________________________

c Conducting evaluation workshop

c Evaluating or providing guidance on how to evaluate a

program component/activity

c Evaluating or providing guidance on how a program or its

activities is/are implemented

c Evaluating or providing guidance on how to conduct an

outcome evaluation (effects of a program in the target

population)

c Other (Specify) 
APPENDIX E

HOW DO YOU RECRUIT AN EVALUATION CONSULTANT?

1. Identify your evaluation needs, and develop a clear scope of work.

The first step in recruiting an external evaluation consultant is to

know what you will evaluate and decide exactly what you need

from that individual. This may be spelled out in a statement of

work (SOW) that describes the intended evaluation task(s), what

products or deliverables you would like to see as a result of the

evaluator’s labor, and the timeframe in which you are operating.

The more specific you are in writing up this SOW, the easier it

will be to locate the individual you need and to negotiate a work

contract with her/him. Some of the categories to take into

account when constructing a contract include procedural

guidelines for conducting the evaluation (e.g., communication

protocols), informational requirements (clear description of the

expected product(s), data collection protocols, safeguarding of

information such as confidentiality of informants, and data

ownership), procedures for data analysis, reporting and timeline

demands, client responsibilities, budgetary guidelines, and

provisions for reviewing and evaluating the evaluator’s work. 

See the sample SOW provided on the next page.

149

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
SOW SAMPLE

EVALUATION CONSULTANT WORK ORDER

WORK ORDER NO. 12345

Period of performance: June 23, 2006 – July 24, 2006

Task Description: Under cooperative agreement X titled “Improving Evaluation Skills”

(IES), the consultant will draft and finalize the evaluation report for this project according

to the instructions specified in the Program Guidance. 

Specifically, the consultant will:

• Draft, revise as needed, and finalize the Executive Summary; Evaluation Purpose;

Evaluation Design and Methods; Results; Conclusions and Recommendations. 

• Prepare required appendices (i.e., logic model, evaluation plan). 

• Draft an abstract if requested by Project Director

Deliverables/Milestones

1. Conference call with Project Director and other stakeholders.

2. Draft of above referenced sections

3. Revised evaluation report

4. Completed evaluation report

Payment Schedule:

• Not to exceed level of effort (5 days) with payments based on days worked and days

worked tied to specific progress in achieving milestones/deliverables.

Not-to-exceed # days at $ /day; Not-to-exceed $ total.

Technical Direction: Jane Doe

Organization X and Consultant agree that the above services will be provided in accordance

with the Organization X Consulting Agreement signed by both parties dated 

.

For (Organization X) Consultant

Signature Date Signature Date

Name Name

Title Social Security Number

150

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

Due Dates

June 27, 2006

July 9, 2006

July 18, 2006

July 20, 2006
2. Search for available evaluation consultants.

Find out about the availability of evaluation consultants by

checking with nearby colleges or universities that have a school

or program of public health or departments of education,

psychology, sociology, or statistical analysis. Evaluators may also

be identified through the website of the American Evaluation

Association (AEA) (www.eval.org). By clicking on Affiliates, you

can find state or regional evaluation members. By clicking on

Find an Evaluator, you can find several evaluation firms located

in different parts of the country with their contact web addresses

and contact information. By clicking on TIGs (Topical Interest

Groups), you can find contact information for evaluators located

at different universities and organizations, as well as their specific

interests. AEA also maintains an online job bank, where you can

post a job description for your ideal evaluator

(http://www.eval.org/JobBank/jobbank.htm). 

3. Develop a set of criteria to identify an appropriate 

evaluator for your program.

Consider using the following checklist as an additional resource

to identify a suitable evaluator for your evaluation activities.

151

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS
152

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

Well 

Qualified

Not Well

Qualified

Cannot

Determine if

Qualified

Evaluator appears to be:

(Check one for each item)

EVALUATION ACTIVITIES

1. To what extent does the formal training of the

potential evaluator qualify her/him to conduct

evaluation studies (Consider degree specialization

and if the candidate has any previous STD evaluation

experience).

2. To what extent does the previous evaluation

experience of the potential evaluator qualify her/him

to conduct evaluation studies? (Consider length of

experience, and relevance of experience)

3. To what extent does the previous performance of the

potential evaluator qualify her/him to meet the needs

for your project? What prior experience does she/he

have in similar settings? (Look at work samples or

contact references)

4. To what extent are the personal styles and

characteristics of the potential evaluator acceptable?

(honesty, character, interpersonal communication

skills, ability to resolve conflicts)

5. Summary: Based on the questions above, to what

extent is the potential evaluator qualified and

acceptable to conduct your evaluation?

CHECKLIST FOR SELECTING AN EVALUATOR 
APPENDIX F

HOW DO YOU DEVELOP A BUDGET FOR YOUR

EVALUATION?

The costs of your evaluation need to be realistic so that the resources

expended can be justified. Stakeholders need to feel confident that

the evaluation will be conducted efficiently and will produce

information of sufficient value to address their needs. To project a

realistic evaluation budget, consider the direct and indirect

evaluation costs of your program. Direct costs for program

evaluation include dollars spent on evaluation activities, employee

benefits, contractors, equipment, supplies, and travel. Indirect costs

include overhead expenses (e.g., building rental and utilities,

custodial services). As you prepare your budget, the cost for the

contractor may appear as a single line item, or it can be itemized by

consultant’s salary, fringe benefits, and the non-personnel costs. The

following guidance includes information you need when developing

a budget for any evaluation you undertake.

A. Salaries and Wages

For each requested position, provide the following information:

name of staff member occupying the position, if available; annual

salary; percentage of time budgeted for this program; total months

of salary budgeted; and total salary requested. Also, provide a

justification and describe the scope of responsibility for each

position, relating it to the accomplishment of program objectives.

153

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

SAMPLE BUDGET

PERSONNEL TOTAL $_______

POSITION TITLE AND NAME ANNUAL TIME MONTHS AMOUNT

Evaluation Coordinator

Jane Doe

$X 100% 12 months $X

Data Collector

John Doe

$X 50% 12 months $X

Data analyst

(Vacant*)

$X 100% 12 months $X
Sample Justification

The format may vary, but the description of responsibilities should be

directly related to specific program objectives.

Job Description: Evaluation Coordinator (Name) 

This position directs the overall operation of the program evaluation

activities and for staff performance evaluations; responsible for

overseeing the implementation of evaluation activities, coordination

with other agencies, development of data collection instruments,

provisions of in service and training on program evaluation,

conducting meetings; designs and directs the gathering, tabulating

and interpreting of required data, and is the responsible authority for

ensuring necessary reports/documentation are submitted to CDC.

B. Fringe Benefits

Fringe benefits are usually applicable to direct salaries and wages.

Provide information on the rate of fringe benefits used and the

basis for their calculation. If a fringe benefit rate is not used,

itemize how the fringe benefit amount is computed.

154

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

SAMPLE BUDGET

25% OF TOTAL SALARIES = FRINGE BENEFITS IF FRINGE BENEFITS 

ARE NOT COMPUTED BY USING A PERCENTAGE OF SALARIES,

ITEMIZE HOW THE AMOUNT IS DETERMINED.

Retirement 5% of $XX,XXX = $X,XXX

FICA 7.65% of $XX,XXX = $X,XXX

Workers Compensation = $X,XXX

FRINGE BENEFITS Total $_______

Example: Evaluator C Salary $45,000

Total =
C. Consultant Costs

This category is appropriate when hiring an individual to give

professional advice or services (e.g., external evaluator, data

collectors, data entry, etc.) for a fee but not as an employee of the

grantee organization. When federal funds are used, written

approval must be obtained from CDC prior to establishing a

written agreement for consultant services. Approval to initiate

program activities through the services of a consultant requires

submission of the following information to CDC:

1. Name of Consultant;

2. Organizational Affiliation (if applicable);

3. Nature of Services to Be Rendered;

4. Relevance of Service to the Project;

5. The Number of Days of Consultation (basis for fee); and

6. The Expected Rate of Compensation (travel, per diem, other

related expenses) - list a subtotal for each consultant in this

category.

If the above information is unknown for any consultant at the time

the application is submitted, the information may be submitted at a

later date as a revision to the budget. In the body of the budget

request, a summary should be provided of the proposed consultants

and amounts for each.

D. Equipment

Provide justification for the use of each item and relate it to

specific program objectives. Maintenance or rental fees for

equipment should be shown in the "Other" category.

155

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

SAMPLE BUDGET

ITEM REQUESTED HOW MANY UNIT COST AMOUNT

Computer Workstation 2 ea. $X,XXX $X,XXX

Fax Machine 1 ea. XXX XXX

EQUIPMENT Total $_______
Sample Justification

Provide complete justification for all requested equipment, including a

description of how it will be used in the program (e.g., (1) One

computer workstation is requested for the evaluation coordinator, who

will be working at the State Department of Health, STD Program office.

At the current time, there are no unused computer workstations

available. The coordinator will use this workstation to design all data

collection instruments, keep minutes of all stakeholder meetings, design

all components of the evaluation plan and write all evaluation reports.

The second computer workstation is requested for the data collector, also

to be housed at the State Department of Health, STD Program office.

This workstation will be used to collect and analyze all data from STD

Program evaluations. (2) The fax machine will be used to receive reports

from the field from interviewers of the qualitative and quantitative

evaluations details on these evaluations may be accessed in the most

current CSPS progress report.).

E. Supplies

Individually list each item requested. Show the unit cost of each

item, number needed, and total amount. Provide justification for

each item and relate it to specific program objectives. If

appropriate, General Office Supplies may be shown by an

estimated amount per month times the number of months in the

budget category.

Sample Justification

General office supplies will be used by staff members to carry out daily

evaluation activities of the program. Qualitative software will be used to

analyze interviews with the target population on their perceptions about

STD services and Word Processing Software will be used to document

program activities, process progress reports, etc.

156

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

SAMPLE BUDGET

General office supplies (pens, pencils, paper, etc.) = $X

12 months x $X/year x X staff

Evaluation Instrument-questionnaire (X copies @) $1 each = $X

Qualitative analysis software (@ $X-specify type) = $X

Word Processing Software (@ $X-specify type) = $X

SUPPLIES Total $_______
F. Travel

Dollars requested in the travel category should be for staff travel

only. Travel for consultants should be shown in the consultant

category. Travel for other participants, advisory committees,

review panel, etc. should be itemized in the same way specified

below and placed in the “Other” category.

In-State Travel - Provide a narrative justification describing the

travel staff members will perform. List where travel will be

undertaken, number of trips planned, who will be making the

trip, and approximate dates. If mileage is to be paid, provide

the number of miles and the cost per mile. If travel is by air,

provide the estimated cost of airfare. If per diem/lodging is to

be paid, indicate the number of days and amount of daily per

diem as well as the number of nights and estimated cost of

lodging. Include the cost of ground transportation when

applicable.

Out-of-State Travel - Provide a narrative justification describing

the same information requested above. Include CDC

meetings, conferences, workshops, if required by CDC.

Itemize out-of-state travel in the format described above.

Sample Justification

The Evaluator and STD Program Director will travel to (location) to

present the evaluation findings. The Evaluator will make an estimated

25 trips to local STD outreach sites to monitor program implementation.

157

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

SAMPLE BUDGET

IN-STATE TRAVEL:

1 trip x X people x X miles r/t x .27/mile = $X

2 days per diem x $X/day x X people = $X

1 nights lodging x $X/night x X people = $X

25 trips x X person x X miles avg. x .27/mile = $X

_________________________________________________

Total = $X

TRAVEL (IN-STATE AND OUT-OF-STATE) Total $______
Sample Justification

The Project Coordinator will travel to CDC, in Atlanta, GA, to attend

the CDC Conference.

G. Other

This category contains items not included in the previous budget

categories. Individually list each item requested and provide

appropriate justification related to the program objectives.

Sample Justification

Some items are self-explanatory (telephone, postage, rent) unless the unit

rate or total amount requested is excessive. If not, include additional

justification. For printing costs, identify the types and number of copies

of documents to be printed (e.g., procedure manuals, annual reports,

materials for media campaign).

H. Contractual Costs

Cooperative Agreement recipients must obtain written approval

from CDC prior to establishing a third-party contract to perform

SAMPLE BUDGET

158

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

SAMPLE BUDGET

1 trip x 1 person x $X r/t airfare = $X

3 days per diem x $X/day x 1 person = $X

1 night= lodging x $X/night x 1 person = $X

Ground transportation 1 person = $X

_________________________________________________

Total = $X

OUT-OF-STATE TRAVEL

Telephone: ($ per month x months x #staff) = $ Subtotal

Postage: ($ per month x months x #staff) = $ Subtotal

Printing: ($ per x documents) = $ Subtotal

Equipment Rental (describe) ($ per month x months) = $ Subtotal

Internet Provider Service ($___ per month x ___ months) = $ Subtotal

OTHER Total $_______
program activities. Approval to initiate program activities through

the services of a contractor requires submission of the following

information to CDC:

1. Name of Contractor;

2. Method of Selection;

3. Period of Performance;

4. Scope of Work;

5. Method of Accountability; and

6. Itemized Budget and Justification.

If the above information is unknown for any contractor at the

time the application is submitted, the information may be

submitted at a later date as a revision to the budget. Copies of the

actual contracts should not be sent to CDC, unless specifically

requested. In the body of the budget request, a summary should

be provided of the proposed contracts and amounts for each.

I. Total Direct Costs $________

Show total direct costs by listing totals of each category.

J. Indirect Costs $________

To claim indirect costs, the applicant organization must have a

current approved indirect cost rate agreement established with the

cognizant Federal agency. A copy of the most recent indirect cost

rate agreement must be provided with the application.

If the applicant organization does not have an approved indirect cost

rate agreement, costs normally identified as indirect costs (overhead

costs) can be budgeted and identified as direct costs.

159

FOCUS YOUR EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

SAMPLE BUDGET

THE RATE IS ___% AND IS COMPUTED ON THE FOLLOWING DIRECT COST BASE OF $________.

Personnel $

Fringe $

Travel $

Supplies $

Other $

Total $ x ___% = Total Indirect Costs
160

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

TOOL 3.3: DEVELOP AND PRIORITIZE EVALUATION

QUESTIONS BASED ON PROGRAM AND

STAKEHOLDERS’ NEEDS AND RESOURCES 

_____________________________________

INTRODUCTION

This tool will help you develop the questions you want the

evaluation to answer about the program component/activity which

you are interested in evaluating. The flowchart below provides a

description of where development and prioritization of evaluation

questions fit with the program planning and program evaluation

process.

ENGAGE

STAKEHOLDERS

THROUGHOUT

EVALUATION

EVALUATION QUESTIONS

UNDERSTANDING OF PROGRAM FOCUS AND PRIORITY AREAS

PROGRAM GOALS

LOGIC MODEL

SMART

PROCESS OBJECTIVES

SMART

OUTCOME OBJECTIVES

INPUTS ACTIVITIES OUTPUTS

PROCESS

SHORT-TERM

OUTCOMES

INTERMEDIATE

OUTCOMES

LONG-TERM

OUTCOMES

OUTCOME

EVALUATION PURPOSE, USERS, USES
161

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

LEARNING OBJECTIVES

Upon completion of this tool, you will be able to:

1. Generate process and outcome evaluation questions.

2. Identify the necessary criteria for prioritizing evaluation

questions.

WHAT IS THE PURPOSE OF EVALUATION QUESTIONS?

Evaluation questions help focus your evaluation and should reflect

the purpose of the evaluation, and the priorities and needs of

stakeholders. Therefore, key stakeholders should be involved in

identifying, discussing and coming to agreement on the questions the

evaluation will answer. Ultimately the answers to these questions

will assist you in making improvements to your program. 

HOW DO YOU DEVELOP AND PRIORITIZE 

EVALUATION QUESTIONS? 

To focus your STD program evaluation, you will need to formulate

and prioritize the evaluation questions. Here is some guidance. 

1. Involve key stakeholders.

• Review, along with stakeholders, why you need to do the

evaluation, how the evaluation results will be used, and by

whom. 

• Make sure that the evaluation can be conducted with the

available resources, including funds and personnel with relevant

expertise. 

• Review the logic model of the program component/activity you

need to evaluate. This will generate discussion and evaluation

questions addressing multiple dimensions (e.g., outcomes,

operations) of the issue to be evaluated. 

2. Brainstorm evaluation questions.

• Brainstorm a list of questions about the specific

component/activity you want to evaluate by posing the

following: “What are the questions we want the evaluation to

answer?” 

• As part of the brainstorming process, encourage STD program

staff and other stakeholders to voice possible questions they have

about program operations, implementation, and outcomes. You

INPUTS ACTIVITIES OUTPUTS SHORT-TERM

OUTCOMES

INTERMEDIATE

OUTCOMES

LONG-TERM

OUTCOMES
162

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

need to ensure that your stakeholders’ range of interests in,

concerns about, and perspectives on your STD program are

represented and that they buy into the entire evaluation process. 

• Use your logic model to help you develop questions. For

instance, if you decided to evaluate the community screening

activity because you suspect the screening is not reaching the

right population, look back on the logic model to identify the

process and outcome components linked to that activity and

generate questions accordingly. 

3. Group questions by theme.

• After you have generated a set of evaluation questions that you

would like to answer, you will need to group the questions by

themes. Grouping questions by theme will allow you to identify

those that measure similar topics and to begin to prioritize.

• Classify the questions by either process or outcome questions.

This will help you decide the type of evaluation (see Tool 3.1)

you will apply. 

Below you will find the evaluation questions identified during the

brainstorming activity., These were grouped by theme (e.g.,

operations, behaviors, etc.) and identified as either a process or an

outcome question. A rationale for the grouping is also provided. 

Sample evaluation questions on outreach activities targeting at-risk

MSM that might result from a brainstorming could include:

• Are the prevention materials (e.g., pamphlets, posters)

appropriate for MSM at high risk?

• Did MSM read the prevention materials? 

• Were new STD screening and counseling pamphlets

distributed to appropriate locations by the disease

intervention specialists (DIS) and outreach workers?

• Did screening and counseling events reach the target

population? 

• Did screening and counseling events reach the target

population? 
163

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

4. Prioritize evaluation questions.

You may need to narrow down the number of evaluation

questions proposed due to the relevance of the questions to the

purpose and uses of your evaluation and your available resources.

For example, you may realize that outcome evaluation questions

would require substantial resources (e.g., evaluation personnel;

time commitments of program staff to collect data) that are not

available. Therefore, prioritize the list of proposed evaluation

Operational Evaluation Questions

• Are the prevention materials (e.g., pamphlets, posters)

appropriate for MSM at high risk? (process)

• Were new STD screening and counseling pamphlets

distributed to appropriate locations by the disease

intervention specialists (DIS) and outreach workers?

(process)

• Did screening and counseling events reach the target

population? (process)

• Consider hiring an evaluation consultant, if funds are

available.

Rationale for grouping: Each of these questions relates to

the operation of the intervention.

Behavioral and Awareness Evaluation Question

• Did at-risk MSM read the prevention material? (outcome)

• Were at-risk MSM aware of the prevention messages

conveyed by the media campaign? (outcome)

• To what extent did the outreach activities increase the

number of at-risk MSM coming to the STD clinic for

testing? (outcome)

Rationale for grouping: Each of these questions relates to

behavioral and awareness outcomes associated with the

exposure to the intervention.

Health Outcome Evaluation Question

• How have the P&S syphilis rates decreased among the

MSM population? (outcome)

Rationale for grouping: This question deals directly with a

health outcome of the target population.
164

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

questions by asking: “Why this question?” Make sure that each

question is clear to all stakeholders. The following criteria can

help you determine the importance, utility, and feasibility of each

question as it pertains to your STD program evaluation needs. 

Ensure that a question:

• Is important to your program staff and stakeholders.

• Reflects key goals and objectives of your program.

• Reflects key elements of your program logic model.

• Will provide information you can act upon to make program

improvements.

• Can be answered using available resources (e.g., budget,

personnel) and within the appropriate timeframe. 

• Will be supported (in terms of resources needed to answer the

question) by the STD program. 

Assess each of your evaluation questions based on how well it

meets these criteria. Questions that do not meet all of the criteria

should be considered lower priority. It is particularly important to

address how each of the questions, once answered, is likely to provide

information with which you can make program improvements. See

the Appendix in this tool for a worksheet to help you apply the

criteria to your evaluation questions. 

You may have to negotiate with stakeholders if it is not feasible

to include some of their evaluation questions. For example, you may

not be able to answer how much jail screening contributes to the

reduction in syphilis cases, but you can determine if jail screening is

resulting in the discovery of new cases, and if not, why not. 

5. Carefully examine and categorize the prioritized 

list of questions.

Categorize the prioritized list of evaluation questions as either

process or outcome evaluation questions. Process evaluation

questions are primarily concerned with the implementation, or

process of delivering your STD prevention and control activities.

Outcome evaluation questions are concerned with the effects of

your STD program delivery and operations in the target

population(s). 

Relate your questions to your logic model. Make sure your

process evaluation questions incorporate key process components

of your logic model (i.e., inputs, activities, outputs) and your

outcome evaluation questions incorporate key outcome

components of the logic model (i.e., short-, intermediate-, and

long-term outcomes). 
165

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

c Involve key stakeholders in discussing reasons for doing the evaluation and how

evaluation results will be used.

c Brainstorm a list of evaluation questions about specific components and

activities you want to evaluate.

c Group your evaluation questions by theme.

c Prioritize evaluation questions by applying the evaluation question criteria to

your list of questions. Criteria include whether the question:

c Is important to your program staff and stakeholders.

c Reflects key goals and objectives of your STD program.

c Reflects key elements of your STD program logic model.

c Will provide information you can act upon to make

program improvements.

c Can be answered using available resources (e.g., budget, personnel) 

and within the timeframe of the STD program funding cycle.

c Will be supported (in terms of resources needed to answer the question)

by the STD program.

c Examine and categorize the prioritized list of questions as process or outcome

evaluation questions.

c Relate your process evaluation questions to the process components of your 

logic model (i.e., inputs, activities, outputs) and your outcome evaluation 

questions to the outcome components of your logic model (i.e., short-term,

intermediate, long-term outcomes).

SUMMARY CHECKLIST: Develop and Prioritize Evaluation Questions

Based on Program and Stakeholders’ Needs and Resources 
166

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

CONCLUSION AND NEXT STEPS

An important step in focusing your evaluation involves developing and

prioritizing evaluation questions that are relevant to your program and

stakeholders’ needs. In this tool you learned how you and your

stakeholders can do this. 

Now that you have focused your evaluation, it is time to consider the

indicators (see Tool 4.1) that will help you answer the evaluation questions. 

ACRONYMS USED IN THIS TOOL

CBO - Community-based organization

DIS - Disease Intervention Specialist

GC - Gonorrhea

MSM - Men who have sex with men

STD - Sexually transmitted disease

KEY TERMS

Outcome evaluation: A type of evaluation that determines the effects

of your program activities in the target population (e.g., changes in:

knowledge, attitudes, beliefs and skills) or organization. The outcome

components of a logic model (the right side) are used to plan an

outcome evaluation.

Outcome evaluation questions: Evaluation questions concerned with

your program outcomes. Such questions can address whether the

desired outcomes of your program were achieved, and whether the

program produced changes in the target population(s)/organization.

Process evaluation: Also referred to as implementation evaluation, is a

type of evaluation that determines whether your program and its

activities are implemented as intended; and why? /why not? The

information gathered is used for refining or modifying these activities

and related procedures. The inputs, activities and outputs of a logic

model (the left side) are used to plan a process evaluation. 

Process evaluation questions: Evaluation questions concerned with

the implementation of your program and specific program

component/activity. You develop process evaluation questions to

examine the development and delivery of services and activities of your

program, as well as its operations and administrative functions.
167

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

CASE SCENARIO

Two years ago, Cactus City, a medium sized city located in the

American southwest, experienced an outbreak of reported gonorrhea

(GC) among Hispanic/Latino males and females; specifically among

Mexican Americans (Americans whose parents are Mexicans).

Increased GC testing in the two STD and multiple family planning

clinics in the city has identified many more infections, but not

reduced the overall morbidity. The City Health Officer, under

pressure from the mayor’s office (because of recent local news media

about the outbreak) has decided to make a concerted effort to bring

GC under control, especially in the Mexican American community.

He called a meeting of the city STD, family planning, laboratory,

surveillance, and budget office directors to develop a plan of action.

The plan of action called for a number of steps to be taken,

including: 1) development of new, or revision of existing STD

prevention materials to be disseminated by all city clinics that serve

the target population; 2) provision of updated information about the

outbreak and reminders to relevant health providers to report all

infections; and 3) expanded efforts by

Hispanic/Latino/Mexican/Mexican American CBOs to distribute

STD prevention awareness materials and condoms to their clients. 

At the start of the effort, program staff and other stakeholders

brainstormed the following possible evaluation questions: 

1. Are the STD prevention materials (e.g., pamphlets, fact sheets,

posters) that were developed or revised for use by the city health

clinics culturally appropriate for the target population (i.e.,

Mexican American males and females)? (process)

2. To what extent did the city health clinics disseminate the STD

prevention materials to the target population? (process)

3. To what extent was updated information about the outbreak

provided to health providers who serve the target population?

(process)

4. What were the barriers for the implementation of the action

plan? (process).

5. Did health providers who serve the target population

consistently report new STD infections according to reporting

guidelines? (outcome)

6. Were Mexican-American clients satisfied with the GC screening

and treatment services they received in city clinics? (process)
168

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

7. Did CBOs have appropriate trained staff to adequately reach

the target population? (process) 

8. To what extent did Hispanic/Latino/Mexican/Mexican

American CBOs distribute STD prevention awareness materials

(e.g., fact sheets; referral information) to Mexican American

clients at risk of infection? (process) 

9. To what extent did Hispanic/Latino/Mexican/Mexican

American CBOs distribute condoms to Mexican American

clients at risk of infection? (process)

10. Did Hispanic/Latino/Mexican/Mexican Americans who

received prevention awareness material find them to be

culturally appropriate? (process)

11. Did Hispanic/Latino/Mexican/Mexican Americans who

received prevention awareness material, satisfied with the

amount of information provided? (process)

12. Did Hispanic/Latino/Mexican/Mexican Americans who

received prevention awareness material share this information

with their family and friends? (outcome)

13. Was there an increase in the awareness of the GC outbreak in

the Mexican-American community? (outcome)

14. Was there an increase in the self-reported consistent use of

condoms by Mexican American males following

implementation of the action plan? (outcome)

15. Did Mexican American females request that their partners were

using condoms consistently following the implementation of

the action plan? (outcome)

16. To what extent did the GC plan of action reduce the reported

prevalence of GC within the Mexican-American population?

(outcome)

17. Was there an increase in Mexican Americans requesting to be

tested for STD, following implementation of the action plan?

(outcome)

After examining these questions, you and your colleagues decided to

group your questions into three themes. Your first theme was called

‘operations’ and included questions 1–4, 6–9. This theme was chosen

because it related to how the action plan operated. The second theme

referred to ‘acceptability’ which includes questions related with target

population’s reaction to the material/activities (10–12). The third

theme that was selected was called “knowledge and behavioral

outcomes.” This theme included questions 5 and 13–17. This theme

was chosen because these questions all deal with knowledge and
169

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

behavioral outcomes related to the action plan. The fourth theme

was labeled ‘health outcome’ and included question 16, a question

related to expected health outcome of the action plan. 

After grouping the evaluation questions by theme, you

determined which evaluation questions you would attempt to

answer. While the City Health Officer had been under pressure from

the mayor’s office to make a concerted effort to bring GC under

control, you and your colleagues knew that your action plan could

not make that happen unless the plan was implemented according to

its design. Therefore, you initially decided to address the questions

related to the theme of ‘operations’ and one of the behavioral

outcome evaluation questions (e.g., question number 14 on

treatment of target population members who had tested positive). 

When you and your colleagues reviewed these evaluation

questions, you determined that not enough resources were available

to address all of the ‘operations’ questions. You decided to address

those that were considered: 1) the most important in assessing the

major objective of implementing the action plan according to its

design: 2) would provide information that the staff could easily act

upon; 3) reflected the key components of the action plan logic

model’s outputs and intermediate outcomes; and 4) could be

supported with resources of the STD program. 

You ended up with the following finalized list of process and

outcome evaluation questions:

1. Are the STD prevention materials (e.g., pamphlets, fact sheets,

posters) that were developed or revised for use by the city health

clinics culturally appropriate for the target population (i.e.,

Mexican American males and females)? (process)

2. To what extent did the city health clinics disseminate the STD

prevention materials to the target population? (process)

3. To what extent was updated information about the outbreak

provided to health providers who serve the target population?

(process)

4.Did CBOs have appropriate trained staff to adequately reach the

target population? (process) 

5. To what extent did Hispanic/Latino/Mexican/Mexican American

CBOs distribute STD prevention awareness materials (e.g., fact

sheets; referral information) to Mexican American clients?

(process) 
170

3 FOCUS THE EVALUATION

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

6. To what extent did Hispanic/Latino/Mexican/Mexican American

CBOs distribute condoms to Mexican American clients? (process)

7. What were the barriers for the implementation of the action

plan? (process)

8. Was there an increase in Mexican Americans requesting to be

tested for STD, following implementation of the action plan?

(outcome)

REFERENCES

King, J. A., Morris, L. L., & Fitz-Gibbon, C. T. (1987). How to assess

program implementation. Thousand Oaks, CA: Sage.

Patton, M. Q. (1997). Utilization-focused evaluation: The new century

text (3rd ed.). Thousand Oaks, CA: Sage.

Rossi, P. H., Freeman, H. E., & Lipsey, M. W. (1999). Evaluation: 

A systematic approach (6th ed.). Thousand Oaks, CA: Sage.
List your evaluation questions in this

column and then decide if each question

meets the six criteria listed in the right 

hand column.

Evaluation Questions

1

2

3

4

5

6

7

8

171

FOCUS THE EVALUATION 3

PRACTICAL USE OF PROGRAM EVALUATION AMONG STD PROGRAMS

WORKSHEET: Criteria to Use When Prioritizing the Evaluation Questions

1 2 3 4 5 6

c c c c c c

c c c c c c

c c c c c c

c c c c c c

c c c c c c

c c c c c c

c c c c c c

c c c c c c

Criteria:

1 = Important to stakeholders.

2 = Reflects key goals and objectives of program.

3 = Reflects key elements of program logic model.

4 = Provides information which can be acted upon to make 

program improvements.

5 = Can be answered with available resources.

6 = Will be supported by the STD program.

Check if evaluation question meets each criteria:

APPENDIX