Describe the Program 
Practical Use of Program Evaluation among Sexually Transmitted Disease (STD) Programs 
STEP 1: ENGAGE STAKEHOLDERS 
1.1	Determine how and to what extent to involve stakeholders in
program evaluation

. STEP 2: DESCRIBE THE PROGRAM 

2.1	Understand your program focus and priority areas 
2.2	Develop your program goals and measurable (SMART) objectives 
2.3	Identify the elements of your program and get familiar with
logic models

2.4	Develop logic models to link program activities with outcomes 
STEP 3: FOCUS THE EVALUATION 

3.1	Tailor the evaluation to your program and stakeholders needs 
3.2	Determine resources and personnel available for your evaluation 
3.3	Develop and prioritize evaluation questions 
STEP 4: GATHER CREDIBLE EVIDENCE 

4.1	Choose appropriate and reliable indicators to answer your
evaluation questions

4.2	Determine the data sources and methods to measure indicators 
4.3	Establish a clear procedure to collect evaluation information 
4.4	Complete an evaluation plan based on program description
and evaluation design

STEP 5: JUSTIFY CONCLUSIONS 

5.1	Analyze the evaluation data 
5.2	Determine what the evaluation findings say about your program 
STEP 6: ENSURE USE OF EVALUATION FINDINGS
AND SHARE LESSONS LEARNED

6.1	Share with stakeholders the results and lessons learned from
the evaluation

6.2	Use evaluation findings to modify, strengthen, and improve
your program

SUGGESTED CITATION: 
SalabarraPea, Y, Apt, B.S., Walsh, C.M. Practical Use of Program Evaluation among Sexually Transmitted Disease (STD) Programs, Atlanta (GA): Centers for Disease Control and Prevention; 2007. 

Describe the Program 
D
escribing your program is the second step in CDCs framework for program evaluation. Note that in this step we describe the program and not the evaluation. A comprehensive program description ensures that program staff, evaluators, and other stakeholders share a clear understanding of what the program entails, and how its goals and objectives will be achieved. This understanding among stakeholders sets the stage for program evaluation and can be helpful in strategic planning, and performance measurement. 
The tools in Step 2 can help you: (1) identify the needs your
program should address, (2) develop clear goals and objectives, (3)
describe other program elements, and (4) develop a picture (logic
model) of how you believe your program works.

	
Tool 2.1 provides guidance on how to assess the STD needs of your project area (e.g., city, state, nation, territory). 

	
Tool 2.2 discusses the difference between goals and objectives and provides guidance on how to develop them. 

	
Tool 2.3 describes the components for creating a logic model. 

	
Tool 2.4 provides an opportunity for you to develop logic models and link program activities to outcomes. 

TOOL 2.1: UNDERSTAND THE YOUR PROGRAM FOCUS AND PRIORITY AREAS 
INTRODUCTION 
Describing your program includes explaining the health problems (e.g., syphilis, gonorrhea, Chlamydia) addressed by your program, how these affect different segments of the population, and any trends that may be occurring. This information can be obtained from a needs assessment and Tool 2.1 can help you understand the importance and benefits of identifying program needs and provides guidance on how to conduct a needs assessment. The flowchart below provides a description of the relationship between the activity of identifying the STD needs of your project area and the activity of engaging your stakeholders throughout the evaluation. 

UNDERSTANDING OF PROGRAM FOCUS AND PRIORITY AREAS 
LEARNING OBJECTIVES 
Upon completion of this tool, you will be able to: 
1. 
Explain to other stakeholders the STDrelated problem(s)
addressed by your program.

2. 
Use appropriate data sources for justifying the STD needs in
your project area (e.g., city, state, nation, territory)

WHY IS IT IMPORTANT TO DETERMINE STDRELATED NEEDS IN YOUR PROJECT AREA? 
Knowing the STD prevention and control needs of your project area provides an important foundation for understanding the focus of your program. It can also help you to develop realistic and measurable objectives and plan your program activities accordingly. A needs assessment is a starting point for program planning. It can help your program staff identify and measure gaps between what is and what ought to be. This knowledge can then be used to set program priorities and to develop program interventions and activities that reflect those priorities. 

Project areas, have undertaken needs assessments for some time since part of your Comprehensive STD Prevention System (CSPS) grant application requires you to provide behavioral and morbidity trends (e.g., number of cases, case rate) for the STDs that affect your project area. You have used that information to determine populations and diseases of high priority in your project area. 
This tool provides information on the process of conducting a needs assessment and it can also help to plan future needs assessments. If you have already gone through a similar process, discuss the most relevant information with the stakeholders. This will increase their familiarity with the focus and priority areas of the STD program. Next, go to Tool 
2.2 where you will review the objectives pertaining to the activity you are interested in evaluating. 
WHAT ARE THE BENEFITS OF CONDUCTING A NEEDS ASSESSMENT? 
A needs assessment can benefit your program by helping you to: 
	
Clearly determine both the met and unmet STD prevention and control needs within your area. 

	
Identify target populations service needs related to preventing and treating STDs. 

	
Clearly define your program purpose and scope. 

	
Identify appropriate goals, objectives, and corresponding
program interventions and activities.

	
Have a baseline from which to measure program achievements over time. 

	
Get community support for your program. 

	
Provide data to meet funder requirements and to seek additional funding. 

WHEN SHOULD YOU CONSIDER CONDUCTING A NEEDS ASSESSMENT IN YOUR PROJECT AREA? 
Consider conducting a needs assessment when: 
	
You are preparing your grant application (e.g., CSPS). 

	
You need to determine the STDrelated risk factors among
target populations.

	
Demographic changes occur within your target populations
(e.g., increase in immigrant populations) and within the
community your program serves.

	
Shifts occur in the incidence and prevalence of STDs within
your community.

	
You are planning new STD prevention program activities
and services.

	
You need additional support to maintain your program. 

HOW DO YOU DO A NEEDS ASSESSEMENT IN YOUR PROJECT AREA? 
A thorough needs assessment will serve as a basis for developing a strategic program and evaluation plan. This section provides information on steps you can take to determine the STD prevention needs in your area. Additional information on conducting needs assessment, including dos and donts of needs assessment can be obtained from the Oxford County Librarys Needs Assessment/ Strategic Planning manual available online1 and additional resources included in the References section of this tool. 
1. Identify stakeholders, staffing, and resources for the
needs assessment.

When possible, it is very important to involve key stakeholders in your needs assessment, including members of your target population (see Tool 1.1, Determine How and to What Extent to Involve Stakeholders in Program Evaluation) to determine what is known about the STD needs in your project area, and agree on the purpose of the needs assessment. Stakeholder involvement can help to ensure that needs are appropriately identified and understood and that this information directly informs program planning. They can provide or collect needed data. Most importantly, they can review the information from the needs assessment and help to set program priorities based on this information. 

Clarify the roles and responsibilities of the key staff within your program who are to plan and carry out components of the needs assessment. Also, map out a timeline and identify needed resources for completing the assessment. 
2. Use existing data or collect new data to determine the STD related problems and their magnitude in the target population(s). To determine the STDrelated problems of the target population, focus on two things: (1) the incidence and prevalence of STDs among target population(s) and (2) the consequences of STDs in terms of both morbidity and mortality. This means that you need to access and use existing data and/or collect new data. 
You can collect information directly from your target
population(s) to fully understand their STDrelated needs.
Such information is known as primary data. Examples include
STD testing and screening information you obtain from
patients directly. Primary data can also be obtained when your
program administers surveys, conducts key informant
interviews, group interviews, or gathers information in
community forums (see Appendix or more information on
methods to collect needs assessment data).

Example of the collection of primary data: 
STD program staff from project area X conducted a focus group with 10 individuals representing high school health programs and juvenile detention facilities. The staff queried about their knowledge of current adolescent sexual behaviors and partnering patterns. The participants consistently confirmed that most high school students and detainees had sexual relations before entering high school and that most had multiple partners. 

You may be able to obtain previously collected local, state, or nationallevel STDrelated information (on STDs or your target populations) from other organizations which was not collected specifically for your needs assessment purposes, but may prove useful. Such information is known as secondary data. You may already have a considerable amount of secondary data that you have gathered to write your STD program grant application such as behavioral trends for the STDs that affect your project area. Examples of secondary data include the following: U.S. Census data, Centers for Disease Control and Prevention (CDC) epidemiological profiles, Morbidity and Mortality Weekly Report (MMWR data), nationallevel risk behavior research data, and data collected by academic institutions. When using secondary data, consider from whom, how, and when data were collected, and their applicability to your area and the community(ies) your program serves. For project areas with limited resources, the use of secondary data may be more feasible and practical than primary data because the latter may be expensive and time consuming. 
You may want to use the following sources of secondary data: 
	
Local HIV program data 

	
Sexually transmitted disease surveillance from CDC (www.cdc.gov/nchstp/dstd/Stats_Trends/Stats_and_Trends.htm). 

	
Data from your state or local health department or CDCs
National Center for Health Statistics (NCHS) on mortality,
morbidity, and determinants of health status. You can find
NCHS data at www.cdc.gov/nchs/hus.htm.

	
Data from your state or local Department of Education 

	
Substance Abuse and Mental Health Services Administration
(SAMHSA) at www.samhsa.gov

	
Data from your State Department of Corrections 

	
Data from your state or county government agencies on socio
demographic characteristics including age, gender, ethnicity,
income, employment, family size, geographic location, and
other social indicators. You can find U.S. Census data at
http://quickfacts.census.gov/qfd.

Examples of the use of secondary data: 
While STD rates among teens in project areaX are much lower than national teen rates, project areaX rates have increased significantly since 1998. National data indicated that in project areaX Gonorrhea rates increased 229% from 18.7 per 100,000 in 1998 to 42.8 per 100,000 in 2002. Chlamydia rates almost doubled from 3% in 1998 to 5.7% in 2002. 
According to the 2003 Youth Risk Behavior Survey (YRBS), 15.9% of middle school students have had sexual intercourse. Nearly 5% had intercourse before age 11, and 5.4% have had sex with three or more people. Of those who had sex, 63.7% used a condom during their last sexual intercourse. 
You can collect primary and secondary data using a number of different methods. Appendix includes methods used to collect needs assessment data. 
3. Identify possible causes of STDs in the target population. 
Data sources that can determine the health status of your target populations may also be used to better understand the causes and associated factors related to acquiring STDs. Such causes and factors could be intrapersonal such as clients lack of knowledge about STDs, inaccurate perceptions of risk for STDs, drugtaking behaviors, and low selfefficacy to negotiate for safer sex practices. Community and social norms (e.g., patterns regarding use of condoms with casual partners) or environmental factors (e.g., availability of free or lowcost condoms) related to risk behaviors and use of health services may play a role in whether the target population engages in safer behaviors. 
4. Determine what other programs are available to address STD related problems. 
The first task is to identify any other current programs  whether from your Department of Health or others  that address STD prevention and control among the target population. Carefully examine and determine how these programs address current needs. Are the programs accessible? Do they adequately reach the target population? 

5. Prioritize needs based on available resources and/or resources that can be obtained or developed. Because of limited resources, it is essential that your program staff, along with stakeholders, prioritize program needs based on a discussion of the more pressing STD needs and available resources (i.e., budget, staff, and time). At the end of this discussion, arrive at an agreement about the purpose of your program and your areas of special emphasis. For example, you may decide to implement a health education program to address the STDrelated needs of adolescents in the county. Based on the available resources, you may narrow this to implementing the program only in high schools. Thus, the purpose of your program can be to address the STD needs of adolescents, with special emphasis on high school adolescents. If you require more assistance with conducting need assessments refer to the references provided in the tool. 
SUMMARY CHECKLIST: Sharing Evaluation Findings 

CONCLUSION AND NEXT STEPS 
Upon completion of this evaluation tool, you will understand the importance and benefits of determining the STD related need in your project area. You will have implemented the prescribed steps of a needs assessment in your STD program and produced information about the target population(s) and disease priorities you will address. 
In Tool 2.2, you will learn how to develop goals and measurable objectives. The tool will address the difference between goals and objectives and provides guidance on the development of process and outcome objectives. 
ACRONYMS USED IN THIS TOOL 
CBO  Communitybased organization CDC  Centers for Disease Control and prevention CPG  Community Planning Group CSPS  Comprehensive STD Prevention System DIS  Disease Intervention Specialist DH  Department of Health HIV  Human Immunodeficiency Virus JDC  Juvenile detention center MSM  Men having sex with men MMWR  Morbidity and Mortality Weekly Report NCHS  National Center for Health Statistics STD  Sexually transmitted disease YRBS  Youth Risk Behavior Survey 
KEY TERMS 
Focus group: A qualitative method used to collect data from a group of people (about 611) who meet for 12 hours to discuss their insights, ideas, and observations about a particular topic with a trained moderator. Participants are selected because they share certain characteristics (e.g., individuals who have been tested for syphilis, women in detention facilities) relevant to the evaluation. 
Incidence: New cases of a disease in a specific population within a defined time period. 

Individual interview: A data collection method which involves dialogue with individuals who are carefully selected for their personal experience and knowledge with the issues at hand. Since these interviews are conducted individually, they are useful when anonymity is an issue or when asking about sensitive topics so participants can feel free to express their ideas 
Morbidity: Sickness or illness. 
Population at risk: Groups that have a high probability of developing an STD or related condition. 
Prevalence: Number of cases of a disease in a population at a given point in time. 
Primary data: Data directly obtained by your STD program (e.g., surveillance, number of sex partners of syphilis cases collected through DIS interviews). 
Risk factor: A factor that increases a persons chances of getting a disease or condition (e.g., having multiple sexual partners, lack of access to health care). 
Secondary data: Information your STD program can use that has been collected by someone else (e.g., national data). This may include epidemiological data, sociodemographics, health risk behaviors, and health policies. 

CASE STUDY

The following is an example of a syphilisrelated needs assessment conducted by the STD program in City X. The program decided to examine the syphilis prevention plan in light of new national findings reported in CDCs MMWR highlighting increasing rates of HIV among young men. The example outlines all the steps used in this tool. The STD program used both primary and secondary data to identify young men (1725 years of age) who have sex with men as the target population. The approaches and methods used in this example are by no means exhaustive and are used only to illustrate a sample needs assessment. 
Following are the steps that program staff and stakeholders in City X implemented in their needs assessment. 
1. Identify stakeholders, staffing, and resources for the needs
assessment.

The STD program identified the program staff and resources for the needs assessment. Stakeholders invited to be part of the project included the STD program director and manager, medical care providers, prevention service providers and clients from a number of communitybased organizations, clinical staff serving populations at high risk of STDs (particularly syphilis), a member of City X HIV Prevention Program, and members of the HIV community planning group (CPG). 
2. Use existing data or collect new data to determine the STD
related problems and their magnitude in the target
population(s).

Program staff used primary and secondary data to determine the syphilisrelated factors of the target population and their magnitude. For programs with limited resources (staff, budget, and time), the program staff may find it more practical and feasible to begin by using secondary data than by engaging in primary data collection. 

Primary Data 
To develop a more comprehensive understanding of syphilis as a public health problem in the community, the STD program, in collaboration with community partners, conducted 75 individual (openended) interviews and 5 focus groups, with nine men attending each. Each focus group was composed of men who met one of five distinct characteristics (i.e., white, Latino/Hispanic, African American, outofschool youth, youth receiving services at the STD clinic). Men were recruited from advertisements in local gay newspapers, through email networks of partner communitybased organizations (CBOs), and through Internet chat rooms. 
Two public health school graduate students, welltrained in interviewing techniques and focus group facilitation, conducted the interviews and focus groups. Each interview lasted approximately 30 minutes. The interviews were conducted over the course of a week, and each participant received a food certificate donated by local businesses. The focus groups took place over a week and lasted 2 hours. Each participant received a movie pass donated by a local movie theatre. 
The interviews and focus group were designed to assess issues related to syphilis knowledge, attitudes, social norms, and behaviors regarding safer sex behaviors (e.g., condom use; number of sexual partners) among men having sex with men (MSM). In addition, interviews and focus groups explored issues relevant to mens perceptions about the accessibility and quality of community health services, and factors that would predispose them to use such services. The interviews and focus groups revealed that some young MSM had inaccurate perceptions of their own vulnerability for syphilis given their sexual practices. In addition, many of the young MSM reported lack of skills to negotiate for safer sex, inconsistent condom use with casual partners, and that they felt highly influenced by social norms to use party drugs and to have sex with multiple partners. 

Secondary Data 
Secondary data were obtained from a recent study conducted by the local school of public health on the overall health of MSM and local epidemiologic profiles. These data showed that among young MSM, 8% were reported to have had syphilis, 10% Chlamydia, and 15% gonorrhea within the past year. 

3. Identify possible causes of STDs in the target population. 
Based on a review of the primary and secondary data,
program staff and stakeholders identified possible causes of
increased rates of syphilis among MSM. They listed the
following risk factors:

	
Unprotected sexual intercourse. 

	
Multiple sexual partners. 

	
Drugtaking behaviors. 

	
Lack of knowledge about syphilis, symptoms,
and treatment options.

	
Inadequate skills to negotiate for safer sex. 

4. Determine what other programs are available to address STD related problems. 
Based on a review of available information, program staff and stakeholders found limited syphilis prevention programs for young MSM within the community. A local STD coalition consisting of the STD program staff, HIV/AIDS Department of Health (DH) staff and two local CBOs committed resources to address syphilis prevention among the young MSM population. The STD program had adequate resources to screen, test, educate, and provide clinical services to the MSM community, but these services were accessed by small numbers of MSM. One of the CBOs had initiated a communitylevel syphilis prevention intervention intended to educate the MSM community about syphilis and to change norms about sexual practices. 

5.Prioritize needs based on available resources and/or resources
that can be obtained or developed.

The program staff and stakeholders came together to prioritize needs based on their needs assessment. It was evident to them that broader, more comprehensive STD prevention efforts needed to be undertaken with MSM in City X. They noted that unmet prevention and control needs included: 
	
Availability of syphilis testing at nontraditional venues and times of day. 

	
MSM knowledge about syphilis and other STDs, and HIV. 

	
Community awareness about the link between HIV infection and syphilis. 

	
Venuebased outreach. 

STD program staff and stakeholders noted that program needs included: 
	
Addressing MSM lack of knowledge about syphilis. 

	
Facilitating MSM acquisition of more realistic selfappraisals of their personal vulnerability for syphilis and other STDs. 

	
Building MSM skills and selfefficacy to communicate and negotiate for safer sex. 

	
Increasing MSM awareness of the syphilis outbreak, symptoms, and treatments. 

	
Promoting communitywide norms that reduce the spread of syphilis. 

The program staff and stakeholders made a number of conclusions based on the needs assessment. Most importantly, they felt that syphilis screening must change in order to serve MSM more effectively. The data reflected that screening needed to be available in alternative locations such as sex venues and mobile sites. In addition, the times of availability for screening needed to be compatible with the activities and schedules of the population served. Later hours during the weekday and weekend hours should be provided in order to increase access to screening. The STD program staff and stakeholders concluded that increased access to screening increased the opportunity to diagnose and treat infections. 

Program staff and stakeholders made the following recommendations: 
1. 
Syphilis screening must be pilottested at nontraditional venues (bath houses, CBOs, bars, etc.) to determine the effectiveness. In addition, already identified MSMfriendly clinics should have hours that more effectively serve their clientele. 

2. 
A targeted media campaign specifically addressing MSM needs and risks should be implemented for syphilis. Increasing the communitys awareness of the connection between HIV infection and syphilis is also a priority. 

3. 
Program planning and implementation should involve pertinent stakeholders, including: 

 
STD and HIV DPH staff and managers 

 
Staff and managers from four local CBOs serving MSM 

 
Local MSM 

 
Bathhouse owners and managers 

REFERENCES 
Academy for Educational Development, Center for CommunityBased Health Strategies. (2000, August). Communitybased health program tip sheet: The needs assessment  Tools for long term planning. [Brochure]. Washington, DC: Academy for Educational Development. 
Centers for Disease Control and Prevention. (1999). Framework for program evaluation in public health. MMWR Recommendations and Reports, 48(RR11). Retrieved February 22, 2005, from 
http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm 
Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Division of Adolescent and School Health. (2004). Evaluation steps tools. Unpublished manuscript. 
Last, J. N. (2001). Dictionary of epidemiology (4th ed.). New York: Oxford University Press. 

MacDonald, G., Starr, G., Schooley, M., Yee, S. L., Klimowski, K., & Turner, K. (2001, November). Introduction to program evaluation for comprehensive tobacco control programs. Atlanta: Centers for Disease Control and Prevention. Retrieved October 17, 2004, from http://www.cdc.gov/tobacco/evaluation_manual/ Evaluation.pdf 
McKenzie, J. F., Jurs, J. L. (1993). Planning, implementing, and evaluating health promotion programs: A primer. New York: Macmillan. 
Modeste, N. N. (1996). Dictionary of public health promotion and education: Terms and concepts. Thousand Oaks, CA: Sage. 
State of Wyoming Public Health Department. (2004, April). Maternal and child health needs assessment: Issue brief  Sexually transmitted diseases in adolescents. Retrieved November 5, 2004, from http://wdh.state.wy.us/mchepi/pdf/briefs/STD.pdf 
Windsor R., Baranowski, T., Clark, N., & Cutter, G. (1999). 
Evaluation of health promotion, health education, and disease 
prevention programs (2nd ed.). Mountain View, CA: Mayfield. 
Witkin, B. R., & Altschuld, J. W. (1995). Planning and conducting needs assessments. Thousand Oaks, CA: Sage. 
W.K. Kellogg Foundation. (1998, January). W.K. Kellogg Foundation evaluation handbook. Retrieved October 17, 2004, from http://www.wkkf.org/Programming/ResourceOverview.aspx?CID= 281&ID=770 

APPENDIX
Methods Frequently Used to Collect
Needs Assessment Data

METHOD  DESCRIPTION OF METHOD  SAMPLE ASSESSMENT ISSUES ADDRESSED BY METHOD  
Focus group  Interviews with small, homogenous group (about 6 11 individuals) intended to elicit insights, attitudes, and perceptions about a focused topic area.   MSM perceived risks of acquiring an STD if they are already HIV+.  Community members opinions of media campaign being designed.  
Individual interviews  Interviews with individuals who are selected because of their relevant personal experience, knowledge, and/or expertise regarding the issues   Identification of best ways to introduce new treatment options for partner referrals.  STD program director and staff  Latino community advocates  
Community forums and public meetings  Structured discussions with community members to elicit insights, opinions, perceptions, and recommendations related to health issues and community services.   Examination and discussion of the barriers to STD screening in community venues.  Discussion of how the local school board views provision of STD prevention education in high school.  Identification of barriers, and solutions to barriers, regarding integrating the local faith community into STD/HIV prevention efforts.  
Survey  Data collection method which usually uses closeended questions, thus providing quantitative data. It can be selfadministered, or administered over the telephone, using a computer, or facetoface.   Identification of demographic characteristics (e.g., race, ethnicity, sexual orientation) of the clients receiving clinical services.  Identification of service utilization patterns (e.g., number of MSM who received a syphilis screening within the last 6 months).  Identification of patterns and trends in risk behaviors (e.g., frequency of unprotected intercourse among heterosexual women).  

TOOL 2.2: DEVELOP YOUR PROGRAM GOALS AND MEASURABLE (SMART) OBJECTIVES 
INTRODUCTION 
You also need to describe your program goals and objectives. Goals and objectives are essential in program planning and they establish criteria and standards against which you can determine your program performance. 
STDrelated program goals and objectives are based on program needs (see Tool 2.1), Healthy People 2010 objectives, and the Division of STD Prevention (DSTDP) Performance Measures. You can maximize the contribution of your stakeholders on your program goals and objectives by engaging them in their development or by sharing these with them. 
Understanding that you have already developed your program goals and objectives, you will need to identify the goals and objectives related with the program component or intervention you plan to evaluate. These goals and objectives should meet the criteria offered in this tool. 
The flowchart below provides a description of where developing program goals and objectives fits in with your past program planning activities. 

SMART SMART
PROCESS OBJECTIVES OUTCOME OBJECTIVES

LEARNING OBJECTIVES 
Upon completion of this tool, you will be able to: 
1. 
Distinguish between a goal and an objective. 

2. 
Develop one or more program goals and related objectives that will provide the basis for determining your programs performance. 

WHAT IS A GOAL? 
A goal is a broad statement about the longterm expectation of what should happen as a result of your program. Wellwritten goals do the following: 1) establish the overall direction for and focus of your program, 2) define the scope of what you want your program to achieve, and 3) serve as the foundation for developing your program objectives. 
Constructing a wellwritten goal involves two steps: 
Step 1: Specify the STD problem or STDrelated health risk factors. 
Programs funded by DSTDP might address a health problem (e.g., Chlamydia, gonorrhea, syphilis) or health risk factors (e.g., individuals low selfperceived risk for STDs; believing that condoms are difficult to use; having multiple sex partners). 
Step 2: Identify the target population(s) for your program. 
DSTDPfunded programs target populations and groups that are disproportionately affected by many STD pathogens. These populations and groups include: African Americans, Hispanics/Latinos, American Indian/Alaska Natives, Asian and Pacific Islanders, adolescents/young adults, women, men who have sex with men (MSM), people who have multiple sex partners, and sex workers (people who exchange sex for money, drugs, or other goods). Populations may also be identified by the location where they access care: correctional facilities, family planning clinics, human immunodeficiency virus (HIV) prevention and care clinics, substance abuse centers, and school health clinics, medical care 

DESCRIBE THE PROGRAM 
facilities. For some STDs that affect the population in general (e.g., genital herpes, human papillomavirus), you may need to define the target population more broadly. 
Following are two examples that illustrate a health problem and the related health risk behaviors, target populations, and goals. 
Example 1 
HEALTH PROBLEM: STD officials in County X reported that Chlamydia (Ct) was 4 times higher in female adolescents (aged 1618 years) screened in juvenile detention facilities than in female adolescents screened in other settings during the year 2002. Among female adolescents entering juvenile detention facilities, the rate for Ct was 16.7%, compared with 4.1% for school attendees. 
HEALTH RISK FACTORS: Unprotected sex; lack of awareness of how STDs are acquired. 
TARGET POPULATION: Female adolescents engaging in highrisk behaviors. 
RELATED GOAL: To reduce the prevalence of Ct among female adolescents in juvenile detention centers in County X. 
Example 2 
HEALTH PROBLEM: STD officials in City Y reported an increased number of syphilis cases among heterosexual males. In 2003, heterosexual males represented 0.9% of syphilis cases in males; in 2004, they made up 30% of male cases. 
HEALTH RISK FACTORS: Unprotected sex; having multiple partners. 
TARGET POPULATION: Heterosexual males engaging in highrisk behaviors. 
RELATED GOAL: To reduce the incidence of primary and secondary syphilis among heterosexual males in City Y. 

WHAT IS AN OBJECTIVE? 
Your program objectives are statements describing the results to be achieved, and the manner in which they will be achieved. Objectives are more immediate than goals; they are the mileposts you will pass on the way to achieving your program goal(s). Because objectives detail your program activities, you usually need multiple objectives to address a single goal. Wellwritten and clearly defined objectives will help you monitor your progress toward achieving your program goals and set targets for accountability. 
HOW CAN YOU WRITE AN OBJECTIVE THE SMART WAY? 
Your objectives will be appropriate and effective if you follow the SMART technique for writing objectives. The attributes of a SMART objective are: Specific, Measurable, Achievable, Relevant, and Timebound. [Note: Be aware that there is a variety of terminology used for the five components of SMART. For instance the A sometimes stands for appropriate or the R sometimes stands for realistic. The bottom line is that the message is the same.] 
Specific. Making your objectives specific means including the who, what, and where of the objective. Who refers to your target population (e.g., Latino adolescents, attendees of STD clinics). What refers to the action (e.g., screen, identify). Where refers to the location of the action (e.g., STD clinic in City X). Be as specific as possible about the target population (e.g., male and female adolescents between the ages of 1519 years, instead of adolescents). 
When describing the action, use only one action verb per activity (e.g., develop a workshop instead of develop and implement a workshop). More than one verb means that more than one action must be measured, which will cause problems when it comes to measuring success. For instance, suppose that your program was able to develop the workshop but did not implement it. Did you meet your objective? Because the objective had two actions, your success would be difficult to measure. 

Also, avoid verbs with vague meanings (e.g., understand, do) when describing expected results. Instead, use verbs that reflect measurable action, such as identify or list (see Appendix A or examples of precise and vague verbs). 
Remember: The greater the specificity, the greater the possibility for measurement. 
Measurable. Your objectives need to be measurable. Here the focus is on how much change is expected. Your objectives should quantify the amount of change you hope to achieve (e.g., Project area X will implement 2 professional development workshops among all STD clinical providers in State X by January 2007.). 2 and all represent the how much of the objective. 
Achievable. Your objectives should be realistic given your program resources and planned implementation. For instance, if you read the following: 100% of women in project area X will be screened for Ct you realize that this is not achievable. Besides the fact that reaching 100% of women is unrealistic, you will be wasting resources because not all women are at risk for Ct. You can use state, county, or local statistics as well as data from similar STD programs to provide context for what is reasonable and to help you ensure that your program objectives are achievable. 
Relevant. Objectives are relevant when they relate directly to the programs goals and together represent reasonable programmatic steps that can be implemented within a specific timeframe. For instance, a program goal is Reduce congenital syphilis in City X. A relevant objective may be: By December 2006, increase the percentage of women (from X% to Y%) in City X receiving a test for syphilis at first prenatal visit. 
Timebound. Your objectives should be defined within a timeframe. Here the focus is on when the objective will be met. Specifying a timeframe in the objective will help you in both planning and evaluating your program (e.g., at the end of laboratory visits; by January 2007). 

Example

GOAL: Reduce STDs among inmates in correctional institutions in 
Flowers City. SMART OBJECTIVE: By August 2008, increase the percentage (from X% to Y%) of clinical staff in all correctional facilities in Flowers City that fully 
adhere to the CDCSTD treatment guidelines. 
	
This objective is specific because it identifies a single action: increase; and focuses on a target group i.e., clinical staff in all correctional facilities in Flowers City. 

	
This objective is measurable because it specifies how much change is expected: increase from X% to Y% of clinical staff in all correctional facilities that fully adhere to the CDCSTD treatment guidelines. 

	
This objective is achievable because it is realistic given the
timeframe.

	
This objective is relevant because it relates to the program goal of
reducing the proportion of inmates with STDs in correctional
institutions in Flowers City by assuring that infected individuals are
treated according to guidelines.

	
This objective is timebound because it provides a specified time by which the objective will be achieved: August 2008. 

WHAT ARE PROCESS AND OUTCOME OBJECTIVES? 
You can write two types of objectives: process and outcome. 
When you write a process objective, you describe the activities/services that will be delivered as part of implementing the program. 

Example of a SMART process objective: 
By (month/year), STD training staff will provide one professional development workshop for disease intervention specialists (DIS) on protocols for timely interviewing of primary and secondary syphilis cases. 

DESCRIBE THE PROGRAM 
When you write an outcome objective, you specify the intended effect of the program in the target population or end result of a program. The outcome objective focuses on what your target population(s) will know or will be able to do at the conclusion of your program/activity. 

Example of a SMART outcome objective: 
By (month/year), the proportion of primary and secondary syphilis cases interviewed within 7 days of the date of specimen collection will increase from X% at baseline to Y% in City X. 
WHAT ARE SHORTTERM, INTERMEDIATE, AND LONGTERM OUTCOME OBJECTIVES? 
You can categorize outcome objectives as shortterm, intermediate, or longterm. They should be logically linked to each other and to the process objectives. 
	
Shortterm outcome objectives are the initial expected changes in your target population(s) after implementing certain activities or interventions (e.g., changes in knowledge, skills, and attitudes). 

	
Intermediate outcome objectives are those interim results that
provide a sense of progress toward reaching the longterm
objectives (e.g., changes in behavior, norms, and policy).

	
Longterm outcome objectives are achieved only after the
program has been in place for some time (e.g., changes in
mortality, morbidity, quality of life).

Example 
Following is an example of linking a program goal with process and outcome (shortterm, intermediate, and longterm) objectives. 
SAMPLE GOAL: Reduce the prevalence of Ct infection among highrisk female adolescents in County Z. 
SAMPLE PROCESS OBJECTIVES: 
1. 
By (month/year), STD program staff will disseminate protocols for Ct screening for female adolescents age 1519 to all family planning clinics in County Z. 

2. 
By (month/year), provide two professional development workshops for all medical staff of County Z family planning clinics on protocols for the Ct screening of female adolescents age 1519. 

SAMPLE SHORTTERM OUTCOME OBJECTIVES: 
1. 
By (month/year), all medical staff who have participated in the Ct
training will report in a pre/post questionnaire, an increase in
knowledge of the Ct screening protocols from X% to Y%.

2. 
By (month/year), medical staff who participated in the Ct training will demonstrate, in a performance session, at the end of the training, an improvement in Ct prevention counseling skills from X% to Y%. 

SAMPLE INTERMEDIATE OUTCOME OBJECTIVES: 
1. 
By (month/year), the percentage of medical staff in family planning clinics in County Z who screen atrisk female adolescents for Ct as a regular part of the health assessment will increase from X% to Y%. 

2. 
By (month/year), the percentage of medical staff who conduct Ct
prevention counseling in family planning clinics in County Z will
increase from X% to Y%.

SAMPLE LONGTERM OUTCOME OBJECTIVE: 
1.	The prevalence of Ct among high risk female adolescents within County Z will have been reduced from X% in (year) to Y% in (year). 
SUMMARY CHECKLIST: Developing WellWritten Goals and Objectives 

CONCLUSION AND NEXT STEPS 
After completing this tool, you will have learned to develop program goals that support the purpose of your program. You will be able to use the SMART technique to generate (1) process objectives related to the implementation of your program, and (2) outcome objectives related to the kinds of changes that you expect to make in the target population(s) served by your program. 
These goals and objectives provide a frame of reference for articulating the other elements of your program and for initially understanding the use of a program logic model, which you will learn more about next in Tool 2.3 (Describe your Program and Get Familiar with Logic Models). 
ACRONYMS USED IN THE TOOL 
Ct Chlamydia CSPS Comprehensive STD Prevention Systems DIS Disease Intervention Specialists DSTDP Division of STD Prevention, CDC HIV Human Immunodeficiency Virus MSM Men who have sex with men SMART Specific, Measurable, Achievable, Relevant, and Timebound objectives STD Sexually transmitted disease 
KEY TERMS 
Goal: A broad statement related to the purpose of your STD program, which states what your program will accomplish (the desired result). 
Objectives: Measurable statements that describe the manner in which your STD program goals will be achieved. They will help you monitor your progress toward achieving program goals. 

Outcome Objectives: Measurable statements specifying the intended effect of your program in the target population(s) or end result of your STD program. 
Process Objectives: Measurable statements describing your STD program activities and the actions involved in their implementation. 
STDrelated risk factors: Specific behaviors, attitudes and/or limited knowledge that put individuals in your project area at risk of STDs 
SMART: An acronym describing criteria used to write objectives that are Specific, Measurable, Achievable, Relevant, and Timebound. 
Note: On the next two pages there are two exercises related to 
program goals and objectives. We encourage you to complete 
these exercises in order to apply what you learned from this tool. 

EXERCISE 1: Matching Goals and Objectives 
Listed below are goals and objectives for two hypothetical STD program components (A and B). 
	
Improve STD surveillance and data management in City X. 

	
Reduce Ct and gonorrhea rates among women in County X. 

	
Provide staff training on data reporting. 

	
To increase knowledge of CDC requirements for data reporting. 

	
Family planning clinics will develop a computerized case management system that facilitates timely followup with clients for treatment appointments. 

	
To increase the proportion of women with positive Ct and
gonorrhea tests who are treated.

Here are the steps for completing the exercise: 
	
Identify which two statements are the STD program component goals, and write them in the spaces provided (Goal A and Goal B). 

	
Identify which statements are objectives, and match each objective with its respective goal. 

	
Among these objectives, determine which are process objectives and which are outcome objectives. Write them in the appropriate places in the list. 

	
Under each process and outcome objective, rewrite the objectives using the SMART technique. 

Program Component A 
Goal A:
Process objective A:
Revised SMART process objective A:
Outcome objective A:
Revised SMART outcome objective A:

Program Component B 
Goal B:
Process objective B:
Revised SMART process objective B:
Outcome objective B:
Revised SMART outcome objective B:

Note: You will find the answers and suggested objectives at the end of this tool. 

EXERCISE 2: Writing Goals and SMART Objectives 
INSTRUCTIONS 
	
Review the information presented below. 

	
Develop in the table on next page at least one process (operations) and one outcome (effects) objective in the SMART way based on the Background Information provided. 

	
Classify your outcome objective(s) into short, intermediate or
long term.

BACKGROUND INFORMATION 
This exercise relates to a professional development (PD) component of a larger Chlamydia (Ct) screening initiative that the STD Program plans to conduct in 3 juvenile detention centers (JDCs) in County 
Z. The goal of the initiative is to reduce the prevalence of Ct among female adolescents (1317 years of age) in JDCs in County Z. 
The PD component will include the provision of two Ct training sessions (i.e., a general and a clinical). The general session (for clinicians and nonclinicians) will provide (1) an overview of the epidemiology of Ct, and (2) STD prevention counseling techniques, particularly as they relate to adolescent females. The topics for the clinical session (only for clinicians) will cover Ct screening protocol (STD clinical guidelines), including the quality of specimen collection, diagnostic methods, and treatment options. Training material will be developed accordingly. 
It is expected that JDC staff (clinicians and nonclinicians) who participate in the general session will increase their knowledge of the epidemiology of Ct and skills in providing STD prevention counseling. It is also expected that JDC clinical staff who participate in the clinical session will increase their skills in screening, diagnosing and treating Ct, according to clinical guidelines (i.e., protocol). In addition, it is expected that after participating in training, the clinical staff will fully apply the Ct screening protocol among eligible detainees, and that the prevalence of Ct among high risk female adolescents within the targeted JDCs will decrease. 

CRITERIA FOR SMART OBJECTIVES 
	
Specific: includes the who, what, and where. Use only one action verb. 

	
Measurable: focuses on how much change is expected. 

	
Achievable: realistic given program resources and planned implementation. 

	
Relevant: relates directly to program/activity goals. 

	
Timebound: specifies date in which the objective will
be achieved.

GOAL: To reduce the prevalence of Ct among female adolescents (1317 years of age) in Juvenile Detention Centers (JDCs) in County Z.  
PROCESS OBJECTIVES  OUTCOME OBJECTIVES  
SHORTTERM  INTERMEDIATE  LONGTERM  

REFERENCES

Academy for Educational Development. (1996). Chapter 4: Goals and objectives. In Planning and evaluating HIV/AIDS prevention programs in state and local health departments. (Document produced for the Centers for Disease Control and Prevention).Washington, DC. 
Aspen Reference Group. (1997). Community health education and prevention: A guide to program design and evaluation. Rockville, MD: Aspen. 
Cantor, J., Kaufman, N., & Klitzner, M. (1984). Four steps to better objectives. Madison, WI: The Wisconsin Clearinghouse. 
Centers for Disease Control and Prevention. (1999). Framework for program evaluation in public health. MMWR Recommendations and Reports, 48(RR11). Retrieved February 22, 2005 from http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm 
Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Division of Adolescent and School Health. (2004). Evaluation steps tools. Unpublished manuscript. 
MacDonald, G., Starr, G., Schooley, M., Yee, S. L., Klimowski, K., & Turner, K. (2001, November). Introduction to program evaluation for comprehensive tobacco control programs. Atlanta: Centers for Disease Control and Prevention. Retrieved October 17, 2005, from http://www.cdc.gov/tobacco/evaluation_manual/Evaluation.pdf 
McKenzie, J. F., Jurs, J. L. (1993). Planning, implementing, and evaluating health promotion programs: A primer. New York: Macmillan. 
Rossi, P. H., Freeman, H. E., & Lipsey, M. W. (1999). Evaluation: A systematic approach (6th ed.). Thousand Oaks, CA: Sage. 

Should you need help or more information regarding this or any other evaluation tool, please contact CDC DSTDP program evaluation staff at (404) 6398276. 

POSSIBLE ANSWERS FOR EXERCISE 1 
Goal A: Improve STD surveillance and data management in City X. 
Process objective A: Provide staff training on data reporting. 
Revised SMART process outcome A: By (month/year), the STD surveillance unit will provide X number of professional development workshops to STD program staff in City X on the importance of data reporting and CDC requirements. 
Outcome objective A: To increase knowledge of CDC requirements for data reporting. 
Revised SMART outcome objective A: By (month/year), the percentage of STD surveillance staff (attending the workshops in City X) who can correctly describe the CDC requirements for complete data reporting will increase from X% to Y%. 
Goal B: Reduce Ct and gonorrhea rates among women in County X. 
Process objective B: Family planning clinics will develop a computerized case management system that facilitates timely followup with clients for treatment appointments. 
Revised SMART process B: By (month/year), X% of family planning clinics in County X will complete the development of a computerized case management system that facilitates the timely followup appointments for treatment of female clients with Ct and gonorrhea. 
Outcome objective B: To increase the proportion of women with positive Ct and gonorrhea tests who are treated. 
Revised SMART outcome B: 
	
By (month/year), the STD Program in County X will increase the percentage (from X% to Y%) of women with positive Ct test results who are treated within 14 days of the date of specimen collection. 

	
By (month/year), the STD Program in County X will increase the percentage (from X% to Y%) of women with positive gonorrhea test results who are treated within 14 days of the date of specimen collection. 

POSSIBLE ANSWERS FOR EXERCISE 2 
GOAL: To reduce the prevalence of Ct among female adolescents (1317 years of age) in Juvenile Detention Centers (JDCs) in County Z.  
PROCESS OBJECTIVES  OUTCOME OBJECTIVES  
SHORTTERM  INTERMEDIATE  LONGTERM  
 By (month/year), the STD program staff will prepare all training materials (i.e., sessions curriculum, presentation, handouts for participants, performance exercise, pre/post questionnaires) to use with JDC staff.  By (month/year), one training session will be conducted with 15 JDC staff (including 6 clinicians) from 3 JDCs on an overview of epidemiology of Ct.  By (month/year), one training session will be conducted with 6 clinical staff in 3 JDCs on screening protocol (i.e., STD clinical guidelines).   By (month/year), the JDC staff (15) participating in the Ct training will demonstrate increased knowledge (from X% to Y%) in the epidemiology of Ct in a pre/post questionnaire.  By (month/year), all 6 clinical staff who participated in the training will report in a pre/post questionnaire, an increase in knowledge of the Ct screening protocol from X% to Y%.  By (month/year), all training participants will demonstrate, in a performance session at the end of the training, their skills in providing STD prevention counseling.   By (month/year), supervisors of the 6 clinicians in 3 JDCs will report an increase in Ct screening (from X% to Y%) among eligible detainees through clinical audits of staff and chart reviews.   The prevalence of Ct among high risk female adolescents within JDCs participating in the Ct training will be reduced from X% in (year) to Y% in (year).  

DESCRIBE THE PROGRAM 
APPENDIX A Examples of Vague and More Precise Verbs 

APPENDIX B
Worksheet for Developing SMART Objectives

This worksheet is intended to help you with developing SMART objectives. Begin by stating one or more program goals, from which the SMART objectives will be developed. 
Program Goal(s): 
1. 

2.

Criteria for goals: 1) Specifies the STD problem or STDrelated health risk factors; 2) Identifies the target population(s) for your program. 
Related SMART Process Objectives: 
1. 

2.

Related SMART Outcome Objectives: 
1. 

2.

Criteria for SMART objectives: 
	
Specific: includes the who, what, and where. Use only one action verb. 

	
Measurable: focuses on how much change is expected. 

	
Achievable: realistic given program resources and planned implementation. 

	
Relevant: relates directly to program/activity goals. 

	
Timebound: specifies date in which the objective will
be achieved.

TOOL 2.3: IDENTIFY THE ELEMENTS OF YOUR PROGRAM AND GET FAMILIAR WITH LOGIC MODELS 
INTRODUCTION 
When you have a clear description of your program, you achieve an important milestone toward designing an evaluation of your program or a program activity. You started to do this when you developed your program goals and objectives. Using this tool, your program staff and stakeholders will be able to describe the program more fully by identifying your programs core components and activities, and their ability to achieve the desired results of the program. 
While your program may have been functioning for awhile and many of these elements are already articulated, it is important to engage program staff and stakeholders in a more lengthy discussion about the program. This will allow everyone who is planning the evaluation to better understand how the program will attain the expected results, and identify where the barriers of doing so might be. This tool will allow you to become familiar with logic models which can assist you in strategic planning, program planning, and program evaluation by graphically illustrating how your program is intended to work. This step is extremely fundamental for program evaluation to the point that some people think that this is where the actual evaluation process starts. The flowchart below provides a description of where the logic model fits in with your past planning activities. 

LEARNING OBJECTIVES 
Upon completion of this tool, you will be able to: 
1. 
Describe the elements of your program. 

2. 
Identify the components of a logic model. 

WHAT ARE THE ELEMENTS OF YOUR PROGRAM? 
The first step in developing a logic model is to consider and identify the elements of your program. Since it may not be feasible to evaluate the entire program, we will refer to a program activity or component you plan to evaluate. The following elements will give you the background information you need to develop a logic model. These include: 
	
Goals and objectives, which are the criteria and standards against which you can determine your program performance (for more information, refer to Tool 2.2.). 

	
Activities, which are those specific actions that your program
plans to implement during the funding cycle.

	
Inputs, which are the resources for conducting program activities (e.g., time, staff, technology, information, money). 

Two additional things to consider before developing a logic model are the stage of development of the program activity to be evaluated and the context in which it operates. 
	Stage of development is the level of maturity of your program activity classified in three stages: planning, implementation and maintenance/outcomes achievement. The stage influences the type of evaluation you may want to conduct as will be discussed in Step 3 (tools 3.1). For example: 
	
If you are planning a program activity, the intent of the evaluation is to refine the program plan. This also applies to new activities or components you plan to add to the program. 

	
If you are already implementing the program activity, the evaluation goal may be to see what works and to improve operations. 

	
If you have implemented your program activity for years, your program may be mature enough for you to evaluate its outcomes (effects). 

	Context refers to the setting and environmental influences in which your STD program activities operates. This can include a broad array of external factors such as STDrelated communitys/states laws, regulations, political climate, cultural norms, and social and economic conditions. These influences can significantly affect how you implement your program activities, whether you reach specified outcomes, and the design and interpretation of evaluation results. 
WHAT IS A LOGIC MODEL? 
A logic model is a picture of how a program works. You may have heard other names for logic models such as theory of change, conceptual map, strategic framework, program hypothesis, causal chain, or roadmap. While logic model terminology may vary, the Division of STD Prevention (DSTDP) uses the terms reflected in Table 1. 
TABLE 1. Logic Model Terms Used by DSTDP 

A logic model for your program will identify its main components and show how they relate to one another. See Figure 1 for an illustration of a basic logic model. The lefthand side of the logic model shows the process components (i.e., inputs, activities, and outputs) that correspond to the actions you plan for your STD program. The righthand side depicts the program outcomes or the effects you hope your STD program will have in your target population(s). The outcomes (i.e., shortterm, intermediate, and longterm) are in sequence, according to the time needed to accomplish them. 

Figure 1: Process and Outcome Components of a Logic Model 

WHY IS A LOGIC MODEL USEFUL? 
Logic models can benefit your program planning by (1) building understanding and clarity about your program among program staff and stakeholders, (2) identifying resources needed for your program, 
(3) identifying the sequencing of activities that should be implemented, and (4) serving as a basis for program evaluation. 
Another important benefit is that when you involve a broad range of stakeholders in constructing a logic model, they can better understand why it takes time to achieve longterm outcomes (short and intermediate outcomes must be achieved first), and help prioritize the activities that need to be evaluated. Your stakeholders involvement in constructing a logic model will promote their commitment to, and shared vision and ownership of the program plan, as well as their initial buyin on an evaluation. 
HOW CAN YOU USE YOUR GOALS AND OBJECTIVES TO DEVELOP A LOGIC MODEL? 
Because you have developed goals and SMART objectives in your CSPS application, you already have the content for the items you will use in the logic model regarding inputs, activities, outputs, shortterm outcomes, intermediate outcomes, and longterm outcomes. 

Essentially, your process objectives contain content that will help you write your inputs, activities and outputs; your shortterm objectives have content for your shortterm outcomes; your intermediate objectives have content for your intermediate outcomes; and your longterm objectives have content for your longterm outcomes. 
The following are two examples that illustrate the connection between goals and objectives and elements of logic modeling. Example 1 is of an everyday process like preparing a peanut butter sandwich and Example 2 is related to an STD program campaign. 
Example 1
Logic Model for Making Peanut Butter Sandwiches

GOAL: To eat homemade peanutbutter sandwich for breakfast. 
PROCESS OBJECTIVE: By (month/year), I will get one loaf of bread, a jar of peanut butter and one jar of jelly of my choice. 
SHORTTEM OUTCOME OBJECTIVE: By (month/year), I will prepare for myself one peanut butter and jelly sandwich for breakfast. 
INTERMEDIATE OUTCOME OBJECTIVE: By (month/year), I will increase the number of sandwiches prepared for my family from 0 to 2 in a week. 
LONGTEM OUTCOME OBJECTIVE: By (month/year), I will increase the number of requests I receive from family members for peanut butter and jelly sandwiches for breakfast. 

Example 2
Logic Model Elements for an STD Program Campaign

This is an example of an STD program campaign designed to inform females 2024 years of age who visit family planning clinics in City X about the risk to their reproductive health from chlamydial (Ct) infections. The goal is to prevent Ct infections among females 2024 years of age in City X. 
USE OF A PROCESS OBJECTIVE TO DEVELOP INPUTS, ACTIVITIES AND OUTPUTS IN A LOGIC MODEL: 
Process Objective  By (month/year), STD program staff will distribute 10 posters and 6,000 educational leaflets on chlamydia (Ct) to 10 family planning clinics that are frequented by females 2024 years of age, in City X. 
	
Inputs: 10 posters; 6,000 educational leaflets, staff time, transportation costs. 

	
Activities: Distribute posters and educational leaflets to family planning clinics
that are frequented by females 2024 years of age.

	
Outputs: Posters and leaflets distributed to family planning clinics that are
frequented by females 2024 years of age.

USE OF A SHORTTERM OUTCOME OBJECTIVE TO DEVELOP A SHORTTERM OUTCOME IN A LOGIC MODEL: 
Shortterm Outcome Objective By (month/year). X% of sexually active females 2024 years in 10 family planning clinics in City X will report in a questionnaire 3 ways of preventing Ct. Shortterm Outcome: Increased knowledge of how to prevent Ct among females 2024 years of age. 

USE OF AN INTERMEDIATE OUTCOME OBJECTIVE TO DEVELOP AN INTERMEDIATE OUTCOME IN A LOGIC MODEL: 
Intermediate Objective: By (month/year), X% of sexually active females 2024 years
of age, in 10 family planning clinics in City X will have reported increased use of
condoms in the past 6 months by X%.
Intermediate Outcome: Increased condom use among females 2024 years of age.

USE OF A LONGTERM OBJECTIVE TO DEVELOP A LONGTERM OUTCOME IN A LOGIC MODEL: 
LongTerm Objective: Chlamydia cases among females 2024 years of age, in family planning clinics in City X will have been reduced from X% in (month/year) to Y% in (month/year). LongTerm Outcome: Reduced prevalence of Chlamydia among females 2024 years of age who attend family planning clinics. 

SUMMARY CHECKLIST: Identify program elements of your program and get familiar with logic models 

CONCLUSION AND NEXT STEPS 
This tool discussed the importance of articulating the elements of your program and how to use them as a starting point for developing a logic model. 
In the next tool  2.4 Develop Logic Models to Link Program Activities with Outcomes  you will use the information on the components of a logic model to actually construct logic models that will help you plan any evaluation activities you intend to undertake. 
ACRONYMS USED IN THIS TOOL 
CSPS Comprehensive STD Prevention System DIS Disease Intervention Specialist DSTDP Division of STD Prevention SMART Specific, measurable, achievable, relevant, and timebound objectives STD Sexually transmitted disease 
KEY TERMS 
Activities: Actual events that take place as part of your STD program (e.g., develop pamphlet, screen patients). 
Context: The setting and environmental influences in which your STD program operates (e.g., laws, regulations, political climate). 
Inputs: Your STD programs resources (e.g., money, staff, materials). 
Intermediate outcomes: Intended effects of your program in the target population that takes longer than shortterm outcomes to occur (e.g., changes in STDrelated policy or in behavior of the target population). 
Logic Model: Also referred to as Global Logic Model, a picture of how your STD program is supposed to work. 

Longterm outcomes: Intended effects of your program in the target population that may take several years to achieve, such as reduced disease transmission and incidence. 
Outcomes: The intended effects or changes of in the target population(s) that result from your STD program. 
Outputs: The direct products of your program activities or services delivered (e.g., pamphlet developed, patients screened). 
Shortterm outcomes: Immediate effects of the program in the target population, such as changes in knowledge, attitudes, skills, awareness, or beliefs of the target population. 
SMART: An acronym describing criteria used to write objectives that are Specific, Measurable, Achievable, Relevant, and Timebound. 

EXERCISE 1:
Translating Objectives into Logic Model Components

Instructions: Listed below are sample process and outcome objectives that were developed for a program being implemented to reduce Gonorrhea among sexually active male adolescents in state X. Review each of these objectives and translate them into appropriate logic model components (as shown in the example on pages 4 and 5 of this tool). [Please note: You will find the correct answers at the end of this tool.] 
Process objective: By (month/year), X% of STD program staff in City Z will attend a workshop on techniques to improve partner elicitation and management services among male adolescents (1418 years of age). 
Inputs: 
Activities: 
Outputs:

Shortterm outcome objective: By (month/year), X% of STD clinics in City Z will have paired experienced DIS (Disease Intervention Specialists) with new DIS (having less than 6months experience) to improve partner elicitation and partner management skills. 
Shortterm Outcome: Intermediate outcome objective: By (month/year), X% of DIS in City Z will have increased their rate of eliciting locatable sex partners among male adolescents (1418 years of age) from Y% to Z%. 

Intermediate Outcome: 
Longterm outcome objective: By (month/year), the prevalence of gonorrhea among male adolescents (1418 years of age) in City Z will be reduced from X% (in month/year) to Y%. 
LongTerm Outcome: 

EXERCISE 2: Matching Program Logic Model Components 
Instructions: Listed below are nine selected items from a logic model depicting an STD screening, treatment, and counseling program. The goal of the program is to reduce Gonorrhea in State 
X. Match each item with one of the six components of a logic model: Inputs, Activities, Outputs, ShortTerm Outcomes, Intermediate Outcomes, and LongTerm Outcomes. Put these items in the correct components in the attached logic model template. 
[Please note: You will find the correct answers at the end of this tool.] 

Items in a logic model for an Gonorrhea screening, treatment, and counseling program 
1. 
Screen, treat and counsel patients. 

2. 
Six medical staff, two DIS, and two administrative staff. 

3. 
Increased rates of condom use among patients. 

4. 
Increased awareness of Gonorrhea transmission and prevention among patients. 

5. 
$XX,XXX in health department funds. 

6. 
Increased rate of abstinence until treatment is complete. 

7. 
Patients screened, treated, and counseled. 

8. 
Decreased prevalence of Gonorrhea. 

Logic Model Template for Exercise 2 
SHORTTERM INTERMEDIATE LONGTERM INPUTS ACTIVITIES OUTPUTS OUTCOMES OUTCOMES OUTCOMES 

REFERENCES 
Centers for Disease Control and Prevention. (n.d.). Oral health infrastructure development tools, Step 2B: Logic models. Retrieved October 17, 2004, from http://www.cdc.gov/oralhealth/library/pdf/logic_models.pdf 
Harvard Family Research Project. (1999). Learnings from logic models: An example of a family/school partnership program. Retrieved November 5, 2004, from http://gseweb.harvard.edu/hfrp/pubs/onlinepubs/rrb/learning.html 
MacDonald, G., Starr, G., Schooley, M., Yee, S. L., Klimowski, K., & Turner, K. (2001, November). Introduction to program evaluation for comprehensive tobacco control programs. Atlanta: Centers for Disease Control and Prevention. Retrieved October 17, 2004, from http://www.cdc.gov/tobacco/evaluation_manual/Evaluation.pdf 
United Way of America. (1996). Measuring program outcomes: A practical approach. Alexandria, VA. Retrieved October 17 2004, from http://national.unitedway.org/outcomes/library/pgmomres.cfm 
W.K. Kellogg Foundation (2001, December). Logic model development guide. Retrieved November 7, 2004, from http://www.wkkf.org/Programming/ResourceOverview.aspx?CID=2 81&ID=3669 POSSIBLE ANSWERS FOR EXERCISE 1 (The answers provided in the answer key are by no means exhaustive of all potential answers.) 

Process objective: By (month/year), X% of STD program staff in City Z will attend a workshop on techniques to improve partner elicitation and management services among male adolescents 1418 years of age. 
 
Inputs: funding for program staff to attend workshop; training schedule; staff time 

 
Activities: program staff signup to attend workshop 

 
Outputs: staff signed up to attend workshop; staff attended the workshop 

Shortterm outcome objective: By (month/year), X% of STD clinics in City Z will have paired experienced DIS (Disease Intervention Specialists) with new DIS (having less than 6months experience) to improve partner elicitation and partner management skills. 
 Shortterm Outcome: Increased instances of DIS working in pairs on partner elicitation and management. 
Intermediate outcome objective: By (month/year), X% of DIS (Disease Intervention Specialists) in City Z will have increased their rate of eliciting locatable sex partners among male adolescents (1418 years of age) from Y% to Z%. 
 Intermediate Outcome: Increased elicitation of locatable sex partners of male adolescents. 
Longterm outcome objective: By (month/year), the prevalence of gonorrhea among male adolescents (1418 years of age) in City Z will be reduced from X% (in month/year) to Y%. 
 LongTerm Outcome: Reduced prevalence of gonorrhea among male adolescents. 

ANSWERS FOR EXERCISE 2 

Logic Model Item 1.	Screen, treat, and counsel patients 
Activities

Logic Model Item 2.	Six medical staff, two DIS, and two
administrative staff  Inputs

Logic Model Item 3.	Increased rates of condom use among
patients  Intermediate Outcome

Logic Model Item 4.	Increased awareness of Gonorrhea transmission and prevention among patients 
 ShortTerm Outcome Logic Model Item 5. $XX,XXX in health department funds  Inputs Logic Model Item 6. Increased rate of abstinence until treatment is complete Intermediate Outcome Logic Model Item 7. Patients screened, treated, and counseled  Outputs 
Logic Model Item 8.	Decreased prevalence of Gonorrhea  Long
Term Outcome

TOOL 2.4: DEVELOP LOGIC MODELS TO LINK YOUR PROGRAM ACTIVITIES WITH OUTCOMES 
INTRODUCTION 
At this point in your evaluation planning process, STD program staff and other stakeholders have a common understanding of the focus and priority areas of the program, formulated goals and objectives, described other program elements, and gotten familiar with what a logic model is. Now is the time for you to construct one or more logic models that will help you plan your evaluation. The flowchart below provides an illustration of where the program logic model fits in with the program planning and program evaluation process. 
UNDERSTANDING OF PROGRAM FOCUS AND PRIORITY AREAS 

LEARNING OBJECTIVES 
After completing this tool, you will be able to:
Develop a logic model for your overall program.
Develop a logic model for a program component/activity you are
interested in evaluating.

WHAT ARE THE CHALLENGES AND REWARDS OF THE LOGIC MODEL PROCESS? 
Developing a logic model may take time because it requires engaging stakeholders in the process, carrying out the actual steps of logic model construction, and adjusting the logic model periodically to reflect changes in programmatic efforts. Nonetheless, program staff and stakeholders can learn a great deal about the STD program or selected program activity, clarify how it is supposed to work, and create a sense of ownership. Also, having a comprehensive description of a program or an activity is an excellent aid in moving programs in a positive direction and deciding what to evaluate. A logic model is a good way to map the expected progress of your program, and focus the evaluation to determine what works, what does not work and how to improve it. 
WHAT TYPES OF LOGIC MODELS CAN YOU CONSTRUCT AND WHY? 
You can develop a logic model to illustrate the big picture of the entire STD program (i.e., a global logic model) or develop a logic model of a segment of the global logic model (i.e., nested logic model). For instance, Figure 1 shows a global logic model illustrating the general process and outcomes of State X STD Prevention Program. This logic model reflects the epidemiology of STDs in State X and is provided as an example, not a model of a global logic model for any STD program. It displays the connections between inputs, activities, outputs and outcomes. It presents the more immediate outcomes on the way to longterm outcomes, which will take years to attain, so that progress can be tracked and improvements along the way can be made by the STD program. State X STD Prevention Program developed and used that global logic model as a management tool and to monitor the implementation of the program plan. 
The logic model in Figure 2 shows a nested logic model developed by program staff and relevant stakeholders of State X STD program. Because it was not feasible to evaluate the entire program, the STD program staff decided to focus on the activity pertaining to Chlamydia screening of adolescent females who are admitted to the countys juvenile detention centers (JDCs). In this logic model, they went into more detail regarding the connections from inputs to outcomes of that particular activity and how the latter is achieved. 

The logic model helped them determine the progress of the processes implemented, and identify additional actions that needed to take place for this activity to produce changes in the target population(s). 
Examples on how Figures 1 and 2 were constructed can be found in the Appendix. The Appendix will show you the connection between goals/objectives/ inputs/activities and logic model components. 

Figure 1: Logic Model for State X Comprehensive STD Prevention Systems (CSPS) Program

INPUTS  ACTIVITIES  OUTPUTS  SHORTTERMOUTCOMES  INTERMEDIATEOUTCOMES  LONGTERMOUTCOMES 
FundsCDC/DSTDP Other federalsources State sources PrivateSourcesAssigned Staff SHD/LHDCDCguidelines andrecommendationsTechnicalassistance andcollaboration Federalagencies State agencies Local agencies NGOs &affiliates   Provide communityand individualbehavior changeinterventions onsyphilis. Provide medical andlaboratory services. Provide chlamydiascreening amongsexually activefemale adolescentsand young women. Ensure syphilispartner services. Promote leadershipand programmanagement. Conduct surveillanceand datamanagement. Provide or ensuretraining andprofessionaldevelopment. Ensure adocumented STDoutbreak responseplan.  Community/ Individual Behavior ChangeInterventions Community and behavioral interventions onsyphilis implemented among at risk MSM.Medical and Lab Services Lab/medical facilities and providers reportingtesting results. Female admittees in juvenile detentionfacilities screened for Chlamydia.Partner Services Syphilis cases partners identified.Leadership and Program Management Strategic plan in place. Program operation plan to monitor programactivities. Appropriate program policies on professionaldevelopment in place.Surveillance and Data Management Reported cases of P&S syphilis andChlamydia sent to CDC within 30 to 60 daysfrom the date of specimen collection.Training and Professional Development Staff training needs regularly assessed. Training opportunities on syphilis andChlamydia provided and individuals trained.STD Outbreak Response Planning Plan includes required elements.  Increasedknowledge: Consequences Safe behaviors Self assessmentof riskIncreased intentionto use condoms  Increased safer sexbehaviors: AbstinenceMutual monogamy Fewerconcurrentpartners   ReducedSyphilisincidence  ReducedChlamydiaprevalence  

Figure 2: Logic Model of Chlamydia (Ct) Screening Program for Adolescent Females in County Z JuvenileDetention Centers (JDCs) 
INPUTS ACTIVITIES OUTPUTS SHORTTERMINTERMEDIATELONGTERMOUTCOMES OUTCOMES OUTCOMES 
JDCs
Reduction of Ctprevalence amongadolescent femalesin JDCs. 

HOW DO YOU CONSTRUCT A LOGIC MODEL? 
Logic models play a critical role in evaluation planning, so it is important to take ample time to develop them. Here are some steps to help you do this in a systematic way. 
1. Convene program staff and other stakeholders to develop a logic model. 
	
Determine the purpose of developing the logic model (e.g., build consensus on how the program or a program activity works; determine if given your activities the desired results can be achieved; show your funder what you are doing; determine when and what to evaluate; and monitor operations). 

	
Schedule sufficient time for your stakeholders to brainstorm ideas for creating the initial draft of the logic model. 

	
You may want to reserve a room with a large wall or workspace where the group can write down their thoughts, physically display them on the wall, and organize them. 

	
Review Tool 2.3 (Identify Elements of Your STD Program and Get Familiar with Logic Models) and discuss the benefits of logic modeling, and how all of you can use process and outcome objectives to develop the components of the logic model. 

	
On a board or on pieces of paper taped to a wall, write the logic model components (i.e., inputs, activities, outputs, shortterm outcomes, intermediate outcomes, and longterm outcomes). The group can organize their thoughts on the board, or on sheets of paper, or on postit notes under each component. 

2. Decide which direction you want to work on the model. 
If you are working with an existing program with activities already in place and are planning an evaluation of one of the activities, it may make more sense to start with the process components of the logic model (i.e., the inputs, activities, and outputs), which are on the left side of the model, then work toward the longterm outcomes (see Figure 3). This is called the Ifthen approach. 

Figure 3. Model for Existing Program with Activities in Place 
IF......THEN IF......THEN IF......THEN IF......THEN IF......THEN 

That is: If you invest time and money (e.g., inputs) to operate your program, then you can develop a syphilis educational plan (e.g., activities). If there is a plan, then the target population can receive information on syphilis and prevention messages,(e.g., outputs). If the target population receives information on syphilis and its prevention, then its knowledge of syphilis symptoms, transmission and prevention will increase (e.g., shortterm outcome). If the target populations knowledge of syphilis symptoms, transmission and prevention increases, then its healthseeking behavior will increase (e.g., intermediate outcome). Ultimately, if these changes take place in the target population, then the incidence and prevalence of syphilis will be reduced (e.g., longterm outcomes). Remember, when working from left to right, first generate inputs, activities, and outputs; then develop the outcomes. Figure 4 depicts a pictorial example of the If then flow. 
Figure 4: Example of Logic Model Flow on a Syphilis
Outbreak Response Project Targeting MSM in City Y

INPUTS ACTIVITIES	OUTPUTS 

	Meetings conducted with gay newspaper 
	1 fulltime, 
 Press releases written and sent to media outlets 2 parttime staff 
	Posters created and distributed to businesses 
 CDC/DSTDP (gay bars and bath houses) funds 
	Bus billboards developed and posted 
	Printing 
 Article written for newsletter and distributed equipment 
to health providers 
	
Condoms 

	
Training Materials 

	Meetings conducted with CBOs 
 HD funds distributed to CBOs
supplies

	
Screening 

	
Training conducted with CBO staff 

	
Educational pamphlets developed and 

	
CBO partners 

distributed to MSM 
	
Laboratory 

 CBO meeting with MSM leaders held
medication

	
Appropriate 

	
MSM and partners screened 

	
Condoms distributed to MSM, partners, clinics and gay bars 

On the other hand, if you are planning a new initiative for which you have determined its end result or goal, it may be easier to work from right to left, and to generate your outcomes and corresponding outputs, activities and inputs (see Figure 5). You would start with your goal(s) and move backward. 
Figure 5. Model to Plan a New Activity or Initiative 

For example, if you are planning to conduct syphilis screening in nontraditional venues the logic model will help you plan this initiative by linking outcomes with outputs. If your longterm outcome is reduced incidence of syphilis in a specific population, you will need to ask the question, What intermediate outcome would I expect before that?. If you decide on the intermediate outcome of increased healthseeking behavior among target population members, you would ask yourself what shortterm outcomes you would expect before this. The same questioning would take place to then develop your shortterm outcomes, outputs, activities and, finally, your inputs that would be needed to support the new activity. You work from right to left, by first stating your goal and then developing the outcomes before working toward the inputs. 
Note that when developing new activities some programs have found it beneficial to first work with their stakeholders to develop the backward logic model and then develop their SMART objectives. They found that understanding the logic of a new activity helped them design realistic and achievable objectives. 

3. Identify elements for each logic model component, and link them. 
	
As you brainstorm about the logic model components, write everyones ideas on the sheets of paper, or postit notes, or board under each component (see Figure 6). Reviewing your objectives can help you identify some of the elements (see Tool 2.3). 

	
After you have generated a list for each of the logic model components, place them in chronological order from top to bottom, if possible. For example, certain activities may need to be completed before other activities can occur. 

	
Then, working across, make sure that each activity is linked to corresponding outputs, and outputs are linked to corresponding outcomes. You may need to cut up or rewrite your lists to make this work. 

4. Periodically revise the logic model, as necessary. As your program or program activity changes, you will also need to change the logic model to include new or revised activities. For example when STD trends change in your project area, your areas of special emphasis and priority activities of your program may change. Therefore, you have to modify your logic model accordingly. In addition, if you are implementing a relatively new activity, you may realize that the related outputs and outcomes are not realistic based on your resources; consequently the logic model should be revised to reflect this. 

SUMMARY CHECKLIST: Things to consider when you develop logic models for your program and activities. 

CONCLUSION AND NEXT STEPS 
Having completed this program evaluation tool, you are aware of the importance of logic models for program planning and evaluation and able to construct one along with your program stakeholders. 
Now that you have engaged your stakeholders and fully described your program, your next step will be to focus your evaluation and decide which components of your logic model you will evaluate. As you proceed to Step 3, you will begin the process of tailoring your evaluation to the needs of your program and stakeholders, as well as determining what resources and personnel are necessary for your evaluation. 

ACRONYMS USED IN THIS TOOL 
CBO  Communitybased organization CDC  Centers for Disease Control and Prevention CLIA  Clinical Laboratory Improvement Amendments CPG  Community planning group CSPS  Comprehensive STD Prevention Systems Ct  Chlamydia DSTDP  Division of STD Prevention JDC  Juvenile detention center LHD  Local health department MSM  Men who have sex with other men NGO  Nongovernmental organization P&S syphilis  Primary and secondary syphilis PTC  Prevention training center SHD  State health department STD  Sexually transmitted disease YRBS  Youth Risk Behavior Survey 
KEY TERMS 
Activities: Actual events that take place as part of your STD program (e.g., develop pamphlet; screen patients). 
Global logic model: A type of logic model which depicts how an entire program operates. 
Goal: A broad statement expressing the purpose of your STD program that states what our program will accomplish (the desired result). 
Inputs: Your STD programs resources (e.g., money, staff, materials). 
Intermediate outcomes: Intended effects of the program in the target population (e.g., changes in STDrelated policy or in behavior of the target population). 
Logic model: A picture of how your STD program is supposed to work. 

Longtermoutcomes: Intended effects of the program in the target population that may take several years to achieve, such as reduced disease transmission and incidence. 
Nested logic model: A type of logic model, which depicts a component of a global logic model and describes the component in more detail than the global model. 
Outcome objectives: Measurable statements specifying the intended effect of your program in the target population(s) or end result of your STD program. 
Outcomes: The intended effects or changes in the target population(s) that result from your STD program. 
Outputs: The direct products of program activities or services delivered (e.g., seminar conducted, infected clients treated ). 
Process Objectives: Measurable statements describing your STD program activities and actions involved in their implementation. 
Shortterm outcomes: Immediate effects of the program in the target population, such as changes in knowledge, attitudes, skills, awareness, or beliefs of the target population. 

EXERCISE
Constructing a Program Component Logic Model

Following is a handson exercise to construct a logic model for the State X STD Programs training and professional development component. Everything you need to construct the logic model is provided, including a description of the component and its goals, inputs, activities, and outputs, along with its process objectives and shortterm, intermediate, and longterm outcome objectives. Also included are worksheets for developing each component of the logic model and for making linkages. You may find these worksheets helpful in developing your own logic models. Finally, a logic model template for the finalized model is provided [see Figure 6]. [Please note: You will find the actual logic model at the end of this tool, Figure 7.] 

Description of State X Leadership and Professional Development Program Component Goals 
	
Improve the quality of STD Prevention services in State X. 

	
Reduce the prevalence and incidence of STDs in State X. 

Inputs 
	
CDC/DSTDP funds. 

	
Staff: STD program training coordinator and syphilis elimination team trainer. 

	
Partner: Area prevention training center (PTC). 

	
Materials: Training curricula and training materials. 

Activities 
	
The STD training coordinator will conduct a training needs
assessment of STD prevention and control staff and share the
results with the area PTC.

	
The STD training coordinator, in collaboration with the STD program manager, will develop a schedule for STD staff to attend pertinent PTC trainings. 

	
STD clinic and disease intervention staff will attend pertinent
trainings conducted by the PTC.

100 

Process Objectives 
	
By (month/year), the STD training coordinator will conduct a training needs assessment of STD prevention and control staff in which he/she will interview (#) field supervisors about staffs skill and knowledge deficiencies. 

	
By (month/year), the STD training coordinator will share the
results of the training needs assessment with area PTC.

	
By (month/year), the STD training coordinator, in collaboration with the STD program manager, will develop a schedule for (#) STD clinical staff and (#) disease intervention staff to attend pertinent PTC trainings. 

	
By (month/year), (#) clinic staff will attend one STD intensive training conducted by the PTC. 

	
By (month/year), (#) disease intervention staff will attend one
training on effective STD interviewing and case management
techniques.

ShortTerm Outcome Objectives 
	
By (month, year), (X%) of STD program supervisors will report that there has been an increased number of professional development (PD) opportunities provided to STD staff. 

	
By (month/year), X% of clinical staff who attend PTC STD intensive training will report in a pre/post instrument administered at the training, an increase in their knowledge of the signs and symptoms of syphilis from (X%) to (Y%). 

	
By (month/year), an increase from X% to Y% of clinical staff who attend PTC STD intensive training will report a score of over 80% in a pre/post instrument on knowledge of Chlamydia screening protocols administered at the training. 

	
By (month/year), (X%) of disease intervention staff who attend PTC training will demonstrate, in a performance exercise, an increase in their interview skills recorded in a skills performance sheet. 

	
By (month/year), (X%) of disease intervention staff who attend PTC training will demonstrate, in a performance exercise, an increase in their case management skills recorded in a skills performance sheet. 

Intermediate Outcome Objectives 
	
By (month/year), X% of STD supervisors will report that the percentage of clinical staff who adhere to Chlamydia screening protocols has increased from (X%) to (Y%). 

	
By (month/year), X% of STD supervisors will report that the percentage of disease intervention staff who adhere to syphilis interview protocols has increased from (X%) to (Y%). 

	
By (month/year), X% of STD supervisors will report that the percentage of disease intervention staff who adhere to case management protocols has increased from (X%) to (Y%). 

LongTerm Outcome Objectives 
	
By (month/year), STD clinics with an unacceptable quality of services score will increase their quality score by X%. 

	
By (month/year), all STD clinics with an acceptable quality of services score will maintain or increase their quality score. 

	
By (month, year), the prevalence of Chlamydia in State X will have been reduced from (X%) to (Y%). 

	
By (month, year), the incidence rate of syphilis in State X will have been reduced from (X%) to (Y%). 

102 

REFERENCES 
Centers for Disease Control and Prevention. (n.d.). Oral health infrastructure development tools, Step 2B: Logic models. Retrieved October 17, 2004, from http://www.cdc.gov/oralhealth/library/pdf/logic_models.pdf 
Harvard Family Research Project. (1999). Learnings from logic models: An example of a family/school partnership program. Retrieved October 17, 2004 from http://gseweb.harvard.edu/hfrp/pubs/onlinepubs/rrb/learning.html 
MacDonald, G., Starr, G., Schooley, M., Yee, S. L., Klimowski, K., & Turner, K. (2001, November). Introduction to program evaluation for comprehensive tobacco control programs. Atlanta: Centers for Disease Control and Prevention. Retrieved October 17, 2004, from http://www.cdc.gov/tobacco/evaluation_manual/Evaluation.pdf 
Poister, T. (2003). Measuring performance in public and nonprofit organizations. San Francisco: JosseyBass. 
United Way of America (1996). Measuring Program Outcomes: A Practical Approach. Alexandria, VA: United Way. Retrieved October 17, 2004, from http://national.unitedway.org/outcomes/library/pgmomres.cfm 
W.K. Kellogg Foundation (2001, December). Logic model development guide. Retrieved October 17, 2004, from http://www.wkkf.org/Programming/ResourceOverview.aspx?CID= 281&ID=3669 

Worksheet for Logic Model Component: INPUTS 
Definition: Inputs are the resources that go into a program (e.g., money, staff, materials). 
List inputs here: 
104 

DESCRIBE THE PROGRAM 
Worksheet for Logic Model Component:
ACTIVITIES

Definition: Activities are the actual events that take place as part
of your program

List activities here: 

Worksheet for Logic Model Component:
OUTPUTS

Definition: Outputs are the direct products of your program activities; in other words what was produced or delivered as a result of the activities (e.g., seminar conducted, infected clients treated). 
Tip: Use your programs process objectives to inform how you write your outputs. Try to link each item in the activity component of your logic model to one or more outputs. 
ACTIVITIES 

OUTPUTS 

106 

Worksheet for Logic Model Component:
SHORTTERM OUTCOMES

Definition: Shortterm outcomes are the immediate effects of the program in the target population(s), such as changes in knowledge, attitudes, skills, awareness, or beliefs. 
Tip: Use your programs shortterm outcome objectives to inform how you write your shortterm outcomes. Try to link each item in the outputs component of your logic model to one or more shortterm outcomes. 
OUTPUTS 

SHORTTERM OUTCOMES 

Worksheet for Logic Model Component:
INTERMEDIATE OUTCOMES

Definition: Intermediate outcomes are the effects of the program in the
target population that take longer to occur than shortterm
outcomes, such as changes in policy or in behavior.

Tip: Use your programs intermediate outcome objectives to inform how you write your intermediate outcomes.Try to link each item in the shortterm outcome component of your logic model to one or more intermediate outcomes. 
SHORTTERM OUTCOMES 

INTERMEDIATE OUTCOMES 

108 

Worksheet for Logic Model Component:
LONGTERM OUTCOMES

Definition: Longterm outcomes are program effects on the target population(s) that may take several years to achieve. 
Tip: Use your programs longterm outcome objectives to inform how you write your longterm outcomes. Try to link each item in the intermediate outcome component of your logic model to one or more longterm outcomes. 
INTERMEDIATE OUTCOMES 

LONGTERM OUTCOMES 

Figure 6: Logic Model Template for State X Leadership and Professional Development Program Component 
SHORTTERMINTERMEDIATELONGTERM
INPUTS ACTIVITIES OUTPUTS 
OUTCOMES OUTCOMES OUTCOMES 
110 
Figure 7: Logic Model for State X Training and Professional Development (PD) Program Component 
INPUTS ACTIVITIES OUTPUTS SHORTTERMINTERMEDIATELONGTERMOUTCOMES OUTCOMES OUTCOMES 

Increased number of
PD opportunities
pertinent to the work
of STD staff.

	Reduced prevalenceof Chlamydia
	Reduced incidenceof syphilis 

APPENDIX

WHAT IS A GLOBAL LOGIC MODEL FOR AN STD PROGRAM? A CASE SCENARIO 
The STD program in State Z applied for funding from CDC/DSTDP to support its Comprehensive STD Prevention Systems (CSPS) program. The application outlined the performance goals, main objectives, and other elements of the program. This information was then used to develop an overall logic model, which provided a framework for program design and expected results as well as a reference for guiding the evaluation. Relevant information on State Z STD Programs goals, inputs, and activities; and its process objectives and shortterm, intermediate, and longterm outcome objectives is listed below. See Figure 1 for the logic model that was developed by program staff and stakeholders. This shows how the list of program information was used to construct the logic model. [Note that the objectives included are not exhaustive, but rather a sample to show you how to connect objectives and logic model components.] 
GOALS, INPUTS, ACTIVITIES, AND OBJECTIVES 

Goals 
	
Reduce STD rates by providing Chlamydia and gonorrhea
screening, treatment, and partner management to women in
publicly funded family planning and STD clinics.

	
Reduce the incidence of primary and secondary syphilis. 

	
Reduce the incidence of congenital syphilis. 

Inputs 
	
Funding from CDC/DSTDP and other federal, state, and local sources. 

	
State health department staff. 

	
CDC guidelines and recommendations. 

	
Partners providing technical assistance and collaboration, including federal, state, and local agencies; communitybased organizations (CBOs) and their affiliates; city HIV community planning group (CPG). 

112 

Activities1 
	
Provide community and individual behavior change
interventions.

	
Provide medical and laboratory services. 

	
Ensure partner services. 

	
Promote leadership and program management. 

	
Conduct surveillance and data management. 

	
Provide or ensure training and professional development. 

	
Ensure a documented STD outbreak response plan. 

Process Objectives2 
	
Community/Behavior Change Interventions By (month/year), (#) community/behavior change interventions will be conducted which serve MSM at high risk for syphilis in State Z. 

	
Medical and Laboratory Services
By (month/year), all onsite STD clinic laboratories will use
Clinical Laboratory Improvement Amendments (CLIA)
standards of testing in State Z.

By (month/year), (X%) of female admittees to X juvenile detention facilities in State Z will have been tested for Chlamydia. 

	
Partner Services By (month/year), at least one partner of (X%) of syphilis cases will have been initiated for followup by the STD program. 

	
Leadership and Program Management
By (month/date), the State Z STD Program will have
implemented a strategic plan to guide their operations.

By (month/year), the STD Program in State Z will have implemented an operation plan to monitor program activities. 
By (month, year), the STD Program in State Z will have implemented appropriate policies on professional development. 
1 
Required activities for funding of CSPS programs as stated in PA 04232. 
2 
Adapted from STD program objectives in the DSTDP Program Operations
Guidelines for STD Prevention and the DSTDP 2005 performance measures.

	
Surveillance and Data Management By (month/year), the percentage of reported cases of P&S syphilis sent to CDC via NETSS within 30 days from the date of specimen collection will have increased from X% to Y% in State Z. 

By (month/year), the percentage of reported cases of Chlamydia sent to CDC via NETSS within 30 days from the date of specimen collection will have increased from X% to Y% in State Z. 

	
Training and Professional Development
By (month/year), STD Program in State Z will have system in
place for the regular assessment of staff training needs.

By (month, year), (#) professional development events will have been provided to (#) program staff on syphilis prevention, screening, and treatment in State Z. 
By (month, year), (#) professional development events will have been provided to (#) program staff on Chlamydia prevention, screening, and treatment in State Z. 
	STD Outbreak Response Planning By (month/year), STD Program in State Z will have an outbreak response plan with required elements which will be updated every 6 months. 
ShortTerm Outcome Objectives (to be accomplished in 1 to 3 years) 
	
By (month/year), (X%) of patients in STD clinics in State Z will report, on a postscreening survey, an increase in their knowledge of STD consequences, safe behaviors and selfassessment of risk. 

	
By (month/year), (X%) of patients in STD clinics in State Z will report, on a postscreening survey, an increase in their intention to use condoms with their sexual partners. 

114 

Intermediate Outcome Objectives3 (to be accomplished in 3 to 5 years) 
	
The percentage of adolescents in grades 9 through 12 who report on the Youth Risk Behavior Survey (YRBS) that they abstain from sex will increase from (X%) in (month/year) to (Y%) in (month/year). 

	
The percentage of patients in STD clinics in State Z who report, on a postscreening survey, that they are in mutually monogamous relationships will increase from (X%) in (month/year) to (Y%) in (month/year). 

	
By (month/year), (X%) of patients in STD clinics in State Z will report, on a postscreening survey, a decrease in number of concurrent sexual partners from the previous year. 

LongTerm Outcome Objectives (to be accomplished in 5 or more years) 
	
Syphilis incidence in State Z will have decreased by at least (X%) from (month/year) to (month/year). 

	
Chlamydia prevalence rates in State Z will have decreased by at least (X%) from (month/year) to (month/year). 

Adapted from Poister, TH. (2003). Measuring Performance in Public and Nonprofit Organizations; pp 178. JosseyBass; San Francisco, CA. 

WHAT IS A NESTED LOGIC MODEL AND HOW DOES IT FIT INTO THE OVERALL LOGIC MODEL? A CASE SCENARIO 
The STD program in County Z has received funding from the States STD program and other sources to support an activity within its medical and laboratory component. The program activity is to conduct Chlamydia screening among adolescent females who are admitted to the countys juvenile detention centers (JDCs). 
In developing the design for the activity, the STD program staff identified the goal of the initiative, the inputs available, the specific activities that needed to be implemented, and the activitys process objectives and shortterm, intermediate, and longterm outcome objectives. This information is provided below. See Figure 2 for the logic model that program staff and relevant stakeholders developed. It shows how the program staff and stakeholders used the listed information to construct the logic model. [Note that the objectives included are not exhaustive, but rather a sample to show you how to connect objectives and logic model components.] 
GOAL, INPUTS, ACTIVITIES, AND OBJECTIVES 

Goal 
	Reduce Chlamydia rates among adolescent females incarcerated in juvenile detention centers in County Z. 
Inputs 
	
Funds. Funds from CDC/DSTDP, state and local sources, and private sources have been made available. 

	
JDC Staff. Administrators, medical providers, and
administrative staff at County Z JDCs will implement the
activity.

	
Partners. Several partners will lend technical direction, assistance, and collaborative energy to the activity. This includes the technical direction of State and County Z STD program staff and health department laboratory technicians who will process specimens. It also includes the organization that has been contracted to train the JDC medical staff in the protocols of Chlamydia screening and treatment, as well as several community stakeholders who will serve on an advisory committee for the project. 

116 

	Materials. Several types of materials will be used including CDC protocols/guidelines for Chlamydia screening and treatment, prevention educational materials, test kits, and treatment medication. 

Activities 
	
Professional Development. In order to adequately prepare JDC medical providers to implement the screening activity, contracted organization (X) will train the providers in Chlamydia screening and treatment. 

	
JDC Support. In order for JDC staff to be appropriately
involved in the development and implementation of the
initiative, the JDCs will authorize the participation of medical
staff and others for the activity.

	
Serving the Target Population. All sexually active female
adolescent admittees to JDCs will be offered Chlamydia
screening, counseling, and treatment if appropriate.

Process Objectives 
	
By (month/year), contracted organization (X) will conduct (#)
professional development events with (#) medical providers in
JDCs in County Z on appropriate Chlamydia screening,
counseling, and treatment with adolescent females.

	
By (month/year), the administrators of County Z JDCs will authorize that (#) JDC staff be allocated (#) hours of their work week to develop and implement the Chlamydia screening initiative. 

	
By (month/year), (Y%) sexually active female adolescents who are admitted to JDCs in County Z will have received Chlamydia screening and counseling. 

ShortTerm Outcome Objectives (to be accomplished in 1 to 3 years) 
	
By (month/year), the medical staff from County Z JDCs who participated in the training will demonstrate, in a pre/post questionnaire, an increase in their knowledge of the Chlamydia screening, counseling and treatment protocols from (X%) to (Y%). 

	
By (month/year), medical staff from County Z JDCs who participated in the training will demonstrate, in a performance exercise, an increase in their skills in conducting Chlamydia screening, counseling and treatment from (X%) to (Y%), as recorded in a skills performance sheet. 

	
By (month/year), the percentage of JDCs in County Z that conduct Chlamydia screening, treatment, and counseling among sexually active adolescent females as a regular part of the admission protocol will increase from (X%) to (Y%). 

	
By (month/year), (X%) of sexually active adolescent females in JDCs in County Z who receive Chlamydia screening will report, in a postscreening questionnaire, an increase in their awareness of how Chlamydia and other STDs are transmitted and prevented. 

	
By (month/year) (X%) of adolescent females in JDCs in County Z who receive Chlamydia screening will report, in a postscreening questionnaire, an increase in their intention to use condoms. 

Intermediate Outcome Objectives (to be accomplished in 3 to 5 years) 
	
By (month/year), (X%) of supervisors of JDC medical units in County Z will report an increase in clinicians adherence to Chlamydia clinical practice guidelines. 

	
By (month/year), (X%) of sexually active adolescent females in JDCs in County Z who receive Chlamydia screening will report, in a postscreening questionnaire, a decrease in their number of sexual partners. 

	
By (month/year), (X%) of sexually active adolescent females in JDCs in County Z who receive Chlamydia screening will report, in a postscreening questionnaire, an increase in their healthseeking behavior. 

LongTerm Outcome Objective (to be accomplished in 5 or more years) 
	By (month/year) Chlamydia prevalence among sexually active adolescent females will be reduced from (X%) to (Y%) in JDCs in County Z. 
118