11Evaluating and Measuring Success
 

Evaluation	is	a	systematic	way	to	improve	and	account	for	public	health	activities.	 

Partnership	evaluation	is	most	effective	when	it	is	an	ongoing	process	that	is	well	 

integrated	into	all	aspects	of	the	partnership’s	work.	Involving	partners	in	all	aspects	of	 

the	evaluation	process	will	help	you	to 

•	 Ensure	that	the	main	partnership	 

issues	of	each	stakeholder	group	are	 

understood	and	addressed	in	your	 

evaluations,	which	will	increase	 

acceptance	of	evaluation	findings. 

•	 Conduct	culturally	competent	 

evaluations	that	provide	accurate	and	 

complete	data	and	feedback.	 

•	 Ask	the	tough	questions.	(Is	your	 

chosen	approach	the	best	way	to	 

Getting Started with Evaluation 

Program evaluation is an essential public 

health practice; however, it can seem 

overwhelming. To make it easier to integrate 

evaluation into your partnership’s activities, 

start by reviewing the CDC’s evaluation model, 

“Framework for program evaluation in public 

health” (1999). 

The model is summarized in this chapter; for 

details on all steps of the model and how to use 

them, visit the CDC’s evaluation website, www. 

cdc.gov/eval/index.htm. 

accomplish	your	goals?	Do	you	need	to	make	major	changes?) 

•	 Evaluate	where	your	partnership	stands	on	the	nine	practices	associated	with	highperformance	partnerships	(see	Chapter	3:	What Successful Health-Related Community 

Partnerships Have in Common). 

•	 Make	major	changes	or	minor	modifications	that	will	significantly	improve	your	 

partnership’s	outcomes. 

•	 Anticipate,	monitor,	and	improve	TB	partners’	perceptions	of	the	partnership	and	its	 

work,	which	will	substantially	increase	their	commitment	over	time. 

•	 Verify	the	partnership’s	accomplishments	and	use	this	information	to	increase	 

financial	and	political	support	for	effective	TB	prevention	and	control	efforts. 

Good	partnership	evaluation	systems	have	both	formal	and	informal	components.	 

Calling	a	partner	who	has	missed	a	few	meetings	to	get	his	or	her	perspective	on	the	 

partnership	and	its	work	is	an	informal,	yet	important	way	to	conduct	a	mini-evaluation.	 

Formal	evaluation	procedures	provide	a	more	complete	view	of	the	partnership	and	its	 

effectiveness.	 

The CDC’s Recommended Evaluation Framework 

The	Centers	for	Disease	Control	and	Prevention	(CDC)	has	created	a	practical	 

evaluation	model,	Framework for Program Evaluation in Public Health (1999),	which	 

Forging Partnerships to Eliminate Tuberculosis: A Guide and Toolkit 8 

Chapter : Evaluating and Measuring Success 
can	be	adapted	for	effective	use	with	all	public	health	programs,	including	partnerships.	 

When	adapted	to	your	partnership’s	individual	circumstances,	the	framework	will	help	 

you	to	answer	the	following	essential	evaluation	questions:	 

•	 What	will	be	evaluated?	(What	is	the	partnership?	What	is	it	trying	to	accomplish?	In	 

what	context	does	it	exist?)	 

•	 What	aspects	of	the	partnership	will	be	considered	when	judging	its	performance?	 

(Levels	of	trust	and	collaboration	among	diverse	partners?	Partners’	satisfaction	with	 

partnership	processes	and	outcomes?	Amount	of	material	resources	accessed	through	 

the	partnership?	Levels	of	awareness	of	 Program evaluation is necessary to TB	symptoms	among	specific	segments	of	 fulfill CDC's operating principles for high-risk	populations?)	 public health, which include: 

•	 What	standards	(e.g.,	type	or	level	of	 • Using science as a basis for 

performance)	must	be	reached	for	the	 decisionmaking and action

• Expanding the quest for social equity partnership	to	be	considered	successful?	 

• Performing effectively as a service agency (What	amount	of	progress	constitutes	 

• Making efforts outcome-oriented success?)	 

• Being accountable 

•	 What	evidence	will	be	gathered	and	used	 

to	indicate	how	the	partnership	has	performed?	(Knowledge,	attitudes,	or	behaviors?)	 

•	 What	conclusions	regarding	partnership	performance	are	justified	by	comparing	the	 

available	evidence	to	the	selected	standards?	 

What is the CDC Framework for Program Evaluation? 

The	CDC	framework	for	program	evaluation	is	a	guide	that	public	health	 

professionals	and	their	partners	can	use	to	develop	and	conduct	useful,	feasible,	 

ethical,	and	accurate	evaluations	of	their	work.	The	framework: 

•	 Summarizes	the	essential	elements	of	program	evaluation	 

•	 Provides	a	common	frame	of	reference	for	conducting	evaluations	 

•	 Clarifies	the	steps	in	program	evaluation	 

•	 Reviews	standards	for	effective	and	practical	program	evaluation 

•	 Addresses	misconceptions	about	the	purposes	and	methods	of	program	evaluation	 

The	framework	is	composed	of	six	steps	(see	diagram	on	the	next	page)	that	must	 

be	taken	in	any	evaluation.	These	steps	are	starting	points	for	tailoring	an	evaluation	 

to	a	particular	public	health	effort	at	a	particular	time.	Because	the	steps	are	all	 

interdependent,	they	might	be	encountered	in	a	nonlinear	sequence;	however,	an	 

order	exists	for	fulfilling	each—earlier	steps	provide	the	foundation	for	subsequent	 

progress.	Thus,	decisions	regarding	how	to	execute	a	step	are	iterative	and	should	not	 

be	finalized	until	previous	steps	have	been	thoroughly	addressed. 

8 Forging Partnerships to Eliminate Tuberculosis: A Guide and Toolkit 

Chapter : Evaluating and Measuring Success 
CDC Framework for Program Evaluation
 

Clarify and agree on 

the purpose of the 

evaluation, questions 

the evaluation will be 

attempting to answer, 

and what methods will 

be used to conduct the 

evaluation. 

Although these steps do not need to be 

performed linearly, any evaluation must 

begin with engaging stakeholders, 

since they will help execute the 

remaining steps. Stakeholders are 

those involved or affected; they are 

the primary intended users of the 

evaluation findings. 

Describe the need that the 

partnership is addressing; 

the expected impact, 

activities, and resources 

of the partnership; the 

stage of the partnership’s 

development; the context 

within which the partnership 

is conducting its activities; 

and the model or approach 

that the partnership uses to 

guide its efforts. 

Deliberate effort is 

needed to ensure that 

the evaluation findings 

translate into informed 

decisionmaking and 

appropriate action. 

Elements of credible evidence include 

specific indicators, relevant sources, 

and valid, significant data gathered in a 

way that is culturally appropriate to the 

source. 

Justifying conclusions 

on the basis of evidence 

includes the following 

five elements: Standards, 

analysis/synthesis, 

interpretation, judgment, 

and recommendations. 

The	second	element	of	the	CDC	framework	for	program	evaluation	is	a	set	of	30	 

standards	for	assessing	the	quality	of	your	partnership’s	evaluation	practices.	The	 

standards	are	organized	into	these	four	categories: 

Forging Partnerships to Eliminate Tuberculosis: A Guide and Toolkit 8 

Chapter : Evaluating and Measuring Success 
Utility—To	ensure that	evaluation	results	serve	the	information	needs	of	intended	 

users	 

Feasibility—To	ensure	that	evaluation	procedures	are	realistic,	prudent,	diplomatic,	 

and	economical	 

Propriety—To	ensure that	the	evaluation	is	conducted	legally,	ethically,	and	with	 

regard	for	the	welfare	of	those	involved	and	those	affected 

Accuracy—To	ensure	that	the	evaluation	reveals	and	conveys	technically	accurate	 

information 

The	CDC	framework	for	program	evaluation	and	other	evaluation	resources	are	 

available	at	www.cdc.gov/eval/index.htm. 

A Collaborative Approach to Evaluation 

A	diverse	team	of	engaged	stakeholders	has	a	greater	probability	of	conducting	a	 

culturally	competent	evaluation	(i.e.,	one	that	understands	and	is	sensitive	to	the	 

persons,	conditions,	and	contexts	associated	with	the	program).	Such	a	collaborative	 

approach	helps	to	reduce	suspicion	and	fear,	increases	awareness	and	commitment,	 

increases	the	possibility	of	achieving	objectives,	broadens	knowledge	base,	teaches	 

evaluation	skills,	increases	the	possibility	that	findings	will	be	used,	and	allows	 

for	differing	perspectives.	All	of	these	will	serve	to	strengthen	your	partnership.	A	 

collaborative	approach	also	provides	you	with	access	to	community	resources	when	 

planning	and	conducting	the	evaluation.	 

Partners	may	have	varying	levels	of	involvement	in	developing	and	implementing	formal	 

evaluations.	To	ensure	that	the	process	runs	smoothly,	it	is	important	to	designate	an	 

overall	evaluation	coordinator.	Here	are	some	examples	of	evaluation	roles	partners	can	 

undertake:	 

•	 Leaders	of	high-risk	populations	can	engage	members	of	their	communities	in	the	 

evaluation	process.	 

•	 When	describing	the	partnership,	those	who	understand	the	partnership’s	history,	 

purpose,	operating	practices,	and	projects	are	needed.	 

•	 Those	with	good	group	facilitation	skills	might	be	asked	to	help	elicit	unspoken	 

expectations	and	concerns	that	partners	may	have	about	evaluations.	 

•	 Partnership	core	leadership	can	help	focus	evaluation	design	on	questions	that	 

address	specific	users	and	uses.	They	can	also	help	to	set	logistic	parameters	for	the	 

evaluation's	scope,	time	line,	and	deliverables.	 

Forging Partnerships to Eliminate Tuberculosis: A Guide and Toolkit 

Chapter : Evaluating and Measuring Success 

88 
•	 Scientists,	particularly	social	and	behavioral	scientists,	can	bring	expertise	to	the	 

development	of	evaluation	questions,	methods,	and	evidence-gathering	strategies.	 

They	can	also	help	evaluate	the	partnership	within	its	organizational	and	community	 

context.	 

•	 Trusted	persons	who	have	no	particular	stake	in	evaluation	results	can	ensure	that	 

participants'	values	are	treated	fairly	when	applying	standards,	interpreting	facts,	and	 

reaching	justified	conclusions.	 

•	 Advocates,	clear	communicators,	creative	thinkers,	and	members	of	the	power	 

structure	can	help	ensure	that	lessons	learned	from	the	evaluation	influence	future	 

decisionmaking	regarding	program	strategy. 

Things to Keep in Mind 

•	 Be	sure	that	your	evaluation	monitors	how	well	your	partnership	conducts	highperformance	partnership	practices	(see	Chapter	3:	What Successful Health-Related 

Community Partnerships Have in Common). 

•	 It	is	important	for	your	partnership’s	core	leadership	to	consistently	support	ongoing	 

effective	evaluation	systems. 

•	 When	seeking	foundation	funds	for	a	partnership	project,	be	sure	that	appropriate	 

evaluation	procedures	and	their	costs	are	included	in	your	proposal. 

•	 Deliberate	effort	is	needed	to	ensure	that	lessons	learned	in	your	evaluation	are	used	 

to	make	lasting	improvements	in	your	partnership	and	your	overall	TB	prevention	 

and	control	efforts. 

Related Resources 

•	 Community Toolbox 

The	University	of	Kansas	has	created	a	Community	Toolbox	with	a	variety	of	 

evaluation	worksheets	and	checklists	that	can	be	used	with	the	CDC	evaluation	 

framework.	The	toolbox	also	includes	overheads	summarizing	important	evaluation	 

concepts.	The	Community	Toolbox	can	be	accessed	at	http://ctb.ku.edu/index.jsp. 

•	 Making Health Communication Programs Work 

This	250-page	publication	contains	a	chapter	entitled	Assessing effectiveness and 

making refinements,	which	addresses	designing	and	using	effective	outcome	 

evaluations.	This	publication	is	available	through	the	National	Cancer	Institute’s	 

Website	at	www.cancer.gov/pinkbook	or	by	calling	1-800-422-6237. 

•	 Practical Evaluation of Public Health Programs (Course # VC00) 

Forging Partnerships to Eliminate Tuberculosis: A Guide and Toolkit 8 

Chapter : Evaluating and Measuring Success 
 

This	is	a	5-hour	distance-learning	course	that	uses	the	CDC	framework.	Developed	 

through	the	CDC’s	Public	Health	Training	Network	(PHTN),	the	course	consists	 

of	two	videotapes	and	a	workbook	that	can	be	used	by	individuals	for	self-study	or	 

by	partnerships.	Continuing	education	credit	is	available	for	this	course.	Additional	 

information	about	the	video	is	available	at	www.phppo.cdc.gov/phtn/default.asp.	 

Course	materials	can	be	purchased	from	the	Public	Health	Foundation	by	calling,	 

toll-free,	877-252-1200,	or	using	the	online	order	form	at		 

http://bookstore.phf.org/prod41.htm.	For	informational	purposes,	the	workbook	can	 

be	viewed	over	the	Internet	at	www.cdc.gov/eval/workbook.pdf. 

•	 Partnership Assessment Tool .0 

This	easy-to-use,	web-based	Tool	gives	a	partnership	a	way	to	assess	how	well	its	 

collaborative	process	is	working	and	to	identify	specific	areas	of	focus	in	order	to	 

make	the	partnership	process	work	better.	The	Tool	is	being	provided	at no	charge 

by	the	Center	for	the	Advancement	of	Collaborative	Strategies	in	Health	at	The	New	 

York	Academy	of	Medicine	with	funding	from	the	W.	K.	Kellogg	Foundation.		 

www.cacsh.org/psat.html. 

0 Forging Partnerships to Eliminate Tuberculosis: A Guide and Toolkit 

Chapter : Evaluating and Measuring Success