Budgeting for Evaluation

2014 AmeriCorps

State and National Symposium
2 

Learning objectives

By the end of this presentation, you will be able to:

• Understand why budgeting for evaluation is a strategic 

investment 

• Recognize how evaluation costs vary by type of 

evaluation

• Explain the key components of an evaluation budget

• Identify approaches for creating an evaluation budget
Emerging trends

• Rule of thumb (5%, 10%) of overall budget for evaluation is 

proving too simplistic 

• Administrative data is emerging as a viable source for lowcost RCTs (and some QEDs) 

• Tiered evidence initiatives (SIF, i3, WIF, etc.) are expanding 

the number of high-quality evaluations, leading to more 

available budget data

• Continuum from performance measurement to evaluation; 

links and overlapping data collection

3 
Challenges and payoff

• Paying for evaluation can be a challenge

– Programs can’t afford it

– Don’t know how to find low-cost evaluation partners

– Funders won’t always pay for it

• CNCS acknowledges these challenges 

– But we believe that evaluation should be viewed as an 

investment and not simply a compliance requirement.

• Consider evaluation to be a strategic investment

– If you can demonstrate that your program works, you will 

be able to attract more funders, grow, and become more 

sustainable. 

4 
Investing in evaluation does pay dividends 

• Spending money on evaluation is not a tradeoff

• Learning for program improvement, so you can: 

– Serve more people better

– Serve more people better, at lower cost

• Increasing your evidence base attracts funders...

...which allows you to serve more people 

• Demonstrating impact is a differentiator in an 

environment of scare resources

– Policymakers and funders are reluctant to spend scarce 

resources on programs without evidence 

5 
Build a long-term research agenda

• What does a long-term research agenda look 

like? 

– What do we want to have learned 5 years from now? 

10 years from now?

– Work backwards: define your destination, then name 

the supporting steps that will get you there

– Each evaluation should build on what you learned 

previously 

– If you invest evaluation money strategically, scarce 

resources can have a big impact

6 
Example of a long-term research agenda

AmeriCorps program provides housing assistance for lowincome families. 

Goal: Demonstrate that the program has a positive impact on 

beneficiaries via a randomized control trial (RCT).

– Step 1: Collect program data, routinely, on family background 

characteristics and number of families served.

– Step 2: Collect pre/post outcome data each year via annual survey.

– Step 3: Implementation study: Is the program being implemented 

with fidelity to the model? 

– Step 4: In addition to data collected from Steps 1&2, collect longterm outcomes data via follow-up survey (1 year post- program).

– Step 5: Demand for the program exceeds supply, so implement 

RCT by randomly assigning families to receive housing assistance. 

Collect background data and survey data from all eligible families.

7 
Guidelines for budgeting

• In general, evaluation budgets should be: 

– Commensurate with stakeholder expectations and 

involvement 

– Appropriate for the research design used and key 

questions to be answered 

– Adequate for ensuring quality and rigor 

– In line with the level of program and organizational 

resources available 

8 
Factors that influence budget estimates

• Program Factors 

− Geography, number of sites, and characteristics of 

target population

− Availability of resources

• Evaluation Design

− RCT vs. QED vs. Non-Experimental 

− Level of evidence 

− Need for evaluation capacity-building 

• Reporting, Dissemination and Use

− Using lessons learned for internal decision making

− External dissemination and communication

9 
Example evaluation costs by design type

01
Evaluation costs by level of evidence

11
Key themes in evaluation budgeting 

• Rule of thumb ratios lead to under-budgeting

 

• Amount budgeted should be considered in its 

own right

• Costs increase with level of evidence 

• Costs vary based on the study design

12
Key themes (continued)

• All design types can potentially be expensive 

− Multisite implementation studies 

− Feasibility studies for generating stronger levels of 

evidence 

• Most evaluation types can be done inexpensively

– Including some randomized control trials (RCTs)

• Determine what you want to learn from your 

evaluation and build a budget that can answer those 

research questions

13
So where can I get the money?

• Tap into local resources

– Local partners

– Universities, community colleges, Cooperative Extension, 

consultants 

• When feasible, consider partnering with similar 

programs for joint evaluation activities 

• Some foundations will fund evaluation

– Brady Education Foundation; Annie E. Casey Foundation; 

WT Grant Foundation

• Evaluation IS an allowable cost at CNCS!

14
15

• Evaluation is one piece of your overall budget

– What do you need to spend money on?

– Do you have a diverse, sustainable fundraising strategy?

Big picture budget discussion
Paying for evaluation

• Use of funds can be restricted or unrestricted, 

regardless of source

– Depends on funder stipulations 

• You may find that you need mostly unrestricted 

money for evaluation

– Examine the universe of possible ways to raise money for 

your program 

– From which sources can you most likely get unrestricted 

money? 

– Which other sources will provide restricted money for other 

portions of your budget? 

 

16
Paying for evaluation across years

• Evaluation activities often cost different amounts 

throughout 3-year grant cycle

– Ex: Yr 1 = Planning; Yr 2 = Data Collection; Yr 3 = 

Reporting and Analysis

– Yr 1 → Smaller evaluation budget; Yr 2 & 3 → Larger 

evaluation budget

17

Year Member Training Evaluation Total CNCS

CNCS Match CNCS Match

1 $25,000 $0 $10,000 $0 $35,000

2 $25,000 $0 $10,000 $25,000 $35,000

3 $0 $20,000 $35,000 $0 $35,000
Overview of evaluation budget planning

Consider questions of:

• Who will conduct the evaluation?

– If external evaluator, consider what services are and are not 

included in their cost

– If own staff, consider cost of time spent on evaluation relative to 

programmatic tasks

• What will it include and how will it be conducted? 

• Will it involve new data collection? 

– If so, at what time points and where?

• Who will manage it? 

• How will the results be communicated? 

18
Components of an evaluation budget

1. Evaluation staff and subcontractor 

salary/benefits, consultant time

2. Travel (airfare, ground transportation, lodging, 

incidental travel costs)

3. Other Direct Costs (e.g. communications 

printing and postage, supplies/equipment) 

4. Overhead Costs and Fees

5. Program Costs to Support Evaluation

19
External evaluator time

• Evaluation staff salary/benefits and consultant time 

– Evaluation planning

– Instrument selection, development, validation

– IRB approval

– Data collection, processing, analysis, reporting

– Project administration

• Common cost drivers are data collection and analysis

– Fielding surveys and conducting interviews

– Analyzing large datasets and/or transcripts

• Don’t underestimate time for planning and reporting

20
Calculating labor costs

Task by time by staff member

21

Team Leader Subject Expert Analyst Assistant

Total Days 

Per Task

Cost Per 

Task ($)

Daily Rate ($)

Tasks Days Cost Days Cost Days Cost Days Cost

Planning 15 10 20 20 65

Preparation 15 10 20 30 75

Data Collection 10 20 50 50 130

Analysis 10 25 20 30 85

Reporting 10 15 20 10 55

Total Days Per Person 60 80 130 140 410
Calculating labor costs (continued)

Staff member by time by task

22

Data Total Days Cost Per 

Planning Preparation Collection Analysis Reporting Per Person Person ($)

People Daily Rate ($) Days Cost Days Cost Days Cost Days Cost Days Cost

Team Leader 15 15 10 10 10 60

Subject Expert 10 10 20 25 15 80

Analyst 20 20 50 20 20 130

Assistant 20 30 50 30 10 140

Total Days Per Task 65 75 130 85 55 410
How to estimate labor rates

• An example of labor rates for private firms 

(loaded govt. rate) and university consulting

23

Labor Rate per Hr.

Labor Rate:

Consulting Firm

Labor Rate: 

University-based

Consulting

Project Manager $204.14 $111.52

Executive/Key Principal $240.95 $162.18

Senior Research/Principal or 

Technical Staff $187.47 $99.50

Mid Level 

Research/Analyst/Technical $102.57 $100.56

Mid-level Program Management $125.73 N/A

Subject Matter Experts $209.43 $167.25-$243.48

Learning consultant $118.98 $76.19

Management Consultant $156.02 $156.02

Support Staff $64.41 $40.23
Travel

• Travel to and from:

– Client site

– Program site for stakeholder meetings and planning

– Program site(s) for site visits, data collection, etc. 

• Travel line item should include:

– Airfare or train tickets, car mileage

– Ground transportation

– Lodging and meals

– Incidental travel costs 

24
Other direct costs

• Communications 

– Conference call lines, WebEx, etc.

• Printing and postage

– Mailing surveys

– Printing consent forms

• Supplies and equipment

– Online survey platform; survey incentives

– Voice recorders and transcription services 

– Renting meeting spaces 

– Purchasing datasets, databases, survey tools, etc.

25
Overhead costs and fees

• Overhead costs for large evaluation firms are 

often built into “loaded” labor rates

• University staff may account for overhead 

differently

• Fees will depend on contract type

– Cost plus fixed fee 

– Firm fixed price 

– Time and materials

26
Program costs to support evaluation

• Staff time to meet regularly with evaluator; 

quality control and monitoring

• Staff time for facilitating connections between 

evaluator and program/site staff

• Resources for additional policy briefs, website 

summaries, travel to conferences, etc.

• Data systems

27
Example evaluation budget

28

Year 1: Planning Year 2: Data Collection Year 3: Analysis and reporting

Planning and 

Evaluation Staff Time (Labor) preparation Data Collection Analysis Reporting Totals

Hours Cost Hours Cost Hours Cost Hours Cost Hours Cost

Project Leader ($100/hr) 25 $2,500 75 $7,500 15 $1,500 10 $1,000 125 $12,500

Senior level analyst ($80/hr) 300 $24,000 500 $40,000 60 $4,800 50 $4,000 910 $72,800

Data analyst ($65/hr) 50 $3,250 100 $6,500 80 $5,200 30 $1,950 260 $16,900

Junior level assistant ($50/hr) 250 $12,500 500 $25,000 50 $2,500 60 $3,000 860 $43,000

Admin support ($35/hr) 75 $2,625 200 $7,000 20 $700 30 $1,050 325 $11,375

700 $44,875 1375 $86,000 225 $14,700 180 $11,000 2480 $156,575

Travel Miles

Auto mileage ($.30/mile) $0 200 $60 $0 0 $0 $60

Other Direct Costs (ODCs)

Printing and copying $0 $500 $0 $1,000 $1,500

Postage $0 $250 $0 $0 $250

Conference lines $100 $200 $25 $50 $375

Transcription service $0 $0 $1,000 $0 $1,000

Incentives $7,500 $7,500

Survey platform $1,000 $5,000 $0 $0 $6,000

$1,100 $13,450 $1,025 $1,050 $16,625

G&A-10% $1,663

$173,260 Sub-total

Fees- 10% $17,326

$190,586 Total

Program Support Costs

Executive Director- $35/hr 60 $2,100 40 $1,400 5 $175 15 $525 $4,200

AmeriCorps Program Manager- $25/hr 60 $1,500 80 $2,000 10 $250 25 $625 $4,375

Supervisors (1 per site, 10 total)- $25/hr 60 $1,500 120 $3,000 10 $250 25 $625 $5,375

$13,950 Sub-total

$204,536 Grand Total
Creating an internal evaluation budget

• Same general cost categories

• Consider additional staff time/resources for

– New or augmented data collection

– Travel outside of normal program operations

– Additional analysis

– Reporting and communication

– Consultants

29
Other budgeting tips

• Create a high and a low estimate

• Refer to evaluations of similar size and scope

• Consult with procurement or budgeting experts

• Talk to staff working in the locations where data 

collection will occur

• Assess the budget with the stakeholders who have been 

engaged in evaluation planning

• Plan for contingencies and adjust as needed

30
Challenges with evaluation “on the cheap”

• Lack of continuity

– Ex: Student groups

• Lack of appropriate expertise

– Internal staff who do not have evaluation training

• Under-powered study

– Sample is so small that you will never detect any 

difference in outcomes due to your program

• Poor communication

– Technical jargon that you can’t interpret or use

• Too many unanswered questions

– Poor design did not shed light on research questions

31
Tips for cutting costs and retaining quality

• Utilize existing program data and administrative data

• Build data collection into routine program operations

• Develop internal staff capacity for evaluation work

• Engage pro bono experts

• Build a long-term research agenda so that each 

evaluation builds upon previous work

• Consider replicating an evidence-based program

32
Using existing data

• Utilize program data

– Data you already collect for performance measures 

– Data you already collect for other funders 

• Utilize administrative data

– Student test scores, attendance records 

– Census data, unemployment insurance claims 

• Build data collection into routine program operations

– Collect data from beneficiaries, AmeriCorps members, and 

staff on a regular schedule 

– Use that data to inform decision-making and for 

continuous improvement 

33
Example: Allocating funds for evaluation

34

• Program has raised funds from CNCS ($500K), state 

grant ($100K, up to 5% for evaluation), unrestricted 

foundation grant ($150K), restricted corporate giving 

($100K), and organizational match ($150K, currently used 

for personnel salary).

• Evaluation will cost $200,000 – where should the money 

come from?

• $5K can come from the state grant

• $150K can come from the foundation grant

• That leaves $45K to come from CNCS

• Where could you shift other costs out of the CNCS share?

• Could you shift salary costs? 
Resources on budgeting for evaluation

• Social Innovation Fund Budgeting Guide: 

http://www.nationalservice.gov/build-yourcapacity/grants/funding-opportunities/2014/socialinnovation-fund-grants-fy-2014/financial-managementsystem-requirements

• U Michigan Evaluation Budgeting Checklist: 

http://www.wmich.edu/evalctr/archive_checklists/evaluati

onbudgets.pdf

• Pell Institute Evaluation Toolkit: 

http://toolkit.pellinstitute.org/evaluation-guide/planbudget/develop-a-budget/

• BetterEvaluation: http://betterevaluation.org/evaluationoptions/calculate_evaluation_costs

35
AmeriCorps Evaluation Resources

https://www.nationalserviceresources.gov/evaluationamericorps

36
Questions?