Report for Minnesota Reading Corps Pre-K Program Cost Analysis

Center for Benefit-Cost Studies of Education 

Teachers College, Columbia University

Brooks Bowden, Maya Escueta, Atsuko Muroga, Viviana Rodriguez, and Henry M. Levin1

Last Updated: January 14, 2018 

1 Our sincere gratitude goes to Dayane Rocha de Pauli, a visiting student at Center for Benefit-Cost Studies of 

Education and a doctoral student and professor at Universidade Federal do Paraná, for her excellent support in 

delivering this work.

1
 

 

 

 

 

 

 

 

 

Table of Contents 

1. INTRODUCTION .....................................................................................................................3

2. BACKGROUND AND CONTEXT ................................................................................................4

The Challenge of Early Literacy ............................................................................................................ 4

Previous Evidence on Early Literacy Interventions..............................................................................5

3. MINNESOTA READING CORPS PRE-K PROGRAM......................................................................6

Minnesota Reading Corps: Theory of Change......................................................................................6

Description of the Program.................................................................................................................. 7

Target Population and Selection Criteria ............................................................................................. 8

Personnel Structure.............................................................................................................................. 8

Key Components of the Program......................................................................................................... 9

4. COST ANALYSIS IN A BROADER EVALUATION FRAMEWORK ..................................................10

Previous Evaluations of the Minnesota Reading Corps Pre-K Program............................................. 10

The Need for Cost Analysis................................................................................................................. 11

Research Questions............................................................................................................................ 12

5. DATA AND METHODS........................................................................................................... 13

Measuring Costs in Education............................................................................................................ 13

The Ingredients Method..................................................................................................................... 14

Data Sources and Data Collection Methods....................................................................................... 15

Missing Data Strategies...................................................................................................................... 16

Prices..................................................................................................................................................19

Estimation of Average Cost Per Site and Cost Per Student................................................................ 20

Distribution of Costs...........................................................................................................................20

6. RESULTS .............................................................................................................................. 21

Main Results ....................................................................................................................................... 21

Personnel Structures .......................................................................................................................... 24

Uncertainty and Sensitivity Analyses.................................................................................................25

Limitations..........................................................................................................................................28

7. CONCLUSION .......................................................................................................................29

REFERENCES ................................................................................................................................ 32

2
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1. INTRODUCTION

Establishing literacy proficiency early in life is fundamental to ensuring academic success, not

only in reading, but across all subjects studied in school. Pre-K programs are a promising avenue

for developing literacy skills early in life. However, not all young students have access to a Pre-K

education, and when they do, not all programs have the resources required to provide the

individualized support that is tailored to struggling readers’ needs.

One promising approach for helping struggling readers is to hire certified teachers or specialists 

as tutors for every classroom (e.g. May et al., 2013; Slavin, Lake, Davis & Madden, 2011). 

Nevertheless, this can be costly and impractical for under-resourced schools. Alternatively, using

volunteers or paraprofessionals as additional support may be a less costly solution; however,

uncertified volunteers may yield lower quality instruction. Additionally, volunteer-based programs

may require intense supervision and support to be effective and can be challenging to bring to scale

(Grossman & Furano, 1999). 

The Minnesota Reading Corps (MRC) program is a statewide AmeriCorps early literacy

initiative that aims to foster emergent literacy skills of children to ensure reading proficiency by

the end of grade 3. MRC and its host organization, Reading & Math, Inc. (RMI), aim to address

the resource gaps within under-resourced schools by bringing AmeriCorps members into preschool

(Pre-K) classrooms to provide literacy enrichment for the whole class and tutoring services for

specific at-risk students.

An impact evaluation of the MRC Pre-K program in the 2013-2014 school year found that the

program had significant positive effects on outcome measures of emergent literacy (e.g. 

phonological awareness and vocabulary) for 3-, 4-, and 5-year-olds (Markovitz, Hernandez, 

Hedberg & Silberglitt, 2015). While evidence of program impact is necessary for understanding 

whether a program is achieving desired outcomes, it is not sufficient for policymakers to make

informed decisions about resource allocation or replication of the program without information on

costs. A cost analysis of the program is required to understand whether the program is worth the

cost, and subsequently, the magnitude of the benefits that are generated as a result of the required 

investments. 

The purpose of this study is to estimate the economic costs of providing the MRC Pre-K

program that are associated with the impacts (i.e., increased reading skills) measured by the 2013-

3
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2014 evaluation. Thus, the descriptions and analysesincluded in this report focus on the Minnesota

Reading Corps Pre-K model as it was implemented in the 2013-14 school year. In addition to

estimating the average costs per site and per student, this cost study will also assess variation in

cost among sites or among different bundles of resources with a specific focus on the portion of

costs borne by schools.

Though previous literature has investigated the cost-effectiveness of early literacy

programs, evidence suggests high variability in costs depending on program design (Hollands et

al., 2013; Simon, 2011). While programs that focus on improving classroom instruction coupled

with providing individualized instruction focused on phonics (Slavin et al., 2011) are promising in

terms of impact, no cost analysis has been conducted on such a program. A careful cost analysis

of the MRC program, which retains many of these promising program components, can help us

better understand the resources required to replicate the program impact, how much of each

resource is necessary, and how cost-effective the program is relative to alternatives. 

2. BACKGROUND AND CONTEXT

The Challenge of Early Literacy

Reading competency is foundational for children’s learning and academic success in all

subjects. Children struggling with reading may experience difficulties in other subjects (e.g., Chall, 

Jacobs & Baldwin, 1990). If not addressed, early challenges in literacy may continue during school

years (e.g., Baydar, Brooks-Gunn, & Furstenberg, 1993; Felton, 1998) and into adulthood (Bruck, 

1998). A critical task assigned to K-12 schools in the United States today is to ensure that every

student becomes a proficient reader. 

Nevertheless, the 2015 National Assessment of Educational Progress (NAEP) reading 

assessment results show that 64 percent of fourth graders are below “Proficient” level, and that 21 

percent are below “Basic” level of reading proficiency (NAEP, 2015a). This challenge is 

exacerbated by evidence of achievement gaps among disadvantaged students. Assessment results

among disadvantaged students indicate that 78 percent of fourth grade students eligible for the

National School Lunch Program are below “Proficient,” and 44 percent are below the “Basic”

level (NAEP, 2015a).

4
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Pre-K programs can be an effective vehicle to reduce these gaps because building literacy

skills early in life can prevent later difficulties in reading. Nevertheless, providing enough support

at the Pre-K level can be challenging because schools with struggling readers are more likely to

lack sufficient resources. Moreover, at the Pre-K level, no “one-size-fit-all” policy works due to

the complexity of how Pre-K programs are organized. In the absence of one overarching entity

that governs “Pre-K education,” public preschools, Head Start programs, and community child 

care centers are all regulated and funded differently. Several different models of reading programs 

have been examined and evaluated at the Pre-K level. Understanding costs of these approaches

together with their effectiveness can contribute to the knowledge base that policymakers can use

for effective decision making.

These challenges are particularly salient in Minnesota, where the state’s reading 

achievement gaps remain some of the highest in the nation. In 2015, only 20 percent of low income

(as defined by eligible for free or reduced lunch) fourth graders and 16 percent of AfricanAmerican fourth graders were proficient in reading as measured by NAEP (2015b). While

addressing these literacy gaps early in life is an important policy objective, access to Pre-K 

programs in Minnesota still remains among the lowest in the nation. In 2015, only 56 percent of

Minnesota’s 3- and 4-year-olds were enrolled in preschool (Bridges, 2015). Hence, understanding

how to address these resource constraints in the Minnesota context, and to do so at a reasonable

cost, is now more important than ever. 

Previous Evidence on Early Literacy Interventions

Rigorous evaluations of literacy interventions show that one-on-one tutoring delivered by

certified teachers is effective in improving reading performance (e.g., May et al., 2013; Slavin, et

al., 2011), even relative to alternative interventions such as small group tutorials, classroom

instructional approaches and computer assisted learning (Slavin et al., 2011). A review of 96

evaluations of alternative literacy interventions for struggling readers in grades K-5 concluded that

a promising approach to addressing gaps in literacy for students lagging behind is combining

improved classroom instruction with individualization focused on phonics (Slavin et al., 2011). 

Nevertheless, providing one-on-one tutoring by trained teachers is both time and resource

intensive. Hiring additional specialists or certified teachers as tutors can be costly and may not be

practical for already resource-constrained schools. An alternative approach to certified teachers is

5
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

that of enlisting volunteers or paraprofessionals, which are less costly, but may yield lower quality

instruction. There is recent literature documenting the effectiveness of placing paraprofessionals 

or volunteer tutors in schools to improve the reading proficiency of students lagging behind (e.g.,

Jacob, Armstrong & Willard, 2015). However, the literature on the effectiveness of this approach

remains thin, and still somewhat mixed. For instance, Slavin et al. (2011) reviews three evaluations

of such interventions—Experience Corps (Morrow-Howell, Jonsen-Reid, McCrary, Lee, &

Spitznagel, 2009), Book Buddies (Meier & Invernizzi, 2001) and HOST (Ramey, 1991)—however,

the findings were inconsistent. Additionally, evidence suggests that bringing to scale interventions

that rely on volunteers may be challenging (Grossman & Furano, 1999; Hager & Brudney, 2004). 

In particular, the success of volunteers can depend heavily on the support that they receive

(Grossman & Furano, 1999). 

Cost analyses of effective early literacy interventions for students in grades K-3 find 

significant variation in the costs required to implement these various interventions (Hollands et

al., 2013; Simon, 2011). Cost variation is found to depend critically on program design and

implementation. An evaluation of a literacy intervention using community volunteers to provide

one-one-one instruction found such interventions can be a low-cost option for schools when the

majority of the costs are in-kind contributions (Jacob, Armstrong, Bowden, & Pan, 2016).

The challenge of addressing early literacy is three-fold: How can early literacy programs

ensure (1) quality instruction, (2) at a reasonable cost, and (3) using resources that can be

operationally viable at scale? To address these challenges in early literacy, the Minnesota Reading 

Corps and its host organization, Reading & Math, Inc., undertook an approach to bring

AmeriCorps members into Pre-K classrooms to provide literacy enrichment for the whole class 

and tutoring services for at-risk students. 

3. MINNESOTA READING CORPS PRE-K PROGRAM

Minnesota Reading Corps: Theory of Change

The Minnesota Reading Corps (MRC) program is a statewide AmeriCorps early literacy 

initiative that aims to foster emergent reading skills of children to ensure reading proficiency by

the end of grade 3. The theory of change underlying the program is designed to specifically address 

three aspects of the early literacy challenge. First, there are resource insufficiencies in some

6
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

schools, by which schools alone are unable to provide students with the individualized support that

they need to become kindergarten ready. Second, even though additional support may be available

through paraprofessionals or volunteers, such resources may yield lower quality instruction than a

certified teacher. Third, even if paraprofessionals or volunteers are effective at a small scale,

bringing such programs to scale across the nation is a significant challenge due to the amount of

support required and uncertainty about the supply of volunteers. 

The MRC Pre-K program addresses these challenges through three key components of the

program: (1) additional classroom support through a full-time or part-time tutor (either a

Community Corps or an Educator Corps) that implements classroom activities and provides oneon-one and group tutoring sessions, (2) a dedicated training and a supervisory support structure

that enhances instructional skills of AmeriCorps members and supports them in their day-to-day

activities; and (3) an interactive and skills-focused literacy curriculum model, called “SEEDS of

Emergent Literacy,” which is based on current research in early childhood. This model includes 

classroom based strategies and daily targeted one-on-one and small group interventions. The

model also uses a Response-to-Intervention (RTI) framework that identifies at-risk students,

through defined benchmarks and regular assessments, who are targeted for individualized

intervention. 

These features of the program enable AmeriCorps members to provide tiered support that

targets students falling behind by assessing students’ learning levels and providing one-on-one or

small group literacy sessions to at-risk students throughout the school day. Through regular

assessment of the students, daily classroom activities for all students, and targeted interventions 

for students lagging behind, the MRC Pre-K program is designed to help students improve their

academic performance in literacy and achieve Minnesota state “Kindergarten Ready” targets 

before entering kindergarten. 

Description of the Program 

The core activities of MRC and its host organization, Reading & Math, Inc. (RMI), are to

recruit, place, train and monitor AmeriCorps members who are placed into Pre-K classrooms to 

implement research-based literacy interventions and tutoring services. The program includes both

an immersive “push-in” component, where AmeriCorps members provide literacy enrichment to 

the entire Pre-K classroom as well as individualized interventions to students struggling with

7
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

emergent literacy skills. Struggling students are identified using a Response-to-Intervention 

framework.

Target Population and Selection Criteria

The Minnesota Reading Corps Pre-K program provides literacy enrichment support to 3-,

4-, and 5-year-old students at already existing preschool settings across Minnesota, including Head 

Start sites, public schools, and community-based programs. Existing Pre-K programs throughout

the state can apply to be a part of the MRC program, and are selected and awarded AmeriCorps

members based on need and ability to supply the required support staff. In the 2013-14 school

year, the Minnesota Reading Corps Pre-K and K-3 programs placed more than 1,100 AmeriCorps

members in 712 elementary schools, Head Start centers, and pre-schools to serve over 30,000 

students across the state.

Personnel Structure

Classroom Personnel. The key personnel supplied by MRC are the AmeriCorps members 

who are placed in Pre-K classrooms to implement the literacy rich schedule and individualized 

interventions. There are two types of AmeriCorps members: Educator Corps and Community

Corps. Educator Corps are personnel currently employed at the site comprised of either a lead 

teacher or assistant teacher who are trained to incorporate Minnesota Reading Corps strategies into

their instruction alongside their regular teaching. Community Corps members are typically

recruited by the community and brought into Pre-K classrooms full-time or part-time to implement

MRC strategies. The lead teachers in classrooms where Community Corps members are placed 

also actively implement MRC activities. Since lead teachers host Community Corps members and

collaborate with them throughout the day, they enter into an agreement that stipulates the roles and

expectations over the course of the program. 

Supervisory Staff. Classroom personnel are supported by Internal Coaches and Master

Coaches, who play an important role by both ensuring the MRC activities are implemented with 

fidelity and by providing in-depth training and regular support to the tutors to optimize their

success with students. Internal Coaches are typically school employees who are trained to provide

on-site literacy support, coaching, and oversight to AmeriCorps members. Master Coaches are

typically literacy experts contracted by the program to support Internal Coaches with regular

8
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

feedback based on monthly observations. All personnel also attend regular training, including an 

annual three-day Summer Training Institute and additional professional development, that are

taught by Master Coaches, and, in some cases, Internal Coaches.

Key Components of the Program

The MRC Pre-K program is designed to address the challenges of early literacy through

three key components: (1) additional tutors in the classroom, (2) curriculum and literacy content

focused on the “Big Five” Essential Early Literacy Predictors, and (3) dedicated training and 

support to standardize the quality of instruction across Pre-K sites and classrooms. 

AmeriCorps Members. The first key aspect of the program embraces the AmeriCorps

members, either Community Corps or Educator Corps, who are put into Pre-K classrooms to

develop their students’ emergent literacy skills. AmeriCorps members complete student

assessments, implement the Literacy Rich Schedule and enact the SEEDS of emergent literacy

curriculum, and provide small group and one-one-one interventions to students falling behind. 

Literacy Curriculum. The second key component is the research based literacy curriculum

designed to address each of the following “Big Five” Essential Early Literacy Predictors (Lonigan 

& Shanahan, 2009): conversational skills, vocabulary and background knowledge, book and print

rules, phonological awareness (i.e., rhyming and alliteration), and alphabetic knowledge (i.e., letter

name recognition and letter sound correspondence). There are three main channels through which

the MRC Pre-K program is designed to develop these Big Five skills. 

First, instructional staff at participating sites together with AmeriCorps members provide

classroom- and evidence-based literacy enrichment strategies (called “Literacy Rich Schedules”) 

to create a stimulating literacy learning environment focused on the Big Five. Literacy Rich

Schedules include 12 distinct activities (i.e., Arrival, Sign-in, Meal Time, Large Group, Daily

Message, Repeated Read Aloud, Tier 1 Small Group, Journal, Choice Time/Active Learning, Tier 

2 or Tier 3 Small Group, Big 5 transitions, and Family) that engage children in daily routines,

conversations, and activities (i.e., games, songs, etc.) aimed at enhancing their Big Five early

literacy skills. 

Second, an instructional approach called “SEEDS of Emergent Literacy” (Horst & Passe,

2004) guides AmeriCorps members and teachers to interact with children in a way that promotes

children’s broader cognitive and social-emotional development. Specifically, high quality

9
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

interactions with children are expected in the areas of: sensitivity to each child’s needs,

encouragement though verbal and non-verbal communications, embedded “Big 5” literacy skills

in daily routines, development of skills through practical application, and support for the child’s

feeling of being respected and capable. Third, MRC offers add-on, tiered support that targets the

needs of at-risk and struggling students in reading.

These tiers of students are determined based on a Response to Intervention (RTI)

framework that matches student needs with an appropriate intensity of additional instruction.

Periodic assessment is an integral part of this framework, and AmeriCorps members use a

standardized, individually administered assessment tool called Individual Growth and

Development Indicators (IGDI) for assessing rhyming, picture naming, and alliteration on a

regular basis. For students identified as Tier 2 (at-risk) and Tier 3 (high-needs), individual or small

group literacy tutoring is offered by AmeriCorps members. These students receive 15 minutes of 

additional instruction each day.

Training and Support. The third key aspect of the program is the training and support

AmeriCorps members receive through interactions with supervisory staff (Internal Coaches and

Master Coaches) and the training received through both the annual Summer Institute training and

the on-going professional development sessions throughout the year. The Summer Institute

training is a three-day training session attended by all AmeriCorps members on an annual basis 

that provides expert training in the evidence-based literacy interventions used by the MRC

program. Members also receive additional professional development, called Pre-K Fundamentals, 

throughout the year through sessions on topics such as oral language, child confidentiality, 

repeated read aloud method, and social and emotional development, as well as additional trainings

on SEEDs. 

4. COST ANALYSIS IN A BROADER EVALUATION FRAMEWORK

Previous Evaluations of the Minnesota Reading Corps Pre-K Program

The impact of the MRC Pre-K program was evaluated in the 2013-2014 school year by the

University of Chicago-based research center, NORC (Markovitz et al., 2015). The evaluation 

studied fifty Pre-K sites in the Minnesota area: twenty-five sites that were receiving the MRC

program (the treatment condition) and twenty-five comparison sites that carried on with “business 

10
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

as usual” operations (the control condition), matched based on a number of educationally

important pre-treatment characteristics. The impact evaluation found that the MRC Pre-K program

had significant positive effects ranging from 0.4 to 0.72 standard deviations on five Individual

Growth and Development Indicators (IGDI) outcome measures of emergent literacy for 4- and 5-

year-olds (letter sound and letter name fluency, rhyming and alliteration fluency, and picture name

fluency), and significant and positive effects on two of the four IGDI measures developmentally

appropriate for 3-year-olds (letter name recognition and alliteration fluency). Effects were equally

distributed across student demographics and type of site, indicating the possibility of replicating

the program with similar impacts in multiple Pre-K settings.

The Need for Cost Analysis

The impact evaluation was one of several complementary studies of the Minnesota Reading 

Corps program conducted to understand both implementation details and program impacts

(Diaconis et al., 2015; Markovitz et al., 2015; Markovitz, Hernandez, Hedberg & Silberglitt, 2014;

Hafford et al., 2013). However, in order to fully explore the resources required to generate the

impacts measured and to better understand the magnitude of the benefits relative to the investments

made, an estimation of the costs of the program is required. This will give policymakers and 

decision makers information to understand how to improve the program for the future, and the

resources required to replicate the program in a new context.

This cost study, conducted by the Center for Benefit-Cost Studies of Education (CBCSE)

at Teachers College, Columbia University, estimates the costs of replicating the MRC Pre-K

program that produced the impacts on students’ reading skills measured by the 2013-2014 NORC

outcome assessment in the twenty-five treatment sites evaluated. In addition to estimating the

average costs per site and per student, this cost study also addresses and quantifies the variation in

costs across sites and the distribution of costs, with a specific focus on the portion of costs borne

by the schools. Consequently, this study will provide an understanding of the resources utilized to 

implement the program, as well as the mechanisms through which these resources contribute to 

the program impacts (i.e., increased reading skills). 

This cost analysis complements the existing impact and process evaluations of the

Minnesota Reading Corps Pre-K program in two key ways. First, it builds upon the rich data

collected in the process assessment (Diaconis et al., 2015) that documents the resources required 

11
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

to implement the Minnesota Reading Corps Pre-K program with fidelity, and the activities 

involved in the program’s delivery. This cost analysis carefully documents all of the resources 

provided by both the program (i.e. RMI) and by the community that enter into providing the

program at Minnesota Pre-K sites. In addition, it identifies who provides these resources and how

they vary along multiple dimensions. In doing so, this cost analysis can provide a bridge between

implementation and impact evaluations by addressing the question of what it takes to implement

the Minnesota Reading Corps Pre-K program in order to achieve the observed impacts. 

Second, a cost analysis can contribute to a better understanding of the program’s 

replicability and scalability. This analysis can contribute to an understanding of whether the

program itself can be implemented with fidelity in other contexts across the United States, and if 

so, which basic conditions must be in place in order to achieve comparable impacts. It can aid in

the identification of ingredients necessary, at both the school and program levels, to ensure fidelity

of program implementation. Finally, it can help uncover potential efficiency gains for cost savings. 

This study will analyze which program components or ingredients are more or less costly and

which may be contributing more or less to program impacts based on an analysis of both site-level

variation and an analysis of whether certain bundles of resources, for example certain classroom

personnel structures, are efficient relative to other potential configurations.

Research Questions

The primary research questions being addressed in this cost analysis are:

 What is the average cost per site and average cost per student for the MRC Pre-K

program?

 How were these costs financed? Specifically, what portion of the costs was born by

schools and what portion by other entities?

 As secondary research questions, we also investigate how costs, descriptions of

ingredients, or allocation of resources might differ among sites in the evaluation. 

12
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

5. DATA AND METHODS

Measuring Costs in Education

Estimating the costs of a program involves identifying and accounting for all of the

resources used to implement activities that generate the program’s impact on the outcomes of

interest regardless of how they are budgeted and financed (Levin, McEwan, Belfield, Bowden, &

Shand, 2018). In this study, we refer to the costs of a program as the value of the resources that

are required to implement and/or replicate an intervention or program. Thus, for the purposes of

this study, costs are explicitly different from finance, which focuses on the way the costs are paid

and who pays for them.

For example, consider an education program that relies on volunteer time for its 

implementation, which is a similar case to the MRC program, which utilizes the service of

AmeriCorps members. For volunteer-based programs, volunteer time is a resource that is

necessary for the implementation of the program and the impact generated on student educational

outcomes. However, this resource will not appear in any budget or financial analysis, since the

time devoted by volunteers is usually unpaid. If one were to replicate the program elsewhere, in

an area where there are no available volunteers, one would need to hire workers to replace

volunteer time. Therefore, restricting costs only to those accounted for in financial budgets would

understate the overall costs of the program because they do not include costs borne by sources 

other than the program itself. 

Economic analysis of costs is based on the foundational concept of opportunity cost, the

value of what is sacrificed by using a specific resource in one way rather than in its next best use

(Levin, 1975, 1983; Levin et al, 2018). The ingredients method is a cost accounting approach 

developed at the Center for Benefit-Cost Studies of Education (CBCSE) that uses this underlying 

concept to address costs (Levin, 1975, 1983; Levin et al., 2018). This method has been widely

validated in the fields of economics and accounting and provides consistency in cost estimation to

allow for comparison across programs and interventions by basing estimates on the economic

principle of opportunity cost (Levin et. al, 2018).

13
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The Ingredients Method

The ingredients method of cost analysis begins with the identification of the resources or 

ingredients that are required and used to implement the program being evaluated. Since one

purpose of this cost analysis is to estimate the resources required to replicate a particular effect in

a cost-effectiveness framework (see Section 4), our focus is on estimating the costs of the program

as actually implemented, not as a theoretical goal. Both quantitative and qualitative characteristics 

of each ingredient are identified in order to identify the true opportunity cost of each resource. For

example, the precise types and amounts of personnel are specified according to their qualifications,

functions, and time commitments. A similar exercise is carried out for facilities, equipment, and 

other program inputs. 

Under the ingredients method framework, ingredients or resources that are required and

used to implement the MRC Pre-K program are identified and valued according to their market

prices or equivalents. With this information, the true economic cost of the intervention can be

estimated and then matched to effectiveness measures obtained from the 2013-14 outcome

assessment in order to understand the resources that are required to obtain the impact realized. For

this study, information on ingredients was obtained from existing documents of the process 

assessment and outcome evaluation of the MRC Pre-K program (Diaconis et al., 2015; Markovitz

et al., 2015), from administrative data provided by RMI, and from both interviews with the

program’s personnel and observations from site visits. Data collection procedures and instruments 

are described in more detail below. All ingredients required for the intervention are identified and 

specified regardless of how they are financed. 

Once the ingredients are identified, the next step entails establishing their economic costs.

Market prices are used to establish the true economic value of each ingredient as the economic

opportunity cost is usually considered to be approximated by the market price, i.e. the price that

equates supply and demand in the competitive marketplace. Nonetheless, many markets do not

have competitive market conditions that provide a true price of the resource and, in some cases, a

market for the resource does not exist (for example, a unique source of talent). A deeper discussion 

of the methods and the assumptions used to estimate the opportunity costs of these resources is 

included below.

14
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Data Sources and Data Collection Methods

Data collection for this study began with developing a hypothesized list of ingredients

under the categories of personnel, facilities, materials and equipment, training, and others, based

on a thorough review of program documentation. These documents include the process assessment

and the outcome evaluation of the MRC Pre-K program (Diaconis et al., 2015; Markovitz et al.,

2015). In addition to these documents, the team had access to administrative data provided by RMI 

with important information of resource use during the implementation of the program in the

academic year of 2013-14. This administrative data included the program’s administrative records

on the allocation of Educator and Community Corps personnel to each site.

Furthermore, reports on training expenses and site-level records of in-kind contributions of 

personnel time devoted to AmeriCorps member supervision, and of facilities and material use

allowed us to recover relevant information for our analysis. From this hypothesized list, we

identified gaps in our knowledge and developed interview protocols with targeted questions for 

Internal Coaches, lead teachers, Master Coaches, and other relevant RMI staff. Additional

information on resource allocation was gained through on-site observation of the implementation 

of the MRC Pre-K program. Finally, current Master Coaches were surveyed via Qualtrics, an

online survey research platform, with questions that allowed a deeper understanding of how 

different personnel structures were allocated across sites and their perceived effectiveness in terms 

of implementation and student outcomes.

The costs estimated here are intended to reflect the contrast between the MRC Pre-K

program and what would have happened in the absence of the program (“business-as-usual”). By

estimating the costs that are incremental to, or above and beyond, business-as-usual, this cost

analysis is designed to uncover the costs related to the production of the program’s documented

impact. While our data are extensive, there are some aspects of program delivery for which the

information was unavailable. In particular, we were not able to obtain the specific personnel

structures within each of the classrooms in the previous evaluation, since the data were organized 

at the site level. Furthermore, we were not able to obtain information on the level of Educator and

Community Corps’ experience with the program; that is, whether they were new or returning 

members. This piece of information was crucial because new members implied longer training 

time and more intensive supervision and support by Coaches. 

15
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Thus, some ingredients data were complemented by assumptions based on information 

provided by the program and regarding typical Pre-K practices in Minnesota. These assumptions 

are explained in greater depth and are tested in a sensitivity analysis in Section 6. In cost analysis, 

it is important to examine sources of uncertainty in estimates and test the robustness of results to

assumptions via sensitivity analysis. Cost analysis is more limited than other forms of economic

evaluation, such as cost-effectiveness or benefit-cost analysis, in that it does not imply a decision

rule. Therefore, we cannot test, for instance, whether changes would alter the efficiency ranking 

of programs according to cost-effectiveness, or the break-even point under which a program’s

benefits would no longer exceed its costs. Rather, we follow Boardman, Greenberg, Vining, and

Weimer (2011) and perform a series of parameter variation tests, by which we select the

assumptions about which we are most uncertain or which are most likely to significantly impact

our results, and test a range of plausible values. We then combine these sensitivity tests into “best

case” (lowest cost) and “worst case” (highest cost) scenarios to estimate upper and lower bounds.

Missing Data Strategies

Even with our extensive data collection efforts as described above, two important pieces

of information critical to perform cost analysis were still missing. These were information related

to: (i) whether each Community and Educator Corps member was new or returning, and (ii)

classroom-level information on members, Internal Coaches and lead teachers. This section 

describes the rationale behind the importance of this information and the strategies we used to 

generate the missing information with the information already in hand.

Member experience

Program documents and interview data from key MRC program personnel indicate that

personnel time allocation across the program is directly linked to whether the AmeriCorps member 

assigned to each site is returning (and therefore experienced) or new. A returning member’s 

experience within the program greatly alleviates Internal and Master Coach supervision time as 

well as implementation time allocation from key school staff. More specifically, during Fall and 

Spring semesters, Internal Coaches would spend 6 hours per returning member per month to 

conduct two classroom observations and provide follow-up support. In contrast, they spent 12

hours per month in the Fall and 8 hours in the Spring for a new member. These interactions 

16
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

between ingredients is an important aspect to estimating idiosyncratic site level costs.

Nevertheless, our data were unable to identify whether a specific member was returning or newly

assigned, or which exact member or personnel was assigned to which classroom within a site.

Therefore, we utilized the data available to conduct a simulation procedure to fill in these missing 

data for our analysis. This section provides a description of the methods used in this procedure. 

Within a specific classroom, each member’s experience level (i.e., returning or new) was 

varied through Monte Carlo simulations, a statistical method often used to account for uncertainty

and missing data. Monte Carlos methods rely on repeated random sampling to obtain numerical

results to problems that are unfeasible to resolve through other statistical approaches (Greene, 

2000). For this study, we used Monte Carlo simulation to draw from a probability distribution that

would allow us to determine the experience of each member of the data at hand. We approximated

the way returning or new members were distributed across sites to a “binominal” probability

distribution—a distribution that can be drawn from a repeated random experiment of two possible

outcomes, such as flipping a coin (i.e., “head” and “tail”). Each of these draws is called a

“Bernoulli” trial. In our case, we conducted one Bernoulli trial of our two outcomes — “returning”

and “new”— for each member at each site 50 times and averaged their time commitments across

each simulation trial to obtain their time allocation across sites. 

One strength of this simulation strategy is its capacity to incorporate data from actual

practice, for example the share of returning and new members. We conducted a survey of all

current Master Coaches (i.e, from the 2017-2018 school year) who were working closely with site

level personnel, and asked questions about the experience of AmeriCorps members that they

supervise. The survey results showed that, on average, 58.37% of AmeriCorps members were new,

implying that more than half of the sites in our sample would receive a greater amount of time

allocated for supervision and support of its members. Assuming that current practice reflected the

practice in the 2013-2014 school year, we used this percentage as an input to our Bernoulli trial.

This assumption about the percentage of new members is varied and tested in the sensitivity

analyses provided in Section 6. 

Personnel structure

Another piece of information that was missing in our data was the classroom-level

personnel structure; that is, which AmeriCorps member or Internal Coach was assigned to which

17
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

classroom within the site. In other words, we had member and Internal Coach assignment

information at the site-level, but the information was not further broken down by classrooms even 

when a site had multiple classrooms. Different personnel structures could potentially generate

variation, both in student outcomes and in costs per site.

For example, each classroom has a lead teacher and he or she can also be an Educator

Corps member. Lead teachers that are not Educator Corps would spend extra preparation time for 

MRC activities in addition to the preparation for their ‘business-as-usual’ activities in order to

incorporate the MRC requirements. However, if the lead teacher is also an Educator Corps

member, such additional preparation time would not be considered “incremental” and would be

accounted for fully with the Educator Corps time commitment. An interview with a Master Coach

indicated that, in another instance, classrooms in which Educator Corps members were also the

lead teachers and were not paired with a Community Corps member for additional support faced 

particular challenges in maintaining implementation fidelity. Such implementation difficulty may

serve to impede the program’s ability to help develop children’s literacy skills. 

In order to account for this missing information at the classroom-level, we performed two

different analyses. First, using a similar Monte Carlo simulation approach, we simulated whether 

the lead teacher in each classroom in each site was also an Educator Corps member. That is, we

made a random draw of two possible outcomes — “lead teacher is Educator Corps” and “lead

teacher is not Educator Corps”— for each classroom in each site. To reflect actual practice in the

simulation, we incorporated the Master Coach survey results about the percentage of lead teachers 

who are also Educator Corps members (i.e., 27.13%). This adjustment allows us to recover

variation in lead teacher time arising from the way that the Educator Corps and lead teacher roles 

may overlap, which in turn can generate variation in cost across sites. The result of this approach

is presented as part of our main analysis in Section 6. 

There are other types of classroom-level personnel structures, such as lead teachers who 

also serve as Internal Coaches, that possibly lead to implementation difficulties. Our second

approach to explore classroom-level personnel structure is intended to shed light on this 

phenomenon. These estimation exercises are not based on simulation, but rather on estimates of

the average costs per classroom per ingredient. Drawing on these estimations allows us to estimate

the costs of different personnelstructures within the average classroom and provide insight on how

18
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

different personnel assignment to each classroom might vary these costs. Results for this analysis

are presented below in Section 6. 

Prices

After quantities and qualitative descriptions of all ingredients were obtained, the next step 

in the ingredients method is to apply an appropriate market or shadow price that represents the

economic value of each ingredient. In this analysis, we utilize national average prices in 2014 US

Dollars. Additional descriptions of the methods and assumptions used in this step are provided 

below. 

We mainly utilized competitive market prices rather than idiosyncratic expenditure data. 

Average market prices represent the economic value for each ingredient based upon revealed 

preference of willingness to pay broadly, rather than reflecting specific prices that may not be

available elsewhere nor reflect the true value of the resource. National prices were utilized here to

ensure comparability across sites in different locations (rural, suburban, and urban) to inform

replication efforts nationally, and to allow for comparison with other program alternatives for 

policy. 

A majority of the national prices used in this analysis were obtained from the CBCSE

Database of Educational Resource prices available through the cost tool, CostOut (Hollands et al., 

2015). Where appropriate, a corresponding fringe benefit cost was added for full-time personnel

such as preschool principals or preschool teachers.

Ingredients in the personnel category include Internal and Master Coach time, as well as

Educator Corps, Program Coordinator, lead teacher, and Principal time. As described above, the

MRC program relies upon AmeriCorps, a nation-wide federally supported volunteer program for

young adults, retirees, or anyone above the age of 18 willing to serve in the community. Given the

scale and availability of this program, this analysis values all Community Corps positions with the

market value of the actual 2013-14 hire packages provided to Community Corps members. If the

program were replicated without the availability of the AmeriCorps program, the price value of

these personnel may change. We explore this by conducting a sensitivity test that applies the

national market rate of a minimum wage position to perform equivalent services. 

Facilities and equipment costs were amortized to reflect the life value of these resources 

beyond the term of this evaluation to provide annualized measures. The prices were estimated 

19
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

using new construction or purchase prices amortized over 30 years for facilities, and 3 years for

printers and laptops (2 years for small printer) at a 3.5% discount rate.

Training was also an important category of ingredients for the MRC program. MRC

provides a standardized training, in partnership with RMI, for AmeriCorps members as well as for 

coaches. The market price for this training was applied in our analyses. Data on training, and

resources used for training related to these specific sites, were provided by RMI. In addition, we

included the opportunity cost of members and coaches attending training sessions in the training

category rather than in the personnel category. The pricing for these were matched with the

corresponding prices in the personnel category.

Estimation of Average Cost Per Site and Cost Per Student

 After we collected the quantity and price data for each ingredient, we produced cost

estimates by multiplying each corresponding pair of quantity and price. Our quantity data were

specific to each site, so the quantity-price multiplication generated a cost for each ingredient for 

each of the sampled sites. In our research questions, we had set out to estimate: (i) average cost

per site and (ii) average cost per student. To yield average cost per site, we simply aggregated 

costs by site, and took the average among the 25 sites. For the average cost per student, we

aggregated all costs regardless of the sites and divided by the total number of students served at

the 25 sampled sites. Both of these estimates should be generalizable to other MRC program

sites in Minnesota because our sample of 25 sites was a representative sample of Pre-K program

sites in Minnesota with three or more years of experience in the program.

Distribution of Costs

In a finalstep of the cost analysis performed in this study, the distribution of the cost burden 

is analyzed across categories of ingredients (i.e., personnel, facilities, materials and equipment,

training, and other) and by perspective of who bears the burden of the costs (i.e., schools, the

program, members, and students’ families). Knowing the resources that feed into a program, their

costs, and who pays for them is valuable information that allows a deeper understanding of how

the impacts of the program are generated, the resources the program leverages, in addition to the

program’s financial resources and how replication of the program is viable elsewhere.

20
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

For this analysis, the primary cost perspective of interest is that of the program, Minnesota

Reading Corps. However, for comparative purposes it is also important to document the total costs 

from a social perspective. This includes all ingredients regardless of who provides them, and

whether there is a monetary cost or if they are provided in-kind by schools, such as personnel time

and facilities space. From a school’s perspective, however, the program may provide great value

due to the resources provided by the program, by members and by families. In addition to analyzing 

costs according to perspective, and total costs from a social perspective, we also analyze how costs

vary according to different bundling of resources, such as by different classroom personnel

structures, and by school site.

6. RESULTS

In this section, we present estimates of the total incremental cost of the Minnesota Reading

Corps Pre-K Program, or the estimated costs above and beyond “business-as-usual,” as well as 

the average per student cost and the ways costs vary by school site, and by bundling of various 

resources, such as personnel structures. We also examine the cost borne by various perspectives, 

such as by the program, by the school sites, by members, and by families.

Main Results

Table 1 presents the total and per student costs of the program. Note that for all reported

findings, figures are presented in 2014 US dollars and rounded to the nearest $10 to avoid false

precision. The total cost of the MRC program as evaluated in 2013-14 in 25 sites, serving 1261 

children, is $2,124,480 per year. The average cost per site is $84,980. Each site serves between 17

to 164 students. On average, the cost per student is $1,690. This average is weighted for site size. 

The average cost per site varies substantially, with a standard deviation of $31,080. This 

variation is due to the fact that site size varies across the sample as some small sites serve 17

students while other larger sites serve 164. In addition, the assignment of Educator Corps and 

Community Corps varies across sites. Some sites have exclusively Community Corps members

assigned to them while others have only Educator Corps. Since these two resources have different

21
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

opportunity costs and different incremental time allocation2, these assignment variations are

translated into variation in total personnel and training costs across sites.

Table 1. Total Costs of Minnesota Reading Corps

Total Cost for 25 sites

Cost Per Student

Average Costs Per Site

 $2,124,480 

$1,690

 $84,980 

 ($31,080) 

Note: Dollars reported in 2014 US $ and rounded to the nearest ten. 

Standard errors of site-level statistics are in parenthesis. 

Table 2 shows the distribution of the costs across categories of ingredients. As is the case

typically in education, most of the costs to deliver the program are in the categories of personnel

and training. These two categories together account for 96% of the costs of the program. Once

again there is great variation in costs across sites within each ingredient category. This variation 

is not driven by the size of each site, rather it reflects differences in the way the program is

delivered across sites. Average personnel costs per site are $51,680 but have a standard deviation 

of $26,290, while average training costs per site are $29,860 and have a standard deviation of

$16,030. This variation is driven by the number of members allocated to each site and by whether 

or not the members are new to the program. 

While personnel and training costs of education programs tend to be larger than that of

other categories, the share of training cost (35%) is higher than usual in the Pre-K MRC program.

The program training costs include costs associated with delivering organized training sessions,

such as the annual Summer Institute, the Make-Up Institute throughout the year (for those who

missed the summer training), the Pre-K Fundamentals training provided throughout the year, and 

an annual week-long SEEDS training. Moreover, resources and materials used for these training

sessions, such as a Literacy Handbook—the primary resource that provides theoretical foundation 

and practical guidance as to how to organize reading activities, are included. Finally, we included

the time of Community and Educator Corps, as well as Internal Coaches, when attending relevant

trainings. Altogether, these account for 35% of the total cost. 

2 A Community Corps hour is valued at $10.94 while an Educator Corps hours is valued at $24.30 which is the

hourly rate of a Child Care Center Teacher at the national level.

22
 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 2. Distribution of Costs by Ingredient Category

Materials & Personnel Facilities Training Equipment Other

Average Costs Per Site

Average Site-level Cost 

Per Student

Percent

 $51,680 $650 $1,730 $29,860 

 ($26,290) ($220) ($1,090) ($16,030) 

$1,180 $20 $40 $780 

($500) ($10) ($20) ($560) 

61% 1% 2% 35%

$1,090

 ($460) 

$30 

($20) 

1% 

Note: Dollars reported in 2014 US $ and rounded to the nearest ten. 

Standard errors of site-level statistics are in parenthesis. 

Table 3 showsthe distribution of costs across schools, the program, members, and students’ 

families. Schools provide in-kind resources such as facilities and materials and key on-site

personnel time that contribute to the implementation of the program. Members also contribute the

time that they allocate to the training sessions in the summer and throughout the school year.

Finally, families contribute to the program by assisting students at home with reading assignments

that are part of the program’s designed activities.

On average, approximately two-thirds of the costs of the implementation of the program

are borne by MRC (73%). The other third of the program’s costs are borne mostly by the school

(18%). Members and families contribute to the remaining 9% of the economic costs of the

program. Cross-site variation of the distribution of who bears the costs is consistent with earlier

patterns as there is substantial variation across sites in how costs are distributed. The majority of

the costs borne by the schools are personnel costs, which include principal time, lead teacher time

and Internal Coach time. 

Table 3. Distribution of Costs by Perspectives

School Program Members Families

Personnel

Facilities

Materials and

Equipment

Training

 $10,190 $34,920 

($5,610) ($17,880) 

$560 $90 -

($210) ($60) -

$1,050 $680 -

($670) ($440) -

$3,900 $24,960 $1,010

$6,590

 ($4,590) 

-

-

-

-

-

23
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 ($1,650) ($15,700) ($950) -

- $1,090 - - Other - ($460) - -

 $15,690 $61,710 $1,010 $6,590 Total

($6,450) ($22,690) ($950) ($4,590) 

Percent 18% 73% 1% 8% 

Note: Dollars reported in 2014 US $ and rounded to the nearest ten. 

Standard errors of site-level statistics are in parenthesis. 

Personnel Structures

This section presents estimates of the costs associated with different types of classroomlevel personnel structures. Information on the perceived efficiency of each classroom personnel

structure was obtained from a survey completed by Master Coaches in the 2017-2018 school year

in which they were asked to rate each classroom structure efficiency from 1 to 5, where 5 indicated

the most efficient structure. Estimates of perceived efficiency are thus 2017-18 measures and must

be taken with caution as they might not reflect the actual efficiency of each personnel structure as

implemented in the 2013-14 school year. 

Table 4 presents the costs estimates of the most common personnel structures as reported 

by Master Coaches. Some personnel structures, for example those that involve the presence of 

both a Community and an Educator Corps member in the classroom, have a great deal of variation 

in costs that depend on whether each member is full-time or part-time and whether they are

returning or not. For these personnel structures, standard errors are presented that reflect the

variation of the costs generated by different ingredient combinations. 

Table 4. Costs Estimates Associated to Different Classroom-level Personnel Structures

Classroom Cost Cost Per Student Efficiency

1 

2 

3 

4 

5 

Community Corps + Lead Teacher + Internal $36,900 $1,845 4.07 Coach

Community Corps + Educator Corps + Lead $43,620 $2,181 4.16 Teacher + Internal Coach ($4,500) -

Educator Corps who is also Lead Teacher + $29,640 $1,482 3.3 Internal Coach

Community Corps + Lead Teacher who is also an $36,170 $1,809 4 Internal Coach

$45,780 $2,289 4*

24
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Community Corps + Educator Corps who is also ($3,800) - Lead Teacher + Internal Coach 

Community Corps + Educator Corps + Lead $42,890 $2,145 6 Teacher who is also an Internal Coach ($4,500) - 5*

Note: Dollars reported in 2014 US $ and rounded to the nearest ten. Standard errors of site-level statistics are in

parenthesis. * indicates that efficiency estimates were recovered from 5 or less observations

The estimates show an important variation of costs per student across different classroom

personnel structures. A classroom structure where there is only an Educator Corps who is also a

lead teacher in the classroom (#3) is the least costly personnel structure with an average cost per 

student of $1,482. However, its perceived efficiency is the lowest of all the personnel structures 

analyzed. The costliest personnel structure is one where a Community Corps member and an

Educator Corps member who is also a lead teacher are present in the classroom (#5). This

personnel structure generates an average cost per student of approximately $2,289, but is not

associated with the highest perceived efficiency. 

In general, there appears to be more cost-efficient options than allocating resources to the

costliest personnel structure. For example, personnel structures where both a Community and an

Educator Corps member are present (#2 and #6) are perceived by Master Coaches as the most

efficient personnel structures in implementing the program, but are not the costliest to provide. 

Uncertainty and Sensitivity Analyses

In estimating our main results, we made a number of assumptions. In this sub-section, we

perform sensitivity analysis to examine four major assumptions that we made: (i) the proportion 

of new versus returning members, (ii) the pricing of Community Corps time, (iii) training costs, 

and (iv) families’ time. Our sensitivity analyses show that, while varying or relaxing certain

assumptions generates some variation in costs, the change in the estimations is not very

substantial. Hence, our main results are generally robust. 

Proportion of new vs returning members

Our analyses utilized missing data simulations to estimate the site-level cost of MRC. By

imputing the years of experience for AmeriCorps members, there are potential implications for the

value of personnel and training based upon a new or returning AmeriCorps member in each

classroom. We test the sensitivity of the assumptions underpinning this simulation by providing

25
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

an upper bound estimate wherein all members are new (rather than nearly 60%). In addition to

providing an upper bound on costs, these estimates can be informative for start-up costs in new 

locations. In this scenario, personnel costs increase as Internal Coach time allocation increases

when guiding new members through the program throughout the school year. If all members are

new, there are also cost implications for training as all new members must attend the Pre-K 

Fundamentals training sessions. Table 5 presents this sensitivity analysis. According to these

estimates, if all members are new, implementation costs would be 3% higher, representing on

average, an increase of $2,590 on the average cost per site. These changes would account for an

increase of 4.7% on total personnel costs per site and an increase of 0.6% on total training costs 

per site. 

Table 5. Changes of the Average Cost Per Site with Variation of Members’ Experience Assumptions

New Cost Main Cost Estimates Changes in Costs Percent Change Estimate

Total Costs $84,980 $87,570 $2,590 3.0%

Personnel $51,680 $54,090 $2,410 4.7%

Training $29,860 $30,040 $180 0.6%

Note: Dollars reported in 2014 US $ and rounded to the nearest ten. 

Pricing of Community Corps time

In addition to varying the experience of the pool of members of the MRC program, we also

perform a sensitivity analysis that varies the pricing of Community Corps time. If Community

Corps members were not available, these positions would need to be replaced by the available

local workforce. We assume the position would be filled as a minimum wage position based upon 

the open requirements and extensive training and support provided. Table 6 presents the average

cost of the program per site under this sensitivity test. In general, these assumptions do not appear 

to affect cost estimates as changes in Community Corps pricing would lead to a decrease of 0.7% 

of the costs, approximately a decrease of $560 on the average site costs. This in turn would

generate a decrease in 1.1% of the average costs per site allocated to personnel ingredients.

Table 6. Changes of the Average Cost Per Site with Variation of Community Corps Pricing Assumptions

New Cost Main Cost Estimates Changes in Costs Percent Change Estimate

26
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Total Costs $84,980 $84,420 -$560 -0.7%

Personnel $51,680 $51,120 -$560 -1.1%

Note: Dollars reported in 2014 US $ and rounded to the nearest ten.

Training costs

We also performed sensitivity analysis on training costs as presented in Table 7. Training

costs represents the second largest category after personnel and hence it is important to examine

some of the major assumptions made in its estimation. We assumed in the main analysis that about

80% of members and coaches attend the Summer Institute while the remaining 20% attend MakeUp Institute sessions. Our sensitivity analysis changed this distribution to 70% for the Summer

Institute and 30% for the Make-Up Institute. This change in assumptions yields a negligible

increase ($10) in the training cost per site. Note that this estimate does not include some of the

training costs that involved assumptions as to whether the member was new or returning (i.e., 

attendance to Pre-K Fundamentals training sessions). It is included in the analysis presented in 

Table 5. 

Table 7. Changes in Training Take-up Assumptions

New Cost Main Cost Estimates Changes in Costs Estimate Percent Change

Total Costs

Training

 $84,980 $84,990 $10

 $29,860 $29,870 $10

0.0%

0.0%

Note: Dollars reported in 2014 US $ and rounded to the nearest ten. 

Standard errors of site-level statistics are in parenthesis. 

Families’ time

Finally, a pricing assumption we used to recover the value of family time dedicated to

home reading was to price this time at the national minimum wage ($7.25 per hour). We decided

to use this price because we assumed the skills needed to perform this task at home could be

performed by someone with minimal qualifications. However, if the impact found in the outcome

evaluation can be thought to be driven by higher skills and their effect on student outcomesthrough 

home reading activities, then the use of the minimum wage would underestimate the costs of the

program. Thus, Table 8 presents sensitivity analysis estimations of the increase in the program

costs if family time were to be valuated at a higher price ($22.5 per hour). Estimates indicate that

27
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

valuing family time at a higher rate would generate an increase of 16% of the total estimated

average costs which would account for a 27% increase in total average personnel costs. 

Table 8. Changes in Family Pricing Assumptions

New Cost Main Cost Estimates Changes in Costs Estimate Percent Change

Total Costs

Personnel

 $84,980 $98,870 $13,890

 $51,680 $65,570 $13,890

16%

27%

Note: Dollars reported in 2014 US $ and rounded to the nearest ten. 

Standard errors of site-level statistics are in parenthesis. 

Limitations

While this study represents a thorough and comprehensive study of the resources required 

to implement the MRC program, and the results of the study are robust to assumptions according 

to sensitivity analyses, there are several possible limitations of this study that should be taken into

account when interpreting the results. First, these estimations were performed retrospectively and 

are subject to measurement error issues insofar as we were not able to account for observed and

detailed implementation practices. By performing several sensitivity analyses, presented above, 

we address this issue by generating possible bounds of how our costs estimates might vary. 

Second, although the cost estimates provided imply information on the potential to

replicate or scale the program in other areas of the country, a generalizability problem still persists.

Programs in more densely populated areas may exhibit economies of scale: fixed costs may be

divided over a larger number of participants, sites may become more efficient by learning from

others nearby, and the program may become large enough to have market power to drive down

prices for its ingredients. These two competing factors offset one another, but the extent to which

they do so is unclear a priori. These general equilibrium effects are not accounted for in the

estimates provided above. Hence, the results presented in the sensitivity analysis must be

interpreted cautiously, taking these considerations into account.

Third, the pricing assumptions used in the estimation assume that a national market exists

for each ingredient considered. However, specific local price contexts may vary due to a number 

of circumstances that are not accounted for in the cost estimates. For example, a program that

requires personnel with specific and rare skills for its implementation might not be able to find

28
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

local workers with such skills in rural areas, because the local market does not attract the kind of

human capital needed. Therefore, implementation would require these professionals to move to

the areas of implementation, which may generate higher recruitment costs than it would if 

implementation were carried out in an otherwise costlier area of the country.

Finally, this analysis does not address other relevant questions that might be useful to 

understand about the MRC Pre-K program. For example, differences in personnel structure is one

of the areas crucial for implementation quality and efficiency indicated by the prior evaluation of 

the MRC program (i.e., process assessment), as well as by our interviews and survey data. 

However, we are unable to know if the variation in costs across sites or across personnel structures

is associated with commensurate differencesin impactsince the impact evaluation did not measure

impacts by site or by personnel structure. 

Despite these limitations, the cost estimates obtained from this analysis can offer useful

information about the return on investment in the program, and specifically the costs borne by

schools in a context in which Pre-K programs are underfunded and where improving early literacy

skills remains an important policy objective. 

7. CONCLUSION

This cost study examined the resources utilized in providing the MRC Pre-K program

during the 2013-14 school year for a sample of schools that was evaluated for impacts on emergent

literacy outcomes for 3-, 4-, and 5-year-olds. The impact evaluation found positive significant

impacts that did not vary significantly by program personnel demographic or by type of site,

indicating potential for replication. Our study compliments this evaluation to measure the costs

associated with the impacts measured. Overall, the costs of MRC are $2.1 million per year to serve

1,261 students across twenty-five schools, or $1,690 per pupil on average. Costs were found to

vary substantially by site, by ingredient category and by who bears the burden of the costs across

the 25 sites evaluated. This variation is primarily driven by the assignment of Educator Corps and

Community Corps across sites. 

Our survey data also suggest that different bundles of resources, or personnel structures,

may have different relative efficiencies that are perhaps not commensurate to the cost savings.

Although the impact evaluation and this accompanying cost analysis cannot directly measure the

29
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

relative cost-effectiveness of different personnel structures, this analysis does indicate that costs

vary between sites based on the number of assigned members, and by classroom personnel

structure. The survey data also indicate that the costliest structures are not necessarily associated

with the highest perceived efficiency. For example, classrooms with one Educator Corps member

alone are perceived by current Master Coaches to have much lower relative efficiency than

providing that Educator Corps member with additional support through either a Community Corps

member or an Internal Coach. 

These different classroom configurations may have implications for impact. While there

are some variations in cost with these different personnel structures, it is possible that the

associated impacts may vary disproportionately. This suggests that focusing program design on

ensuring Community Corps and Educator Corps members are optimally supported in the

classroom may have significant implications for impact. However, it should be noted that our

survey data only capture perceived efficiency from a small sample, and are not rigorous data on 

impact; therefore, this hypothesis has not been tested yet. To continue improving on program

design, MRC might consider conducting future evaluations of the impact of different personnel

structures or resource bundles within sites and classrooms to understand which personnel

structures and combination of resources optimize the quality of classroom instruction. 

The costs measured in this study represent investments made by MRC, schools, members, 

and families. The portion of the costs borne by the schools was about 18% of the total. Thus,

schools are able to receive additional personnel, who are trained and supported, to supplement

standard classroom practices without financing 100% of the costs of the intervention. In dollars,

the average cost per site for resources provided or financed by the school ranged from $8,570 to 

$33,300. Depending on the alternatives available, the MRC Pre-K program may provide a lowcost option for Pre-K schools to enhance reading support for young students. This is particularly 

valuable given a setting where Pre-K education is systematically budget-constrained.

Minnesota faces budget limitations in Pre-K funding, and as they increase investments

toward a targeted approach, policymakers need information to understand which programs provide

the greatest return on investment. Thus, it is important to interpret these findings within the context

of traditional Pre-K funding in Minnesota. In 2016, student expenditure in Pre-K programs in 

30
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Minnesota was $7,924 per student (NIEER, 2016).3 The average cost per student of $1,690

indicates that the program is relatively resource intensive, which largely reflects a program model

that includes intensive training and continued support. However, from the perspective of the

school, MRC is adding about $370 per student rather than the full amount. Thus, the program

offers a low-cost approach for schools even though the program provides about a 25% increase in 

the investment on schooling received in public Pre-K. 

Given previous literature on the effectiveness of various early literacy interventions, underresourced schools within Minnesota face an important decision of how to address resource

constraints to provide the individualized attention that struggling readers might need. This gap can 

either be filled by additional certified teachers, who are costly and may be in low supply, or

volunteers, who likely require intensive support to ensure quality of instruction and scalability. To 

be effective, volunteer based-programs likely need to be relatively resource-intensive. The results 

of this study indicate that the MRC Pre-K program provides the resources and a support structure

with the necessary intensity to ensure the instructional quality of AmeriCorps members, and

delivers impacts on emergent literacy with little required additional investment from underresourced schools. Identifying such a design that delivers impact and at a relatively small cost to

schools is important for decision makers aiming to address the challenges of early literacy faced 

not only by Minnesota, but by the entire nation.

3

 http://nieer.org/wp-content/uploads/2017/05/Minnesota_YB16.pdf

31
 

 

REFERENCES

Baydar, N., Brooks-Gunn, J., & Furstenberg, F. F. (1993). Early warning signs of functional

illiteracy: Predictors in childhood and adolescence. Child Development, 64(3), 815-829.

Boardman, A. E., Greenberg, D. H., Vining, A. R., & Weimer, D. L. (2011). Cost-benefit

analysis: Concepts and practice (4th ed.). New York, NY: Cambridge University Press. 

Bridges, K. K. (2015). KIDS COUNT Data Center. Retrieved from:

http://datacenter.kidscount.org/

Bruck, M. (1998). Outcomes of adults with childhood histories of dyslexia. In C. Hulme & R. M. 

Joshi (Eds.), Reading and spelling: Development and disorders (179- 200). Mahwah, NJ:

Lawrence Erlbaum Associates, Inc., Publishers.

Chall, J. S., Jacobs, V. A., & Baldwin, L. E. (1990). The reading crisis: Why poor children fall

behind. Cambridge, MA: Harvard University Press. 

Diaconis, A., Estrera, E., Hafford, C., Hernandez, M., Markovitz, C., Muyskens, P. (2015).

Process assessment of the Minnesota Reading Corps PreK program. Chicago, IL: NORC

at the University of Chicago.

Felton, R. H. (1998). The development of reading skills in poor readers: Educational

implications. In C. Hulme & R. M. Joshi (Eds.), Reading and spelling: Development and 

disorders (219-233). Mahwah, NJ: Lawrence Erlbaum Associates, Inc., Publishers. 

Greene, W. H. (2000). Econometric Analysis. Upper Saddle River, NJ: Prentice Hall.

Grossman, J. B., & Furano, K. (1999). Making the most of volunteers. Law and Contemporary

Problems, 62(4), 199-218. 

Hafford, C., Markovitz, C., Hernandez, M., Hedberg, E., Silberglitt, B., Langerman, H.,... Kiss, 

M. (2013). Process assessment of the Minnesota Reading Corps. Chicago, IL: NORC at

the University of Chicago.

Hager, M. A., & Brudney, J. L. (2004). Balancing act: The challenges and benefits of volunteers. 

Washington, DC: The Urban Institute. 

Hollands, F. M., Pan, Y., Shand, R., Cheng, H., Levin, H. M., Belfield, C. R., ... Hanisch-Cerda,

B. (2013). Improving early literacy: Cost-effectiveness analysis of effective reading

programs. New York, NY: Center for Benefit-Cost Studies of Education, Teachers

College, Columbia University.

Horst, K., & Passe, A. (2004). Creating Literacy Rich Classrooms for Preschool Children (Ages 

32
 

0-5). Presented at the 2004 CEED Symposium. Minneapolis, MN.

Jacob, R., Armstrong, C., & Willard, J. (2015). Mobilizing volunteer tutors to improve student

literacy: Implementation, impacts, and costs of the Reading Partners program. New 

York, NY: MDRC. 

Jacob, R., Armstrong, C., Bowden, A. B., & Pan, Y. (2016). Leveraging volunteers: An 

experimental evaluation of a tutoring program for struggling readers. Journal of Research

on Educational Effectiveness, 9(sup1), 67-92.

Levin, H. M. (1975). Cost-effectiveness analysis in evaluation research. In M. Guttenbag & E. L. 

Struening (Eds.). Handbook of evaluation research (Vol. 2, Chap. 5). Beverly Hills, CA:

Sage.

Levin, H. M. (1983). Cost-effectiveness analysis. Beverly Hills, CA: Sage.

Levin, H. M., & McEwan, P. J. (2001). Cost-effectiveness analysis: Methods and 

applications (Vol. 4). Beverly Hills, CA: Sage.

Levin, H. M., McEwan, P. J., Belfield, C., Bowden, A. B., & Shand, R. (2018). Economic

evaluation in education: Cost-effectiveness and benefit-cost analysis. Beverly Hills, CA:

Sage.

Lonigan, C. J., & Shanahan, T. (2009). Developing Early Literacy: Report of the National Early

Literacy Panel. Executive Summary. A Scientific Synthesis of Early Literacy

Development and Implications for Intervention. Washington, DC. National Institute for 

Literacy. 

Markovitz, C. E., Hernandez, M. W., Hedberg, E. C., & Silberglitt, B. (2014). Impact evaluation 

of the Minnesota Reading Corps K-3 program. Chicago, IL: NORC at the University of 

Chicago. 

Markovitz, C. E., Hernandez, M. W., Hedberg, E. C., & Silberglitt, B. (2015). Outcome

evaluation of the Minnesota Reading Corps PreK program. Chicago, IL: NORC at the

University of Chicago. 

May, H., Gray, A., Gillespie, J. N., Sirinides, P., Sam, C., Goldsworthy, ... Tognatta, N. (2013). 

Evaluation of the i3 scale-up of Reading Recovery: Year one report, 2011-12. 

Philadelphia, PA: Consortium for Policy Research in Education. 

Meier, J. D., & Invernizzi, M. (2001). Book Buddies in the Bronx: Testing a model for America

Reads. Journal of Education for Students Placed at Risk, 6(4), 319–333.

33
 

 

Morrow-Howell, N., Jonson-Reid, M., McCrary, S., Lee, Y., & Spitznagel, E. (2009). Evaluation 

of Experience Corps: Student reading outcomes (CSD Research Report 09-01). St. Louis, 

MO: Washington University, Center for Social Development. 

NAEP. (2015a). National Center for Education Statistics. The Nation’s Report Card: National

Results Overview for Reading. National Center for Education Statistics, Institute of 

Education Sciences, U.S. Department of Education, Washington, D.C. 

NAEP. (2015b). The Nation’s Report Card – 2015 Reading State Snapshot Report, Minnesota, 

Grade 8, Public Schools. U.S. Department of Education, Institute of Education Sciences, 

National Center for Education Statistics, National Assessment of Educational Progress 

(NAEP), various years, 1998-2015 Reading Assessments. 

NIEER. (2016). National Institute for Early Education Research. Minnesota Report. Retrieved 

from: http://nieer.org/wp-content/uploads/2017/05/Minnesota_YB16.pdf

Ramey, M. (1991). Compensatory Education Sustained Gains from Spring 1988 to Spring 1990. 

Report No. 91-1. 

Simon, J. (2011). A cost-effectiveness analysis of early literacy interventions. Columbia

University.

Slavin, R. E., Lake, C., Davis, S., and Madden, N.A. (2011). Effective programs for struggling

readers: A best-evidence synthesis. Educational Research Review, 6(1), 1-26.

34