The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page i

Outcome Evaluation of the Minnesota Reading Corps PreK Program 

February 2015 

Authors 

This study was conducted by researchers from NORC at the University of Chicago and TIES:

Carrie E. Markovitz, Ph.D., Principal Research Scientist, NORC at the University of Chicago

Marc W. Hernandez, Ph.D., Senior Research Scientist, NORC at the University of Chicago 

Eric C. Hedberg, Ph.D., Senior Research Scientist, NORC at the University of Chicago 

Benjamin Silberglitt, Ph.D., Director of Software Applications, TIES 

This report represents the work and perspectives of the authors and is the product of professional research. It does 

not represent the position or opinions of CNCS, the federal government, or the reviewers. 

Acknowledgements

Many individuals and organizations contributed to the design and implementation of the Outcome Evaluation of the Minnesota 

Reading Corps PreK Program. While it is not possible to name everyone, we would like to acknowledge some of the individuals 

and organizations who played a significant role in the successful execution of this study: 

■ The PreK evaluation sites, their staff, and the AmeriCorps members serving in the Minnesota Reading Corps PreK 

program sites for agreeing to participate in the evaluation. The comparison sites deserve extra recognition; this 

evaluation was possible only because their administrators and teachers were willing to trust our field assessors to enter 

their classrooms and collect assessment data from their students several times during the school year. 

■ The site liaisons who managed the data collection at individual PreK sites: Athena Diaconis, Elc Estrera, and Molly 

Jones; as well as Heidi Whitmore, who served as Task Manager for the data collection. We would also like to 

recognize the exceptional contributions of the field assessors who travelled throughout the state of Minnesota during 

one of the worst winters on record to collect assessment data from the PreK students at comparison sites. 

■ The CNCS Office of Research and Evaluation, who provided guidance and support throughout the design and 

implementation of the study. 

■ Finally, the PreK outcome evaluation would not have been possible without the strong support and assistance from the 

Minnesota Reading Corps program and ServeMinnesota, especially Audrey Suker, Sadie O’Connor, and Kate Horst. 

Citation 

Markovitz, C.; Hernandez, M.; Hedberg, E.; Silberglitt, B. (2015). Outcome Evaluation of the Minnesota Reading 

Corps PreK Program. NORC at the University of Chicago: Chicago, IL. 

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. Upon request, this 

material will be made available in alternative formats for people with disabilities.
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page ii

Table of Contents 

Executive Summary ........................................................................................................................................................................ v

Key Study Findings ................................................................................................................................................................. v

About the Minnesota Reading Corps .....................................................................................................................................vi

Outcome Evaluation Methodology ....................................................................................................................................... viii

Findings and Conclusions ......................................................................................................................................................xi

Final Thoughts ..................................................................................................................................................................... xiii

I. Introduction .......................................................................................................................................................................... 1

II. About Minnesota Reading Corps ....................................................................................................................................... 4

A. Statewide Implementation of the Minnesota Reading Corps: 2003-2013 .................................................................... 4

B. Foundational Framework and Staffing Structure in the Minnesota Reading Corps...................................................... 5

C. Summer Institute Training ............................................................................................................................................ 9

D. The Role of Assessment Data in the Minnesota Reading Corps Program Implementation and Improvement .......... 10

III. Outcome Evaluation of the Minnesota Reading Corps PreK Program ......................................................................... 13

A. Evaluation Logic Model .............................................................................................................................................. 13

B. PreK Outcome Evaluation Research Questions ........................................................................................................ 14

C. Site Selection ............................................................................................................................................................. 15

D. Data Collection ........................................................................................................................................................... 24

E. Analysis ...................................................................................................................................................................... 25

F. Limitations of the Study .............................................................................................................................................. 27

IV. Study Findings ................................................................................................................................................................... 29

Overall Findings for 4- and 5-year Old Students.................................................................................................................. 29

Overall Findings for 3-year Old Students ............................................................................................................................. 35

Findings by Major Demographic Groups ............................................................................................................................. 40

Site Level Effects ................................................................................................................................................................. 45

V. Conclusions ....................................................................................................................................................................... 48

Program Implications from Conclusions .............................................................................................................................. 50

Recommendations for Future Research .............................................................................................................................. 51
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page iii

List of Figures

Figure 1. Minnesota Reading Corps Response to Intervention Tiers ..............................................................................vii

Figure 2. Example of differential growth in 4- and 5-year old PreK program and comparison students’ emergent literacy 

outcomes: Letter Sound Fluency ......................................................................................................................xii

Figure II.1. Minnesota Reading Corps Response to Intervention Tiers ............................................................................... 6

Figure II.2. Minnesota Reading Corps Supervisory Structure ............................................................................................. 8

Figure III.5. Illustration of PreK Evaluation Data Hierarchy ................................................................................................ 26

Figure IV.1. Growth in Letter Sound Fluency for 4- and 5-year old PreK program and comparison students .................... 31

Figure IV.2. Growth in Rhyming Fluency for 4- and 5-year old PreK program and comparison students .......................... 32

Figure IV.3. Growth in Letter Name Fluency for 4- and 5-year old PreK program and comparison students ..................... 33

Figure IV.4. Growth in Picture Name Fluency for 4- and 5-year old PreK program and comparison students ................... 34

Figure IV.5. Growth in Alliteration Fluency for 4- and 5-year old PreK program and comparison students ........................ 35

Figure IV.6. Growth in Rhyming Fluency for 3-year old PreK program and comparison students ..................................... 37

Figure IV.7. Growth in Letter Name Recognition for 3-year old PreK program and comparison students ......................... 38

Figure IV.8. Growth in Picture Name Fluency for 3-year old PreK program and comparison students .............................. 39

Figure IV.9. Growth in Alliteration Fluency for 3-year old PreK program and comparison students ................................... 40
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page iv

List of Tables

Table 1. PreK QED Matching Variables ..........................................................................................................................ix

Table II.1. Minnesota Reading Corps PreK Assessments and Winter/Spring Targets for 4- and 5-Year Old Students .. 12

Table III.1. PreK QED Matching Variables ........................................................................................................................ 16

Table III.2. Characteristics of schools participating in the Minnesota Reading Corps PreK Outcome Evaluation (Fall 

2013) ............................................................................................................................................................... 18

Table III.3. Program and comparison student participants for the Minnesota Reading Corps PreK Outcome Evaluation 

(Fall 2013) ....................................................................................................................................................... 23

Table III.4. Differences between Comparison and Program group students by age (Fall 2013) ....................................... 24

Table IV.1. Average growth among 4- and 5-year old PreK students by Spring ............................................................... 29

Table IV.2. Average Growth Among 3-Year Old PreK Students by Spring ....................................................................... 36

Table IV.3. Moderator effects of subgroup variables on 4- and 5-year old PreK students by Spring ................................ 41

Table IV.4. Average growth among male and female 4- and 5-year old PreK students by Spring .................................... 42

Table IV.5. Average growth among White and non-White 4- and 5-year old PreK students by Spring ............................. 43

Table IV.6. Average growth among DLL and non-DLL 4- and 5-Year old PreK students by Spring ................................. 44

Table IV.7. Average growth among 4 and 5-Year old PreK students at Public Schools and Head Start Centers by 

Spring .............................................................................................................................................................. 46

Table IV.8. Site pair intraclass correlations for Spring-Fall differences ............................................................................. 47
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page v

Executive Summary

Minnesota Reading Corps is the largest AmeriCorps State program in the country. Its mission is to help every Minnesota child 

become a proficient reader by the end of third grade. To meet this goal, the Minnesota Reading Corps and its host organization, 

ServeMinnesota Action Network, engages a diverse group of AmeriCorps members to provide evidence-based literacy 

enrichment and tutoring services to preschool (PreK) students and at-risk Kindergarten through third grade (K-3) elementary 

school students. 

This report, commissioned by the Corporation for National and Community Service (CNCS), describes the results of a quasiexperimental design (QED) outcome evaluation of 1,534, 3-, 4-, and 5-year old PreK students who attended either a Minnesota 

Reading Corps PreK program site or a comparison site during the 2013-2014 school year. Students were enrolled at 25 

Minnesota Reading Corps PreK sites and 25 comparison PreK sites matched to each Minnesota Reading Corp on specific sitelevel characteristics. The goal of the outcome evaluation was to determine the effects of the Minnesota Reading Corps program 

on PreK students’ emergent literacy skills. 

Key Study Findings 

PreK students in the Minnesota Reading Corps program achieved significantly higher emergent literacy 

assessment scores by the end of the school year than did students enrolled at matched-comparison sites.

The 4- and 5-year old students at Minnesota Reading Corps PreK sites outperformed students at matched-comparison sites on 

all five emergent literacy outcomes assessed: letter sound and letter name fluency (alphabet knowledge), rhyming and 

alliteration fluency (phonological awareness), and picture name fluency (vocabulary). The effect sizes associated with these 

differences were not only significant, but substantial in magnitude. Further, 4- and 5-year old Minnesota Reading Corps PreK 

students met or exceeded end of school year targets for all five emergent literacy outcomes, suggesting that they were 

Kindergarten-ready on these critical skills. 

Three-year old students at Minnesota Reading Corps PreK sites outperformed students at matched-comparison sites on two of 

four emergent literacy outcomes measured: rhyming fluency (phonological awareness) and picture name fluency (vocabulary). 

The effect sizes associated with these differences were not only significant, but substantial in magnitude. Three-year old 

students’ scores on the remaining two outcomes, letter name recognition (alphabet knowledge) and alliteration fluency 

(phonological awareness), grew significantly during the school year, but not significantly more than students in comparison 

classrooms. The results suggest that 3-year old students enrolled in Minnesota Reading Corps PreK classrooms make 

meaningful growth in key emergent literacy outcomes associated with later reading comprehension. 

The Minnesota Reading Corps PreK program was equally effective for PreK students regardless of gender, 

race/ethnicity or Dual Language Learner (DLL) status. 

Students from all subgroups examined produced significant growth in emergent literacy outcomes regardless of gender, 

race/ethnicity, or DLL status. In instances where statistically significant differences were found, students at Minnesota Reading 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page vi

Corps PreK sites traditionally at-risk for academic failure (i.e., DLLs, non-White students) grew more over the course of the year 

than similar types of students at comparison sites. 

The Minnesota Reading Corps PreK program was effective across multiple sites and site types; thus, the 

program is highly replicable in multiple preschool settings.

The Minnesota Reading Corps PreK program was consistently effective across the individual sites where the students received 

program services, regardless of the type of school setting in which the program was implemented. The lack of school-level 

effects on student outcomes validates the Minnesota Reading Corps’ approach to training, coaching and supervision. 

About the Minnesota Reading Corps 

The Minnesota Reading Corps program was started in 2003 to provide emergent literacy enrichment and tutoring to students in 

four Head Start programs. In 2005, it expanded to serve elementary school students in Kindergarten through third grade (K-3). 

Today, the Minnesota Reading Corps is the largest AmeriCorps State program in the country. The core activities of the program, 

and its host organization, ServeMinnesota Action Network, are to recruit, train, place and monitor AmeriCorps members who 

implement evidence-based literacy interventions for at-risk K-3 students and preschool children. 

AmeriCorps members in the Minnesota Reading Corps program serve in school-based settings to implement Minnesota Reading 

Corps literacy enrichment strategies and conduct interventions with PreK-3 students using a Response to Intervention (RtI) 

framework. The key features of the Minnesota Reading Corps RtI framework are: 

■ Clear literacy targets at each age level from PreK through grade 3 

■ Benchmark assessment three times a year to identify students eligible for one-on-one or small group interventions 

■ Scientifically based interventions 

■ Frequent progress monitoring during intervention delivery 

■ High-quality training and coaching in program components, and literacy assessment and instruction 

In the RtI framework, data play the key roles of screening students’ eligibility for services and then monitoring students’ progress 

towards achieving academic goals (i.e., targets). The Minnesota Reading Corps screens students for program eligibility three 

times a year (i.e., Fall, Winter, Spring) with two sets of grade-specific, literacy-focused general outcome measures (i.e., IGDI 1.0 

for PreK and FAST for K-3) that possess criterion-referenced grade- and content-specific performance benchmarks. Program 

staff use scores from these general outcome measures to categorize students into one of three possible tiers (i.e., proficiency 

levels; see Figure 1): Tier 1 students score at or above benchmark and benefit from typical classroom instruction (75-80% of 

students score in this category); Tier 2 students score below benchmark and require specific supplemental interventions until 

they meet benchmarks (15-20% of students fall into this category); and Tier 3 students require intensive intervention provided by 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page vii

a special education teacher or literacy specialist and often have individualized educational plans (5-10% of students qualify for 

this category).1

Figure 1. Minnesota Reading Corps Response to Intervention Tiers 

The Minnesota Reading Corps PreK program includes both an immersive “push-in” component, where members provide wholeclass literacy enrichment for all students (i.e., Tier 1), as well as targeted small group and one-to-one component, where 

members provide more individualized interventions to students struggling with emergent literacy skills (i.e., Tiers 2 and 3). The 

PreK program focuses on integrating the “Big Five” Essential Early Literacy Predictors outlined by the National Early Literacy 

Panel (2008)2into all aspects of the daily classroom routine. The “Big Five” for preschool students include: 1) conversational 

skills, 2) vocabulary and background knowledge, 3) book and print rules, 4) phonological awareness (i.e., rhyming and 

alliteration), and 5) alphabet knowledge (i.e., letter name recognition and letter sound correspondence). Classroom teachers and 

AmeriCorps members are tasked with creating a Literacy Rich Classroom using evidence-based practices assessed in the 

ELLCO (Early Language & Literacy Classroom Observation) tool and implementing a Literacy Rich Schedule. In addition, 

AmeriCorps members serving in the PreK program are responsible for enacting the SEEDS of Emergent Literacy,3

 which 

provides both members and teachers with specific strategies to enhance literacy instruction for all children in the classroom. 

 

1

 The percentage of children testing into Tiers 1-3 are national estimates based on the literature: 

http://www.rtinetwork.org/essential/tieredinstruction/tiered-instruction-and-intervention-rtimodel; http://www.hanoverresearch.com/insights/developing-a-maximum-impact-response-to-intervention-program/?i=k-12-education; Hoover, 

J.J., & Patton, J.R. (2008). The role of special educator in a multi-tiered instructional system. Intervention in School and Clinic, 43(4), 195-202.

2 National Early Literacy Panel. (2008). Developing early literacy: Report of the National Early Literacy Panel. Washington, DC: National 

Institute for Literacy.

3

 Horst, K., & Passe, A. (2004). Creating Literacy Rich Classrooms for Preschool Children (Ages 0-5). Presented at the 2004 CEED 

Symposium. Minneapolis, MN. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page viii

Outcome Evaluation Methodology 

The PreK outcome evaluation is one of several complementary studies of the Minnesota Reading Corps program: a process 

assessment of program implementation in 20 PreK and K-3 sites;4

 a Phase II process assessment focused solely on the PreK 

program5

; a randomized controlled trial (RCT) impact evaluation of the program on K-3 students’ literacy outcomes; 6 and a 

survey of AmeriCorps members. The PreK outcome evaluation focused on the following three research questions: 

1. What is the effect of the Minnesota Reading Corps program on student literacy outcomes? 

2. Does the effect on literacy outcomes vary by student characteristics/demographics? 

3. Do effects on literacy outcomes vary by site/school? 

The methodology used was informed by the 2008-2009 and 2010-2011 annual Minnesota Reading Corps evaluations.7

School and Student Selection 

The site selection process was designed to ensure representation of the diverse schools participating in the Minnesota Reading 

Corps PreK program. Therefore, site selection was both purposive and involved a multi-step stratified sampling process. As it 

was not our intention to assess how well sites adopted/executed the program during their first year of participation, the site 

sampling process began by limiting the sampling frame to only sites that had fully implemented the Minnesota Reading Corps 

PreK program for at least one year. Next, to ensure geographical diversity, sites were stratified by urbanicity (i.e., urban, 

suburban, and rural), institution type (i.e., public preschool, community-based program and Head Start center), and student ages 

served within a classroom (i.e., 4- and 5-year old students only or a mix of 3-, 4-, and 5-year old students). A total of 40 sites 

were selected, using Probability Proportional to Size (PPS), whereby larger sites based on student enrollment (defined as the 

number of PreK students served by the Minnesota Reading Corps during the 2012-2013 school year) had a higher probability of 

selection. Using PPS ensured a statistically adequate sample size to conduct the PreK outcome evaluation. It is important to 

note that while the Minnesota Reading Corps program staff encouraged sites to participate, participation in the evaluation was 

voluntary. In the end, 39 sites agreed to participate in the PreK outcome evaluation during the 2013-2014 school year.8

To identify comparison sites for the PreK QED, the study followed U.S. Department of Education’s Institute of Education 

Sciences’ (IES) What Works Clearing House (WWC) guidelines for a QED that “Meets Evidence Standards with Reservations.”

PreK programs were eligible for consideration for the comparison group if: 1) the program had no history of implementing the 

Minnesota Reading Corps program; and 2) the program staff had not been trained in the SEEDS of Emergent Literacy 

 

4

 Hafford, C., Markovitz, C., Hernandez, M, et al. (February 2013). Process Assessment of the Minnesota Reading Corps Program. (Prepared 

under contract to the Corporation for National and Community Service). Chicago, IL: NORC at the University of Chicago. 

5

 Hafford, C., Markovitz, C., Hernandez, M, et al. (October 2014). Phase II Process Assessment of the Minnesota Reading Corps Program. 

(Prepared under contract to the Corporation for National and Community Service). Chicago, IL: NORC at the University of Chicago. 

6

 C., Markovitz, C., Hernandez, M, et al. (March 2014). Impact Evaluation of the Minnesota Reading Corps K-3 Program. (Prepared under 

contract to the Corporation for National and Community Service). Chicago, IL: NORC at the University of Chicago. 

7

 Bollman, K. & Silberglitt, B. (2009). Minnesota Reading Corps Final Evaluation 2008-2009. Minneapolis, MN: MRC.; Bollman, K. & Silberglitt, 

B. (2011). Minnesota Reading Corps Final Evaluation 2010-2011. Minneapolis, MN: MRC. 

8

 The school that did not participate referenced staff shortages as the reason. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page ix

approach.9 From this group of eligible PreK sites, a comparison group was identified by matching the site to a Minnesota 

Reading Corps PreK program site on a select group of educationally important pre-intervention characteristics2

10 (i.e., urbanicity, 

institution type, classroom composition of student ages, percentage of students eligible for FRPL,11 and percentage of students 

who were DLLs). Additional secondary variables were also considered when matching program and comparisons sites (e.g., site 

enrollment, instructional time, and student to teacher ratio), though they were primarily used to sharpen the match when multiple 

potential match sites were identified. The final set of variables is provided in Table 1. 

Table 1. PreK QED Matching Variables 

Order Variable Description

Primary Matching Variables

1 Urbanicity Urban, suburban or rural

2 Institution Type Public preschool, Head Start Center, Community-based program

3 Classroom Composition of 

Student Ages Served Only 4/5 year olds or mixed 3,4 and 5 year olds

4 Socio-Economic Status (SES) % students eligible for Free or Reduce Price Lunch (FRPL) and/or 

with children in the census tract within which the site is located 

% poverty of families 

5 Dual Language 

Status

Learner (DLL) % of students who do not speak English as a first language.

Secondary Matching Variables

6 Site Enrollment Number of students enrolled in the PreK program at the site.

7 Instructional Time Number of hours of instruction per week (e.g., half vs. full day; number of days per week)

8 Student to Teacher/Adult Ratio The number of students for every teacher or adult in the classroom.

Once potential comparison sites were identified, the evaluation team approached site administrators to gain their participation in 

the study through a prescribed recruitment process. Sites typically agreed to participate because they would benefit from 

obtaining access to the assessment data we collected on their students upon completion of the study. In the end, the study team 

was able to match 29 schools to Minnesota Reading Corps programs and recruit 25 sites to participate in the study. Thus, 25 

pairs (25 program sites and 25 comparison sites) for a total of 50 sites participated in the outcome evaluation. The total student 

sample size was determined by the number of students enrolled in a PreK classroom. If a comparison site had more classrooms 

and students than was required to achieve sufficient statistical power (i.e., 30 students across two classrooms), the comparison 

site classrooms that best mirrored the matched-Minnesota Reading Corps program classroom on amount of instructional time 

(i.e., days per week, hours per day) were selected. 

 

9

 Since SEEDS is an integral part of the Minnesota Reading Corps program, including SEEDS trained sites in the evaluation would have 

eliminated the possibility of detecting an effect of this critical program element.

10 These variables were based on WWC recommendations, administrative data collected by Minnesota schools at the time, the literature on 

predictors of preschool emergent-literacy outcomes, and consultation with the project’s Technical Working Group (TWG).

11 A measure of household poverty and socio-economic status 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page x

Data Collection

The evaluation team employed two methods of data collection for the PreK outcome evaluation: 1) administrative data retrieval 

from program sites and 2) original data collection from comparison sites. Because the Minnesota Reading Corps PreK program 

collects Fall, Winter, and Spring IGDI benchmark data from all PreK students in program classrooms, NORC requested access 

to all 2013-2014 school year assessment and demographic data from the 25 Minnesota Reading Corps PreK sites selected for 

participation in the outcome evaluation. This administrative data served as the primary outcome data for the program sites. 

NORC was responsible for collecting equivalent IGDI data three times a year (Fall, Winter, Spring) from PreK student at the 

comparison sites. In order to collect this data, the evaluation team sent field testers, who would be responsible for assessing 

students at comparison sites, to the same Minnesota Reading Corps training as that attended by AmeriCorps members. This 

process ensured that the field testers: 1) used the same IGDI data collection instruments for collecting student-level benchmarks 

from comparison site students as those used to collect data from students at the Minnesota Reading Corps program sites; and 2) 

received the same standardized training in assessment administration and scoring as the AmeriCorps members. This common 

use of instrumentation and training ensured commonality of assessment practices across both program and comparison sites. 

Analysis 

The student-level IGDI assessment and demographic data NORC collected from comparison sites were combined for analysis 

with de-identified IGDI data from the Minnesota Reading Corps administrative dataset. The resulting analytic dataset contained 

all the assessment and demographic data collected from comparison sites and existing data extracted for participating Minnesota 

Reading Corps PreK program sites. 

As discussed above, each Minnesota Reading Corps program site was paired with a comparison site on several key variables. 

During data analysis, these variables were equated using statistical controls so as to adjust for pre-intervention differences 

among groups. This produced a “site pair,” comprised of one Minnesota Reading Corps program site and one matched 

comparison site. The analysis was conducted separately for 3-year-old students and for 4- and 5-year-old students due to 

several factors: 1) 3-year-old children are developmentally distinct from 4- and 5-year old children; 2) there is more variability in 

prior exposure to PreK instruction among 4- and 5-year old students than among 3-year old students; and 3) the assessments 

used to measure letter name fluency differed by age group. 

Three specific and separate analytic approaches were used to address the three major research questions of the evaluation: 

1. To assess the average treatment effect of the Minnesota Reading Corps PreK program (RQ1), the evaluation team 

examined differences within each site pair by each student age group by employing an ANCOVA (Analysis of 

Covariance) regression using fixed effects for site pairs. The ANCOVA regression predicted the Spring IGDI 

benchmark score (outcome) as a function of the student-level covariates, namely the Fall benchmark score and age 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page xi

(i.e., 3-year old and 4/5-year old), in addition to the site-level Minnesota Reading Corps PreK program participation 

variable (indicator).12 

2. To determine whether the effect on emergent literacy outcomes varied as a function of student demographics (RQ2), 

the evaluation team employed two types of subgroup analyses.13 First, specific covariates and moderators were added 

to the ANCOVA model and jointly tested to determine if there were significant differences in effects across major 

subgroups. The second analysis involved subdividing the data by major subgroup and applying the same analysis as 

for the entire sample. Subgroups were defined based on characteristics provided at baseline and each level was 

determined by individual characteristics, including gender, race/ethnicity, and Dual Language Learner (DLL) status. 

3. Given the diversity of early childhood education programs (e.g., curricula, attendance time, student/family 

demographics, funding source, etc.), it was unclear if and how effective the Minnesota Reading Corps PreK program 

might be at different sites and site types. To estimate whether program effects varied by site (RQ3), the evaluation 

team conducted two analyses: 1) a subgroup analysis on site type (i.e., public preschool or Head Start Center14) and 2) 

an intraclass correlation (ICC) analysis to assess site pair-level effects. The site type subgroup analysis estimated the 

differential impact of the Minnesota Reading Corps PreK program in public preschools and Head Start centers. The 

ICC analysis examined whether there was variation in the effect of the program across pairs of sites (program vs. 

comparison). 

Findings and Conclusions 

Below, the evaluation team offers our conclusions based on the study findings and organizes them by the three major research 

questions, followed by final thoughts on the implications of these findings for the future of the Minnesota Reading Corps PreK 

program. 

Research Question #1: What is the effect of the Minnesota Reading Corps program on student literacy outcomes?

The results of our analysis showed that preschool students who attended Minnesota Reading Corps PreK classrooms achieved 

significantly higher emergent literacy assessment scores by the end of the school year than did students who did not attend a 

Minnesota Reading Corps classroom. The Minnesota Reading Corps PreK program had statistically significant effects across all 

five IGDI outcome measures, letter sound fluency, rhyming fluency, letter name fluency, picture name fluency ,and alliteration 

fluency, for 4- and 5-year old students between Fall benchmark (September 2013) and Spring benchmark (May 2014) . In 

addition, medium to large effect sizes were found on each measure, ranging from .40 to .72, indicating that the program has a 

comprehensive, positive effect on students’ emergent literacy skills. Despite the substantially smaller sample size for 3-year old 

 

12 The analysis team also conducted an analysis of the Winter IGDI benchmark scores, and the results are available in Appendix B to this 

report. 

13 Subgroup analyses were only conducted on the 4- and 5-year old student sample. They were not conducted on the 3-year old student 

sample due to small sample sizes.

14 Head Start Centers restrict enrollment to low-income households (i.e., at or below federal poverty level for household size). Thus, enrollment 

in a Head Start Center is a proxy for lower socio-economic status (SES). 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page xii

students, the Minnesota Reading Corps PreK program was found to have a significant and medium to large effects on 3-year old 

PreK students’ rhyming fluency and picture name fluency scores between Fall and Spring benchmark. Although growth was also 

found for 3-year old students on the letter name recognition and alliteration fluency outcome measures, 3-year old PreK students 

in Minnesota Reading Corps PreK classrooms did not attain significantly higher scores on these measures at Spring benchmark 

than PreK students in comparison classrooms. 

Figure 2. Example of differential growth in 4- and 5-year old PreK program and comparison students’ emergent literacy 

outcomes: Letter Sound Fluency 

0

2

4

6

8

10

12

Fall Winter Spring

Number of Letter Sounds

Time of Benchmark

Program

Comparison

Letter Sound Fluency Benchmark

Overall, the results of the student outcome analysis revealed that the Minnesota Reading Corps PreK Program is achieving its 

goal of creating Kindergarten–ready 4- and 5-year old students, as measured against the IGDI Spring benchmarks. Of particular 

interest was the large, significant effect that the Minnesota Reading Corps program produced in the growth in children’s 

vocabulary, as measured by picture name fluency. Vocabulary is one of the strongest predictors of later reading comprehension. 

The Minnesota Reading Corps PreK program’s attention to not only decoding skills (i.e., letter name recognition and letter sound 

correspondence) but also oral language skills (i.e., vocabulary) is a key program feature and likely a primary reason for the 

demonstrated effectiveness of the program. The contextualized language and literacy instruction emphasized by the program, as 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page xiii

best evidenced by extended adult-child discourse and rich dialogic reading sessions, is a proven method for growing children’s 

emergent literacy skills. 

Research Question #2: Does the effect on literacy outcomes vary by student characteristics/demographics?

For the most part, students from all subgroups examined produced significant growth in emergent literacy outcomes regardless 

of gender, race/ethnicity, or DLL status. Of note, students typically at risk for not being Kindergarten-ready, non-Whites and Dual 

Language Learners, posted significantly greater growth in vocabulary, as measured by picture name fluency, at Minnesota 

Reading Corps PreK sites than at comparison sites. In contrast to students at program sites, the vocabulary of non-White 

students and DLL students at comparisons sites did not grow significantly from Fall to Spring. This finding implies that the 

program elements that together comprise the Minnesota Reading Corps PreK program have a significant and meaningful impact 

on a critical skill (vocabulary) at program sites that is otherwise unaffected at comparison sites. 

Research Question #3: Do effects on literacy outcomes vary by site/school?

The Minnesota Reading Corps program appears to be consistently effective no matter the type of preschool setting in which the 

program is implemented, either public school or Head Start center. Students enrolled at both types of preschools evidenced 

greater growth in emergent literacy outcomes at Minnesota Reading Corps PreK program sites than comparison sites. Of note, 

students at Minnesota Reading Corps Head Start centers achieved significantly greater growth on letter sound fluency and 

alliteration fluency than their peers at comparison Head Start centers who did not improve on these measures during the school 

year. These results suggest that not only is the Minnesota Reading Corps PreK program effective in both public schools and 

Head Start centers, but that the program may be particularly effective in developing some emergent literacy skills for students 

enrolled in Head Start centers. 

Final Thoughts

Overall, the results of this study show that the Minnesota Reading Corps PreK program is an effective model for improving 

preschool students’ emergent literacy skills. Thus, the Minnesota Reading Corps PreK program is contributing to the nationwide 

goal of improving 3rd grade reading proficiency by preparing preschool-aged students to be kindergarten-ready. 

The Minnesota Reading Corps PreK program’s comprehensive scope, including both whole-class instructional strategies and 

differentiated interventions, results in class-wide impacts. The coordination between the two forms of instruction is likely a key 

source of the accelerated growth we observed in students’ emergent literacy outcomes. As outlined in the Phase II Process 

Assessment,15 the Minnesota Reading Corps PreK program provides a common infrastructure for coordinated, intentional 

instruction through the Literacy Rich Schedule, a common pedagogy for interacting with PreK students via SEEDS, and proven 

strategies for embedding language and literacy instruction at times during the school day that are often overlooked (e.g., arriving 

at the beginning of the day, lining up for bathroom breaks or the bus, transitioning between circle time and choice time). 

 

15 Hafford, C., Markovitz, C., Hernandez, M.W., Diaconis, A., Estrera, E., & Muyskens, P. (2014). Phase II Process Assessment of the 

Minnesota Reading Corps PreK Program. (Prepared under contract to the Corporation for National and Community Service). Chicago, IL: 

NORC at the University of Chicago. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page xiv

Furthermore, the program is most effective when all adults in a classroom coordinate their practice; therefore, it is critical that the 

lead teacher understand and fully commit to the Minnesota Reading Corps PreK model. The Minnesota Reading Corps 

program’s use of a multi-layered supervisor structure, including site-based Internal Coaches and program-provided Master 

Coaches, effectively supports the implementation of such a comprehensive program. In sum, the complete Minnesota Reading 

Corps PreK program, implemented with fidelity, is both highly replicable and effective in improving diverse PreK students’ 

emergent literacy outcomes. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 1

I. Introduction

Minnesota Reading Corps is a statewide initiative with a mission to help every Minnesota child become a proficient reader by the 

end of third grade. The Minnesota Reading Corps program engages a diverse group of AmeriCorps members to provide literacy 

enrichment and tutoring services to preschool children (PreK) and at-risk Kindergarten through third grade (K-3) elementary 

school students. As of the 2013-2014 school year, more than 1,100 AmeriCorps members16 implemented the program in 712 

schools or sites17 and 213 school districts across the state of Minnesota.18 

This report, funded by the Corporation for National and Community Service (CNCS), describes the findings of a quasiexperimental design (QED) outcome evaluation of 1,534 three-, four-, and five-year old students who attended either a 

Minnesota Reading Corps PreK program sites or a comparison site during the 2013-2014 school year. Students were enrolled at 

a representative sample of 25 Minnesota Reading Corps programs and 25 comparison sites matched to each Minnesota 

Reading Corps preschool or Head Start program on specific site-level characteristics. The goal of the outcome evaluation was to 

determine the effects of the Minnesota Reading Corps program on preschool students’ emergent literacy skills.19

 

The PreK outcome evaluation is one of several complementary studies of the Minnesota Reading Corps program: a process 

assessment of the Minnesota Reading Corps program in 20 PreK and K-3 sites (completed in Spring 2013)20 a randomized 

controlled trial (RCT) impact evaluation of the Minnesota Reading Corps K-3rd program (Spring 2014)21 and a survey of 

AmeriCorps members (Fall 2013). In addition to these studies, a Phase II process assessment focusing specifically on the 

Minnesota Reading Corps PreK program was completed in Fall 2014, the results of which were intended to inform the findings 

presented in this report. 

During a feasibility study for designing the Minnesota Reading Corps evaluation, we concluded that the random assignment of 

individual PreK students to program and comparison groups within a school was not feasible. The Minnesota Reading Corps 

PreK program is designed to serve an entire classroom of students and transform teachers’ instructional practices, which affects 

all students. As such, it was not possible to randomly assign either students within classrooms or classrooms within schools to 

 

16 324 members served in the PreK program during the 2013-2014 school year; 5,174 Minnesota Reading Corps members have served from 

2003 through 2014. 

17 According to the Minnesota Department of Education (MDE), during the 2013-2014 school year, 1618 public schools served grades K-12. Of 

those schools, 933 offered PreK services. The total number of preschools in the state of Minnesota (i.e., public schools and non-public schools) 

was not available. http://w20.education.state.mn.us/MDEAnalytics/Summary.jsp

18According to MDE, during the 2013-2014 school year, there were 328 public operating elementary & secondary independent school districts, 

3 intermediate school districts, and 150 charter schools (which are considered public school districts in Minnesota). 

19Specifically, letter sound fluency and letter name recognition or fluency (alphabet knowledge), rhyming and alliteration fluency (phonological 

awareness), and picture name fluency (vocabulary) 

20 Hafford, C., Markovitz, C., Hernandez, M, et al. (February 2013). Process Assessment of the Minnesota Reading Corps Program. (Prepared 

under contract to the Corporation for National and Community Service). Chicago, IL: NORC at the University of Chicago. 

21 C., Markovitz, C., Hernandez, M, et al. (March 2014). Impact Evaluation of the Minnesota Reading Corps K-3 Program. (Prepared under 

contract to the Corporation for National and Community Service). Chicago, IL: NORC at the University of Chicago. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 2

program and comparison conditions.22 Therefore, the best alternative, a rigorous QED, was employed. The Minnesota Reading 

Corps PreK outcome evaluation focuses on the following research questions: 

1. What is the effect of the Minnesota Reading Corps program on student literacy outcomes? 

2. Does the effect on literacy outcomes vary by student characteristics/demographics? 

3. Do effects on literacy outcomes vary by site/school? 

To address these questions, we begin in Chapter II by presenting a brief overview of the Minnesota Reading Corps program and 

its role in the recruitment, training, placement and monitoring of AmeriCorps members as they implement the program in 

preschool and elementary school settings. We then describe the PreK component of the Minnesota Reading Corps program 

(which is the focus of this evaluation), its multi-layered supervisory structure, and the Summer Training Institute. Chapter III then 

provides information on the outcome evaluation’s methodology for selecting sites to participate in the study, procedures for 

matching Minnesota Reading Corps (program) and comparison sites, data collection and use of program data, and analysis of 

findings. 

This background information sets the context for the presentation of findings from our analyses of assessment data from the 

Minnesota Reading Corps PreK program in Chapter IV. These findings are based on a comparison of Spring outcome scores for 

students attending a Minnesota Reading Corps program site and similar students enrolled in a comparison program site. Our 

examination of the program outcomes also includes analyses of key subgroups, including gender, race/ethnicity, and Dual 

Language Learner (DLL) status. We did not examine students by Free or Reduced Price Lunch (FRPL) eligibility, which is a 

measure of socio-economic status, because these data were either not collected or considered confidential; however, the 

majority of our sample attended either a Head Start program (100% FRPL eligible) or public schools with high percentages of 

elementary school-aged children with who were FRPL eligible. Therefore, it is reasonable to assume that the majority of the 

children in our study would likely meet eligibility criteria for FRPL. In this chapter, we also examine whether outcomes vary due to 

the type of site where the intervention took place (i.e., public school or Head Start center). 

In addition to the presentation of findings on the study results in Chapter IV, we provide in Appendix D a separate analysis of the 

data tailored to the requirements of the U.S. Department of Education's Institute of Education Sciences’ What Works 

Clearinghouse (WWC). We share the WWC’s goal of providing educators with the information they need to make evidencebased decisions. Therefore, we have developed this appendix to specifically demonstrate that our study meets WWC’s 

standards for a QED study that “meets evidence standards with reservations.”

We conclude our report in Chapter V by returning to the research questions. The evaluation team addresses whether the 

Minnesota Reading Corps PreK program appears to have an effect on students’ emergent literacy skills and whether there are 

differential effects by age, gender, race/ethnicity, and/or DLL status based on growth between Fall and Spring benchmark on 

 

22 In addition to randomizing students or classrooms, another option which was considered was to randomize schools or sites. However, there 

are not enough new school applicants to the Minnesota Reading Corps program each year to be able to adequately power an impact 

evaluation where the unit of randomization is schools/sites. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 3

age-appropriate literacy measures. Finally, we discuss the implications of the findings for the Minnesota Reading Corps program. 

A glossary of terms to assist the reader is provided in Appendix E. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 4

II. About Minnesota Reading Corps 

A. Statewide Implementation of the Minnesota Reading Corps: 2003-2013 

Minnesota Reading Corps is the largest AmeriCorps State program in the country. The goal of the Minnesota Reading Corps is 

to ensure that students become successful readers and meet reading proficiency targets by the end of the third grade. The 

Minnesota Reading Corps program was started in 2003 to provide reading and literacy tutoring to children in four preschool 

(PreK) Head Start programs. In 2005, the Minnesota Reading Corps expanded its program to serve students in Kindergarten 

through third grade (K-3). The core activities of the Minnesota Reading Corps, and its host organization, ServeMinnesota Action 

Network, are to recruit, train, place and monitor AmeriCorps members to implement research-based literacy interventions for atrisk preschool children and K-3 students. 

The Minnesota Reading Corps is a strategic initiative of ServeMinnesota. ServeMinnesota is the state commission for all 

AmeriCorps State programs in Minnesota, including the Minnesota Reading Corps, and helps leverage federal, state and private 

dollars to operate the Minnesota Reading Corps. As a catalyst for positive social change and community service, 

ServeMinnesota works with AmeriCorps members and community partners to meet critical needs in Minnesota. As a nonprofit 

organization, it supports thousands of individuals to improve the lives of Minnesotans by offering life-changing service 

opportunities that focus on education, affordable housing, employment, and the environment. The ServeMinnesota Action 

Network serves as fiscal host to provide statewide management and oversight for the Minnesota Reading Corps program. The 

Action Network is a nonprofit organization and serves as a home to incubate, replicate and scale evidence-based AmeriCorps 

programs that address critical state priorities. In addition, the Saint Croix River Education District (SCRED) and TIES23 have 

been funded by ServeMinnesota to conduct an annual evaluation of the Minnesota Reading Corps program. 5F

24

AmeriCorps members in the Minnesota Reading Corps program serve in school-based settings to implement Minnesota Reading 

Corps literacy enrichment strategies and conduct interventions with PreK-3 students. Minnesota Reading Corps members serve 

as AmeriCorps members, bound to the program’s call to service. As a direct service program, the Minnesota Reading Corps 

engages its members in service to work towards the solution of a social issue. In exchange for their service of 1,700 hours a year 

(full-time) or 900 hours a year (part-time), members receive benefits that include a bi-weekly stipend, student loan forbearance, 

and an education stipend for the first two years of service. 

Minnesota Reading Corps AmeriCorps members are supported by a multi-layered supervisory structure. One or more on-site 

Internal Coaches mentor members during their year of service, continually monitoring fidelity of program implementation, and 

ensuring effective tutoring. Internal Coaches are typically specialists, teachers, or curriculum directors employed by the site or 

school. Expert-level Master Coaches are assigned to each Internal Coach to provide consultation on literacy interventions and 

assessment, as well as ensure fidelity to the Minnesota Reading Corps model. The Minnesota Reading Corps Program 

 

23 http://ties.k12.mn.us/

24 ServeMinnesota 2011, Background document. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 5

Coordinators provide administrative support to individual sites (Principals, Internal Coaches, and Master Coaches) and assist 

members with their AmeriCorps responsibilities. 

In the 2013-14 school year, the Minnesota Reading Corps program’s more than 1,100 AmeriCorps members served over 30,000 

students in 712 elementary schools, Head Start centers, and preschools, making it the largest AmeriCorps programs in the 

country. Based on the early success of the Minnesota Reading Corps program, replication is underway in Colorado, 

Massachusetts, Michigan, Santa Cruz County, CA, Washington, DC, Virginia, Iowa, and North Dakota. 

B. Foundational Framework and Staffing Structure in the Minnesota Reading Corps 

The Minnesota Reading Corps program utilizes a Response to Intervention (RtI) approach to objectively identify and provide 

additional instructional support to at-risk students. RtI is based on a problem solving approach, which was incorporated into the 

2004 Individuals with Disabilities Education Act (IDEA), and has been gaining popularity among educators, policymakers, 

administrators, teachers, and researchers. The key features of the Minnesota Reading Corps RtI framework are: 

■ Clear literacy targets at each age level from PreK through grade 3 

■ Benchmark assessment three times a year to identify students eligible for one-to-one and small group interventions 

■ Scientifically based interventions 

■ Frequent progress monitoring during intervention delivery 

■ High-quality training and coaching in program components, literacy assessment and instruction 

In RtI, assessment data play the key roles of screening students’ eligibility for additional services and then monitoring students’ 

progress towards achieving academic goals (i.e., benchmarks). The Minnesota Reading Corps screens students for program 

eligibility three times a year (i.e., Fall, Winter, Spring) with two sets of grade-specific, literacy-focused general outcome measures 

(i.e., IGDI 1.0 for PreK and FAST for K-3) that possess criterion-referenced grade- and content-specific performance 

benchmarks. Program staff use scores from these general outcome measures to categorize students into one of three possible 

tiers (i.e., proficiency levels; see Figure II.1): Tier 1 students score at or above benchmark and benefit from typical whole class 

instruction (75-80% of students score in this category); Tier 2 students score below benchmark and require supplemental 

intervention until they meet benchmarks (15-20% of students fall into this category); and Tier 3 students require intensive one-toone intervention provided by a special education teacher or literacy specialist and often have individualized educational plans (5-

10% of students qualify for this category).25

 

 

25 The percentage of children testing into Tiers 1-3 are national estimates based on the literature: 

http://www.rtinetwork.org/essential/tieredinstruction/tiered-instruction-and-intervention-rtimodel; http://www.hanoverresearch.com/insights/developing-a-maximum-impact-response-to-intervention-program/?i=k-12-education; Hoover, 

J.J., & Patton, J.R. (2008). The role of special educator in a multitiered instructional system. Intervention in School and Clinic, 43(4), 195-202. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 6

Figure II.1. Minnesota Reading Corps Response to Intervention Tiers 

The Minnesota Reading Corps PreK program includes both an immersive “push-in” component, where members provide wholeclass literacy enrichment for all students (i.e., Tier 1), as well as targeted small group and one-to-one components, where 

members provide more individualized interventions to students struggling with emergent literacy skills (i.e., Tiers 2 and 3). The 

Minnesota Reading Corps K-3 program provides one-to-one tutoring where members provide supplemental individualized 

literacy interventions to primarily Tier 2 students in Kindergarten through third grade. Although the Minnesota Reading Corps 

program provides both PreK and K-3 interventions to students, the focus of this evaluation is on the Minnesota Reading Corps 

PreK program. Therefore, the remainder of this report will focus on describing the PreK program and evaluation. As previously 

mentioned, the findings from a randomized controlled trial (RCT) evaluation of the Minnesota Reading Corps K-3 program were 

published in Spring 2014. 

Overview of PreK Program Literacy Focus and AmeriCorps Members’ Role 

At the PreK level, the Minnesota Reading Corps program focuses on integrating the “Big Five” Early Literacy Predictors outlined 

by the National Reading Panel26 into all aspects of the daily classroom routine. The “Big Five” for preschool students include: 1) 

conversational skills, 2) vocabulary and background knowledge, 3) book and print rules, 4) phonological awareness (i.e., rhyming 

and alliteration), and 5) alphabetic knowledge (i.e., letter name recognition and letter sound correspondence). AmeriCorps 

members are tasked with creating a Literacy Rich Classroom using evidence-based practices assessed in the ELLCO (Early 

Language & Literacy Classroom Observation) tool, and implementing a Literacy Rich Schedule. Along with implementing 

classroom-based strategies, members provide targeted individual or small group literacy tutoring for Tier 2 and Tier 3 students 

 

26 National Early Literacy Panel. (2008). Developing early literacy: Report of the National Early Literacy Panel. Washington, DC: National 

Institute for Literacy.
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 7

until they meet program-specified targets that predict end-of-year proficiency. Members implement 5-to 10-minute scripted 

interventions focused on students’ emergent literacy skills.

The Minnesota Reading Corps Literacy Rich Schedule is designed to provide children with daily routines that embed early 

literacy predictors into fun and meaningful learning. As such, each classroom’s day is structured around a Literacy Rich 

Schedule that is integrated with the school’s curriculum. The schedule includes 12 activities, in order: Arrival, Sign-in, Meal Time, 

Large Group, Daily Message, Repeated Read Aloud, Tier 1 Small Group, Journal (weekly), Choice Time/Active Learning, Tier 2 

or Tier 3 Small Group, Big 5 transitions, and Family (through Talk, Read, and Write with Me!). Within and between each 

scheduled activity, teachers and members strive to integrate Minnesota Reading Corps expected routines, including “Strive for 5” 

conversations using an overarching theme, functional vocabulary, and “Big 5 Transitions.” The Big 5 Transitions occur as 

children move from one part of the literacy rich schedule to another. Members and teachers engage the children in an activity, 

such as rhyming games or letter sound songs, focused on one of four emergent literacy skills: oral language, phonological 

awareness, letter names, and letter sounds. 

AmeriCorps members serving in the PreK program are also responsible for enacting the SEEDS of Emergent Literacy,27 which 

provides both members and teachers with specific strategies to enhance literacy instruction for all children in the classroom. The 

SEEDS model is interactive, skills-focused and based on current research in early childhood education, child development, 

emergent literacy, and effective teaching. SEEDS is a relationship-based instructional approach that maps out for teachers five 

ways to intentionally interact with children in order to promote academic growth and social-emotional well-being. SEEDS high 

quality interactions include the following five elements: 

 

■ Sensitivity: Look, listen, and ask questions to become aware of each child’s needs, thoughts, abilities and feelings; 

■ Encouragement: Use intentional affirmations and positive non-verbal communication to create a shared positive learning 

environment; 

■ Education: Embed the “Big 5” literacy skills in daily routines (vocabulary, conversation, phonological awareness, book and 

print rules, and letter knowledge); 

■ Development of Skills Through Doing: Help children explore their world through hands-on learning; and 

■ Self-Image Support: Balance the SEEDS quality interactions to support a child’s feeling of being respected and capable.

Within the PreK program, there are different roles for instructional staff and AmeriCorps members. Each PreK classroom has a 

“lead teacher” (a licensed teacher who is the students’ primary instructor) and an AmeriCorps member. Two types of AmeriCorps 

members may be assigned to a classroom: Educator Corps28 or Community Corps. Community Corps members are recruited 

from the community and are embedded into a classroom to collaborate with teaching staff. Educator Corps members are current 

employees of the service site who receive additional training and accept new responsibilities to become a non-stipend 

AmeriCorps members. Both Community Corps and Educator Corps members’ share the same goal of developing their students’ 

 

27 Horst, K., & Passe, A. (2004). Creating Literacy Rich Classrooms for Preschool Children (Ages 0-5). Presented at the 2004 CEED 

Symposium. Minneapolis, MN. 

28 In previous years, Educator Corps members were referred to as Professional Corps members. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 8

emergent literacy skills in preparation for Kindergarten. Their role in the classroom is to provide children with high quality 

language and literacy learning opportunities by completing assessments, implementing the literacy rich schedule, engaging in 

SEEDS quality interactions, and conducting Tier 2 and 3 small-group and one-to-one interventions. 

Supervisory Staff 

The Internal Coaches and Master Coaches play important roles in Minnesota Reading Corps program implementation (see 

Figure II.2 for an illustration of the complete Minnesota Reading Corps supervisory structure). The Internal Coach is a school 

employee who is trained to provide on-site literacy support and oversight to AmeriCorps members serving as literacy tutors at the 

site. In order to ensure fidelity to the Minnesota Reading Corps model, the Internal Coach conducts integrity observations of 

members as they assess students (three times per year) and implement interventions with students (once a month). The Internal 

Coach provides the member with feedback for continual improvement based on these observations. The Internal Coach also 

ensures that the member is accurately reporting student data in edSpring, an internet-based data entry system for storing 

progress monitoring data on students. Throughout the school year, the Internal Coach works with assistance from the Master 

Coach to select appropriate interventions for each student and to determine if students are ready to exit from Tier 2 or 3 

intervention. The Master and/or Internal Coaches also complete an Early Language and Literacy Classroom Observation 

(ELLCO) in the Fall and Spring to provide objective feedback and suggestions to the teaching team to improve the classroom’s 

language and literacy environment. The Internal Coach works closely with Minnesota Reading Corps program staff and school 

administration to address any concerns about member performance and to conduct disciplinary action if necessary. The 

Minnesota Reading Corps estimates that the time commitment for Internal Coaches is 6-9 hours per member per month. The 

additional time commitment for required training is 56 hours for new PreK Internal Coaches and 24 hours for returning PreK 

Internal Coaches.29

Figure II.2. Minnesota Reading Corps Supervisory Structure 

The Master Coach is a literacy expert employed by the Minnesota Reading Corps who serves as a literacy consultant to the 

Internal Coach and member(s). The Master Coach supports the Internal Coach and the member in making decisions about 

 

29 http://minnesotareadingcorps.org/sites/default/files/PreK%20Guide%20for%20Sites%2014-15.pdf
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 9

student eligibility for Tier 2 or 3 interventions by reviewing benchmark IGDI data. The Master Coach also helps to ensure fidelity 

to the Minnesota Reading Corps model. The Master Coach visits schools at different frequencies throughout the year depending 

on the schools’ degree of experience implementing the Minnesota Reading Corps program, ranging from once a month for 

schools that have recently implemented the program to three times a year for schools where the program is well-established. 

Visits last approximately one hour per member, during which the Master Coach, Internal Coach and member(s) examine student 

assessment data, discuss interventions choices, review student progress, and discuss implementation challenges. 

Other Master Coach responsibilities include communicating with the Internal Coach and member(s) about preparing for 

benchmarking; performing member fidelity checks along with the Internal Coach to ensure appropriate administration of 

benchmark assessments and interventions; providing consultation as needed regarding the identification and prioritization of 

students to receive small group or one-to-one interventions; reviewing student progress; and providing program updates to the 

Internal Coach and member. If the Internal Coach cannot answer a member’s question, the Master Coach can often provide 

advice. The Master Coach can also answer questions about topics such as data entry or scheduling. 

For administrative issues, such as questions about training schedules and timesheets, the Internal Coach or member can contact 

their Minnesota Reading Corps Program Coordinator. The Program Coordinator also helps members answer questions about 

their community service requirement and requested leaves of absence. Program Coordinators are also notified about all member 

disciplinary issues. 

C. Summer Institute Training 

Each summer, the Minnesota Reading Corps hosts a multi-day Summer Institute for training returning and new Master Coaches, 

Internal Coaches, and AmeriCorps members.30 ServeMinnesota and Minnesota Reading Corps staff orchestrates the 

organizational and administrative aspects of the Summer Institute, while literacy experts conduct training sessions. This 

intensive, information-filled conference provides expert training in the evidence-based literacy interventions employed by the 

Minnesota Reading Corps. In its most basic form, the Summer Institute is a learning forum for literacy interventions and teaching 

techniques. However, the Summer Institute also serves an important role in developing member, coach, and school adherence 

to the Minnesota Reading Corps model. Speeches from former and current members, funders, parents, and officials from the 

Minnesota Department of Education and local school districts encourage this process and enhance the inspirational atmosphere 

of the training sessions. At the Summer Institute, the members also meet with their Internal Coach, and sometimes Master 

Coach, with whom they will be working throughout the upcoming school year. 

During several intensive sessions at the Summer Institute, members learn the essential skills, knowledge, and tools needed to 

serve as effective literacy tutors. These sessions introduce members to the Minnesota Reading Corps PreK program model, the 

Literacy Rich Schedule, the SEEDS approach to high quality adult-child interaction, the interventions used in the Tier 2 and 3 

small group and one-to one tutoring sessions, as well as the underlying research and theories supporting the interventions and 

 

30 Members attend all four days of the Summer Institute (one day orientation and three days of training). New Coaches attend three days, and 

returning Coaches attend one day. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 10

program model. Importantly, members are provided with a detailed Literacy Handbook to serve as a constant resource for 

supporting program implementation. The Handbook provides an introduction to the Minnesota Reading Corps program, 

information on policies and procedures and service requirements, procedures for the benchmarking and progress monitoring of 

students, and specific direction and materials for implementing the Literacy Rich Schedule and conducting small group and oneto-one interventions. In addition, members are provided with online resources that mirror the contents of the Literacy Handbook 

and supplement it with other resources such as exemplar photos of a Literacy Rich Classroom, and videos of model 

interventions and best practices. Both the Handbook and website are intended to provide members with just-in-time support once 

they are out in their service sites, as well as with opportunities for continued professional development and skill refinement. 

At the Summer Institute, PreK members receive training on the SEEDS of Emergent Literacy approach, which serves as the 

pedagogical framework within which members and teachers create a literacy rich classroom environment. In addition to the 

Summer Institute, PreK members participate in multiple smaller trainings throughout the school year. These sessions include 

additional training on SEEDS, instruction in administering and scoring student assessments, and implementation of large and 

small group interventions. The trainings are generally one-day long and are held in multiple regional locations throughout the 

state. Each PreK site’s Internal Coach is expected to attend the sessions with their member(s). 

The Summer Institute is the primary training venue not only for members, but also for Internal Coaches. At the Summer Institute, 

Internal Coaches receive a comprehensive orientation to the Minnesota Reading Corps PreK program, including program and 

early literacy background, intervention delivery, benchmarking, and progress monitoring. Internal Coaches also learn about their 

roles and responsibilities, including ensuring fidelity to the Minnesota Reading Corps model, orienting the member to the school, 

introducing school staff to the member, and coordinating school-based professional development opportunities for their 

members. Internal Coaches also are oriented to the layers of support provided by the Minnesota Reading Corps program, 

including the Master Coach and Program Coordinator. 

D. The Role of Assessment Data in the Minnesota Reading Corps Program Implementation and 

Improvement 

In the Minnesota Reading Corps PreK program, AmeriCorps members collect general outcome measure data on all PreK 

students in a classroom in the Fall, Winter, and Spring of each school year using the age-appropriate Individual Growth and 

Development Indicators (IGDIs) assessments. Internal Coaches review this data to then objectively determine eligibility for Tier 2 

or 3 interventions based upon their benchmark score. Members collect monthly progress monitoring data on those students 

selected to receive small group or one-to-one interventions. The IGDIs assess the following five critical emergent literacy skills: 

1. Letters Sound Fluency (LSF) assesses how many lowercase letter sounds a student can identify and say (e.g., l, f, 

m) within one minute. Students receive one point for every correct letter sound provided within one minute. The total 

number of correct letter sounds is the Letter Sound Fluency score. Letter sounds fluency is considered a relatively 

difficult alphabetic knowledge skill and is therefore only assessed among 4- and 5-year old students, not 3-year old 

students. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 11

2. Rhyming Fluency (RF) tests students’ phonological awareness by assessing how many rhyming words a student can 

correctly identify (e.g., bat and cat, car and jar, ham and jam) within two minutes. Students receive one point for every 

correct rhyme identified within two minutes. The total number of correct rhymes is the Rhyming Fluency score. 

3. Letter Name Recognition (LNR) is measured using two different methods depending upon the age of the child. For 3-

year old students, children are shown upper case letters and children are asked to name the letter. Students receive 

one point for every letter named correctly. The total number of letters named correctly is the Letter Name Recognition 

score. This test is untimed and there is a limited number of letters, thus the 3 year old letter name test is not a fluency 

test. In contrast, 4- and 5-year old students complete a Letter Name Fluency (LNF) test designed to assess how 

many upper case letter names a student can identify within one minute. Students receive one point for every letter 

named correctly within one minute. The total number of letters named correctly is the Letter Name Fluency score. Both 

LNR and LNF assess students’ alphabel knowledge. Given the difference in testing methods, however, the results of 

the 3- and 4/5-year old letter name assessments cannot be directly compared. 

4. Picture Name Fluency (PNF) tests students’ vocabulary by assessing how many pictures a child can name correctly 

within one minute. Students receive one point for every picture named correctly within one minute. The total number of 

pictures named correctly is the Picture Name Fluency score. 

5. Alliteration Fluency (AF) tests students’ phonological awareness by assessing how many alliteration matches (e.g., 

hill and heart, cat and can, bike and ball) a child can correctly identify within two minutes. Students receive one point 

for every correct alliteration identified within two minutes. The total number of correct alliterations is the Alliteration 

Fluency score 

Students’ scores on the IGDIs are used to identify PreK students who are eligible for Tier 2 or 3 interventions. Members then 

collect monthly progress monitoring data on those Tier 2 and 3 students to whom they provide individualized or small-group 

interventions. Table II.1 lists the specific IGDI assessments and corresponding benchmark scores used to identify eligible 4- and 

5-year old PreK students for Tier 2 and 3 interventions. The benchmarks for Winter and Spring are the same for 4- and 5-year 

old PreK students. These benchmark scores correspond to empirically-derived target scores based on research from the 

Minneapolis Public Schools and the St. Croix River Education District, who both have a long history of implementing the IGDI 

assessments at scale and with fidelity (Bollman, 2003).31 The benchmark scores indicate the level of literacy skill that needs to 

be demonstrated in order to be considered “on-track” to developing literacy skills, which in turn is an indicator of high likelihood of 

reaching grade level standard on a high-stakes state reading assessment in third grade. For example, in order for a 4-year old to 

be “on-track” to be a successful reader by third grade, they need to have a score of at least 26 on the Spring picture name 

 

31 Bollman, K. (2003). Analysis of IGDI performance in Minneapolis Public Schools and St. Croix River Education District, for 

target setting with 4- and 5-year old students. [Technical Data Analysis] Rush City, MN. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 12

fluency assessment. Research to set benchmarks for IGDIs for 3-year old students is more limited, so we did not use benchmark 

scores for the 3-year old students participating in our analysis.32

Table II.1. Minnesota Reading Corps PreK Assessments and Winter/Spring Targets for 4- and 5-Year Old Students 

Fluency Assessment Winter/Spring Target

Letter Sounds 10

Rhyming 12

Letter Names 14

Picture Names 26

Alliteration 8

*The research to set benchmark targets for IGDIs for 3-year-old students is more limited and, therefore, we only provide benchmarks for 4- and 5-year-old 

students. 

The Minnesota Reading Corps program uses the edSpring internet-based data entry system to record and store IGDI benchmark 

and progress monitoring data on all students served by the program. Monthly progress monitoring allows members to chart 

growth in emergent literacy outcomes among students receiving Tier 2 or 3 interventions. Progress monitoring data enables 

members to evaluate the effectiveness of their interventions, gauge whether students might benefit from a change in 

interventions, or determine whether they are ready to exit from intensive intervention. Every student’s progress monitoring scores 

are graphed and then reviewed monthly by a collaborative team consisting of the member, Internal Coach and Master Coach. 

The Master Coach, Internal Coach, and AmeriCorps member discuss each student’s assessment results over time before 

deciding to exit the student from Tier 2 or 3 tutoring. 

The data intensive orientation of the Minnesota Reading Corps program provides members, coaches, teachers and 

principals/directors with a consistent, objective means of identifying students who would benefit from intensive intervention, 

tracking their progress toward achieving academic goals related to critical emergent literacy skills, and informing instruction. The 

assessment data play an important role in garnering site-wide support from site staff, particularly as they see quantitative 

improvement in student outcomes. The data also provide members and coaches with objective information about the efficacy of 

the interventions with individual students, which can in turn be used to tailor the most effective instruction for the student’s skill 

level. 

 

32 Due to these limitations on available benchmarks, 3-year old students are sometimes identified for Tier 2 and 3 interventions using 4-year old 

benchmarks. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 13

Outcome Evaluation of the Minnesota Reading Corps PreK Program

Building on the Minnesota Reading Corps program background provided in Chapter II, this chapter provides a detailed 

description of the methodology used to implement the Minnesota Reading Corps PreK outcome evaluation. The methodology 

was informed by the 2008-2009 and 2010-2011 annual Minnesota Reading Corps evaluations.33 The chapter begins with the 

presentation of a logic model for the Minnesota Reading Corps program outlining key program and school inputs and activities, 

as well as the program’s desired short-term outcomes and long-term goals. After establishing the key components of the 

Minnesota Reading Corps PreK program, the evaluation team presents the three primary research questions for assessing the 

program’s effect on PreK students’ emergent literacy skills (i.e., the logic model’s PreK short-term outcome). These research 

questions guided all aspects of the evaluation design, study implementation, and data analysis. 

Following our presentation of the research questions, the next section in this chapter describes the process for sampling 

Minnesota Reading Corps sites for the PreK evaluation. Next, the quasi-experimental design and site-level matching procedures 

for the study are provided, which includes key characteristics/demographics of the participating schools. The section also 

presents the pre-intervention (i.e., Fall benchmark) characteristics/ demographics for students in classrooms participating in the 

study. A baseline analysis, which confirms the integrity of the matching procedures (i.e., establishes the equality of the program 

and comparison students at the start of the first semester in Fall 2013), is also provided. 

Finally, data collection methods and the use of administrative data are discussed, along with the analytic methods and statistical 

models for the quasi-experimental design study. The chapter concludes with a discussion of the limitations of the Minnesota 

Reading Corps PreK outcome evaluation. 

A. Evaluation Logic Model 

A logic model for the Minnesota Reading Corps program illustrating key program and school inputs and activities, as well as the 

program’s desired short-term outcomes and long-term goals is provided in Appendix A.1.34 The Minnesota Reading Corps logic 

model was developed jointly by the evaluation team and Minnesota Reading Corps program staff and served as the conceptual 

framework for the design of the PreK outcome evaluation. The logic model presents a comprehensive illustration of the complete 

Minnesota Reading Corps program, and includes inputs, activities, short-term outcomes and long-term goals for four primary 

program constituencies: PreK students, K-3 students, AmeriCorps members, and schools. 

The focus of the PreK outcome evaluation was to assess the effect of Minnesota Reading Corps program participation on 

preschool students’ emergent literacy skills. As such, the evaluation focused on only those components of the logic model 

relevant to PreK students. These components include Minnesota Reading Corps program and school-based inputs and 

 

33Bollman, K. & Silberglitt, B. (2009). Minnesota Reading Corps Final Evaluation 2008-2009. Minneapolis, MN: Minnesota Reading Corps.;

Bollman, K. & Silberglitt, B. (2011). Minnesota Reading Corps Final Evaluation 2010-2011. Minneapolis, MN: Minnesota Reading Corps.

34 A comprehensive description of the Minnesota Reading Corps logic model is available in the following two reports: Feasibility Study of the 

Minnesota Reading Corps (2013) and Process Assessment of the Minnesota Reading Corps (2013)
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 14

resources, Internal Coach and AmeriCorps member coaching and supervision, AmeriCorps member individualized literacy 

interventions for students, and student proficiency outcomes. 

Four key Minnesota Reading Corps program and school-based inputs and resources are essential to successful PreK program 

implementation: 1) Minnesota Reading Corps program selection of schools based on degree of student need and school 

capacity to partner effectively with the program; 2) school identification of Tier 2 and 3 PreK students within the classroom based 

on benchmark assessment of students’ emergent literacy skills; 3) web-based data management systems to track and monitor 

student progress with literacy interventions (i.e., edSpring); and 4) school use of research-based core literacy curriculum. 

In addition to the Minnesota Reading Corps program and school-based inputs, three important Minnesota Reading Corps 

program inputs related to AmeriCorps members included: 1) joint Minnesota Reading Corps and school recruitment, screening 

and placement of members in schools; 2) comprehensive Minnesota Reading Corps training of members and Internal Coaches 

in literacy interventions, assessment, data-driven decision-making and program rules; and 3) school identification and 

assignment of dedicated Internal Coaches to support and monitor the members. The logic model also illustrates the multiple 

layers of supervision and coaching the Minnesota Reading Corps program provides to its school-based Internal Coaches and 

AmeriCorps members. 

As shown in the logic model, the Minnesota Reading Corps PreK program’s primary activities include: 1) supporting classroom 

instruction and enriching the literacy environment within the classroom; 2) conducting benchmark testing three times per year 

(Fall, Winter and Spring) to identify students in need of small group or one-to-one intervention (i.e., Tier 2 and 3 students); 3) 

delivering interventions to eligible students; 4) assessing and charting monthly student progress on age-specific emergent 

literacy skills using the IGDI assessment and edSpring (i.e., progress monitoring); 5) “exiting” students from Tier 2 or 3 

interventions once they achieve assessment score targets; and 6) identifying any new students eligible for small group 

interventions. The intended short-term outcomes of these activities are demonstrated improvement on IGDI measures at the 

subsequent benchmarking period (i.e., Winter, Spring). The desired long-term outcome of the Minnesota Reading Corps PreK 

program is for preschool students to achieve Kindergarten-ready targets by the end of the school year. 

Thus, when reviewing the logic model, note that the PreK outcome evaluation assessed the cumulative effect of three specific 

program elements on PreK students’ emergent literacy outcomes: 1) the Minnesota Reading Corps central operations and 

school-based inputs and resources; 2) Internal Coach and AmeriCorps member coaching and supervision; and 3) AmeriCorps 

member whole class enrichment and Tier 2 and 3 interventions. 

B. PreK Outcome Evaluation Research Questions 

As the logic model presented above illustrates, the Minnesota Reading Corps program’s short-term objective is to improve 

students’ literacy skills so they are Kindergarten-ready. The primary goal of the Minnesota Reading Corps PreK outcome 

evaluation was to independently assess the effect of the program on PreK students’ emergent literacy scores. To achieve this 

goal, the PreK outcome evaluation focused on the following three research questions: 

RQ1: What is the effect of the Minnesota Reading Corps program on student literacy outcomes? 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 15

RQ2: Does the effect on literacy outcomes vary by student characteristics/demographics? 

RQ3: Do effects on literacy outcomes vary by site/school?35 

To explore these research questions, the evaluation team analyzed age-appropriate literacy assessment scores from a sample 

of 1,534 PreK students enrolled at 50 diverse public elementary schools, private community-based institutions, and Head Start 

programs during the 2013-2014 school year. Half of the sites (25) were experienced implementers36 of the Minnesota Reading 

Corps program (i.e., program sites). The remaining 25 sites (i.e., comparison sites) were selected for participation in the study 

based on a prescribed set of educationally significant baseline characteristics which matched those of the sampled Minnesota 

Reading Corps program sites (e.g., urbanicity, program type, student age mix, etc.). Thus, 25 pairs of sites (i.e., 25 program sites 

and 25 comparison sites) participated in the evaluation. Students who attended Minnesota Reading Corps program sites were 

enrolled in PreK classrooms where one or more AmeriCorps members served. Data from the Fall, Winter, and Spring 

benchmarks were used to answer the evaluation’s primary research questions, which estimate the effect of the Minnesota 

Reading Corps program on students’ emergent literacy outcomes. 

C. Site Selection 

The site selection process was designed to ensure representation of the diverse schools participating in the Minnesota Reading 

Corps PreK program. Therefore, site selection was both purposive and involved a multi-step stratified sampling process. Since 

the Minnesota Reading Corps program began 10 years ago in 2003 and expanded rapidly over the years, there was a wide 

range in the number of years sites had participated in the program. As it was not our intention to assess how well sites 

adopted/executed the program during their first year of participation, the site sampling process began by limiting the sampling 

frame to only sites that had fully implemented the Minnesota Reading Corps PreK program for at least one year. Next, to ensure 

geographical diversity, sites were stratified by urbanicity (i.e., urban, suburban, and rural), institution type (public school, 

community-based program and Head Start),37 and student ages served within a classroom (4- and 5-year old students only or a 

mix of 3-, 4-, and 5-year old students).38 Thus, the sampling frame consisted of 175 sites, with 12 stratum for sampling within 

urbanicity, institution type, and age group.39 A total of 40 sites were selected, using Probability Proportional to Size (PPS), 

 

35 We were also interested in examining the effects of AmeriCorps member type (i.e., Educator Corps or Community Corps) on student 

emergent literacy outcomes. However, given the small sample size among Minnesota Reading Corps PreK sites (N=25) participating in this 

study, as well as the fact that we did not specifically sample for member type, we could not examine this question. We did conduct a balance 

test among the 25 program schools to determine if it might be possible to conduct an exploratory analysis. Unfortunately we found significant 

lack of balance in the types of members at sites, preventing us from conducting such an analysis. 

36 That is, the program had been in operation at the site for at least one full school year. 

37 Although there were originally five types of institutions, including private and charter schools, only public schools, community-based, and 

Head Start programs were included in the sample. Only four schools were charter and private schools and could not be easily combined with 

other categories due to the fact that they are not similar to any of the other school types. Therefore, these four schools were deleted from the 

sampling frame. 

38 Only three schools served 3-year old students only, so they were excluded from the sampling frame.

39 When a crosstab of schools was run by institution type and urbanicity, there were very few schools in some categories. As a result, suburban 

community-based, suburban head start, and rural community-based schools were deleted from the sampling frame; they only comprised seven 

schools in total. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 16

whereby larger sites based on student enrollment (defined as the number of PreK students served by the Minnesota Reading 

Corps during the 2012-2013 school year) had a higher probability of selection. Using PPS ensured a statistically adequate 

sample size to conduct the PreK outcome evaluation. It is important to note that while the Minnesota Reading Corps program 

staff encouraged sites to participate, participation in the evaluation was voluntary. In the end, 39 sites agreed to participate in the 

PreK outcome evaluation during the 2013-2014 school year.40

To identify a comparison group for the PreK QED, the study followed U.S. Department of Education’s Institute of Education 

Sciences’ (IES) What Works Clearing House (WWC) guidelines for a QED that “Meets Evidence Standards with Reservations.”

PreK programs were eligible for consideration for the comparison group if: 1) the program had no history of implementing the 

Minnesota Reading Corps program; and 2) the program staff had not been trained in the SEEDS of Emergent Literacy 

approach.41 From this group of eligible PreK sites, a comparison group was identified by matching the site to a Minnesota 

Reading Corps PreK program site on a select group of educationally important pre-intervention characteristics2

42 (i.e., urbanicity, 

institution type, classroom composition of student ages, percentage of students eligible for FRPL,43 and percentage of students 

who were DLLs). Additional secondary variables were also considered when matching program and comparisons sites (e.g., site 

enrollment, instructional time, and student to teacher ratio), though they were primarily used to sharpen the match when multiple 

potential match sites were identified. The final set of variables is provided in Exhibit III.1. 

Table III.1. PreK QED Matching Variables 

Order Variable Description

Primary Matching Variables

1 Urbanicity Urban, suburban or rural

2 Institution Type Public school, Head Start program, Community-based program

3

Classroom Composition of 

Student Ages Served Only 4/5-year old students or mixed 3-,4- and 5-year old students

4 Socio-Economic Status % students eligible for Free or Reduce Price Lunch (FRPL) and/or % poverty of families 

with children in the census tract within which the site is located 

5

Dual Language Learners (DLL) 

Status % of students who do not speak English as a first language.

Secondary Matching Variables

6 Site Enrollment Number of students enrolled in the PreK program at the site.

7 Instructional Time Number of hours of instruction per week (e.g., half vs. full day; number of days per week)

8 Student to Teacher/Adult Ratio The number of students for every teacher or adult in the classroom.

 

40 The school that did not participate referenced staff shortages as the reason. 

41 Since SEEDS is an integral part of the Minnesota Reading Corps program, including SEEDS trained sites in the evaluation would have 

eliminated the possibility of detecting an effect of this critical program element. 

42 These variables were based on WWC recommendations, administrative data collected by Minnesota schools at the time, the literature on 

predictors of preschool emergent-literacy outcomes, and consultation with the project’s Technical Working Group (TWG).

43 A measure of household poverty and socio-economic status
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 17

Once potential comparison sites were identified, the evaluation team approached site administrators to gain their participation in 

the study through a prescribed recruitment process. Several sites agreed to participate because they would benefit from 

obtaining access to the assessment data we collected on their students after the study period was complete. In the end, the 

study team was able to match 29 schools to Minnesota Reading Corps programs and recruit 25 sites to participate in the study. 

Thus, 25 pairs (25 program sites and 25 comparison sites) for a total of 50 sites participated in the outcome evaluation. The total 

student sample size was determined by the number of students enrolled in a participating PreK classroom at one of the 50 

selected sites. If a comparison site had more classrooms and students than was required to achieve sufficient statistical power 

(i.e., 30 students across two classrooms), the comparison site classrooms that best mirrored the matched-Minnesota Reading 

Corps program classroom on amount of instructional time (i.e., days per week, hours per day) were selected. The list of 

participating schools and their key characteristics are provided in Table III.2 below. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM PAGE 18

Table III.2. Characteristics of schools participating in the Minnesota Reading Corps PreK Outcome Evaluation (Fall 2013) 

School MRC

Location in 

Minnesota Urbanicity1

Program 

Type Classroom Type % DLL % FRPL

School 

Enrollment

2012-13

Study 

Participants

2013-14 

(N)

Child Care Resource and Referral Head Start: 

Brookside

Austin Head Start

Kootasca Head Start: Itasca Resource Center

Backus Head Start

Duluth Head Start: Stowe Elementary

Bemidji Head Start

Kootasca Head Start: Deer River (King Elementary)

Blackduck Head Start

Minneapolis Public Schools: Waite Park Community

Brimhall & Emmett Elementaries

Becker Early Childhood

Cannon Falls Preschool

Albany Area Schools: Early Childhood Center 

(Avon)

Cold Spring Elementary

Saint Paul Public Schools: Heights Community 

School

Southeast

Southeast

Northcentral

Northcentral

Northeast

Northcentral

Northcentral

Northcentral

Metro

Metro

Central

Southeast

Central

Central

Metro

Rural

Rural

Rural

Rural

Rural

Rural

Rural

Rural

Urban

Urban

Rural

Rural

Rural

Rural

Rural

Head Start

Head Start

Head Start

Head Start

Head Start

Head Start

Head Start

Head Start

Public

Public

Public

Public

Public

Public

Public

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

4- and 5-year old students 

only

3- year old students; 4- and 

5-year old students only

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

4- and 5-year old students 

only

4- and 5-year old students 

only

4- and 5-year old students 

only

4- and 5-year old students 

only

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

4- and 5-year old students 

only

4- and 5-year old students 

only

4- and 5-year old students 

only

46

55.5

0

0

0

0

0

13

18

32

1

6.7

0

17

55

100

100

100

100

100

100

100

100

70

75

19

18

30

31

-

75

68

34

42

32

98

34

30

40

30

84

95

49

84

40

65

24

31

26

30

28

17

12

34

29

80

18

52

23

39
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM PAGE 19

School MRC

Location in 

Minnesota Urbanicity1

Program 

Type Classroom Type % DLL % FRPL

School 

Enrollment

2012-13

Study 

Participants

2013-14 

(N)

Community of Peace

Elk River Public Schools: Rogers Elementary

Delano Wee Tiger

Probsfield Early Learning

Dilworth Elementary

Community Action Partnerships of Ramsey & 

Washington Counties: Mt. Airy

Families Together

Duluth Head Start: Laura MacArthur

Hermantown Head Start

YWCA of Minneapolis: Downtown

Joy Academy

New London Spicer: Cub Kids

Humphrey Elementary

Burnsville Public School District: Diamondhead 

Education Center

Longfellow Elementary

North Branch Early Childhood: North Branch Early 

Childhood

Metro

Metro

Metro

Northwest

Northwest

Metro

Metro

Northeast

Northeast

Metro

Metro

Southwest

Metro

Metro

Southeast

Central

Urban

Suburban

Suburban

Rural

Rural

Urban

Urban

Rural

Rural

Urban

Urban

Rural

Rural

Suburban

Suburban

Rural

Public

Public

Public

Public

Public

Head Start

Head Start

Head Start

Head Start

Community

Community

Public

Public

Public

Public

Public

4- and 5-year old students 

only

4- and 5-year old students 

only

4- and 5-year old students 

only

4- and 5-year old students 

only

4- and 5-year old students 

only

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

26

6

0

1

0

78

0

0

5.3

3

-

0

1

56

23

5

31

23

14

11

30

40

100

100

8

45

51

26.4

28

38.2

48

27

42

53

114

75

40

-

44

64

33

32

20

20

68

121

-

167

26

31

28

33

36

74

26

63

36

43

9

44

37

164

29

141
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM PAGE 20

School MRC

Location in 

Minnesota Urbanicity1

Program 

Type Classroom Type % DLL % FRPL

School 

Enrollment

2012-13

Study 

Participants

2013-14 

(N)

Fergus Falls Public Schools: Otter Preschool

Oakdale Elementary

Melrose Elementary

Kerkhoven-Murdock Sunburg School District: 

Murdock

Paynesville Elementary

Pine City Elementary

Pillager Elementary

TCU (TriCity United) School Readiness: Lonsdale 

School Readiness

Rainbow Preschool

YMCA of Greater St. Paul: Midway

St. Paul Eastside YMCA

Fair Oaks Elementary

Talahi Elementary

Duluth Head Start: Meyers Wilkens Elementary

Walker Head Start

Elk River Public Schools: Zimmerman Elementary

Northwest

Metro

Central

Southwest

Southwest

Northeast

Central

Metro

Metro

Metro

Metro

Metro

Central

Northeast

Northcentral

Metro

Rural

Suburban

Suburban

Rural

Rural

Rural

Rural

Rural

Rural

Urban

Urban

Suburban

Suburban

Rural

Rural

Suburban

Public

Public

Public

Public

Public

Public

Public

Public

Public

Community

Community

Public

Public

Head Start

Head Start

Public

Mix of 3-, 4-, and 5-year old 

students

4- and 5-year old students 

only

4- and 5-year old students 

only

4- and 5-year old students 

only

4- and 5-year old students 

only

4- and 5-year old students 

only

4- and 5-year old students 

only

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

4- and 5-year old students 

only

3- year old students; 4- year 

old students; Mix of 3-,4-

year old students

4- and 5-year old students 

only

4- and 5-year old students 

only

Mix of 3-, 4-, and 5-year old 

students

Mix of 3-, 4-, and 5-year old 

students

4- and 5-year old students 

only

3

32

33.3

2

0

0

0

0

1

10

.05%

50

39.4

0

0

8

43

32

50

27

38

43

54

43

25

85

-

83

91.7

100

100

23

200

32

29

31

36

27

29

49

55

20

78

36

-

62

30

68

30

34

34

42

26

29

28

47

34

19

17

18

28

65

24

31
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM PAGE 21

Study 

School Participants

Location in Program Enrollment 2013-14 

School MRC Minnesota Urbanicity1 Type Classroom Type % DLL % FRPL 2012-13 (N)

4- and 5-year old students Winsted Elementary Metro Suburban Public only 4.4 42 - 30

Saint Paul Public Schools: Dayton's Bluff 4- and 5-year old students 

Elementary Metro Urban Public only 48 91 40 35

4- and 5-year old students Zanewood Elementary Metro Urban Public only 41.3 83 40 23
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 22

Table III.3 presents descriptive statistics by age for the PreK students included in the evaluation. Demographics include gender, 

race/ethnicity, Dual Language Learner (DLL) status, and Fall benchmark scores. During the school year, some students left the 

school area (i.e., moved) or were chronically absent and did not complete all three benchmark assessments. These students 

were removed from the analytic sample. Thus, the final sample of students included in the evaluation totaled 1,534 students.44 

The evaluation team conducted a power analysis prior to developing the evaluation’s sampling plan, in which the number of 

students required to detect a difference between program and comparison groups of students was calculated. The power 

analysis relied on the student, member, and site population numbers reported by the Minnesota Reading Corps program from 

the 2012-2013 school year. For this analysis, we estimated the minimum detectable effect (MDE) for a given number of schools, 

assuming 30 students per program and comparison school would participate in the evaluation. The power analysis 

employed previous Minnesota Reading Corps program data on PreK student-level outcomes to estimate the necessary design 

parameters (specifically, the effects of clustering students in schools). Our analysis revealed that, assuming power of 0.80 and 

an alpha level of 0.05, 25 schools in each condition (program and comparison) would produce an MDE of about 0.2 standard 

deviations. Thus, assuming 25 schools with 30 students in each school would result in a required sample size of 750 students to 

detect a substantively significant difference between program and comparison groups. As with most cluster level effects, the 

power results are relatively robust to minimal loss of students between data collection periods; however, the MDE increases as 

the number of schools decrease (that is, we lose sensitivity as we lose schools participating in the study). 

 

44 For each grade, we demonstrate low levels of attrition assuming the “liberal” standard outlined in the WWC Evidence Review Protocol for 

Early Childhood interventions (Version 2): http://ies.ed.gov/ncee/wwc/pdf/reference_resources/ece_protocol_v2.0.pdf
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 23

Table III.3. Program and comparison student participants for the Minnesota Reading Corps PreK Outcome Evaluation 

(Fall 2013) 

3-year old students 

(N= 286)

4- and 5-year old students

(N= 1,480 )

Percent Percent

Female 48% 47%

Race/Ethnicity

White 52% 66%

Black 15% 11%

Asian 10% 7%

Hispanic 10% 8%

Other 14% 9%

Dual Language Learner (DLL) 18% 14%

Table III.3 shows that demographics somewhat varied by age group. Among 3-year old students, a little more than half were 

White (52%), 15% Black, and 14% “Other.” Only 10% of 3-year old students were Asian and 10% Hispanic. Among 4- and 5-year 

old students, an even higher percentage of students were White (66%) with smaller percentages of children of other races 

represented (11% Black, 7% Asian, 8% Hispanic, and 9% “Other”). These statistics are not unexpected given that the state of 

Minnesota has higher percentages of White students than the broader American population.45 Also, a relatively small percentage 

of the analytic sample included students who were classified by their sites as Dual Language Learners (DLLs), which was not 

unexpected given the low percentage of minority children in the sample. 

Table III.4 compares key demographic variables between students enrolled in the Minnesota Reading Corps program and 

comparison sites, including average Fall benchmark (baseline) scores. The 4- and 5-year old students at program and 

comparison sites showed no significant differences on key demographic variables, thus confirming that sites were adequately 

matched on most variables. For 3-year old students, some differences were found, specifically regarding smaller racial groups 

and DLL status. However, these differences may result from the substantially smaller sample sizes for 3-year old students. While 

few major differences were found by student demographics, some comparison sites reported higher average Fall benchmark 

scores than their matched Minnesota Reading Corps sites on some outcome measures. As a result, our statistical approach 

controlled for the Fall benchmark scores to account for this imbalance. Importantly, the comparison group students posted 

slightly higher Fall benchmark scores on average than students in the program group, so these differences at baseline would 

benefit the comparison sites rather than Minnesota Reading Corps program sites. 

 

45 http://quickfacts.census.gov/qfd/states/27000.html
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 24

Table III.4. Differences between Comparison and Program group students by age (Fall 2013) 

3-year old students 4-year old students 5-year old students

C-P SE(C-P) Sig. C-P SE(C-P) Sig. C-P SE(C-P) Sig.

Female 0.04 0.05 -0.02 0.03 0.07 0.07

Race/Ethnicity

White -0.08 0.05 0.03 0.03 0.00 0.06

Black 0.10 0.04 ** 0.02 0.02 0.00 0.04

Asian 0.06 0.03 * 0.01 0.01 0.02 0.02

Hispanic -0.01 0.03 0.01 0.01 0.06 0.03

Other -0.10 0.03 ** -0.01 0.01 0.02 0.02

Dual Language Learner (DLL) 0.13 0.04 ** -0.02 0.02 -0.01 0.05

Fall Scores

Letter Sounds 0.32 0.30 1.09 1.19

Rhyming 1.19 0.36 ** 0.76 0.31 * 3.87 1.00 **

Letter Names 2.80 0.87 ** 2.47 0.59 ** 2.42 2.12

Picture Names 0.44 0.95 0.90 0.47 1.80 1.23

Alliteration 0.24 0.30 0.70 0.22 ** 2.33 0.77 **

Key: C=Comparison, P=Program (Minnesota Reading Corps), SE=Standard Error 

*p<.05; **p<.01 

D. Data Collection 

As previously discussed, IGDI assessment scores are the primary sources of data for PreK student emergent literacy outcomes 

at Minnesota Reading Corps sites. The evaluation team obtained all benchmark assessment data from the Minnesota Reading 

Corps program for all 25 Minnesota Reading Corps PreK sites selected for participation in the evaluation. As mentioned 

previously, AmeriCorps members collect benchmark data three times a year on all students in their Minnesota Reading Corps 

PreK classrooms to determine eligibility for Tier 2 or 3 small group or one-to-one intervention. Given that Members already 

collect this data, and so as not to duplicate assessment efforts, the evaluation team requested access to the 2013-2014 Fall, 

Winter, and Spring benchmark data for our analysis of student outcomes. 

The evaluation team obtained benchmark assessment data directly from the Minnesota Reading Corps program. At the end of 

the 2013-2014 school year, the complete dataset was de-identified and exported to the evaluation team for analysis. The 

Minnesota Reading Corps provided the evaluation team with other administrative data including student and AmeriCorps 

member demographics. All assessment and demographic data was combined into a single dataset that is the source of all 

analyses and results presented in this report. 

The evaluation team was responsible for collecting equivalent IGDI data three times a year (Fall, Winter, Spring) from PreK 

students at the comparison sites. In order to collect this data, the evaluation team sent field testers, who would be responsible for 

assessing students at comparison sites, to the same Minnesota Reading Corps IGDI data collection training as that attended by 

AmeriCorps members. This process ensured that the field testers: 1) used the same IGDI data collection instruments for 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 25

collecting student level benchmarks on comparison site students as those used to collect data from students at the Minnesota 

Reading Corps program sites; and 2) received the same standardized training in assessment administration and scoring as the 

AmeriCorps members. This common use of instrumentation and training ensured commonality of assessment practices across 

both program and comparison sites. 

In addition to the use of common measures and receipt of common training, whenever possible the same field testers and 

research staff were assigned to collect data from the same students at the same sites throughout the school year. This allowed 

the field testers and research staff to build strong and trusting working relationships with the administrators, teachers and 

students at participating comparison sites. The students at comparison sites became familiar and comfortable with the field 

testers over the course of the year, which further facilitated testing and data collection. 

Upon completion of data collection, the student-level IGDI data collected from comparison sites were combined for analysis with 

de-identified IGDI data from the Minnesota Reading Corps dataset described earlier. The resulting dataset contained all the 

assessment and demographic data collected from comparison sites and existing data extracted for participating Minnesota 

Reading Corps program sites. The final dataset was processed, cleaned, and coded in preparation for data analysis. 

E. Analysis 

In this section, we provide a summary of the methods and analytic approaches used to analyze the student assessment data 

collected in Fall, Winter, and Spring. As discussed above, each Minnesota Reading Corps program site was paired with a 

comparison site on several key variables. During data analysis, these variables were equated using statistical controls so as to 

adjust for pre-intervention differences among groups. This produced a “site pair,” comprised of one Minnesota Reading Corps 

program site and one matched comparison site. The hierarchy of the data is illustrated in Figure III.5, where each site pair 

contains two sites, and each site contains a set of students. The analysis was conducted separately for 3-year old students and 

then for 4- and 5-year old students, because 1) 3-year old students are developmentally distinct from 4- and 5-year old students, 

2) there is more variability in prior exposure to PreK among 4- and 5- year old students than among 3-year old students, and 3) 

the letter name assessments differed for 3-year old students versus 4- and 5-year old students. A more detailed methodology 

section is provided in Appendix A.3. As explained in our presentation of the research questions in section III.B above, the 

analysis was structured to answer three primary research questions, which each address the effect of the program on PreK 

students’ emergent literacy outcomes. Other analyses included robustness checks on the variability of the treatment effects and 

an intraclass correlation analysis of Fall to Spring gains. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 26

Figure III.5. Illustration of PreK Evaluation Data Hierarchy 

Outcome Analysis 

The overall goal of the study was to obtain a measure of the average treatment effect of the Minnesota Reading Corps program 

within each site pair by each student age group. We examined differences by employing an ANCOVA (Analysis of Covariance) 

regression using fixed effects for site pairs. The ANCOVA regression predicts the Winter or Spring benchmark score (outcome) 

as a function of the student level covariates, namely the Fall benchmark score and age (3-year old and 4/5-year old), in addition 

to the site level Minnesota Reading Corps program participation variable (indicator). The Minnesota Reading Corps indicator 

coefficient is then the average difference in the difference between the dependent score (Winter or Spring) and the Fall score for 

an average student. By centering the Fall score on the grand mean, the intercept of the model is the growth for students at 

comparison sites, and the coefficient for the Minnesota Reading Corps indicator is the difference in the growth for program 

students relative to comparison students. 

Two types of subgroup analyses were employed by the evaluation team. First, we added specific covariates and moderators to 

the ANCOVA model and jointly tested the effects to determine if there were significant differences in effects across major 

subgroups, implying that the Minnesota Reading Corps program may be more or less beneficial for selected subgroups of 

students. For the second analysis (for 4- and 5-year old students only), we used the moderators of the major subgroup indicators 

(e.g., females, males, etc.) to estimate the net average treatment effect of the Minnesota Reading Corps program for 4- and 5-

year old students, essentially subdividing the data by major subgroup and applying the same analysis as for the entire sample. 

Due to sample size constraints we were unable to conduct subgroup analysis on the 3-year old student sample. Subgroups were 

defined based on characteristics provided at baseline and each level was determined by individual characteristics, including 

gender, race/ethnicity, and Dual Language Learner (DLL) status. 

Robustness Analysis 

The evaluation team also tested the variability of the program effect at the assigned site pair level. For this analysis, we 

estimated the Minnesota Reading Corps effect between program and comparison sites, and then estimated whether, on 

average, this difference was positive. The findings from the robustness analysis are provided in Appendix C. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 27

Site-Level Effects on Assessment Scores 

The Minnesota Reading Corps PreK program operates in a diverse range of educational contexts (i.e., public schools, Head 

Start centers, and community-based preschools). An important research question for the Minnesota Reading Corps program is 

whether there are differences in student-level effects of the program depending by site type. Therefore, we conducted subgroup 

analysis on one site-level variable-- whether students attended a public school or a Head Start program. We did not have enough 

community-based sites in the sample to conduct subgroup analyses on students attending this type of site. Given the diversity of 

early childhood education programs (e.g., curricula, attendance time, student/family demographics, funding source, etc.), it was 

unclear if and how the Minnesota Reading Corps PreK program might interact with the type of site (i.e., public school or Head 

Start center). The results of this analysis allowed us to examine whether the Minnesota Reading Corps PreK program is more or 

less effective in public preschools or Head Start centers. 

One additional analysis was performed to examine whether student outcomes varied by site-- an intraclass correlation (ICC) 

analysis was conducted to determine whether pair-level random effects influenced program impacts for students attending a 

Minnesota Reading Corps PreK program site. In other words, we examined whether there was variation in the effect of the 

program on students across the 25 pairs of sites (program vs. comparison). The site pair-level ICC measures the degree to 

which students, who attended one site within the same pair, correlate with students in other site pairs. We examined pair-level 

effects rather than site-level effects because Minnesota Reading Corps program and comparison sites were matched on key site 

characteristics prior to baseline. 

F. Limitations of the Study 

The primary objective of the Minnesota Reading Corps PreK evaluation was to assess the effect of the Minnesota Reading 

Corps program on preschool students’ emergent literacy outcomes. In order to achieve this objective, the evaluation was 

designed to measure Minnesota Reading Corps program effects in a rigorous quasi-experimental design study. However, 

unavoidable limitations inherent in the study design and in working with schools and students constrained some aspects of the 

evaluation’s design, implementation, and analysis. 

The single most important limitation of the study design is its inability to control for unobserved differences between the 

Minnesota Reading Corps program and comparison sites. Despite implementing a thoughtful approach to the matching process 

using educationally significant baseline measures, it is not possible to obtain or match on all possible site-level factors which may 

bias student outcomes. For example, the strongest predictor of student performance on outcome measures is their performance 

at baseline (i.e., Fall benchmark [pretest] score). Our matching process was conducted without access to any baseline data 

because it had yet to be collected, thereby, possibly resulting in bias within our site-level pairs. To examine these differences and 

test the integrity of our matching process, we conducted statistical tests of average Fall scores within pairs (simple t-tests) and 

found the vast majority of our site-level pairs did not result in significant differences at baseline across the five outcome 

measures. For further information on our balance tests (school matching validity analysis), please see Appendix A.2. 

Another major constraint is that site participation in the evaluation was voluntary. As such, it is plausible that our estimates may 

not generalize to the population. We first addressed this issue by using stratified random sampling to select Minnesota Reading 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 28

Corps PreK sites; however, we were unable to find reasonable matches for all of our sampled schools resulting in a possibly 

biased sample. We conducted sensitivity analyses to better identify differences between the Minnesota Reading Corps sites that 

participated in the study and those that did not, mainly due to our inability to identify a matched comparison school. The findings 

from these analyses, available in Appendix A.2, confirm the representativeness of our sample. 

Unequal sample sizes across the two age groups (3-year old vs. 4- and 5-year old) were another program-induced constraint 

that resulted in differences in statistical power to detect program effects within age groups. The different sample sizes by age 

were unavoidable as very few sites in our original sampling frame had 3-year old only classrooms. This forced us to stratify the 

sample by the two larger groups of 4- and 5-year old classrooms and mixed 3-, 4-, and 5-year old classrooms; thus, less 3-year 

old students participated in the study. 

Another limitation inherent in the execution of the evaluation was the necessity to assess students three times during the school 

year at both the program and comparison sites. It is unclear what effect if any the act of being assessed might have had on 

student outcome scores. It is possible that simply participating in these assessments might positively affect student proficiency. If 

such a test effect exists it is likely to be small. Further, the effect should benefit the comparison group, increasing their scores 

above what one would typically expect had they not been assessed. 

More common limitations in education studies that the evaluation team anticipated included: student attrition, data integrity, and 

tracking alternate interventions. As mentioned above, the evaluation team implemented a quasi-experimental design using 

matched comparison sites. Benchmark assessment scores were then taken three times during the school year on all students 

within participating classrooms and sites. Statistical tests showed that students in the program and comparison groups did not 

differ on important demographic variables and proficiency measures at the beginning of the school year. As is inevitable, 

students moved away or were chronically absent (i.e., attrition). When this occurred, the affected student was dropped from the 

study’s analytic sample. The deletion of cases was moderate, reducing the sample size from 1,766 to 1,534 students (a loss of

about 24 percent) and could result in an imbalance between the program and comparison groups. However, a more central 

concern is the differential attrition between program and comparison groups. The comparison sites had attrition of about 18 

percent of students whereas the Minnesota Reading Corps program sites had about 27 percent attrition. For a more thorough 

analysis of the differences between program and comparison groups, please see our analysis developed for the What Works 

Clearinghouse (WWC) located in Appendix D. 

Another common limitation related to interpretation of outcome evaluation results is concerns as to whether alternative 

interventions similar to the treatment (in this case, the Minnesota Reading Corps’ Literacy Rich Schedule and SEEDS approach 

to instruction) could similarly affect student outcomes at comparison programs. To overcome this limitation, the evaluation team 

excluded sites where staff had been trained in the SEEDS of Emergent Literacy, which is an integral part of the Minnesota 

Reading Corps PreK program. No other similar alternative interventions were identified at the PreK comparison sites. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 29

Study Findings 

This section of the report presents the findings from the analysis of IGDI assessment data collected in Fall and Spring from PreK 

students enrolled at the program or comparison sites during the 2013-2014 school year. The findings from the analysis of Fall 

and Winter benchmark scores are provided in Appendix B.1. As discussed earlier, the program group consisted of preschool 

children who attended a classroom where the Minnesota Reading Corps PreK program was being implemented during the 2013-

2014 school year, while the comparison group consisted of preschool students who attended sites that had not participated 

previously or currently in the program. 

To address the first research question (RQ1) of whether the Minnesota Reading Corps program had an effect on PreK students’ 

literacy outcomes, the evaluation team compared the Spring assessment scores of students who attended a program site to the 

scores of those who attended a similar comparison preschool site that was not implementing the Minnesota Reading Corps. As 

discussed in Chapter III, the evaluation team estimated the average treatment effect by developing a single ANCOVA model for 

each outcome measure and age group (3-year old students and 4- and 5-year old students). In addition to providing estimated 

differences between program and comparison groups, we also present standard errors, confidence intervals and statistical tests. 

Below the evaluation team provides the findings from the Spring benchmark scores across the five IGDI outcome measures by 

age group. (Findings for Winter benchmark scores are provided in Appendix B.1.) Following the presentation of findings for all 

students, we provide the results of our subgroup analysis for gender, race/ethnicity, and DLL status (RQ2). We conclude this 

chapter with the presentation of results from our examination of student effects by AmeriCorps member type and site (RQ3). 

More detailed tables on the models used are provided in Appendix B.2. 

Overall Findings for 4- and 5-year Old Students 

The Minnesota Reading Corps PreK program had statistically significant and large effects across all five IGDI outcome measures 

of emergent literacy skills for 4- and 5-year old students. Below we provide a discussion of the findings in growth between Fall 

and Spring benchmarks for 4- and 5-year old students. To better illustrate the findings on each outcome measure, we also 

provide several figures showing the average scores on the Fall, Winter, and Spring benchmarks for PreK students in Minnesota 

Reading Corps program and comparison site classrooms. Average growth for the school year, which is provided in Table IV.1, is 

calculated by taking the difference between the Spring and Fall scores for program and comparison site students, while 

controlling for the Fall (baseline) score. We refer to the findings in both Table IV.1 and the various figures in reporting the study 

findings. 

Table IV.1. Average growth among 4- and 5-year old PreK students by Spring 

Treatment Average 

IGDI Fluency Assessment Assignment N Growth SE Effect Size Confidence Interval

Letter Sounds

Program+

Comparison

Difference

783

484

1267

8.22***

4.42***

3.80***

0.33

0.58

0.90 0.71 0.36 1.05
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 30

Rhyming

Letter Names

Picture Names

Alliteration

Program

Comparison

Difference

Program

Comparison

Difference

Program

Comparison

Difference

Program

Comparison

Difference

789

553

1342

802

552

1354

793

550

1343

792

486

1278

7.28***

3.77***

3.51***

14.01**

9.87***

4.15**

8.68***

4.65***

4.03***

5.37***

2.68***

2.68***

0.23

0.31

0.52

0.45

0.72

1.17

0.28

0.39

0.66

0.17

0.28

0.44

0.66

0.40

0.49

0.72

0.46

0.17

0.32

0.48

0.87

0.63

0.65

0.97

***<.001, **<.01, *<.05; +Minnesota Reading Corps 

What is the effect of the Minnesota Reading Corps program on 4- and 5-year old students’ letter sound 

fluency? 

The Minnesota Reading Corps program had a significant and large effect on 4- and 5-year old PreK students’ letter sound 

fluency scores between Fall benchmark (September 2013) and Spring benchmark (May 2014). As shown in Table IV.1 above, 

both groups of students’ letter sound fluency scores increased significantly from Fall to Spring, however by the end of the school 

year, PreK students in a Minnesota Reading Corps classroom were able to produce an average of 3.8 more letter sounds 

correctly in a one-minute period than PreK students in a comparison classroom. This represents a large effect size of about .71 

standard deviations. Furthermore, as shown in Figure IV.1 below, by the end of the school year, the Minnesota Reading Corps 

PreK students’ average score of 10.6 letter sounds was above the Spring benchmark of 10, whereas the comparison PreK 

students’ average score of 6.8 letter sounds was well below the benchmark. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 31

Figure IV.1. Growth in Letter Sound Fluency for 4- and 5-year old PreK program and comparison students 

0

2

4

6

8

10

12

Fall Winter Spring

Number of Letter Sounds

Time of Benchmark

Program

Comparison

Letter Sound Fluency

Benchmark

What is the effect of the Minnesota Reading Corps program on 4- and 5-year old students’ rhyming fluency?

In addition to letter sounds, 4- and 5-year old PreK students in Minnesota Reading Corps classrooms had significantly higher 

scores in rhyming fluency at the end of the school year than did PreK students in comparison classrooms. Table IV.1 shows that 

both groups of students’ rhyming fluency scores increased significantly from Fall to Spring, however by the end of the school

year PreK students in Minnesota Reading Corps classrooms were able to identify an average of 2.1 more rhymes in a two 

minute period than PreK students in comparison classrooms. This represents a large effect size of about 0.66 standard 

deviations. Figure IV.2 shows that once again the Minnesota Reading Corps PreK students’ average score (13.4) reached the 

Spring benchmark of 12 words, while the comparison students’ average score (8.7) did not reach benchmark by the end of the 

school year.
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 32

Figure IV.2. Growth in Rhyming Fluency for 4- and 5-year old PreK program and comparison students 

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Fall Winter Spring

Number of Words

Time of Benchmark

Program

Comparison

Rhyming Fluency

Benchmark

What is the effect of the Minnesota Reading Corps program on 4- and 5-year old students’ letter name fluency?

As with letter sound fluency and rhyming fluency, a significant effect was also observed in letter name fluency among 4- and 5-

year old PreK students in Minnesota Reading Corps classrooms. By the end of the school year, both groups of students’ letter 

name fluency scores increased significantly from Fall, however PreK students in Minnesota Reading Corps classrooms were 

able to identify an average of 4.2 more letters correctly in a one minute period than PreK students in comparison classrooms. 

This represents a considerable effect size of about .40 standard deviations. One noticeable difference between the findings on 

letter name fluency and other outcome measures is the performance of the 4- and 5-year old PreK students in the comparison 

group (shown in Figure IV.3). In this case, 4- and 5-year old PreK students in both Minnesota Reading Corps and comparison 

classrooms produced an average letter name fluency score (22.7 and 18.6, respectively) which well exceeded the Spring 

benchmark of 14 letters. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 33

Figure IV.3. Growth in Letter Name Fluency for 4- and 5-year old PreK program and comparison students 

0

2

4

6

8

10

12

14

16

18

20

22

24

Fall Winter Spring

Number of Letter Names

Time of Benchmark

Program

Comparison

Letter Name Fluency

Benchmark

What is the effect of the Minnesota Reading Corps program on 4- and 5-year old students’ picture name 

fluency? 

The Minnesota Reading Corps program also had a significant effect on 4- and 5-year old PreK students’ picture name fluency 

scores between Fall and Spring benchmark. By the end of the school year, both groups of students’ picture name fluency scores 

increased significantly from Fall, however PreK students in Minnesota Reading Corps classrooms were able to identify an 

average of 4.0 more pictures correctly in a one minute period than PreK students in comparison classrooms. As with the other 

outcome measures, this difference represents a substantial effect size of about .49 standard deviations. Figure IV.4 below shows 

that, by the end of the school year, the Minnesota Reading Corps PreK students’ average score of 28.3 pictures was above the 

Spring benchmark of 26, whereas the comparison PreK students’ average score of 24.2 pictures was just below benchmark.
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 34

Figure IV.4. Growth in Picture Name Fluency for 4- and 5-year old PreK program and comparison students 

0

2

4

6

8

10

12

14

16

18

20

22

24

26

28

30

Fall Winter Spring

Number of Pictures

Time of Benchmark

Program

Comparison

Picture Naming Fluency

Benchmark

What is the effect of the Minnesota Reading Corps program on 4- and 5-year old students’ alliteration 

fluency? 

The final outcome examined for 4- and 5- year old students was alliteration fluency, the ability to identify words beginning with 

the same letter sound (e.g., cat and car). Between Fall and Spring benchmark, both groups of students’ alliteration fluency 

scores increased significantly, however PreK students in Minnesota Reading Corps classrooms were able to identify an average 

of 2.7 more alliterations correctly in a two minute period than PreK students in comparison classrooms. This represents a large 

effect size of about .72 standard deviations. Furthermore, as shown in Figure IV.5 below, by the end of the school year, the 

Minnesota Reading Corps PreK students’ average score of 8.0 alliterations reached the Spring benchmark of 8, whereas the 

comparison PreK students’ average score of 5.3 sounds was well below the benchmark.
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 35

Figure IV.5. Growth in Alliteration Fluency for 4- and 5-year old PreK program and comparison students 

0

1

2

3

4

5

6

7

8

9

Fall Winter Spring

Number of Words

Time of Benchmark

Program

Comparison

Alliteration Fluency

Benchmark

Overall Findings for 3-year Old Students 

The Minnesota Reading Corps PreK program had a statistically significant and positive effect on 3-year old students’ emergent 

literacy skills on two of the four IGDI measures developmentally appropriate for 3-year old students.46 As described in Chapter II, 

similar assessments were administered to the 3-year old PreK students as for 4- and 5-year old students, with the exception of 

letter sound fluency, which is not considered developmentally appropriate for 3-year old children. Also, the letter name 

assessment for 3-year old students was untimed.47

Our analysis was conducted by age, due to developmental differences between 3-year old PreK students and 4- and 5-year old 

PreK students. However, it is important to note the substantially smaller sample size for the analysis of 3-year old students’ 

emergent literacy outcomes compared to that for 4- and 5-year old students. Smaller sample sizes substantially reduce the 

power to detect statistically significant differences, and the study was not designed to conduct these analyses with sufficient 

power. The lower sample size for 3-year old students is due to the fact that most Minnesota Reading Corps sites had classrooms 

 

46 Letter sound fluency was not administered to 3-year old students. 

47 As mentioned previously, the 4- and 5- year old version of the letter name assessment was a fluency measure. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 36

that either only enrolled 4- and 5-year old students or a mix of 3-, 4-, and 5-year old students. Thus, these smaller sample sizes 

are one possible explanation for the lack of significance found on two of the four IGDI measures. 

Below we provide a discussion of the findings for 3-year old students, and Table IV.1 presents a summary of the data. As we did 

for the 4- and 5-year old findings above, several figures are provided, which show the average scores on the Fall, Winter, and 

Spring benchmarks for PreK students in Minnesota Reading Corps program and comparison classrooms. 

Table IV.2. Average Growth Among 3-Year Old PreK Students by Spring 

IGDI Assessment

Treatment 

Assignment N

Average 

Growth SE Effect Size Confidence Interval

Rhyming Fluency

Letter Name Recognition

Picture Name Fluency

Alliteration Fluency

Program+

Comparison

Difference

Program

Comparison

Difference

Program

Comparison

Difference

Program

Comparison

193

114

307

194

115

309

196

105

301

189

189

3.17***

0.86

2.30***

6.52***

5.11***

1.41

6.94***

3.43**

3.52**

1.96***

0.54

0.35

0.48

0.37

0.50

1.02

1.41

0.44

0.88

1.06

0.22

0.66

0.43

-

0.42

0.28

-

0.14

0.59

-

0.71

Difference 281 1.43 0.68 - - -

***<.001, **<.01, *<.05; +Minnesota Reading Corps 

What is the effect of the Minnesota Reading Corps program on 3-year old students’ rhyming fluency? 

The Minnesota Reading Corps program had a significant and large effect on 3-year old PreK students’ rhyming fluency scores 

between Fall benchmark (September 2013) and Spring benchmark (May 2014). As shown in Table IV.2 above, by the end of the 

school year, PreK students in Minnesota Reading Corps classrooms could identify an average of 2.3 more rhyming words 

correctly in a two minute period than PreK students in comparison classrooms. This represents a substantial effect size of about 

.43 standard deviations. Although we provide an analysis of students’ progress relative to the benchmark target scores for the 4- 

and 5-year old PreK students, due to the limited research on appropriate target scores for 3-year old students, we focused on 

growth only at that age level. However, it is worth noting in Table IV.2 above that the 3-year old students in Minnesota Reading 

Corps program classrooms demonstrated significant growth between Fall and Spring Benchmark while students in the 

comparison classrooms did not. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 37

Figure IV.6. Growth in Rhyming Fluency for 3-year old PreK program and comparison students 

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

Fall Winter Spring

Number of Words

Time of Benchmark

Program

Comparison

What is the effect of the Minnesota Reading Corps program on 3-year old students’ letter name 

recognition? 

Although large growth effects were found for 3-year old students in rhyming fluency, 3-year old PreK students in Minnesota 

Reading Corps classrooms did not attain significantly higher letter name recognition scores at Spring benchmark than did PreK 

students in comparison classrooms. By the end of the school year, the 3-year old Minnesota Reading Corps PreK students 

correctly identified an average of 1.4 more letters than PreK students in comparison classrooms. PreK students in both 

Minnesota Reading Corps program and comparison classrooms experienced significant growth between Fall and Spring 

benchmark in their average letter name recognition scores (6.5 and 5.1 letter names, respectively). Also, the average scores at 

Spring benchmark for 3-year old students in Minnesota Reading Corps program and comparison classrooms, shown in Figure 

IV.7, were similar (10.9 and 9.5, respectively). 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 38

Figure IV.7. Growth in Letter Name Recognition for 3-year old PreK program and comparison students

0

2

4

6

8

10

12

14

16

18

20

22

24

Fall Winter Spring

Number of Letter Names

Time of Benchmark

Program

Comparison

What is the effect of the Minnesota Reading Corps program on 3-year old students’ picture naming 

fluency?

Three year old PreK students in Minnesota Reading Corps classrooms attained significantly higher growth in picture naming 

fluency scores between Fall and Spring benchmark than did PreK students in comparison classrooms. The picture naming 

assessment measures child vocabulary and is an important indicator of future reading comprehension proficiency. As shown in 

Table IV.2, by the end of the school year, the 3-year old Minnesota Reading Corps program students correctly identified twice as 

many picture names as PreK students in comparison classrooms (6.9 and 3.4, respectively), resulting in a difference of 3.5 more 

pictures named correctly. Similar to rhyming fluency, this represents a substantial effect size of about .42 standard deviations. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 39

Figure IV.8. Growth in Picture Name Fluency for 3-year old PreK program and comparison students

0

2

4

6

8

10

12

14

16

18

20

22

24

26

28

30

Fall Winter Spring

Number of Pictures 

Time of Benchmark

Program

Comparison

What is the effect of the Minnesota Reading Corps program on 3-year old students’ alliteration fluency?

For the final outcome measure of alliteration fluency, 3-year old PreK students in Minnesota Reading Corps classrooms did not 

experience significantly higher growth in scores between Fall and Spring benchmarks than did PreK students in comparison 

classrooms. By the end of the school year, the 3-year old Minnesota Reading Corps PreK program students correctly identified 

an average of 1.4 more alliterative words than PreK students in comparison classrooms. Importantly, while the difference in 

growth between program and control sites was not significantly different, Table IV.2 above shows that the 3-year old students in 

Minnesota Reading Corps classrooms demonstrated significant growth from Fall to Spring Benchmark, whereas students in the 

comparison classrooms did not. Also, the average alliteration fluency score at Spring benchmark for 3-year old students in 

Minnesota Reading Corps classrooms, shown in Figure IV.7, is almost twice as high as the average for students in comparison 

classrooms (3.1 and 1.7, respectively). 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 40

Figure IV.9. Growth in Alliteration Fluency for 3-year old PreK program and comparison students

0

1

2

3

4

5

6

7

8

9

Fall Winter Spring

Number of Words

Time of Benchmark

Program

Comparison

Findings by Major Demographic Groups 

Our findings include a set of subgroup analyses of student study participants. The evaluation team identified subgroups for 

further examination based on the major demographic groups generally of interest in education research and also of interest to 

CNCS and the Minnesota Reading Corps program. We provide results by gender, race/ethnicity, and Dual Language Learner 

(DLL) status.48 Subgroups were defined based on characteristics provided at baseline and each level was determined by 

individual characteristics (e.g., male vs. female; White and non-White; DLL vs. non-DLL). 

Below the evaluation team provides two types of subgroup analysis. First, we added the identified moderators to the ANCOVA 

model and jointly tested the effects to determine if there are differential results across subgroups for 4- and 5-year old students, 

implying that the Minnesota Reading Corps PreK program may be more or less beneficial for selected subgroups of students.49

 

48We did not examine students by Free and Reduced Price Lunch (FRPL) status, which is a measure of socio-economic status, 

because these data are not often collected on preschool children; however, the majority of our sample attended schools with 

high percentages of children with FRPL status, so it is reasonable to assume that the majority of the children in our study would 

likely meet the eligibility criteria for FRPL. 

49 Due to small sample sizes, subgroup analyses were not conducted for 3-year old students. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 41

Results from the first set of subgroup statistical tests are provided in Table IV.3 below. The results show no statistically 

significant differences in the effect of the program for most subgroup variables across outcome measures, with the exception of 

DLL for picture naming. That is, the Minnesota Reading Corps PreK program is equally effective for both boys and girls, White 

and non-White students, and Dual Language Learners or English speaking students. One difference was found between nonDLL and DLL students for picture name fluency, where DLL students appear to have benefited from the program more so than 

non-DLL students on this outcome. 

Table IV.3. Moderator effects of subgroup variables on 4- and 5-year old PreK students by Spring 

IGDI Fluency Assessment Gender (Female)

Race/Ethnicity 

(Non-White) Dual Language Learner (DLL)

Letter Sounds 

Rhyming 

Letter Names

Picture Names

Alliteration

1.04

(N=511)

-0.16

(N=578)

-0.39

(N=587)

1.11

(N=580)

-0.69

(N=522)

0.01

(N=242)

1.07

(N=289)

2.52

(N=289)

2.26

(N=289)

-0.24

(N=241)

1.14

(N=127)

1.16

(N=147)

0.33

(N=148)

3.65**

(N=148)

-1.18

(N=128)

**<.01 

For the second subgroup analysis, we subdivided the 4- and 5-year old data by major subgroup (e.g., males, females, etc.), 

estimated the differences between the Minnesota Reading Corps program and comparison groups, and conducted significance 

testing to examine differences in patterns of findings by major subgroup. The results from this second subgroup analysis should 

be interpreted with caution, in particular when the results are not significant because sub-sectioning the data results in much 

smaller sample sizes than may be sufficient to detect differences between the program and comparison groups. As discussed 

earlier, smaller sample sizes substantially reduce the power to detect statistically significant differences, and the study was not 

designed to conduct these analyses with sufficient power. The results of both subgroup analyses are presented below. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 42

Gender 

Both types of subgroup analyses found no differences in the effects of the program for male and female students. The moderator 

test presented in Table IV.3 above showed no significant differences in effects by gender for 4- and 5-year old students across 

all five literacy measures (letter sound fluency; rhyming fluency, letter name fluency, picture name fluency, and alliteration 

fluency). When the sample was segmented by gender, significant findings persisted for 4- and 5-year old students for both males 

and females. Furthermore, Table IV.4 shows that the average difference between the program and comparison groups at Spring 

benchmark were relatively similar for males and females. Although differences between 4- and 5-year old students in Minnesota 

Reading Corps classrooms compared to students in comparison classrooms were slightly higher for female than male students 

in letter sound fluency and picture name fluency, most of the five outcome measures show similar amounts of growth in 

emergent literacy skills across both gender subgroups. 

Table IV.4. Average growth among male and female 4- and 5-year old PreK students by Spring 

IGDI Fluency 

Assessment

Treatment 

Assignment

Male Students Female Students

N Average Growth SE N

Average 

Growth SE

Letter Sounds

Program+

Comparison

7.54***

4.23***

0.33

0.63

8.95***

4.60***

0.44

0.78

Difference 583 3.31** 0.89 511 4.35*** 1.15

Program 6.83*** 0.32 7.77*** 0.33

Rhyming Comparison 3.22*** 0.50 4.32*** 0.35

Difference 641 3.62*** 0.71 578 3.45*** 0.59

Program 13.14*** 0.48 14.94*** 0.61

Letter Names Comparison 8.78*** 0.85 10.96*** 0.92

Difference 648 4.36** 1.20 587 3.98* 1.44

Program 8.58*** 0.37 8.79*** 0.00

Picture Names Comparison 5.09*** 0.50 4.20*** 0.48

Difference 642 3.49*** 0.71 580 4.59*** 0.66

Alliteration

Program

Comparison

4.97***

1.97***

0.29

0.50

5.78***

3.46***

0.31

0.53

Difference 588 3.00*** 0.71 522 2.32** 0.76

***<.001, **<.01, *<.05; +Minnesota Reading Corps 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 43

Race/Ethnicity 

Due to small sample sizes for individual racial and ethnic subgroups, we conducted our analysis by examining differences 

between White and non-White students, which allowed us to collapse all minority groups into one subgroup category. In our 

initial test, where we examined whether race/ethnicity is a significant moderator of program effects, we found that race/ethnicity 

was not a significant factor in explaining variations in all five of the student outcomes for 4- and 5-year old students. 

Consistent with this finding, few differences were found in the patterns of effects when we subgrouped the sample into White and 

non-White and compared these findings to the subgroup analysis of moderator effects. Table IV.5 shows significant average 

growth on four of the five outcome measures (all except letter sound fluency) among 4- and 5-year old students in Minnesota 

Reading Corps PreK classrooms compared to similar students in comparison classrooms, despite White or non-White subgroup 

membership. Despite a significant finding for White students and no significant finding for non-White students, the differences in 

actual growth in letter sound fluency between students in Minnesota Reading Corps classrooms and those in comparison 

classrooms is about the same among White and non-White students (3.5 in both subgroups). Therefore, the lack of significance 

in letter sound fluency among non-White students is most likely due to the smaller sample size for this subgroup compared to 

White students (N=242 and N=807, respectively). 

Although the findings were statistically significant for both White and non-White students on four out of the five outcomes 

measured, it is also worth noting that larger average differences in growth were found between non-White students in Minnesota 

Reading Corps program classrooms and comparison classrooms compared to White students in Minnesota Reading Corps 

program and comparison classrooms across most measures. Therefore, the Minnesota Reading Corps program appears to have 

produced greater gains among non-White students relative to White students. Differences in outcomes were particularly evident 

in picture name fluency, a critically important measure of child vocabulary and a strong predictor of future reading 

comprehension proficiency. Non-White 4- and 5-year old students in Minnesota Reading Corps program classrooms achieved a 

larger amount of growth in letter name fluency (5.7 more letter names among non-White students compared to 3.2 more letters 

among White students) and picture name fluency (5.9 more picture names among non-White students compared to 3.6 more 

pictures among White students) compared to White students. 

Table IV.5. Average growth among White and non-White 4- and 5-year old PreK students by Spring 

IGDI Fluency Treatment 

White Students Non-White Students

Average 

Assessment Assignment N Average Growth SE N Growth SE

Program+ 8.16*** 0.32 8.07*** 1.38

Letter Sounds Comparison 4.65*** 0.59 4.55** 1.73

Difference 807 3.51*** 0.87 242 3.52 2.60

Program 7.66*** 0.32 6.27*** 0.61

Rhyming Comparison 4.34*** 0.44 1.89* 0.71

Difference 874 3.32*** 0.70 289 4.39*** 0.99

Program 13.90*** 0.53 14.24*** 1.12
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 44

Letter Names Comparison 10.72*** 0.94 8.54*** 2.53

Difference 892 3.18* 1.40 289 5.70** 2.94

Program 9.00*** 0.32 7.85*** 0.96

Picture Names Comparison 5.40*** 0.50 2.00 0.94

Difference 877 3.59*** 0.75 289 5.85*** 1.37

Program 5.64*** 0.24 4.59*** 0.59

Alliteration Comparison 2.84*** 0.36 2.03** 1.09

Difference 823 2.80*** 0.55 241 2.56*** 1.48

***<.001, **<.01, *<.05; +Minnesota Reading Corps 

Dual Language Learners 

Some differences in the effects of the program were found in both subgroup analyses for DLL and non-DLL students. The 

moderator test showed no significant differences in effects by DLL status for 4- and 5-year old students in four of the five 

emergent literacy measures. Only one outcome measure, picture name fluency, was significant (p<.01).

When the student sample was divided into DLL and non-DLL, significant findings persisted for 4- and 5-year old students for both 

DLL and non-DLL on two of five measures. Table IV.6 shows that the average difference between the Minnesota Reading Corps 

program and comparison groups at Spring benchmark were significant for DLL and non-DLL students in rhyming and picture 

name fluency. The significance for picture name fluency may appear to contradict the finding from the first subgroup analysis, 

which indicates that DLL status is a significant moderator of program effects in picture naming. However, the substantial 

difference in size of the effect (difference) by subgroup (3.8 for non-DLL and 7.5 for DLL) are likely responsible for the significant 

finding in the moderator test. Growth in picture name fluency scores is particularly important for DLL students, as it is a measure 

of English vocabulary. 

Outcome measures in letter sound fluency, letter name fluency, and alliteration were significant for non-DLL students, but not for 

DLL students. However, on all three of these measures, students who attended Minnesota Reading Corps PreK sites were found 

to grow significantly more over the course of the year than their peers at comparison sites. Therefore, substantially smaller 

sample sizes for DLL students compared to non-DLL students may be a contributing factor to the lack of significance found on 

these measures. 

Table IV.6. Average growth among DLL and non-DLL 4- and 5-Year old PreK students by Spring 

IGDI Fluency Treatment 

Non-DLL Students DLL Students

Average 

Assessment Assignment N Average Growth SE N Growth SE

8.17*** 1.38

Program+ 8.27*** 0.30

Letter Sounds Comparison 4.54*** 0.56

3.3 1.73

4.87 2.60

Difference 967 3.73*** 0.83 127

6.05*** 0.61

Program 7.55*** 0.26
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 45

Rhyming

Letter Names

Picture Names

Alliteration

Comparison

Difference

Program

Comparison

Difference

Program

Comparison

Difference

Program

Comparison

Difference

1072

1087

1074

982

4.03***

3.52***

14.08***

9.94***

4.14**

8.94***

5.13***

3.81***

5.57***

2.71***

2.86***

0.36

0.57

0.49

0.78

1.20

0.31

0.43

0.69

0.18

0.29

0.43

147

148

148

128

1.37

4.68***

13.70***

9.22**

4.47

8.22***

0.76

7.46***

3.8***

2.11

1.68

0.71

0.99

1.12

2.53

2.94

0.96

0.94

1.37

0.59

1.09

1.48

***<.001, **<.01, *<.05; +Minnesota Reading Corps 

Site Level Effects 

An additional analysis of interest to CNCS is the examination of whether student assessment scores varied as a function of the 

type of site the student attended (i.e., public school or Head Start center).50

 We were only able to examine differences for public 

schools and Head Start programs because only one pair of schools in the sample (one program and one comparison school) 

was community-based. For this analysis, we used a similar approach as for the second subgroup analysis for student-level 

demographics, whereby we subdivided the 4- and 5-year old data by school type (public school or Head Start center), estimated 

the differences between program and comparison groups, and conducted significance testing. Finally, we examined whether 

student outcomes varied by site by calculating an intraclass correlation (ICC) across pairs of matched sites (program and 

comparison). 

Site Type 

As shown in Table IV.7, significant effects of the program were found regardless of the type of site where the Minnesota Reading 

Corps program was being implemented. Students attending Minnesota Reading Corps classrooms in both public schools and 

Head Start centers showed significantly higher growth across all five outcome measures than their peers in comparison 

classrooms. While most differences on individual outcome measures were similar, 4- and 5-year old students in Head Start 

centers tended to show greater growth in letter name fluency than average public school students in Minnesota Reading Corps 

classrooms (6.2 and 3.5 letter names, respectively) relative to their peers in comparison classrooms. That said, the standard 

error for the difference in letter name fluency between program and comparison classrooms was comparatively large, suggesting 

significant variability in this outcome across sites. 

Also of interest, students attending Minnesota Reading Corps Head Start Centers demonstrated significantly greater growth in 

letter sound fluency and alliteration fluency than peers at comparison Head Start Centers. Students at Minnesota Reading Corps 

 

50 It is important to note that students enrolled in Head Start centers must be eligible for Free and Reduced Price Lunch (FRPL), a measure of 

poverty; whereas, public school students vary in their eligibility for FRPL. As such, children attending Head Start programs may be qualitatively 

different than those enrolled in public school programs. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 46

Head Start program sites identified significantly more letter sounds by the end of the school year than at the beginning (7.07), 

whereas students at Head Start comparison sites did not identify significantly more letter sounds from the beginning of the school 

year (2.14). Likewise, students at Minnesota Reading Corps Head Start program sites identified significantly more alliterations 

by the end of the school year than at the beginning (4.78), whereas as students at Head Start comparison sites did not (1.13). 

These results show that the Minnesota Reading Corps program significantly improved growth in letter sound fluency and 

alliteration fluency at Head Start sites. 

Table IV.7. Average growth among 4 and 5-Year old PreK students at Public Schools and Head Start Centers by Spring 

IGDI Fluency 

Assessment

Treatment 

Assignment

Public School Head Start Center

N Average Growth SE N

Average 

Growth SE

Program+ 8.44*** 0.45 7.07*** 0.42

Letter Sounds Comparison 4.86*** 0.68 2.14 1.32

Difference 841 3.58** 1.13 211 4.93** 1.28

Program 7.40*** 0.31 7.14*** 0.3

Rhyming Comparison 3.66*** 0.39 3.79*** 0.56

Difference 919 3.74*** 0.67 248 3.35** 0.83

Program 14.07*** 0.58 14.5*** 0.85

Letter Names Comparison 10.61*** 0.89 8.32*** 1.27

Difference 937 3.47* 1.46 245 6.18* 2.11

Program 8.83*** 0.38 8.42*** 0.37

Picture Names Comparison 4.78*** 0.45 3.52** 0.61

Difference 921 4.05*** 0.82 249 4.91** 0.93

Program 5.46*** 0.22 4.78*** 0.26

Alliteration Comparison 3.03*** 0.29 1.13 0.88

Difference 855 2.43*** 0.49 211 3.65* 1.13

***<.001, **<.01, *<.05; +Minnesota Reading Corps 

Site Pair-Level ICCs 

An additional analysis of interest to CNCS is the examination of whether student differences in growth varied as a function of the 

site pair (program or comparison) they attended. We examined this question by calculating site pair-level intraclass correlations 

(ICC) using the differences in students’ Spring benchmark scores to determine whether the student outcome data indicated 

substantial variation in site pair-level effects. Intraclass correlations measure the portion of the total variation in an outcome that 

is associated with various levels of a variable, such as site assignment. In this case, the site pair-level ICC measures the degree 

to which students within the same site pair (program and comparison) correlate on their program effects (i.e., differences in 

growth). 

As shown in Table IV.8, the site pair-level ICCs are relatively low or not significant, indicating they are not meaningful. The ICCs 

for program effects vary by outcome measure, with the only statistically reliable ICCs found among 4- and 5-year old students in 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 47

letter sound fluency, picture name fluency, and alliteration fluency. However, the ICCs are relatively small at .06, .06, and .04, 

respectively. Therefore, it is possible to conclude that there is no significant variation in program effects across pairs of sites. 

Table IV.8. Site pair intraclass correlations for Spring-Fall differences 

3-year old students 4- and 5-year old students

IGDI Fluency Assessment

(N= 286) (N= 1,480 )

ICC SE ICC SE

Letter Sounds - - 0.055* 0.022

Rhyming Fluency 0.011 0.025 0.026 0.013

Letter Names 0.001 0.017 0.083 0.029

Picture Names 0.047 0.036 0.060* 0.023

Alliteration 0.046 0.036 0.038* 0.017
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 48

Conclusions

The findings from the Minnesota Reading Corps PreK outcome evaluation provide important evidence for addressing the studies’ 

key research questions (presented in Chapter III). Below, the evaluation team offers our conclusions based on these findings and 

organizes them by major research questions. Following our assessment of the questions is a discussion on the implications of 

our findings for the Minnesota Reading Corps program. 

1. What is the effect of the Minnesota Reading Corps program on student literacy outcomes?

The results of our analysis showed that preschool students who attended Minnesota Reading Corps classrooms achieved 

significantly higher emergent literacy assessment scores by the end of the school year than did students who did not attend a 

Minnesota Reading Corps classroom (see Chapter IV for more detail on these findings). The Minnesota Reading Corps PreK 

program had statistically significant and large effects across all five IGDI outcome measures of emergent literacy skills for 4- and 

5-year old students and on two of the four IGDI measures appropriate for assessing 3-year old students. 

The Minnesota Reading Corps PreK program had statistically significant effects across all five IGDI outcome measures, letter 

sound and letter name fluency (alphabet knowledge), rhyming fluency and alliteration fluency (phonological awareness), and

picture name fluency (vocabulary), for 4- and 5-year old students between Fall benchmark (September 2013) and Spring 

benchmark (May 2014) . In addition, substantial effect sizes were found on each measure, ranging from .40 to .72, indicating that 

the Minnesota Reading Corps program has a comprehensive, positive effect on students’ emergent literacy skills. 

Despite the substantially smaller sample size for 3-year old students, the Minnesota Reading Corps program was found to have 

a significant and substantial effect on 3-year old PreK students’ rhyming fluency (phonological awareness) and picture name 

fluency (vocabulary) scores between Fall and Spring benchmark. Although growth was also found for 3-year old students on the 

letter name recognition (alphabet knowledge) and alliteration fluency (phonological awareness) outcome measures, 3-year old 

PreK students in Minnesota Reading Corps classrooms did not attain significantly higher letter name recognition and alliteration 

fluency scores at Spring benchmark than PreK students in comparison classrooms. The lack of statistical significance in letter 

name recognition is likely due to the fact that both the program and comparison PreK students experienced significant growth in 

their average letter name recognition scores. Letter name recognition the most widely emphasized emergent literacy skills in 

early childhood education. As such, it was not surprising to observe equally significant growth at both the Minnesota Reading 

Corps program and comparison sites. 

With respect to alliteration, 3-year old PreK students in neither Minnesota Reading Corps program classrooms nor comparison 

classrooms experienced significant growth in their scores between Fall and Spring benchmarks. Alliteration is typically a later 

developing phonological awareness skill, one which teachers may wait to engage with until children master precursor skills such 

as rhyming. The fact that large, statistically significant effects were found in alliteration fluency among 4- and 5- year old children 

at Minnesota Reading Corps sites strongly suggests that the program can effectively influence this skill but that it most 

developmentally appropriate for children older than 3-years of age. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 49

Overall, the results of the student outcome analysis revealed that the Minnesota Reading Corps PreK Program is achieving its 

goal of creating Kindergarten–ready 4- and 5-year old students, as measured against the IGDI Spring benchmarks. Of particular 

interest is the large, significant effect that the Minnesota Reading Corps program produces in the growth in children’s oral 

language, specifically vocabulary as measured by the picture name fluency measure. Vocabulary is one of the strongest 

predictors of later reading comprehension. The Minnesota Reading Corps PreK program’s attention to not only decoding skills 

(i.e., letter name recognition and letter sound correspondence) but also oral language skills is a key program feature and likely a 

primary reason for the demonstrated effectiveness of the program. As witnessed during our Phase II PreK Process 

Assessment,51 the program’s emphasis on intentionally integrating decoding and oral language skills in all aspects of daily 

interactions, as well as whole class, small group, and individualized instruction, results in numerous opportunities for children to 

connect text and language with personally meaningful content and experiences. This contextualized language and literacy 

instruction, as best evidenced by extended adult-child discourse and rich dialogic reading sessions, is a proven method for 

growing children’s emergent literacy skills.

2. Does the effect on literacy outcomes vary by student characteristics/demographics?

After establishing the overall positive effect of the Minnesota Reading Corps program on preschool students’ emergent literacy 

outcomes, we conducted analyses to examine whether differential effects of the program existed for specific subgroups of 4- and 

5-year old students based on the following student characteristics: gender (male/female), race/ethnicity (White/non-White), and 

Dual Language Learner (DLL) status (yes/no). For the most part, students from all subgroups examined produced significant 

growth in emergent literacy outcomes regardless of gender, race/ethnicity, or DLL status. In the instances where differential 

growth rates were observed, students who attended Minnesota Reading Corps PreK sites were often found to grow significantly 

more over the course of the year than their peers at comparison sites. Of note, students typically at risk for not being 

Kindergarten-ready, non-Whites and Dual Language Learners, posted significantly greater growth in vocabulary, as measured by 

picture name fluency, at Minnesota Reading Corps sites than at comparison sites. In fact, the vocabulary of non-White students 

and DLL students at comparisons sites did not grow significantly from Fall to Spring. Given the importance of vocabulary as a 

powerful predictor of later reading comprehension proficiency, the significant growth among non-White and DLL students at 

Minnesota Reading Corps PreK sites is a promising finding. It implies that the program elements that together comprise the 

Minnesota Reading Corps PreK program have a significant and meaningful impact on a critical skill (vocabulary) at program sites 

that is otherwise unaffected at comparison sites. Thus, two key conclusions for the evaluation are that the PreK program appears 

to have a positive effect on all students’ emergent literacy skills and that the program may be particularly effective in improving 

non-White and DLL students’ vocabulary skills.

3. Do effects on literacy outcomes vary by site/school?

The Minnesota Reading Corps program appears to be consistently effective no matter the type of school setting in which the 

program occurs, either public school or Head Start center. Students enrolled at both types of PreK programs evidenced greater 

 

51 Hafford, C., Markovitz, C., Hernandez, M.W., Diaconis, A., Estrera, E., & Muyskens, P. (2014). Process Assessment of the Minnesota 

Reading Corps PreK Program: Phase II Extension. (Prepared under contract to the Corporation for National and Community Service). Chicago, 

IL: NORC at the University of Chicago. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 50

growth in emergent literacy outcomes at Minnesota Reading Corps program sites than comparison sites. Of note, students at 

Minnesota Reading Corps Head Start centers achieved significantly greater growth on letter sound fluency and alliteration 

fluency than their peers at comparison site Head Start centers who did not improve on these measures during the school year. 

These results suggest that not only is the Minnesota Reading Corps PreK program effective in both public schools and Head 

Start centers, but that the program may be particularly effective in developing some emergent literacy skills for students enrolled 

in Head Start centers. 

The ICC findings indicate that the PreK program is effective across the individual sites where the students received Minnesota 

Reading Corps services. The low levels of variability in student outcome scores corroborate the high levels of fidelity of program 

implementation we observed in the Phase II Process Assessment.52 The Minnesota Reading Corps program’s high quality 

training and program resources, use of validated formative assessments (i.e., IGDIs) to inform instruction, multi-layered 

supervisory structure, ongoing professional development and coaching sessions, and clear, objective method for measuring 

fidelity (i.e., ELLCO) are key program features that have led to such low levels of variability. These results support the conclusion 

that the Minnesota Reading Corps program is replicable in multiple school settings.

Program Implications from Conclusions 

Overall, the results of this study show that the Minnesota Reading Corps PreK program is an effective model for improving 

preschool students’ emergent literacy skills. Specifically, 4- and 5-year old students in Minnesota Reading Corps classrooms 

demonstrated significantly higher growth in their emergent literacy skills compared to their peers in comparison classroom across 

all five literacy measures. Furthermore, 3-year old students showed significantly higher growth on two of the four IGDI measures 

used to assess younger students. For the most part, the program was equally effective for all students and did not vary by 

gender, race, or DLL status. Likewise, for the most part, no major differences by site location or school type (public school or 

Head Start center) were found. In instances where differences were found, they favored at-risk students (i.e., non-White 

students, Dual Language Learners, Head Start centers). As such, the results show that the program is both effective and highly 

replicable in multiple settings with different types of students. 

The Minnesota Reading Corps PreK program model combines the human capital from at least 9 months of AmeriCorps service 

with evidence-based program elements, including a RtI approach to differentiated instruction, regular formative assessment, the 

Literacy Rich Schedule, the SEEDS approach to high quality adult-child interactions, and intensive small-group and one-to-one 

intervention with Tier 2 and 3 students. 

The Minnesota Reading Corps PreK program’s comprehensive nature, including both whole class instructional strategies and 

differentiated interventions, results in class-wide impacts. The coordination between the two forms of instruction is likely a key 

source of the accelerated growth we observed in student emergent literacy outcomes. As outlined in the Phase II Process 

 

52 Hafford, C., Markovitz, C., Hernandez, M.W., Diaconis, A., Estrera, E., & Muyskens, P. (2014). Process Assessment of the Minnesota 

Reading Corps PreK Program: Phase II Extension. (Prepared under contract to the Corporation for National and Community Service). Chicago, 

IL: NORC at the University of Chicago. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 51

Assessment,53 the Minnesota Reading Corps program provides a common infrastructure for coordinated, intentional instruction 

through the Literacy Rich Schedule, a common pedagogy for interacting with PreK students via SEEDS, and proven strategies 

for embedding language and literacy instruction at times during the school day that are often overlooked (e.g., arriving at the 

beginning of the day, lining up for bathroom breaks or the bus, transitioning between circle time and choice time). Because the 

Minnesota Reading Corps PreK program is comprehensive, its success requires a strong commitment from site administrators 

and lead teachers. Furthermore, the program is most effective when all adults in a classroom coordinate their practice; therefore, 

it is critical that the lead teacher understand and fully commit to the Minnesota Reading Corps PreK model. 

The Minnesota Reading Corps program’s use of a multi-layered supervisor structure, including site-based Internal Coaches and 

program-provided Master Coaches, effectively supports the implementation of such a comprehensive program. The PreK 

program’s focus on objective methods for monitoring both student and AmeriCorps member performance (i.e., IGDI assessments 

and ELLCO tool, respectively) supports fidelity of program implementation and further enforces a strong commitment to continual 

improvement and professional development throughout the year. In sum, the complete Minnesota Reading Corps PreK program, 

implemented with fidelity, is both highly replicable and effective in improving diverse PreK students’ emergent literacy outcomes. 

Recommendations for Future Research 

Future research should more deeply examine differences in student outcomes across the model used at various PreK sites. 

Differences in the type and mix of staff used to support a PreK program are both interesting and relevant to assisting in bringing 

the Reading Corps program to scale. Whether the site includes Educator Corps only, Community Corps only, or a mix of the two 

has potential effects on student outcomes that could not be fully explored in this analysis. 

Future research should also explore the degree to which the impact of the Reading Corps model is seen in PreK sites that are 

relatively new to the Reading Corps (i.e., 1-3 years), versus sites that have been in the Reading Corps for a longer period time 

(i.e., 4+ years). The Reading Corps program provides additional intensity of service to recently established sites, and it might be 

helpful to the program if they could better understand at what point this additional intensity may no longer be needed, so 

resources can be more efficiently allocated to higher needs sites. 

One difficulty in completing this study was the degree of saturation of the PreK Reading Corps model and SEEDS training across 

PreK sites in Minnesota. Potential comparison sites were challenging to identify, especially in the only urban area in the state 

(i.e., Minneapolis-St. Paul). When potentially good candidate were identified, they were often found to have been “contaminated” 

because the site staff had previously received SEEDS training, a foundational component of the Reading Corps model. While the 

widespread dissemination of the effective SEEDS training is beneficial for Minnesota preschoolers, it was a significant hurdle in 

identifying comparison sites for the current study. Future research could replicate this study design in other states, where the 

presence of the Reading Corps PreK program and prevalence of SEEDS training is less widespread. This would allow for a 

potentially larger sample size, permitting additional points of inquiry to explore questions such as the impact of the program in 

 

53 Hafford, C., Markovitz, C., Hernandez, M.W., Diaconis, A., Estrera, E., & Muyskens, P. (2014). Process Assessment of the Minnesota 

Reading Corps PreK Program: Phase II Extension. (Prepared under contract to the Corporation for National and Community Service). Chicago, 

IL: NORC at the University of Chicago. 
The Corporation for National and Community Service | 2015

OUTCOME EVALUATION OF THE MINNESOTA READING CORPS PREK PROGRAM Page 52

year one, two, and three separately (versus later years), differences in effects across staffing models, the program’s effect 

across a broader range of student demographics, and the effect of the program model in states with less rigorous standards for 

early childhood education.