First 5 Service Corps Project Evaluation 

First 5 Santa Cruz County

Reading Corps Members Make a Difference

davisconsultantnetwork

Francesca Wright, MPA

Sheri Hembree, PhD

Jerome Braun, PhD

January 2017
2 

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Acknowledgements 

The Davis Consultant Network would not have been able to develop this report without the vision, 

leadership, data collection, and cooperation of the following individuals. We thank each of you for your 

cooperation, service, and deep commitment to young children. 

Agency 

First 5 Association of 

California 

Moira Kenney Executive Director 

Prevent Child Abuse 

California 

Sheila Boxley President and CEO 

Stephanie Biegler Chief Program Officer 

Sara Fung Director – AmeriCorps 

Ian Hadley AmeriCorps Statewide Senior Program Manager 

Summer Rolfe-Irish First 5 Service Corps Project Manager 

First 5 Santa Cruz David Brody Executive Director 

Vicki Boriack Senior Program Officer 

Irene Freiberg Master Literacy Coach 

Christine Sieburg Santa Cruz Reading Corps Manager 

Jeanne Carriere Literacy Coaches 

Leah Rogers 

Phoebe Boyd 

Santa Cruz Reading 

Corps Members

2015-2017 

Theodora Alexander, Chelsey Anderson, Olivia Arstein-Kerslake, Maribel 

Bravo, Jessica Brym, Elizabeth Cann, Michael Collins, Rayne Delos Santos, Fin 

Finely, Myrsha Flores, Abigail Grattidge, Marilyn Guerra, Casandra Guillen, 

Teran Henderson Bonds, Whitney Heuschkel, Caitlin Klein, Devika Kumar, 

Christina Martin, Laura McRae, Rachel Paul, Ayelen Paulson, Whitney Perry, 

Vista Pickett, Ross Platts, Alexandra Robinson, Margaret Ruiz, Pedro Sierra, 

Megan Stettner, Alicia Stewart, Sean Todd, Joelle Vann, Sasha Voight, 

Delphine Wartelle, and Esdras Zarazua. 
3 

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Contents 

Acknowledgements...............................................................................................................................2

Introduction ..........................................................................................................................................4

Research Design ....................................................................................................................................5

Program Description .............................................................................................................................6

Data Collection ......................................................................................................................................8

2015-2016 School Year Study .............................................................................................................10

2016-2017 School Year Study .............................................................................................................17

Conclusions .........................................................................................................................................23

Appendix 1: Logic Model.....................................................................................................................24

Appendix 2: Evaluation Plan ...............................................................................................................25
4 

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Introduction 

In 1998, voters passed Proposition 10 which levied a tax on tobacco products to fund programs for 

California’s children prenatal through age 5 years and their families. In 2001, First 5 California, the First 5 

Association, CaliforniaVolunteers (the State Commission for the Corporation for National and 

Community Service (CNCS)), and Prevent Child Abuse California (PCA CA) formed a partnership to 

implement a statewide AmeriCorps Initiative. Since 2002, the First 5 Service Corps (F5SC) has been 

deploying AmeriCorps members to help children enter kindergarten developmentally ready to learn and 

better-prepared to succeed in school. PCA CA administers F5SC program and provides technical 

assistance for successful placement of AmeriCorps members into school readiness programs run by 

county level First 5 Commissions throughout the state. 

CNCS has provided leadership in furthering collection of evaluative data and the production of 

increasingly rigorous evaluations. The request for F5SC evaluation services for the period of 2012 

through 2014, sought to narrow the scope to programs that had already collected data suitable to 

conduct a quasi-experimental design (QED) as defined by the What Works Clearinghouse (WWC) as 

follows: 

1. They collect valid and reliable outcome measures;

2. They train and monitor for standardized measurement procedures;

3. There is a comparison group with baseline equivalency; and

4. There are limited attrition rates of target and comparison groups.

The Davis Consultant Network screened two programs for that QED study, the Think Together program 

in Orange County and the Reading Corps program in Santa Cruz. A full QED study was performed for 

Think Together1. The Santa Cruz Reading Corp program demonstrated significant positive change in 

literacy scores for participating children, but there were no data collected from a comparable group, 

which prevented analysis using QED. 

In this subsequent evaluative cycle, staff from First 5 Santa Cruz recognized an opportunity to meet the 

qualifications for QED design due to an AmeriCorps member placement challenge. One of its program 

sites was fully prepared to host two Reading Corps AmeriCorps members who for personal reasons had 

to drop out of the program. The staff of Starlight State Preschool agreed to allow collection of the early 

literacy assessment data, despite having no Reading Corps members placed on their teaching team. This 

provided the program with a comparison group, thus making a QED design possible in studying the 

impact of placing Reading Corps members within high-quality literacy-rich preschool programs.

1https://www.nationalservice.gov/sites/default/files/evidenceexchange/FR_CaliforniaChildrenandFamiliesFoundat

ion_First5ServiceCorps.pdf 
5 

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Research Design 

First 5 Santa Cruz was interested in a very specific evaluation question: How does the delivery of 

targeted early literacy interventions by trained Reading Corps members impact reading readiness in 

high-quality preschool classrooms? 

The DCN research team was familiar with the Reading Corps program from the 2014 AmeriCorps 

evaluation. First 5 Santa Cruz County staff provided information on all programmatic changes since 

2014, which were incorporated into the updated program logic model found in Appendix 1. First 5 Santa 

Cruz County hosted a site visit for the DCN research team on February 10, 2017. The visit provided an 

opportunity to confirm and clarify programmatic and data issues, as well as an opportunity to observe 

classrooms with and without Reading Corps members. While at the YWCA State Preschool program, the 

team observed all three tiers of interventions and two Reading Corps members conducting assessments. 

The evaluation methods included key informant interviews with program staff, classroom observations, 

a review of classroom quality indicators for all sites included in the study, and a quasi-experimental 

design (QED) study. Early literacy was assessed in the Reading Corps classrooms by Reading Corps 

members using the Individual Growth and Developmental Indicators of Early Literacy (IGDI). A Reading 

Corp Literacy Coach performed the IGDI assessments at the control sites. 

IGDI scores were collected and provided by First 5 Santa Cruz County for two program years, 2015-2016 

and 2016-2017. Scores from each program year were analyzed separately in 5 steps: 

1. Total Sample - The entire sample demographics were summarized for child gender, ethnicity,

home language and age.

2. Comparison of Group Demographics - The control and comparison group demographics were

reported and compared between groups. T-test and chi-square analyses was performed to

determine whether the differences were significant. Additionally, a Fisher exact test was

performed for the 2015-2016 sample.

3. Pre/Post Scores - Paired t-tests of pre/post assessment scores were performed separately for

control and comparison groups to demonstrate gains for each IGDI measure over the 2015-2016

school year. This was done only for the intervention sample with data collected in the 2006-

2017 school year.

4. Group Comparison - Independent t-tests for each group were performed at each assessment

and differences between mean scores reported. This provided a comparison at baseline when

the children entered the preschool program, as well as the subsequent two assessment periods.

A univariate linear model was used to analyze group differences for 2016-17 IGDI scores.

5. Differences of Differences - Finally, independent t-tests were performed to assess differences in

the changes in in mean early literacy scores from fall to winter and from fall to spring between

students in classrooms with and without a Reading Corps member.
6 

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Program Description 

First 5 Santa Cruz initiated the Santa Cruz Reading Corps program in 2012. The program deploys trained 

Reading Corps AmeriCorps members, to State preschool and transitional kindergarten classrooms to 

engage in early literacy activities throughout the daily routine. 

The lead teachers have all been trained to use the SEEDS of Early Learning2strategies within a literacyand math-rich daily schedule. SEEDS of Early Learning is an evidence based professional development 

program that prepares educators to help children develop the social, emotional, language and literacy 

skills they need to be ready for kindergarten. Their classrooms are assessed twice per year using the 

Early Language and Literacy Classroom Observation Tool (ELLCO) to identify the environment’s early 

literacy strengths and areas for program improvement. 

SEEDS is an acronym for the five qualities of teacher/ student interaction: 

1. to be Sensitive

2. to be Encouraging;

3. to Educate;

4. to “Develop through doing” by promoting

active learning; and

5. to help develop a positive Self-image.

The program also trains “the big 5 early literacy 

skills”: 

1. conversation and oral language;

2. phonological awareness;

3. books and print rules;

4. alphabet knowledge; and

5. vocabulary and meaning of words.

Early childhood professionals who have taken SEEDS of 

Early Learning are trained to assess and create classrooms 

that offer many opportunities for developing literacy skills 

and integration of literacy-rich exchanges with their 

students throughout the day. 

Classrooms that have teachers that have been SEEDS 

trained also use the Raising a Reader3 book bag program to 

ensure that all children have access to appropriate 

children’s literature at home. Raising a Reader is another 

evidence-based program that bridges at-home and atschool readiness activities. 

2

 http://www.seeds-learning.com/ 

3

 http://www.raisingareader.org

Minnesota Reading Corps Program Model 
7 

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

The Santa Cruz Reading Corps is a national Reading Corps replication partner, which grew out of the 

Minnesota Reading Corps model. This model utilizes individual assessments to monitor each child’s 

early literacy skills in the predictors of reading success (picture naming, letter names, letter sounds, 

rhyming, and alliteration). 

The children with the lowest benchmark scores receive additional support from the Reading Corps 

member in the form of an individualized daily intervention. This can happen in a small group (Tier 2) or 

one-on-one (Tier 3). The length of the intervention session is dependent on the strategy being used by 

the Reading Corps member. 

The most common intervention used in Tier 1 and Tier 2 is the Repeated Read Aloud. In this intervention 

members utilize intervention scripts to increase vocabulary in the context of a story which is repeated 

daily. Each day three new words are introduced, repeated, used in context, defined, highlighted in a 

book, and discussed. As children learn the words, new words are introduced in subsequent days. 

Each Reading Corps member receives over 

200 hours of training, ongoing supervision 

and coaching throughout their twelve 

months of service. Each member is also 

supported by a Literacy Coach, contracted 

with First 5 Santa Cruz County. The Literacy 

Coach observes each Reading Corps member 

twice a month as they engage in the Literacy 

and Math Rich Schedule. These coaching 

sessions include feedback on the strategy 

observed as well as goal setting. First 5 Santa 

Cruz County provides overall program 

support with a Program Manager who 

handles administrative issues and a Master 

Literacy Coach who provides training and 

ongoing content support. 

During coaching sessions, Reading Corps 

members are observed conducting multiple 

interventions (including Tier 2 and Tier 3) and 

are scored for fidelity to the model and 

coached as needed. Similarly, coaches 

address interrater reliability between Reading 

Corps members by observing and scoring their 

implementation of assessments, providing 

coaching until they achieve the standards. 

Tier 3: 5-15% will 

need intensive 

individual plan 

with ongoing 

monitoring

Tier 2: 20-25% will need 

interventions through 

small groups with 

ongoing monitoring 

Tier 1: 70-80% of the children will make 

adequate progress to key benchmarks 

of learning when teachers use researchbased curriculum, embedded and 

explicit teaching, and universal 

screening

Figure 1: Reading Corps is a Response to Intervention (RTI) 

model which uses periodic assessment to determine which 

tier of intervention is most appropriate. 
8 

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Data Collection 

Sample Sites 

Only the First 5 Santa Cruz Reading 

Corps sites with half-day preschool 

programs serving 4- and 5-year old 

children were selected for the study to

have comparability in preschool 

program dosage. All lead teachers held 

site supervision permits and had been 

trained in the SEEDS of Learning. All 

sites also participated in the Raising a 

Reader book bag program. The target 

and comparison sites were all state 

preschool programs Data from eleven 

program sites were included in this 

study. 

The Starlight State Preschool program 

did not participate in the Reading 

Corps program in either the 2015-2016 

or 2016-17 school years but provided a 

similar literacy-rich preschool program 

without any Reading Corps members. 

The children at this site served as a 

comparison group. They were assessed by the Master Literacy Coach using the IGDIs three times in the 

2015-16 school year and in the winter and spring only in the 2016-17 school year. Data from both 

school years were separately analyzed and compared with assessment scores from sites with with 

Reading Corps members. 

In the 2015-2016 program year, the literacy environments of all sites in this study, including the two 

control group classrooms at Starlight State Preschool were assessed. These were performed by the 

Literacy Coach using the Early Language and Literacy Classroom Observation Tool (ELLCO) (Smith, 

Dickenson, et al, 2002) in spring 2016. Aggregate average scores ranged from 3.8 to 4.9 with a mean of 

4.3. The two comparison Starlight 

classrooms achieved scores above the 

mean in their spring assessment, at 4.7 

and 4.4. Since there was continuity of 

staffing, classrooms, and curriculum, 

the two Starlight Classrooms were 

deemed to have a comparable literacy

environment as the other classrooms 

in the study as measured by the 

ELLCO. 

Site Name

Site Lead 

Teacher 

 Target Site  Reading Corps Members  Comparison Site  Spring ELLCO Score  Site Supervision Permit  SEEDS Trained 

1. Bradley State Preschool   4.2 ✓ ✓

2. Calabasas State Preschool   4.1 ✓ ✓

3. Del Mar State Preschool   5.0 ✓ ✓

4. Green Acres State Preschool   4.9 ✓ ✓

5. Landmark State Preschool   3.8 ✓ ✓

6. Linscott State Preschool   3.2 ✓ ✓

7. Live Oak State Preschool   4.5 ✓ ✓

8. Rolling Hills State Preschool   4.0 ✓ ✓

9. Salsipuedes State Preschool   4.4 ✓ ✓

10. Watsonville State Preschool   3.8 ✓ ✓

11. Starlight 1 State Preschool  4.7 ✓ ✓

 Starlight 2 State Preschool  4.4 ✓ ✓

Mean ELLCO Score 4.3

ELLCO Assessment Domains

Organization of Classroom Phonological Awareness 

Contents of Classroom Organization of Book Area 

Classroom Management Characteristics of Books 

Personnel Books for Learning 

Approaches to Curriculum Approaches to Book Reading 

Opportunities for Child Choice & Initiative Quality of Book Reading 

Recognizing Diversity in the Classroom Early Writing Environment 

Discourse Climate Support Children's Writing 

Efforts to Build Vocabulary Environmental Print 

Opportunities for Extended Conversations 
9 

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Early Literacy Measures 

The measures for early literacy used in this study are those assessed using the Individual Growth and 

Developmental Indicators of Early Literacy (IGDIs). The assessments are performed by Reading Corps 

members who have received an initial full-day training, as well as up to 18 hours of practice using the 

assessment. Reading Corps literacy coaches observes each member using an integrity checklist to insure 

fidelity to the tool. 

The IGDIs assess early literacy in five domains: 

1. alliteration (phonological awareness);

2. letter names;

3. picture naming (oral language);

4. letter sounds/sound identification (alphabet knowledge); and

5. rhyming (phonological awareness).

The IGDIs assessment tool was developed by faculty from the University of Minnesota in the late 1990s 

based upon ten years of research in measurement procedures. The tool has been empirically evaluated 

for reliability and validity and is considered psychometrically sound4. 

4 Missal, Kristen and S. McConnell. Psychometric Characteristics of Individual Growth & Development Indicators: 

Picture Naming, Rhyming, and Alliteration. University of Minnesota, April 2004 

https://pdfs.semanticscholar.org/45b0/d05ce608ef2b0556ce8acf4d8f55b4f1d409.pdf 
10

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

2015-2016 School Year Study 

Total Sample Demographics 

There were 472 total students attending 11 different State Preschool classroom sites in 19 different 

sessions. The total sample was 53.6% female, 90.5% Hispanic/Latino, and 46.8% Spanish-speaking, with 

a mean age of 50.3 months at the start of the school year. Scores from one child whose age was greater 

than 65 months at the stat of the school year were disqualified from being included in the study. 

Frequency Percent 

Child Gender

female 253 53.6 

male 219 46.4 

Total 472 100.0 

Latino/Hispanic 

no 45 9.5 

yes 427 90.5 

Total 472 100.0 

Ethnicity

Asian 4 .8

Hispanic/Latino 427 90.5 

Multi-Racial 6 1.3 

Native Hawaiian/Pacific Islander 3 .06 

White 32 6.8 

Total 472 100.0 

Home Language 

English 245 51.9 

Spanish 221 46.8 

other non-English 6 1.3 

Total 472 100.0 

Mean (SD) Minimum Maximum 

Age in Months 50.32 (5.68) 37 68
11

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Group Comparisons 

A quasi-experimental design was employed to determine if Reading Corps member support resulted in 

increased literacy readiness. The children enrolled in the Starlight classrooms that shared a similar 

learning environment without the support of Reading Corps members were identified as the control 

group. This group (n=67), were then compared on all available literacy scores to the intervention group 

(n=404) of children in classrooms with Reading Corps members. 

The control and intervention group demographics were compared using Fisher exact test and twosample t-test to examine equivalency between groups. Tests show no significant differences between 

classrooms with a Reading Corps member and the control group classroom in gender and race. 

However, a large percentage of the control group children (64.2%) came from households where 

Spanish was the primary language spoken at home than in the intervention group (44.1%). This is likely 

associated with the observed difference in ethnicity in the intervention group (89.1% Hispanic versus 

98.5% in the control group). In addition, the control group was about 3 months older on the average 

than the intervention group. Attendance data were not available for the 2015-2016 sample. 

Control Group Demographics 

Demographics - Control Group Only (N=79) 

Age N Minimum Maximum Mean Std. Dev 

Fall age in months 67 39 68 53.01 5.86 

Demographic Variable Frequency Percent 

Gender

male 29 43.3 

female 38 56.7 

Total 67 100.0 

Latino/Hispanic 

no 1 1.5 

yes 66 98.5 

Total 67 100.0 

Ethnicity 

Multi-Racial 1 1.5 

Hispanic/Latino 66 98.5 

Total 67 100.0 

Home Language 

English 24 35.8 

Spanish 43 64.2 

Total 67 100.0 
12

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Intervention Group Demographics 

Demographics - Reading Corps Member Group Only (N=461) 

Age N Minimum Maximum Mean Std. Deviation 

Fall age in months 404 37.00 60.00 49.86 5.54

Demographic Variable Frequency Percent

Gender 

female 215 53.2 

male 189 46.8 

Total 404 100.0 

Latino/Hispanic 

no 44 10.9 

yes 360 89.1 

Total 404 100.0 

Ethnicity 

Asian 4 1.0 

Hispanic/Latino 360 89.1 

Multi-Racial 5 1.2 

Native Hawaiian/ Pacific Islander 3 .7

White 32 7.9 

404 100.0 

Home Language 

English 220 54.5 

Spanish 178 44.1 

other non-English 6 1.5 

Total 404 100.0 
13

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Pre/Post Early Literacy Comparison within Groups 

To address whether the children in the either program showed improved early literacy over the 

preschool program year, pre-post (paired t-test) comparisons were performed for fall to winter and fall 

to spring scores on the five literacy scales: Alliteration, Letter Sounds, Letter Names, Rhyming, and 

Picture Naming for both groups. 

Control Group 

In the control group, children’s scores increased in Alliteration, and Letter Names (p<.05) between fall 

and winter assessments and between Alliteration, Letter Names and Picture Naming between fall and 

spring with medium to large effect size (Cohen’s d score). Out of a total of 67 possible pairs, the 

percentage of missing pairs ranged from 1.5% to 67.2% for the five outcome variables, and missing data 

may have affected these results.

Control Group IDGI Outcome Composites 

Measure Assessment Mean SD

Included Missing 

n

# % # % 

Alliteration Fall 0.16 0.81 63 94.0 4 6.0 67

Winter 1.44 2.38 25 37.3 42 62.7 67

Spring 2.50 3.15 24 35.8 43 64.2 67

Letter Names Fall 3.50 5.91 54 80.6 13 19.4 67

Winter 4.64 6.91 28 41.8 39 58.2 67

Spring 6.00 7.07 25 37.3 42 62.7 67

Letter Sounds Fall 0.83 2.41 52 77.6 15 22.4 67

Winter 1.08 2.64 25 37.3 42 62.7 67

Spring 1.52 3.65 25 37.3 42 62.7 67

Picture Names Fall 6.40 7.81 65 97.0 2 3.0 67

Winter 8.32 7.77 28 41.8 39 58.2 67

Spring 10.76 9.11 25 37.3 42 62.7 67

Rhyming Fall 1.52 3.33 66 98.5 1 1.5 67

Winter 0.23 0.75 22 32.8 45 67.2 67

Spring 2.30 4.25 23 34.3 44 65.7 67

Control Group Pre-Post Paired T-Tests 

IGDI Measure 

Fall to Winter Fall to Spring 

t-value p n Cohen’s d* missing t p n Cohen’s d* missing 

Alliteration 2.478 0.0214 23 0.52 63 3.132 0.0052 21 0.68 67

Letter Names 3.382 0.0023 27 0.65 50 4.518 0.0002 24 0.92 56

Letter Sounds 0.485 0.6330 22 0.10 58 1.162 0.2582 22 0.25 58

Picture Names 1.918 0.0662 27 0.37 58 3.939 0.0007 24 0.80 63

Rhyming -2.163 0.0429 21 -0.47 68 1.244 0.2272 22 0.27 67

*Cohen’s d scores from .5 to .8 are considered to have medium effects. Scores of .8 or more are considered to have a large 

effect size.
14

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Intervention Group 

In the intervention group, children’s scores increased in all five measures (p<.05) between fall and 

winter assessments and between fall and spring assessments. The gains over the course of the school 

year were statistical significant (p<.0001) and large to very large (Cohen’s D >.8). Out of a total of 404 

possible pairs, the percentage of missing pairs ranged from 7.2% to 47.8% for the five outcome 

variables. 

Intervention Group IDGI Outcome Composites 

Measure Assessment Mean SD

Included Missing 

n 

# % # % 

Alliteration Fall 1.27 2.40 335 82.9 69 17.1 404

Winter 2.25 3.63 370 91.6 34 8.4 404

Spring 4.33 4.43 343 84.9 61 15.1 404

Letter Names Fall 4.23 7.63 226 55.9 178 44.1 404

Winter 8.30 9.36 258 63.9 146 36.1 404

Spring 14.31 11.81 215 53.2 189 46.8 404

Letter Sounds Fall 1.82 4.14 221 54.7 183 45.3 404

Winter 4.68 6.51 238 58.9 166 41.1 404

Spring 8.64 8.35 211 52.2 193 47.8 404

Picture Names Fall 8.90 6.91 320 79.2 84 20.8 404

Winter 13.51 8.14 375 92.8 29 7.2 404

Spring 19.64 8.37 347 85.9 57 14.1 404

Rhyming Fall 1.68 3.27 331 81.9 73 18.1 404

Winter 3.46 4.66 369 91.3 35 8.7 404

Spring 5.97 6.06 344 85.1 60 14.9 404

Intervention Group Pre-Post Paired T-Tests 

IGDI Measure 

Fall to Winter Fall to Spring 

t-value p n Cohen’s D Missing t-value p n Cohen’s D Missing 

Alliteration 5.542 0* 317 0.31 5 12.611 0* 289 0.74 14

Letter Names 8.221 0* 211 0.57 7 15.142 0* 195 1.08 14

Letter Sounds 8.766 0* 208 0.61 6 12.906 0* 185 0.95 16

Picture Names 17.132 0* 305 0.98 5 30.618 0* 281 1.83 12

Rhyming 8.655 0* 312 0.49 6 13.730 0* 286 0.81 14

*All p-values were less than .0001.
15

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Group Comparisons 

The control and intervention group mean scores were compared. Independent t-tests showed 

significantly higher baseline scores (fall) in the group with Reading Corps members in the three areas of 

Alliteration, Letter Sounds, and Picture Names. 

There was greater equivalency in baseline scores for Letter Names and Rhyming at the initial assessment 

in the fall. In these two measures, significantly greater gains were seen in the intervention group in both 

the winter and spring assessments. 

Direct Comparisons of Mean Scores Across Groups 

Measure Assessment Reading 

Corps Control Difference t Value 

p 

Value 

Degrees of 

Freedom 

Cohen’s 

d 

Alliteration Fall 1.27 0.16 1.11 6.664 0.0000 291 0.50 

Winter 2.25 1.44 0.81 1.583 0.1232 32 0.23 

Spring 4.33 2.50 1.83 2.664 0.0124 30 0.42 

Letter 

Names 

Fall 4.23 3.50 0.73 0.773 0.4416 100 0.10 

Winter 8.30 4.64 3.66 2.555 0.0147 39 0.40 

Spring 14.31 6.00 8.31 5.107 0.0000 42 0.73 

Letter 

Sounds 

Fall 1.82 0.83 0.99 2.282 0.0241 132 0.26 

Winter 4.68 1.08 3.60 5.320 0.0000 62 0.58

Spring 8.64 1.52 7.12 7.654 0.0000 60 0.89 

Picture 

Names 

Fall 8.90 6.40 2.50 2.394 0.0188 86 0.35 

Winter 13.51 8.32 5.19 3.400 0.0018 32 0.64 

Spring 19.64 10.76 8.88 4.731 0.0001 27 1.05 

Rhyming Fall 1.68 1.52 0.17 0.375 0.7088 92 0.05 

Winter 3.46 0.23 3.23 11.117 0.0000 175 0.71 

Spring 5.97 2.30 3.66 3.878 0.0006 28 0.61 
16

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Change Scores 

The change in mean scores were compared between the two groups, revealing significantly higher gains 

in Letter Naming, Letter Sounds, Picture Naming, and Rhyming in the Reading Corps member group as 

compared to the control group. The effect sizes were large to very large for both fall to winter and even 

greater from fall to spring in these four measures. 

Fall to Winter Mean Change Score Comparison 

IGDI Measure Reading Corps Control Group Difference t p df Cohen’s 

d 

Alliteration 1.23 1.39 -0.16 -0.272 0.7875 29 -0.06

Letter Names 4.15 1.48 2.67 3.994 0.0001 116 0.82

Letter Sounds 3.00 0.14 2.86 6.458 0.0000 105 1.46

Picture Names 5.26 1.93 3.33 3.171 0.0034 31 0.64

Rhyming 1.96 -1.90 3.87 4.254 0.0003 23 0.96

Fall to Spring Mean Change Score Comparison 

IGDI Measure Reading Corps Control Group Difference t p df Cohen’s 

d 

Alliteration 3.27 2.14 1.13 1.545 0.1344 26 0.35 

Letter Names 9.98 3.29 6.69 6.808 0.0000 70 1.48

Letter Sounds 6.74 0.50 6.24 9.223 0.0000 103 2.10

Picture Names 11.03 4.92 6.12 4.708 0.0001 27 1.00

Rhyming 4.34 0.68 3.66 5.785 0.0000 37 1.28
17

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

2016-2017 School Year Study 

Sample Demographics 

There were 462 total students attending 11 different State Preschool sites in 21 different sessions. The 

total sample was evenly divided by gender, 92% Hispanic/Latino, and 64% Spanish-speaking, with a 

mean age of 49.8 months at the start of the year. 

Frequency Percent 

Child Gender

male 231 50.0 

female 231 50.0 

Total 462 100.0

Latino/Hispanic 

no 35 7.6 

yes 427 92.4 

Total 462 100.0

Ethnicity 

White 31 6.7 

Hispanic/Latino 427 92.4 

American Indian 1 .2

Multi-Racial 3 .6

Total 462 100.0

Home Language 

English 163 35.3 

Spanish 297 64.3 

other non-English 2 .4

Total 462 100.0

Mean (SD) Minimum Maximum 

Age in Months 49.80 (5.99) 35.00 60.00 
18

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Pre/Post Literacy Comparison for Intervention Sample 

To address whether the children in the program showed improvement over time, pre-post (paired ttest) comparisons were performed from fall to spring scores on the five literacy scales: Alliteration, 

Letter Sounds, Letter Names, Rhyming, and Picture Naming for the entire sample. Results indicate that 

there were significantly (p<.001) higher scores in spring for all literacy scores, with effect size (Cohen’s d) 

scores ranging from .62 to 1.27, which can be considered as medium or large effects. Out of a total of 

383 possible pairs, the percentage of missing pairs ranged from 26% to 53% for the five outcome 

variables, and missing data may have affected these results, especially for Letter Sounds. 

Pre-Post Comparisons Fall to Spring 

IGDI Measure Assessment Mean N SD t Effect Size 

(Cohen’s d)

Letter Naming Spring 13.84 281 12.36 18.50 .827 

Fall 4.35 281 8.33 (large) 

Letter Sounds Spring 9.05 179 9.50 11.98 .620 

Fall 2.34 179 5.28 (medium) 

Picture 

Naming 

Spring 20.96 282 8.48 27.21 1.27 

Fall 10.45 282 8.06 (large)

Rhyming Spring 6.34 280 6.08 14.34 .863

Fall 1.80 280 3.40 (large) 

Alliteration Spring 4.11 279 4.52 11.58 .773 

Fall 1.17 279 2.38 (medium)

Effects are significant p<.001 
19

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Group Comparisons 

A quasi-experimental design was employed to determine if having Reading Corps members support 

resulted in increased literacy readiness. Once again, the Starlight classroom children were identified as 

the control group, who had a similar learning environment, but who were without the support of 

Reading Corps members. This group (n=79), were then compared to the remaining group (n=383) with 

Reading Corps members on winter and spring literacy scores. No fall scores were collected at the 

control site. 

Chi-square and t-tests were performed to examine equivalency between groups. Tests show no 

significant differences between intervention and control group samples in age, ethnicity, gender, home 

language. However, the control group children attended their preschool program significantly more 

program days (M =155.57) than the group of children receiving Reading Corps member interventions 

(M= 132.59). t (428) =4.008, p<.001). 

Demographics - Control Group Only (N=79) 

Age N Minimum Maximum Mean Std. Dev 

Fall age in months 79 37.00 60.00 51.00 6.387 

Demographic Variable Frequency Percent 

Gender

male 40 50.6 

female 39 49.4 

Total 79 100.0 

Latino/Hispanic 

no 4 5.1 

yes 75 94.9 

Total 79 100.0 

Ethnicity 

White 4 5.1 

Hispanic/Latino 75 94.9 

Total 79 100.0 

Home Language 

English 32 40.5 

Spanish 47 59.5 

Total 79 100.0 
20

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Demographics - Reading Corps Member Group Only (N=461) 

Age N Minimum Maximum Mean Std. Deviation 

Fall age in months 462 35.00 60.00 49.80 5.99 

Demographic Variable Frequency Percent

Gender 

male 191 49.9 

female 192 50.1 

Total 383 100.0 

Latino/Hispanic 

no 31 8.1 

yes 352 91.9 

Total 383 100.0 

Ethnicity 

White 27 7.0 

Hispanic/Latino 352 91.9 

American Indian 1 .3

Multi-Racial 3 .8

Total 383 100.0 

Home Language 

English 131 34.2 

Spanish 250 65.3 

other non-English 2 .5

Total 383 100.0 
21

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Pre/Post Early Literacy Comparisons, Group Comparison and Change Scores 

Only winter and spring assessments were performed in the control group, allowing for comparison at 

only these time points. Independent t tests showed significantly higher scores in the group with Reading 

Corps members for spring scores for Alliteration, Letter Naming, Letter Sounds, and Picture Naming. 

There were also significantly higher winter scores in the Reading Corps member group in Letter Naming, 

Letter Sounds and Picture Naming. 

Because fall scores were missing, it was not possible to assess a true (fall) baseline in a quasiexperimental design. If winter is used as the baseline, it is difficult to determine whether group 

differences in spring are attributable to initial differences in fall, or to differences associated with 3 

months of program attendance. In these cases, change/gain scores are sometimes used to adjust for 

those initial differences. Change scores were therefore compared between the two groups, revealing 

significantly higher gains in Letter Naming, Picture Naming, and Letter Sounds in the Reading Corps 

member group as compared to the Control (no member) group. 

Groups N Mean 

Std. 

Deviation 

Literacy Scores 

Alliteration - Winter Control 74 1.50 2.48 

Reading Corps 348 2.09 3.40 

Alliteration - Spring Control 75 2.44 3.52 

Reading Corps 341 3.78 4.50 

Letter Naming - Winter Control 78 4.29 7.87 

Reading Corps 348 8.06 11.09 

Letter Naming - Spring Control 74 6.77 8.74 

Reading Corps 342 12.61 12.03 

Letter Sounds - Winter Control 78 1.12 2.35 

Reading Corps 221 4.38 6.87 

Letter Sounds - Spring Control 66 2.00 5.04 

Reading Corps 214 8.12 9.10 

Picture Naming - Winter Control 78 11.44 8.63 

Reading Corps 350 14.70 9.36 

Picture Naming - Spring Control 77 13.64 9.34 

Reading Corps 344 19.82 9.37 

Rhyming - Winter Control 78 2.08 3.93 

Reading Corps 350 3.21 5.08 

Rhyming - Spring Control 74 4.62 4.64 

Reading Corps 342 5.82 6.15 
22

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Groups N Mean Std. Deviation 

Change Scores

Change Alliteration 

(Winter to Spring) 

Control 70 1.0857 3.57423 

Reading Corps 319 1.6897 3.83720 

Change Letter Naming 

(Winter to Spring) 

Control 73 2.7945 5.43384 

Reading Corps 320 4.3875 6.23670 

Change Picture Naming 

(Winter to Spring) 

Control 76 2.5000 6.27907 

Reading Corps 323 4.9319 5.65534 

Change Rhyme 

(Winter to Spring) 

Control 73 2.6986 4.19220 

Reading Corps 321 2.6947 4.89007 

Change Letter Sounds 

(Winter to Spring) 

Control 65 .9538 4.15869 

Reading Corps 201 3.8308 5.28500 

T Test Comparisons

t df Significant Findings 

Literacy Scores

Alliteration - Winter -1.406 ns 420

Alliteration - Spring -2.426* 414 Reading Corps Group > Control 

Letter Naming - Winter -2.841** 424 Reading Corps Group > Control 

Letter Naming - Spring -3.954*** 414 Reading Corps Group > Control 

Letter Sounds - Winter -4.111*** 297 Reading Corps Group > Control 

Letter Sounds - Spring -5.224*** 278 Reading Corps Group > Control 

Picture Naming - Winter -2.821** 426 Reading Corps Group > Control 

Picture Naming - Spring -5.240*** 419 Reading Corps Group > Control

Rhyming - Winter -1.844ns 426

Rhyming - Spring -1.580ns 414

Change Scores 

Change Alliteration -1.207ns 387

Change Letter Naming -2.014* 391 Reading Corps Group > Control 

Change Picture Naming -3.301** 397 Reading Corps Group > Control 

Change Rhyme .006 ns 392

Change Letter Sounds -4.004*** 264 Reading Corps Group > Control 

* p <.05; **p<.01; ***p<.001
23

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Conclusions

This multi-year study demonstrates that targeted early literacy interventions by trained First 5 Reading 

Corps members contribute to gains in early literacy skills for children in high quality preschools. 

Consistently over two program-years, within a 4-5-month period, children in preschool programs with 

Reading Corps members demonstrated greater gains than children in comparison classrooms in three 

emergent literacy skills: letter names, letter sounds and picture names. Data from the first year of the 

study found this to be true for rhyming as well. This is important because these skills help children 

decode words and develop their vocabulary.

The Santa Cruz Reading Corps program serves lower income families that meet the household income 

thresholds to qualify for California State Preschool. Over half of the population studied came from 

families where Spanish was the primary home language.5 These findings of this study suggest that the 

Reading Corps model can contribute to improved early literacy for a large segment of California’s early 

learners, our English learner population.

The comparison between control and intervention groups strongly suggests that the inclusion of Reading 

Corps members improves overall early literacy scores of preschool children. The study separately 

analyzed early literacy assessment scores from two program years using a quasi-experimental design.

The 2015-2016 program-year control and intervention groups were similar in gender and race. The 

control group had a greater percentage of Spanish speakers and were on the average 3 months older 

than the average age in the intervention sites. Despite a large percentage of missing control group 

assessment data, there was evidence of gains in the control group in alliteration, letter names and 

picture names over the course of the program-year. Strong pre-post gains in all five measures were 

found for children in the Reading Corps classrooms. There was a lack of baseline equivalency in 

alliteration, letter sounds, and picture naming between the two groups rendering direct mean score 

comparison of spring scores between the control and intervention groups invalid. In letter names and 

picture names there was baseline equivalency and mean scores in the spring were significantly greater 

in Reading Corps classrooms than the control site. When gains were compared between the two 

groups, overall, students from Reading Corps classrooms saw significantly greater gains in letter names, 

letter sounds, picture names and rhyming.

The 2016-2017 program-year control and intervention groups were equivalent in age, gender, ethnicity 

and home language. Control group children attended more school days than the intervention group. In 

the available 2016-17 sample there were no true baseline (fall) scores of the control group to assess 

whether there were pre-existing skill differences in the groups at the start of the program year. The 

mid-year (winter) assessments did not show equivalency between the groups by that point in time. 

While children in the Reading Corps member group were significantly better at winter, the lack of 

baseline equivalency renders direct comparisons of spring scores between the control and intervention 

groups invalid. The analysis of the change scores did reveal significant gains in some of the scores but 

this is not a perfect solution to the problem of lack of initial equivalency. This lack of baseline 

equivalency is a methodological weakness in the study of 2016-2017 data. 

Together, the findings from two distinct program years builds a strong case that the Reading Corps 

program is an effective model for building specific early literacy skills in high quality preschools.

5

Statistical comparison of gains by home language will be performed by February 2018.
Appendix 1: Program Logic Model 

24

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

6 There may be multiple teachers in a classroom 

7

 Plus 2 additional members who left in the winter were not replaced 

8

 One session equals one cohort of children 

9

 Lead Teachers signed Lead Teacher Agreements with the Santa Cruz Reading Corps

Appendix 1: Logic Model - Santa Cruz Reading Corps Program Years 2015-17

Resources Activities Outputs Outcomes

First 5 Staff 

▪ $460,000 total contribution (F5SCC

staff, contacted literacy coaches,20

members living allowance, all

trainings & materials, Reading Corps

fee of $18,000

▪ 1 FTE First 5 Santa Cruz Reading

Corps Manager (Christine Sieburg)

▪ .60 FTE SEEDS Master Coach

(Irene Freiberg)

F5SC: AmeriCorps member recruitment, screening, interviewing, enrollment, 

coordination of trainings and all program expectations with PCA, Reading 

Corps and program partners. 

Reading Corps Member Training/ Professional Development/ Data Driven 

Decision-making meetings from various providers. ~ 170 hours 

- 4-day AmeriCorps Orientation (Sept) & Active Citizenship Training (March),

and Life After AmeriCorps (April)

- 3-day Reading Corps Institute (Sept)

- 2 hr. Raising a Reader Orientation (Sept)

- 4 hr. Santa Cruz Reading Corps Orientation

- 8 hr. SEEDS Training: Response to Intervention (October)

- 8 hr. SEEDS Training: Progress Monitoring (November)

- 4 hr. “Challenging Behaviors” training by First 5

- 4 hr. “Child behavior management” trained by First 5

- 24 hr. SEEDS of Early Learning sessions (Sept-Feb.)

- 2 hour/month “Monthly Meetups” with RC members (Oct-May)

- Approximately 18 hours of IGDIs assessment practice

- Data team meetings twice a year

- 8 hours PCA Leadership training (Spring)

Reading Corps Tutors are embedded in preschool and TK sessions with 

hours ranging from 3-6 hours of instruction per session per day, 5 days per 

week. They provide: 

- daily whole classroom SEEDS of Early Learning literacy strategies in

English (Tier 1)

- small groups (Tier 2) instruction 5-10 minutes daily

- RTI intervention strategies for 5-7 children in small groups (Tier 2) or oneon-one (Tier 3).

- 0ne-on-one tutoring (Tier 3) 3-5 minutes daily

- assess all students using IGDI and FAST 3x/yr.

- monitor progress of students identified for Tier 2 or 3 monthly, and enter 

progress into online data management tool (edSpring/ RCDMS)

28 Lead Teachers6integrate Reading Corps program expectations and 

SEEDS of Early Learning Strategies into the literacy- and math-rich schedule in 

coordination with literacy tutors, coaches, First 5 staff and district 

administration. They review and address Early Language and Literacy 

Classroom Observation Tool (ELLCO) standards. 

2 Literacy Coaches (each coach supports10 tutors) 

- participate in trainings, meet-ups, and data review team meetings;

- conduct Pre- and Post-ELLCO assessment in the fall and spring for each

classroom

- Observe assigned classrooms 2x/mo.

- Provide tutor coaching 2x/mo. (total 6-9 hours for each tutor)

- Check tutors’ integrity on all assessments, literacy strategies and

interventions.

Master Coach 

- Provides SEEDS of Early Learning training

- Facilitates data review with literacy coaches, lead teachers, and tutors

2x/year for instructional decisions.

- Observes and coaches Literacy coaches and tutors monthly

- Enters tutor integrity checks into an online database system (Quickbase)

All children receive Raising a Reader books to take home weekly.

187 trained 

AmeriCorps 

members each 

engaged in a 

minimum of 900 

hours of literacy 

tutor activities 

(Sept-May) 

33 Preschool 

and TK 

Sessions with 

enriched 

literacy 

environments 

and activities 

demonstrated 

by ELLCO 

ratings on 19 

measures 

~700 children

per year 

received high 

quality 

preschool and 

TK experiences 

▪ attended at

least 3 hours

in literacy

enriched

program and

engaged in

Tier 

1instruction

▪ assessed

with

Individual

Growth &

Development

Indicators(IG

DI) and

FAST

assessments

3x/year

▪ 5 -7 children

per session

receive Tier 

2&3

interventions

daily.

Increased 

early literacy 

skills as 

measured by 

the IGDI, FAST 

in: 

▪ conversation

skills

▪ vocabulary

▪ book and

print rules

▪ visual

discriminatio

n 

▪ phonological 

awareness 

▪ early writing

& alphabetic 

knowledge 

2 Literacy Coaches 

Contracted through First 5 Santa Cruz 

County

AmeriCorps Members 

20 AmeriCorps members serving as 

Reading Corps members (32-35 

hours/week) 

900 service hours in a 9-month period 

(September 1- May 31) 

5 School District / Program Partners 

• Pajaro Valley School District, 13

classrooms/ 18 sessions8

• Live Oak School District, 6

classrooms/ 9 sessions

• YWCA 2 classrooms/ 4 sessions

• Gault Elementary, 1 classroom/ 1

session

• Community Bridges Child

Development, 1 classroom / 1

session

Lead Teachers9 in classrooms were 

SEEDS –trained: 

-PreK teachers all with Site Supervision

Permit.

-TK teachers, all with Multi-Subject

Credential

• Reading Corps Replication pre-k

coaching & assessment model

• SEEDS of Early Learning

o Big 5 Early Literacy skills

(conversation &oral language, 

alphabet knowledge, book & print 

skills, phonological awareness, 

vocabulary & background knowledge.)

o Relationship-based (sensitive, 

encouraging, educational, develop by 

doing/ modeling, self-esteem)

• Raising a Reader (RAR) book bag

program
Appendix 2: Evaluation Plan 

25

Santa Cruz Reading Corps Members Make a Difference davisconsultantnetwork

Appendix 2: Evaluation Plan

Outcome Indicators Data Collection 

Methods & Tools 

Frequency/Schedule of Data 

Collection Analytical Methods 

Increased 

Early Literacy 

1. Picture

naming

2. Rhyming

3. Alliteration

Letter 

naming

4. Letter 

sounds

IGDI 

All children in the 11 qualifying 1⁄2 day 

programs classes 3x/year; 

Tier 2 and Tier children every 4 weeks. 

First benchmark assessment last week in 

Sept. or first week in Oct. 

Verification of group comparability by age, primary language, ethnicity, 

gender. Use t-test and chi-square analyses to determine whether the 

differences are significant. Account for any significant site differences 

for these variables in subsequent analyses. A univariate linear model 

will be used to analyze group differences for IGDI and FAST scores. A 

multivariate linear model will be used to further account for significant 

site differences as determined by preliminary analyses, as well as 

potential interactions of group response with the covariates. 

Comparability 

of Sites 

All site teachers 

have been 

SEEDs trained. 

Sites attain 

average ELLCO 

Scores of 3.4 or 

greater 

ELLCO Fall and Spring Each Year Descriptive statistics of site ELLCO scores.