California Children and Families Foundation 

15AC170571

First 5 Service Corps Project Evaluation

2012-2014
 


 

Orange County Children and Families Commission 

THINK Together Program...................................Pages 3 through 29

First 5 Santa Cruz County Commission
 

Reading Corps Program....................................Pages 31 through 59
 

DavisConsultantNetwork 

November 2014 
 

California Children and Families Foundation 

15AC170571

2 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

 

California Children and Families Foundation 

15AC170571

First 5 Service Corps Project Evaluation 

Orange County THINK Together Preschool Program 

DavisConsultantNetwork 

Francesca Wright, MPA 

Sheri Hembree, PhD 

Jerome Braun, PhD 

November 2014 
 

California Children and Families Foundation 

15AC170571

Acknowledgements 

The Davis Consultant Network would not have been able to develop this report without the 

vision, leadership, data collection, and cooperation of the following individuals. We thank each 

of you for your cooperation, service, and deep commitment to young children. 

4 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 

Agency 

First 5 Association / Moira Kenney Executive Director 

California Children and 

Families Foundation 

Prevent Child Abuse Sheila Boxley President and CEO 

Stephanie Biegler Chief Program Officer 

Sara Fung Program Manager – AmeriCorps 

Initiatives 

Ian Hadley First 5 Service Corps Project Manager 

Children and Families Christina Altmeyer Executive Director 

Commission of Orange Kim Goll Strategy and Operations Director 

County Laura Arevalo Contract Administrator 

Muckenthaler & Ass oc, Inc Cinda Muckenthaler CFCOC Program Consultant. 

THINK Together Tiffany Alva Early Literacy & Math Director 

Rochelle Santacruz Early Learning Program Manager 

Josh Dorman Evaluation Manager 

THINK Together/PCA CA Ivonne Abuebeid 2013-2014 AmeriCorps Early Literacy 

and Math Instructors at THINK 

Together 

Valentino Blancarte 

Teresa Bonilla 

Emerald Bouldin Burton 

Edith Brito 

Emily Chu 

Alyssa Manalo 

Michael Martel 

Sydney McNamara 

Angelina Nguyen 

Carmen Reynaga 

Christina Ruiz 

Carlos Salazar 

Martina Som 

Ana Torres 

Justin Tracy 

Stephanie Zuniga 
 

 

 

 

 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Introduction 

In 1998, voters passed Proposition 10 which levied a tax on tobacco products in order to fund 

programs for California’s children prenatal through age 5 years and their families. In 2001, First 

5 California, the First 5 Association, which represents the 58 County First 5 Commissions, 

CaliforniaVolunteers, the State Commission for the Corporation for National and Community 

Service (CNCS), and Prevent Child Abuse California (PCA CA) formed a partnership to 

implement a statewide AmeriCorps Initiative. First 5 Service Corps (F5SC) first received CNCS 

funding in 2002 to engage AmeriCorps members to help children enter kindergarten 

developmentally ready to learn and better-prepared to succeed in school. PCA CA administers 

F5SC and provides technical assistance for successful placement of AmeriCorps into school 

readiness programs being offered by 13 county First 5 Commissions. 

The request for F5SC evaluation services for the period of 2012 through 2014, sought to narrow 

the scope to programs that had already collected data suitable for quasi-experimental design 

(QED) as defined by the What Works Clearinghouse (WWC) as follows: 

1. They collect valid and reliable outcome measures; 

2. They train and monitor for standardized measurement procedures; 

3. There is a comparison group with baseline equivalency; and 

4. There are limited attrition rates of target and comparison groups. 

Two researchers from the Davis Consultant Network visited the THINK Together program site 

at Marjorie Veeh Elementary School where they observed the program in session; interviewed 

program staff, AmeriCorps members. They also met with the Children and Families 

Commission of Orange County evaluation and program administrators; and asked questions 

about the program models and assessment methods. From this visit and follow-up calls, 

program logic models and evaluation plans were drafted, reviewed and finalized with program 

staff. Data sets were received analyzed and serve as the basis of this report. 

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 5 | P a g e 
 

 

 

 

 

California Children and Families Foundation 

15AC170571

THINK Together Outcome Evaluation 

Program Description 

The THINK Together preschool program has been 

designed to increase reading, math and social emotional 

readiness for children with no prior preschool experience. 

Families whose children will be enrolling in pre-K or 

kindergarten the following year in Buena Park, 

Huntington Beach City, Santa Ana or Tustin School 

Districts are eligible to apply to the THINK Together (TT) 

early learning Program. Priority is given to children with 

no prior preschool experience and those residing in lower 

income elementary school catchment areas. 

 

 

 

Table 1: The THINK Together preschool program was designed to draw from elementary catchment 

areas from district schools with lower income student populations, demonstrated by eligibility for free 

or reduced lunch. 

% Free or Reduced Lunch 

School District Target Elementary Schools Target School District 

Buena Park Carl E. Gilbert 94.8% 73.9 % 

Huntington Beach City Joseph R. Perry 52.1% 18.2% 

Santa Ana Hoover 92.2% 88.1% 

Monroe 94.3% 

Jose Sepulveda 92.1% 

Tustin Marjorie Veeh 76.2% 40.7% 

The THINK Together Preschool model includes three complementary evidence-based 

programs. This includes a 15-week High Scope preschool program, with the Raising a Reader 

take-home book bag program, and a 14-week Incredible Years parent education program. 

Children and one parent enroll in a 15-week session which meets twice a week for 3 hours. 

Classroom teachers are trained to deliver a high-quality preschool experience. Each lead 

teacher has a Bachelor’s degree in child development or equivalency, a minimum of 3 years prek teaching experience, and is trained in the High Scope Preschool approach. 

The lead teachers supervise 4-6 AmeriCorps members who: 

participate in activity planning and preparation; lead small 

group activities; help manage free choice-time activities; 

and participate in daily team reflection. Each AmeriCorps 

volunteer has 19 days of AmeriCorps member training 

from various providers. They participate in the 5-day 

AmeriCorps Orientation at the start of their service year. 

This includes 6 hours of training in how to administer the 

6 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

 

 

 

California Children and Families Foundation 

15AC170571

GOLD Assessment for school readiness which is used in this study. The THINK Together 

program provides 4 days of program orientation, a two-hour Raising a Reader training, 8 days 

of High Scope training, and other professional development activities. 

Classrooms have a very high 1:4 adult-child ratio, with 1 lead teacher, 5-6 AmeriCorps 

members. Additionally, for 1 hour of each session, parents are in the room observing and 

joining in activities with their child. They leave to another room for 2 hour of Incredible Years 

parent education. 

Evaluation Design & Tools 

The evaluation design sought to measure comprehensive program impacts using existing data 

collection tools and procedures. The study examined the impact of the program participation 

on the following three school readiness outcomes: literacy skills, numeracy skills, and socioemotional skills. 

The child outcomes analysis is supplemented with AmeriCorps member perceptions of their 

service experience. These findings can be found in Appendix IV. 

Children were administered two types of tests of school readiness. The Teaching Strategies 

GOLD1 is an early learning school-readiness assessment. It is administered at the beginning 

and end of each program year. The Phonological Awareness Literacy Screening (PALS) 

assesses 8 areas of pre-literacy. It is administered at the end of the session. These tests were 

individually administered by trained AmeriCorps volunteers. 

Evaluation Summary 

A full description of evaluation methods may be found in Appendix I, Data Analysis Overview. 

Three different analytical approaches were taken to measure school readiness: a simple 

attainment of a benchmark score; matched pre/post; and quasi-experimental design. All three 

demonstrated that participation in the 15-week THINK Together program positively impacted 

readiness. 

PALS Benchmarks

A simple benchmark for pre-literacy readiness was measured by PALS in the last week of the 

program. The majority of 4-year olds were assessed as “on track” in all 8 pre-literacy skill areas. 

1 http://teachingstrategies.com/assessment/ 

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 7 | P a g e 
 

 

 

 

 

California Children and Families Foundation 

15AC170571

Pre-Literacy Readiness 

Figure 1: Percent of children assessed as “On Track” using the PALS pre-literacy assessment 

Nursery Rhyme Awareness 

Name Writing 

Upper Case Alphabet Recognition 

Lower Case Alphabet Recognition 

Print and Word Awareness 

Beginning Sound Awareness 

Rhyme Awareness 

Letter Sounds 

84% 

76% 

64% 

63% 

58% 

57% 

53% 

52% 

Pre/Post Teaching Strategies GOLD 

All children were assessed twice with Teaching Strategies GOLD. 

Early Literacy 

Five early literacy skills (phonological knowledge, alphabet knowledge, print knowledge, prereading skills, pre-writing skill, and an aggregate literacy score) were assessed using GOLD. 

Significant gains were found in all area using paired t-tests at p<.001. 

Pre- Post- Literacy Gains 

Figure 2: Changes in literacy from entry to completion of 15-week program, using GOLD assessment. 

 

 

TOTAL LITERACY 

Pre-writing skills 

Pre-reading skills 

Print awareness 

Alphabet knowledge 

Phonological knowledge 

Pre 

Gains 

0 1 2 3 4 

Numeracy 

Two numeracy skills (special concepts and number concepts) were assessed using GOLD. 

Significant gains were found in all area using paired t-tests at p<.001. 

8 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Pre/Post Numeracy Gains 

Figure 3: Changes in numeracy from entry to completion of 15-week program, using GOLD 

assessment. 

TOTAL NUMERACY 

Spatial Concepts Pre 

Number Concepts Gain 

0 2 4 6 8 

Social Emotional 

Three social-emotional skills (cooperation with others, positive relationships, and selfregulation) were assessed using GOLD. Significant gains were found in all area using paired ttests at p<.001. 

Pre/Post Socio-Emotional Gains 

Figure 4: Changes in socio-emotional skills from entry to completion of 15-week program, using 

GOLD assessment.
 

TOTAL SOCIO-EMOTIONAL 

Cooperates with Others Pre 

Positive Relationships Gain Self-Regulation 

0 1 2 3 4 5 

Quasi-Experimental Design (QED) 

While pre-post comparisons revealed significant improvements in GOLD scores over the course 

of the program session, a QED comparison provides a more rigorous test of program 

effectiveness. Because there were two sessions of the program, we were able to approximate 

such a comparison by comparing the cohort of children in Session 1 at the end of their session 

(post scores) with cohort of children in Session 2 at the beginning of their session (pre scores). 

Comparability of Comparison Groups 

The two groups were analyzed using t-test and Chi-square analyses to determine comparability. 

This revealed that the groups were not significantly different on: gender, ethnicity, language, 

maternal education level, family income, or parental marital status. However, children without 

program experience (Session 2, pre) were somewhat younger (.3 year difference of mean age) 

than children with program experience (Session 1, post). This finding required that analytical 

methods account for child age. 

GOLD Scores were then examined for significant associations with demographic variables. 

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 9 | P a g e 
 

 

 

 

California Children and Families Foundation 

15AC170571

Analysis revealed that child age, gender, mother’s education, primary language were predictors 

of test scores. This finding required that these variables be addressed in analyses of program 

effects. 

Analyses Accounting for Covariates 

Two types of statistical analysis were conducted to account for the differences in target and 

comparison groups, using regression and linear model analysis. 

Accounting for the contribution of age, language, gender and maternal education, these 

analyses also found that children who participated in the THINK Together preschool program 

demonstrated better literacy, numeracy and social-emotional skills than like-children that had 

not participated in the program. 

0 1 2 3 4 5 

Figure 5: Mean scores differences between cohorts with and without THINK Together preschool 

experience, demonstrate significant gains (p<.001) while accounting for initial differences in age, 

language, gender and maternal education. 

Socio-Emotional 

Literacy 

Numeracy Before 

After 

Limitations 

The investigators were introduced to this project in the spring of the program year, and 

therefore reliant on existing data. The design assumed faithful implementation of three 

evidence-based programs (High-Scope, Raising a Reader, and Incredible Years) but did not 

assess fidelity or attempt to identify how each program element may have been responsible for 

improvements. The design also assumes validity and reliability of assessments. It was not 

possible to use random assignment to the two sessions, which would provide a higher 

experimental design. 

10 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

 

 

 

California Children and Families Foundation 

15AC170571

Conclusions 

The integrated approach of high quality evidence-based preschool program, meaningful 

parental education and engagement, and providing children’s literature for at-home use is 

effective to achieve improved school readiness in a 90-hour 15-week program. 

Considerations for Future Investigation 

1.	 Validate inter-rater reliability of the assessments. 

2.	 Measure fidelity to each of the three program component models. 

3.	 Consider the development of an experimental design by using waitlists of children 

whose parents indicate interest in enrolling in THINK Together to randomly assign 

participants to sessions for comparison. 

4.	 Consider process evaluation to describe the roles AmeriCorps members play. 

5.	 The engagement of parents in the classroom and their participation in the Incredible 

Years program was observed at the site visit. The First 5 Service Corps evaluation of the 

Santa Cruz Reading Corps program discovered significant associations between 

parental behaviors and school readiness. This study did not address the contribution of 

the parent engagement component to the overall impact. Consider assessing changes in 

parental knowledge of their role in school readiness and children’s development or 

another means to examine the contribution of this component. 

6.	 Work with program staff in advance to implement strategies to collect more complete 

data. 

7.	 Discuss, coordinate and plan for how the data will be matched and cleaned. 

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 11 | P a g e 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

DATA ANALYSIS OVERVIEW 

2013-14 Orange County THINK Together Program Data 

SAMPLE 

The initial sample included 439 preschool aged participants (M age = 3.93 years at program 

entry) from a single program year (2013-14) for the THINK Together program. The program 

runs two 15-week sessions over the academic year, as well as a summer session (not evaluated 

in this report). There were data available for 206 children in session 1 and 233 children in 

session 2. A portion of children (n=88) participated in more than one session, and while both 

sets of scores for these children were evaluated for changes across session in pre-post 

comparisons, these children’s second set of scores were excluded from independent 

comparisons between groups. Thus, a total of 351 individual children contributed data for these 

analyses. Below are demographic descriptors for the resulting sample. 

Table 1: Demographic Information for Total sample (n=351) 

Categorical Variable Chart 

Ethnicity 

(n=339) 

7.7 % White 

57.8% Latino/a 

0.6% Afrn-American 

12.1% Asian 

1.2% Am. Indian 

0.6% Pacific Islander 

17.1% Bi-racial 

2.9% Other 0.6% 

0.6% 

1.2% 

2.9% 

7.7% 

12.1% 

17.1% 

African American 

Pacific Islander 

American Indian 

Other 

White 

Asian 

Bi-racial 

Latino/a 57.8% 

Hispanic 

(n=331) 

22.4% No 

77.6% Yes 

22.4% 

77.6% 

Non-Hispanic 

Hispanic 

Gender 

(n=350) 

47.4% Male 

52.6% Female 52.6% 47.4% Male 

Female 

Parents’ 

Marital 

Status 

(n=187) 

72.2% Married/Domestic 

Partnership 

8.6% Single Parent 

19.3% Cohabitating 72.2% 

8.6% 

19.3% Married/DP 

Single Parent 

Cohabitating 

Primary 

Language 

(n=349) 

51.6% Spanish 

39.5% English 

8.9% Other 51.6% 39.5% 

8.9% Spanish 

English 

12 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

Continuous Variables Mean 

(std) 

Chart 

Age at Entry in Years 3.81 

(n=351) (.57) 

Mother’s Educational 3.28 

Attainment (1.90) 

(7 point scale) 

(n=195) 

Family Income 

(10 point scale) 4.19 

(n=162) (2.70) 

 

 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

19% Less than GED 

35% HS/GED 

23% Some technical school or AA 

23% Bachelors or more 

over $100,000 1% 

$75,000 - $99,999 

$50,000-$74,999 

$40,000- $49,999 

$30,000- $39,999 

$25,000- $29,999 

$20,000- $24,999 

$15,000- $19,999 

$10,000 - $14,999 

Less than $10,000 22% 

18% 

6% 

10% 

12% 

5% 

12% 

10% 

4% 

MEASURES AND DATA COLLECTION 

Children were administered two types of assessments of school readiness, the Teaching 

Strategies GOLD at the beginning and end of each session and the Phonological Awareness 

Literacy Screening (PALS) at the end of the session. These tests were individually administered 

by trained AmeriCorps volunteers. Additionally, parents provided demographic data. 

Teaching Strategies GOLD 

The Teaching Strategies GOLD is an observation measure designed to assess 38 objectives 

linked to curriculum standards that are likely to “predict next steps in every area of 

development and learning. The objectives cover 10 areas of development and learning, 

including broad developmental areas, content areas, and English language acquisition”.2 Each 

item is rated on a 1-9 scale by trained AmeriCorps volunteers. 

THINK Together children were assessed on a subset of these objectives, including: 

2 From Web site: http://teachingstrategies.com/assessment/ 

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 13 | P a g e 
 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

 Socio-emotional skills (objectives 1-3) – regulation, cooperative behavior, and positive 

relationships; 

 Numeracy (objectives 20-21) – spatial concepts and number concepts; and, 

 Literacy (objectives (15-19) - phonological knowledge, alphabet knowledge, print 

knowledge, pre-reading skills, and pre-writing skills; 

PALS 

The PALS-PreK Assessment was administered once at the end of each session. The PALS-PreK 

“is a scientifically-based phonological awareness and literacy screening that measures 

preschoolers’ developing knowledge of important literacy fundamentals and offers guidance to 

teachers for tailoring instruction to children’s specific needs. The assessment reflects skills that 

are predictive of future reading success and measures name writing ability, upper-case and 

lower-case alphabet recognition, letter sound and beginning sound production, print and word 

awareness, rhyme awareness and nursery rhyme awareness. The assessment scores indicate 

children’s strengths and those areas that may require more direct attention. The assessment is 

designed to be administered to 4-year-olds in the fall of Pre-K in order to guide instruction 

during the year.”3 Developmental ranges for 4-year olds are provided by test developers to 

establish whether a child is “on track” for literacy readiness.

Demographic and Classroom Data 

Classroom data included teacher, classroom group, and session as well as child gender, 

ethnicity, and language. Additional demographic data were collected from parents at the time 

of the PALS assessment, including family income, marital status, maternal education, caregiver 

ethnicity, and maternal educational level. 

Missing Data 

Pre- and post-test GOLD data were evaluated for missing values and attrition. There were a 

total of 457 cases available for the two sessions combined. Cases were examined for (a) whether 

the pre-test score was missing (i.e., incomplete data) (b) whether the post test score was missing 

(i.e., attrition) and (c) whether both scores were missing (i.e., the student was not ever assessed 

on that item). The results varied slightly by outcome and are presented in Table 2. At < 20% 

these attrition rates are generally acceptable by WWC standards. 

3 From Website: https://www.palsmarketplace.com/assessments/pals_prek/ 

14 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

Table 2: Frequencies for Missing Pre-Post Data and Data Attrition 

Outcome Complete Data Incomplete Pair Missing Pair Attrition 

(used in analyses) (missing pre - score) (missing pre - and (missing post -

post - scores) score) 

Social Emotional 329 (72%) 46 (10%) 22 (5%) 60 (13%) 

Literacy 335 (73%) 69 (15%) 23 (5%) 53 (12%) 

Numeracy 316 (69%) 35 (8%) 24 (5%) 82 (18%) 

PALS data were available for 221 (48%) of the children, child demographics available for 351 

(77%) and parent report of family demographics was available for 195 (43%) children. 

Missing data within cases were handled in one of two ways. First, total scale scores were 

created by taking a mean of scale items, thereby averaging across any missing items within the 

scales. Second, missing data were excluded per each analysis, rather than listwise, i.e., all 

available data for a given analysis were used, rather than excluding cases for which there were 

any missing data. 

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 15 | P a g e 
 

 

 

=

- ”

California Children and Families Foundation 

15AC170571

Appendix I 

Descriptive Results For PALS Literacy Data 

A total of 221 (76 three to four- year-olds and 145 four years and older) THINK Together (TT) 

Children were administered the PALS at the end of their program sessions. Given the short 15-

week program duration, the program did not administer a pre-test. Neither was the PALS 

administered to a comparison group. As sucah, no direct evaluation of program effectiveness is 

possible using these data. However, descriptive analyses are possible, and comparisons can be 

made with norm ranges provided by the test developers. The results of these analyses are 

summarized in Table 3. 

Table 3: Results of End-of-Session PALS Assessments 

PALS Item Mean Score Maximum PALS % “On Track 

(std) (n 220) Score Developmental 

Ranges* 

Name Writing 4.59 (2.45) 7 5-7 57.0% 

Upper Case Alphabet 

Recognition 

12.72 (10.00) 26 12-21 48.9% 

Lower Case Alphabet 

Recognition 

10.56 (9.63) 26 9-17 48.9% 

Letter Sounds 6.10 (7.68) 26 4-8 38.5% 

Beginning Sound 

Awareness 

4.56 (3.95) 10 5-8 46.6% 

Print and Word 

Awareness 

5.65 (3.05) 10 7-9 44.3% 

Rhyme Awareness 4.70 (3.14) 10 5-7 46.2% 

Nursery Rhyme 

Awareness 

6.74 (2.90) 10 6-10 47.8% 

*Developmental Ranges designed for use in the spring of the four year old year and represent “a range of

development associated with later reading success”.

Because PALS developmental ranges are specific to 4-year-olds, PALS data were also evaluated 

by age group. As shown in the Table 4, separating PALS scores by age group reveals higher 

“on-track” rates on PALS literacy outcomes for the 4-year and older group, the children likely to 

be entering school the following year. 

16 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

Table 4 : PALSResults by Age Group 

Mean Score Mean Score “On - “On -

PALS (std) (std) Track” Track 

developmental 3 -4 -year -olds 4 -5 year -olds 3 -4 year - 4 and 

PALS Item 

Name writing 

ranges* 

5-7 

(n=75) 

2.45 (2.08) 

(n=145) 

5.69 (1.83) 

olds 

21.1% 

older 

75.7% 

Upper Case Alphabet 12-21 7.15 (8.83) 15.65 (9.37) 21.1% 63.9% 

Recognition 

Lower Case Alphabet 9-17 5.24 (7.81) 13.35 (9.37) 22.4% 63.2% 

Recognition 

Letter Sounds 

Beginning Sound 

4-8 

5-8 

2.21 (4.70) 

2.61 (3.08) 

8.15 (8.16) 

5.56 (3.99) 

13.2% 

26.3% 

52.1% 

56.9% 

Awareness 

Print and Word 7-9 3.95 (2.99) 6.54 (2.71) 19.7% 57.6% 

Awareness 

Rhyme Awareness 

Nursery Rhyme 

5-7 

6-10 

3.25 (2.65) 

5.52 (3.40) 

5.43 (3.13) 

7.37 (2.39) 

32.9% 

65.8% 

52.8% 

84.0% 

Awareness 

California Children and Families Foundation 

15AC170571

Appendix I
 

*Developmental Ranges designed for use in the spring of the four year old year and represent “a range of

development associated with later reading success”.

 

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 17 | P a g e 
 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

Pre/Post Comparison of Readiness Outcomes 

LITERACY 

OUTCOME 1: “As a result of program participation, there will be an increase in early 

literacy skills in English as measured by the AmeriCorps GOLD assessment” including: 

 book and print rules (PRINT, PREREAD); 

 phonological awareness (PHONE); and 

 early writing & alphabetic knowledge (ALPHA, PREWRITE). 

Table 5: GOLD Summary Literacy Variables for time 1 (beginning of 

program) and time 2 (end of program), computed as mean of relevant items 

for each objective. 

GOLD Variable 


 


 

Skill 

PHONE1, 

PHONE2

ALPHA1,

ALPHA2
 

PRINT1,
 

PRINT2
 

PREREAD1
 

PREREAD2 

PREWRITE1 

PREWRITE2 

LITERACY1 

LITERACY2 

Phonological knowledge (obj 15) 

Alphabet knowledge (obj 16) 

Print knowledge (obj 17) 

Pre-reading skills (obj 18) 

Pre-writing skills (obj 19) 

Total literacy (obj 15-19) 

Results of Comparisons of GOLD Pre-Post (difference) Scores 

As shown in Table 3, paired sample t-tests revealed significant differences in Pre- and Postscores for all literacy measures (p <.001), such that children scored significantly higher on 

literacy at the end of the program. 

Table 6: Pre-Post Comparisons for Literacy Outcomes 

Outcome Variable n Mean (std) Time 1 Mean (std) Time 2 Paired t* 

Phonological knowledge 311 1.34 (1.23) 2.58 (1.84) 13.47 

Alphabet knowledge 311 1.67 (1.44) 2.98 (1.89) 15.81 

Print awareness 312 2.03 (1.40) 3.64 (1.55) 18.14 

Pre-reading skills 311 1.64 (1.29) 3.05 (1.77) 18.29 

Pre-writing skills 311 2.19 (1.60) 3.52 (1.66) 18.49 

TOTAL LITERACY 312 1.72 (1.22) 3.09 (1.60) 20.04 

*Note: All ts are significant at p<.001 

18 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

NUMERACY 

 Number concepts 

 Spatial relationships 

OUTCOME 2: “As a result of program participation, there will be an increase in early 

numeracy skills as measured by the AmeriCorps GOLD assessment, including” 

Summary GOLD numeracy variables for time 1 (beginning of program) and time 2 (end of 

program), computed as mean of relevant items for each objective) included as follows: 

Table 7: GOLD Variables 

GOLD Variable SKILL 

NUMBER1 

NUMBER2 

Number concepts (obj 20) 

SPATIAL1 

SPATIAL2 

Spatial concepts (obj 21) 

NUMERACY1 

NUMERACY2 

Total numeracy (obj 20-21) 

As shown in Table 8, Paired ts revealed significant differences between pre- and post-test scores 

for numeracy items (p<.001), such that children scored significantly higher on number and 

spatial concepts at the end of the program. 

Table 8: Pre-post Comparisons for Numeracy Outcomes 

Outcome Variable n Mean (std) time 1 Mean (std) time 2 Paired t* 

Number Concepts 328 3.51 (1.86) 4.94 (1.20) 17.83 

Spatial Concepts 328 2.62 (2.10) 4.41 (1.55) 18.72 

TOTAL NUMERACY 329 2.82 (1.93) 4.45 (1.51) 19.76 

*Note: All ts are significant at p<.001 

SOCIO-EMOTIONAL 

OUTCOME 3: “As a result of program participation, there will be an increase in early socioemotional skills as measured by the AmeriCorps GOLD assessment, including”

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 19 | P a g e 

 Self-regulation (REGULATE) 

 Establishes Positive relationships (POSREL) 

 Positive interactions with peers (COOPER) 
 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

Pre/Post Comparisons of Socio-Emotional Outcomes 

GOLD summary variables for socio-emotional skills for time 1 (beginning of program) and time 

2 (end of program), computed as mean of relevant items for each objective were as follows. 

Table 9: GOLD Socio-Emotional Variables 

GOLD Variable SKILL 

REGULATE1 

REGULATE2 

Regulates, manages feelings (obj 1) 

POSREL1 

POSREL2 

Establishes positive relationships (obj 2) 

COOPER1, 

COOPER2 

Cooperates with others (obj 3) 

SOCEMOT1 

SOCEMOT2 

Total social emotional (obj 1-3) 

As shown in Table 10, paired ts revealed significant differences between pre- and post-test 

scores for all socio-emotional items (p<.001), such that children showed significantly higher 

GOLD socio-emotional scores at the end of the program. 

Table 10: Pre/Post Comparisons for Socio-Emotional Outcomes 

Outcome Variable n Mean (std) time 1 Mean (std) time 2 Paired t* 

Self-regulation 328 3.51 (1.86) 4.94 (1.20) 17.83 

Positive Relationships 328 2.62 (2.10) 4.41 (1.55) 18.72 

Cooperates with others 329 2.20 (1.89) 3.82 (1.51) 17.84 

TOTAL SOCIO-EMOTIONAL 329 2.82 (1.93) 4.45 (1.51) 19.76 

*Note: All ts are significant at p<.001 

DOSAGE 

Attendance data in the form of minutes attended and days of attendance were available for 158 

children. These two measures were highly correlated and therefore redundant. Pearson 

correlations were computed between change scores for all GOLD outcome variables and service 

days attended revealed significant associations between days attended and change cores in 

spatial concepts (r= .234, p= .004) and total numeracy (r= .189, p= .020) only. Attendance data 

was not associated with any other outcomes. 

20 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

Quasi-Experimental Comparison 

While pre/post comparisons revealed significant improvements in GOLD scores over the 

course of both program sessions, a controlled comparison provides a more rigorous test of 

program effectiveness, and is necessary to conform to WWC standards. Because there were two 

sessions of the program, we were able to conduct a quasi-experimental static-group comparison 

(Campbell & Stanley, 1963) by comparing children in session 1 at the end of their session with 

children in session 2 at the beginning of their session. We therefore formed two groups 

representing Children with Program Experience (session 1 post scores) and Children without 

Program Experience (session 2 pre-scores). For children who were repeating the program, we 

excluded the second set of (duplicate) scores (n=88) for this comparison. This resulted in a total 

sample of 351 children - 206 in session 1 (post) and 145 in session 2 (pre). We compared these 

two groups using GOLD outcome scores for total socio-emotional, total literacy, and total 

numeracy outcomes. 

Comparability of Comparison Groups 

The first task in a quasi-experimental comparison is to establish to whatever extent possible that 

the groups are comparable. At the outset, there is reason to think that these two groups are 

relatively comparable. The children were drawn from the same neighborhoods, and the 

teachers and sites were comparable across the groups. In some cases, children who would have 

been served in the first session were waitlisted to the second session. GOLD assessments for 

post-test session 1 and pre-test session 2 were conducted within a few months of each other by 

trained AmeriCorps members. 

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 21 | P a g e 

Table 11:Demographic Characteristics of Comparison Groups 

Categorical WITH Program WITHOUT program 

Variable Experience (Session 1 Post) Experience (Session 2 Pre) 

Ethnicity 

Gender 

Marital Status 

Primary 

Language 

 8.2 % White 

58.5% Latino/a 

14.0% Asian 

19.3% Bi/multi-racial/other 

46.9% Male 

53.1% Female 

72.4% Married/Partnership 

 9.2% Single Parent 

18.4% Cohabitating 

51.1% Spanish 

40.9% English 

 4.0% Other 

10.0% White 

50.8% Latino/a 

13.3% Asian 

25.8% Bi/multi-racial/other 

45.1% Male 

54.9% Female 

77.9% Married/Partnership 

 6.5% Single Parent 

16.6% Cohabitating 

45.1% Spanish 

41.8% English 

 4.9% Other 
 

 

Continuous Mean (std) Mean (std) 

Variables WITH Program Experience WITHOUT program experience 

Age in Years 4.10 (.49) 3.81 (.62) 

Mother Education 

(7 point scale) 3.48 (2.05) 3.37 (1.84) 

Income 

(10 point scale) 4.27 (2.64) 4.54 (2.76) 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

Group differences were assessed using t-tests and Chi-square tests. The groups were not 

significantly different in gender composition, child ethnicity and language, as well as maternal 

education level, family income, and marital status. However, children without program 

experience (session 2 pre) were somewhat younger than children with program experience 

(session 1 post), t(298) = 4.56, p<.001, thus indicating that analysis of any differences between 

these groups should account for child age. 

Further mean comparisons were conducted to analyze differences in baseline (pretest scores) 

between the two groups. Results indicated that the no program experience group had 

somewhat higher pre-test scores than the group with program experience, that is the noexperience group started the program with higher socio-emotional, literacy, and numeracy 

scores. Such differences at baseline should actually work against the finding significant 

program effects, so did not require analytical adjustments. 

Initial Results of Quasi-Experimental Comparison 

Mean GOLD scores for the two groups were compared using simple independent groups ttests, revealing significantly higher scores on all three GOLD outcome areas for children who 

had participated in the program as compared to children who had not yet participated in the 

program (Table 12). 

Table 12: Mean Differences in GOLD Outcome for Comparison Groups 

Outcome Variable 

Total Socio-Emotional After program 

n 

163 

Mean 

(std dev) 

4.25 (1.49) 

Before program 116 2.11 (1.47) 

Total Literacy After program 

Before program 

171 

107 

2.84 (1.25) 

1.22 (.99) 

Total Numeracy After program 

Before program 

171 

107 

3.41 (1.27) 

1.64 (1.14) 

Note: All mean differences are significant at p<.001 

Associations between Demographic Variables and GOLD Scores 

GOLD Scores were then examined for significant associations with demographic variables 

22 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

(covariates). Initial bivariate correlations (Pearson r) and t-tests revealed the following 

significant predictors of test scores: age, gender, mother education, and child language. Thus, 

these variables were controlled as covariates in subsequent analysis of program effects. 

Univariate and Multivariate Analysis Accounting for Covariates 

A univariate linear model was used to analyze group differences on total numeracy, literacy, 

and socio-emotional scores while adjusting for the main effects of child age, gender, language, 

and mother’s education level. In these analyses, group membership (experience vs. no 

experience) remained a significant predictor of socio-emotional (β = -.533, p<.001), literacy (β = -

.487, p<.001), and numeracy (β = -.447, p<.001) scores, after accounting for age, gender, 

language, and maternal education. 

A multivariate linear model was used to further account for the possible dependence among the 

scores, and potential interactions of group response with the covariates. Estimated marginal 

means from this analysis are listed in Table 10. The pattern of responses is similar to the 

unadjusted means, with slightly smaller differences between groups at equal covariate levels. 

Table 10: Mean Differences Adjusted for Covariates and Existing Differences 

Estimated Standard error 

Outcome Marginal Mean of the mean 

Total Socio-Emotional After program 4.34 .187 

Before program 2.71 .111 

Total Literacy After program 2.90 .145 

Before program 2.04 .086 

Total Numeracy After program 3.74 .153 

Before program 2.58 .105 

*Means significantly different (p<.001). 

Means are adjusted child age, maternal education, child language, and child gender. 

 

 

Figure 6: Mean scores differences between cohorts with and without THINK Together preschool experience, 

demonstrate significant gains (p<.001) while accounting for initial differences in age, language, gender and 

maternal education. 

Numeracy
 

Literacy
 

Before After 

Socio-Emotional 

0 1 2 3 4 5 

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 23 | P a g e 
 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

Accounting for Attrition 

While overall attrition rates for the total sample were acceptable (see Table 2) , per WWC 

standards attrition rates need to be considered in making comparisons between groups. For 

example, if there are large differences in attrition rates between comparison groups, then the 

comparison may not be valid. In the current static comparison design, it is possible that the 

post test scores used to represent the program experience group may be inflated because poorer 

scorers might be more likely to drop out before they were assessed at the end of the program. 

This would not be the case with the pretest scores used in the no experience group. 

As a way to account for attrition in this design, analyses were therefore conducted again, 

eliminating the scores of those children in the no experience group who eventually dropped out 

(or who were not assessed) at the end of the session. Results indicated that the pattern of 

responses is similar to previous results. There is a main effect for group membership even after 

controlling for potential attrition effects. 

Conclusion 

Overall, these results provide good evidence that children with program experience show 

better socio-emotional, literacy, and numeracy skills than similar children who have not yet 

attended the program, even after accounting for demographic differences and covariates. 

24 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix II 

THINK Together Logic Model 

Resources Activities Outputs Outcomes 

THINK Together Preschool-Based annual 

budget of $565K 

CFCOC AmeriCorps administration 

CFCOC & TT: AmeriCorps member recruitment, screening, hiring. 

District partners distribute program participant recruitment flyers. 

17 trained AmeriCorps 

members each engaged 

in ~1700 hours of 

Preschool Program 

Increased early 

literacy skills as 

measured by the 

Children and Families Commission of Orange AmeriCorps GOLD 

County(CFCOC): Early Literacy and Math activities (Sept-May) assessment, PALS 

Program grant; AmeriCorps costs & program 19 days of AmeriCorps member training from various 

administration. providers: 16 sessions of 3 hour  conversation 

(Funded various TT programs at $1M/year; in - 5-day AmeriCorps Summer Institute (Sept) which includes 6 classes 2x/week for 15 skills 

Feb 2012 invested $10M/10 years to include hours in PCAC GOLD Assessment. 12-hour online professional weeks of preschool  vocabulary 

marketing, communications, fund development, 

building purchase.) 

development. .6 FTE trainer 

- 4 days of program orientation sessions: 2 hour RAR training 

- 8 days High Scope training 

- Professional development, CalTech 2-day STEM Conference 

17 AmeriCorps members actively plan & prepare, support 

teacher, teach in small groups of 5 students, share in managing 

choice time activities, and reflect daily as a team. 

4 Host Teachers- High-Scope-trained lead classroom teachers; 

work with AmeriCorps in lesson planning, preparation, and 

reflection. 

with enriched literacy 

and math environments 

and activities . 

~500 children ages 3 

and 4years, per year, 

receive ~90 hours of 

high quality preschool 

# of parents who 

accompany their child in 

preschool and 

 book and print 

rules 

 visual 

discrimination 

 phonological 

awareness 

 early writing & 

alphabetic 

knowledge 

 English language 

Increased at-home 

17 AmeriCorps Members 

4 School District Partners: Buena Park; 

Huntington City; Santa Ana, and; Tustin. 

Provide 16 host classrooms 

THINK Together 

.8 FTE Early Learning Program Manager 4 FTE 

preschool teachers (High-Scoped & PALS 

trained, BA in Child development or 

Early Learning Program Manager provides oversight, 

coordinates training, liaisons with district, site coaching. Manages 

RAR program (1600 bags). Oversee IY parent educator. 

participate in # hours of 

Incredible Years parent 

education 

reading frequency 

as demonstrated in 

RAR pre/post 

equivalency, min. 3 years Pre-K experience.) 2 15-week preschool sessions, 3 hrs/day, 2 days/week parent survey. 

2 FTE Incredible Years facilitators 

~500 Children receive Raising a Reader books to take home 

weekly for at home reading. 

~ 373 parents participate in 14 week Incredible Years program for 

2 hours/week. 

AmeriCorps 

volunteers report 

satisfying service 

experience which 

Evidence-based programs 

 High Scope Preschool Model (began in Teaching Strategies GOLD assessment conducted for each contributed to 

2014) child within 1st three weeks and in 15th week. community and 

 Raising a Reader (RAR) bag program their professional 

(ongoing since 2008) 

 Incredible Years parent education program 

Phonological Awareness Literacy Screen (PALS) conducted by 

head teacher in weeks 14 and 15. growth as reported 

in AmeriCorps (began in 2014) survey.  Prior to Jan. 2014, program used locally 

developed program. 

Data Sources 

1.	 DEMOGRAPHIC: TT intake form includes self report of family income, # of household members, home language, and a TT child-identifier 

that can be used to match intake to other child-level data. (Will add prior pre-k experience in form in fall 2014.) 

2.	 RAISING A READER: Pre/Post standard assessment, entered by TT and available in excel or SPSS, linked to TT child-student identifier. 

Can be used to demonstrate changes in at-home reading behavior. 

3.	 INCREDIBLE YEARS Program (IY) is using the IY session feedback, but not pre/post IY parent scale. 

4.	 AMERICORPS MEMBER SURVEY is created and administered by Tiffany at the end of each program year. She is open to us adding 

additional questions. 

5.	 ATTENDANCE data: collected by TT and can be linked to child. Can also be used to identify children who have attended multiple 

sessions. IY parent attendance also available. 

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 25 | P a g e 
 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix III 

THINK Together Pre-K School Readiness Evaluation Plan 

Out- Indicators Data Collection Frequency/Schedule Sample Size Analytical Methods 

come Methods & Tools of Data Collection a y & Math Skills & Social/Emotional Development

1. 

2. 

3. 

4. 

5. 

Name writing 

Alphabet knowledge 

Beginning sound awareness 

Print word awareness 

Rhyme awareness 

PALS Post 

(PRE-Test NOT 

IMPLEMENTED) 

By Head Teacher, 1x 

within last week of 

program 

~500 

(100% service 

population) 

Compare TT with 

Buena Park 4 week 

summer preschool. 

- Manages feelings 

- Follows limits & expectations 

- Takes care of own needs 

- Forms relationship with adults 

- Responds to emotional cues 

- Interacts with peers 

- Makes friends 

- Balances needs and rights of self and others 

- Solves social problems 

- Demonstrates phonological awareness: a) notices and discriminates: a) rhyme; b) alliteration; c) 

smaller & smaller units of sound. 

Teaching Strategies GOLD 

Assessment 

By AmeriCorps, within 

first 2 weeks and last 

week of each program 

~500 

(100% service 

population) 

Paired T test for 

pre/post 

Compare statistically 

(linear mixed effects) 

with other AmeriCorps 

program Gold 

Assessment scores 

when accounting for 

child age, language, 

gender. 

Increased Early Liter c

- Demonstrates knowledge of the alphabet: a) Identifies and names letters (PALS 2); and b) uses letter 

knowledge 

- Demonstrates knowledge of print and its uses: a) uses and appreciates books; b) uses print concepts 

- Comprehends and responds to books and other texts: a)Interacts during read-aloud & book 

conversations; b)Uses emergent reading skills; and c)Retells stories 

- Demonstrates emergent writing skills: a) writes name (PALS1); b) writes to convey meaning 

- Uses number concepts & operations: a) counts; b) quantifies; c) connects numerals with their 

quantities 

- Explores and describes spatial relationships: a) understands spatial relationships; b) understands 

shapes 

At-Home 

Reading

- Increased at-home reading frequency 

- Increased use of child engagement strategies 

- Increased reading routines 

- Increased library usage 

RAR pre/post parent survey 

(NOT IMPLEMENTED) 

At start and end of 15 

week program. 

~500 

(100% service 

population) 

T test for pre/post 

Contribution

- Self report of perceived contribution to community from “not at all” to “great”; Narrative explanation 

- Self-report of perceived benefit for career path from “not at all” to “great”. Narrative explanation. 

AmeriCorps Survey 1x, in last month of 

service. 

All AmeriCorps 

serving TT 

Pre-K program 

Qualitative narrative, 

simple response rates. 

26 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix IV 

AmeriCorps Member Perceptions 

While the central purpose of this evaluation was to ascertain the effectiveness of the Think 

Together preschool program for preparing children for kindergarten, the researchers were 

interested in how the AmeriCorps members who were so instrumental in service delivery felt 

about their experience. Since the THINK Together Early Learning Program Manager was 

already planning to implement a feedback survey at the end of the program year, she agreed to 

include several additional questions. All 17 First 5 Service Corps members responded. While 

for most (90%) it was their first year of service, there was one 3rd year and one 4th year member. 

11% 

16% 

37% 

37% 

Poor 

Below Average 

Average 

Good 

Excellent 

Usefullness of Training for 

Assignment 

21% 

79% 

Poor 

Below Average 

Average 

Good 

Excellent 

Meaningful & Beneficial Service 

5% 

11% 

47% 

37% 

Poor 

Below Average 

Average 

Good 

Excellent 

Usefullness of Training for Career 

Goals 

59% 

35% 

6% 

Consistent Support 

Sufficient 

Insufficient 

Adequate Support to Provide 

High Scope Method 

5% 

32% 

63% 

Poor 

Below Average 

Average 

Good 

Excellent 

Feeling of Accomplishment 

11% 

21% 

68% 

Poor 

Below Average 

Average 

Good 

Excellent 

Opportunity for Personal Growth 

F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 27 | P a g e 
 

California Children and Families Foundation 

15AC170571

The AmeriCorps members described their personal goals for participating in the THINK 

Together program to include: providing community service; first-hand experiences of classroom 

teaching; and exploring working with young children. All but one reported that their 

experience met their expectations, with 53% saying it exceeded their initial hopes. First 5 Service 

Corps Project Evaluation. 

28 | P a g e F i r s t 5 S e r v i c e C o r p s T H I N K T o g e t h e r F i n d i n g s 
 

 

 

 

California Children and Families Foundation 

15AC170571

First 5 Service Corps Project Evaluation
 

Santa Cruz Reading Corps Preschool Program
 

DavisConsultantNetwork 

Francesca Wright, MPA 

Sheri Hembree, PhD 

November 2014 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 29 | P a g e 
 

 

California Children and Families Foundation 

15AC170571

Acknowledgements 

The Davis Consultant Network would not have been able to develop this report without the 

vision, leadership, data collection, and cooperation of the following individuals. We thank each 

of you for your cooperation, service, and deep commitment to young children. 

Agency 

First 5 Association / Moira Kenney 

California Children & Executive Director 

Families Foundation 

Prevent Child Abuse Sheila Boxley President and CEO 

California Stephanie Biegler Chief Program Officer 

Sara Fung Program Manager – AmeriCorps 

Ian Hadley First 5 Service Corps Project Manager 

First 5 Santa Cruz David Brody Executive Director 

County Commission Vicki Boriack Senior Program Officer 

Irene Freiberg Master Literacy Coach 

Holly Maclure Research and Evaluation Analyst 

Christine Sieburg Santa Cruz Reading Corps Coordinator 

First 5 Service Corps 

Members 

Sophia Bassett 

Santa Cruz Reading Corps Tutors 

2013-2014 

Lauren Carriere 

Melody Curl 

Aundrella Dennings 

Rachel Griffin 

Marilyn Guerra 

Cassondra Guzman 

Esdras Juarez 

Natalia Loken 

Vista Pickett 

Heather Purchase 

Caroline Simmons 

Cover Photo by Shmuel Thaler/Santa Cruz Sentinel 

30 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

California Children and Families Foundation 

15AC170571

Introduction 

In 1998 California voters passed Proposition 10 which levied a tax on tobacco products in order 

to fund programs for California’s children prenatal through age 5 years and their families. In

2001, First 5 California and the First 5 Association, which represents the 58 County First 5 

Commissions, CaliforniaVolunteers, the State Commission for the Corporation for National and 

Community Service (CNSC), and Prevent Child Abuse California (PCA CA) formed a 

partnership to implement a statewide AmeriCorps Initiative. The First 5 Service Corps (F5SC) 

first received CNCS funding in 2002 to engage AmeriCorps members to help children reach 

school-age developmentally ready to learn and better-prepared to succeed in school. PCA CA 

administers F5SC and provides technical assistance for successful placement of AmeriCorps 

into school readiness programs being offered by 13 county First 5 Commissions. 

The request for evaluation services for the F5SC 2012 through 2014 years, sought to narrow the 

scope to programs that had already collected data suitable for quasi-experimental design as 

defined by the What Works Clearinghouse. 

This portion of the comprehensive evaluation focuses on the Santa Cruz Reading Corps 

program which was developed and administered by the First 5 Santa Cruz County 

Commission. 

Two researchers from the Davis Consultant Network met with program administrators who 

provided information about the program history, the Reading Corps Model, implementation 

across sites, and assessment protocols. They then observed the program in action at the Linscott 

State Preschool program in Watsonville. At the site they observed the classroom environment, 

arrival, parent interactions, implementation of the Reading Corps intervention model, and 

interviewed the AmeriCorps member who was working in that classroom. From this visit and 

follow-up calls, a program logic model and evaluation plan were drafted, reviewed and 

finalized with staff. Data was received and analyzed. This report addresses those findings. 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 31 | P a g e 
 

 

 

 

 

California Children and Families Foundation 

15AC170571

Santa Cruz Reading Corps Outcome Evaluation 

Program Description 

In 2012, First 5 Santa Cruz County initiated the Santa Cruz Reading Corps. This program places 

trained AmeriCorps members, called Reading Corps Tutors, in state preschool and transitional 

kindergarten classrooms with SEEDS trained staff to provide early literacy strategies and 

provide small group and one-on-one interventions. 

First 5 Santa Cruz County had already been training early child educators in the SEEDS of Early 

Literacy Curriculum4 methods since 2007. SEEDS is a mnemonic device to teach five qualities of 

teacher/ student interaction: 

1. to be Sensitive 

2. to be Encouraging; 

3. to Educate; 

4. to “Develop through doing” by promoting active learning; and

5. to help develop a positive Self-image. 

The program also trains “the big 5 early literacy skills”: 

1. conversation and oral language; 

2. phonological awareness; 

3. books and print rules; 

4. alphabet knowledge; and 

5. vocabulary and meaning of words. 

Figure 7: SEEDS of Early Learning School Readiness Qualities and “Big 5” early literacy skills. 

4 http://www.seeds-learning.com/ 

32 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Santa Cruz Reading Corps tutors were placed in classrooms in which at least one of the early 

childhood professionals on the teaching team was trained in SEEDS of Early Literacy. SEEDS 

trainings include how to evaluate and create a classroom that offers many opportunities for 

developing literacy skills, and how to integrate literacy-rich exchanges with their students 

throughout the day. 

Reading Corps classrooms also use the Raising a Reader5 book bag program to ensure that all 

children have access to appropriate children’s literature at home through a rotating book bag 

system which is facilitated by the Reading Corps Tutor. Raising a Reader is another evidence 

based program that engages families by bridging at-home and at-school readiness activities. 

When the Reading Corps program was launched in 2012, State Preschool programs from the 

Child Development Divisions of Pajaro Valley Unified School District and Live Oak School 

District agreed to participate. The 23 classrooms in these two districts which qualified to 

received Reading Corps Tutors had existing SEEDS-trained staff who had already implemented 

the “Literacy and Math Rich Schedule” into their programs. This daily schedule provides the 

comprehensive and structured implementation of all of the components of SEEDS. 

The Santa Cruz Reading Corps replicated the Minnesota Reading Corps6 program model which 

uses individual assessments to monitor each child’s early literacy skills. This model has a 

Response to Intervention (RtI) component which offers supplemental individual instruction 

from the Reading Corps Tutor every day for the five children with the lowest initial scores. 

This may be delivered through a “Tier 2” small group intervention for 5-10 minutes, or a “Tier 

3” on-on-one intervention for 3-5 minutes. All children in the classroom participate in the “Tier 

1” daily routine of the SEEDS literacy and math rich schedule. 

Figure 8: Reading Corps is a “Response to Intervention” model which uses periodic assessment to 

determine which Tier of intervention is most appropriate. 

Tier 3: 5-15% will need 

intensive individual plan 

with ongoing monitoring 

Tier 2: 20-30% will need interventions 

through small groups with ongoing 

monitoring 

Tier 1: In high quality early childhood classrooms: 70-80% of 

the children will make adequate progress to key benchmarks 

of learning when teachers use research-based curriculum, 

embedded and explicit teaching, and universal screening 

5 http://www.raisingareader.org 

6 www.minnesotareadingcorps.org 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 33 | P a g e 
 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Figure 9: Minnesota Reading Corps Program Model 

During their 9 months of service, each Reading Corps Tutor receives over 200 hours of training 

in SEEDS strategies, child assessments, AmeriCorps training, and professional development. 

(See Logic Model in Appendix II). Ongoing supervision and coaching are provided throughout 

the year. At each school site, a designated staff member also serves as the Internal Coach. She 

observes and mentors the Reading Tutors on the implementation of SEEDS strategies. First 5 

Santa Cruz County provides overall program support with a Program Coordinator who 

handles administrative issues and a Master Coach who provides training and ongoing program 

support. 

Figure 10: Santa Cruz Reading Corps Coaching Model 

34 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

 

 

California Children and Families Foundation 

15AC170571

Evaluation Design & Tools 

The evaluation design sought to measure comprehensive program impacts on early literacy 

using existing data collection tools and procedures. The child outcomes analysis was to be 

supplemented with AmeriCorps member perceptions of their service experience. This can be 

found in Appendix V. 

Three assessment tools were administered to collect data on school readiness; the 

“IGDIs/FAST”, the “Child Snapshot”, and a measure of school readiness taken from the 

Desired Results Developmental Profile (DRDP) assessment summary. 

The first tool was created from elements of the Individual Growth and Development Indicators7 

(IGDIs) and the Formative Assessment System for Teachers 8(FAST) to assess a child’s 

competency in: 

1. picture naming; 

2. rhyming; 

3. alliteration; 

4. letter sounds ; and 

5. letter naming 

The Reading Corps Tutors administer the IGDIs/FAST assessments three times a year: once in 

late September-early October, once in late January - early February, and once in May. These 

assessments help Tutors select children who receive tailored literacy-based intervention on a 

daily basis (Tier 2 small group and Tier 3 one-on-one), as well as to clarify the needs of the 

whole class (Tier 1). Each of the selected children’s progress is monitored monthly. Before 

conducting each of the IDGIs and FAST benchmarks (fall, winter and spring) with children, all 

tutors are observed by program staff on their assessment procedures, which is measured with a 

fidelity checklist by their coach on each of the five assessments. 

Each child also had a Transition to Kindergarten Child Snapshot (Snapshot) which included 

data from forms completed by parents and from the preschool teacher at the end of the 

preschool year. The parent report section includes demographic information and home-based 

readiness activities, participation in transition to kindergarten activities, participation in other 

community support activities, and child health. The preschool teacher support section includes 

narrative description of child’s learning styles, and progress in language development, and

overall attendance. 

Additionally, the teachers attach the DRDP summary form which is a simple 1-page teacher 

observation score sheet of school readiness competencies: self-care and motor skills; self7 http://www.myIGDIss.com/ 

8 http://www.fastforteachers.info/ 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 35 | P a g e 
 

 

 

 

California Children and Families Foundation 

15AC170571

regulation; social expression and kindergarten academics. Desired Results Developmental 

Profile (DRDP)9 is used for public preschool programs in California to assess school readiness. 

Evaluation Findings 

A full description of evaluation methods may be found in Appendix I, Data Analysis Overview. 

The evaluation used statistical analyses to measure: pre/post changes; rates of changes for the 

students participating in Tiers 2 and 3 versus students who participate only in Tier 1; rates of 

achievement of school readiness benchmarks; relationships between school readiness and 

parent reports of at-home early learning activities; and a descriptive comparison of readiness of 

children in SEEDS programs in Santa Cruz County, with and without the Reading Corps 

component. The data were examined for potential quasi-experimental design analysis, but 

there was not a qualifying set of comparison group data. 

Pre/Post Paired T Analysis 

A paired t-test was conducted which compared pre- and post-scores for the IGDIs/FAST for the 

total sample. This method examines the difference between paired scores and accounts for the 

repeated assessments of the same children. 

There were significant (p <.001) improvements in pre- and post-scores for all literacy measures. 

The children who participated in the Santa Cruz Reading Corps program scored significantly 

higher in picture naming; rhyming; alliteration; letter sounds; and letter naming at the end of 

the program, for both program years. The greatest gains were in letter naming, picture naming 

and letter sounds. 

Figure 11: Gains (difference between post and pre scores) were seen in all 5 early literacy skills 

assessed. 

PrePost Gains on IGDI/FAST Scores 

Picture Naming 10.5 9.1 

Letter Naming 5.4 10.9 

Rhyming 2.9 5.0 

Letter Sounds 2.8 7.1 

Alliteration 1.9 3.3 

Comparison of Tiers 2 & 3 with Tier 1 only Scores 

Based on the Reading Corps Response to Intervention model, by definition, the children in Tiers 

2 or 3 started the school year with low scores. With additional individualized attention, would 

9 http://www.desiredresults.us/form_drdp.htm 

36 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

California Children and Families Foundation 

15AC170571

their IGDIs/FAST change scores be comparable with the Tier 1 only students? We examined 

differences in performance for intervention and non-intervention students. 

As may be expected, children receiving the Tier 2 or 3 intervention also had significantly lower 

final scores than children not receiving the supplemental interventions (See Figure 6). 

However, children in both groups experienced significant gains in literacy and the individual 

rates of change of children in Tiers 2 and 3 were similar to the non-intervention children in 

letter-naming, rhyming, and alliteration. 

 

 

 

Figure 12: Paired sample statistics (only including children with a pre- and post- IGDIs/FAST assessment) 

demonstrates that each group achieved significant gains in early literacy skills. Students in Tiers 2 & 3, by 

definition, started with lower scores. Gains are the difference between mean post scores and mean pre scores. 

 

 

 

 

 

 

 

341% 

239% 

159% 

348% 

179% 

239% 

66% 

0.0 5.0 10.0 15.0 20.0 

Mean Pre 

r

s 

Tiers 2 or 3 

tte

nd

Le

ouS

Tier 1 only 

n atio

Tiers 2 or 3 

terlliA

Tier 1 only 

ing 

Tiers 2 or 3 

Rhym

Post Gains Tier 1 only 

% Gain 

rtte

ing 

Tiers 2 or 3 

Le

Nam

Tier 1 only 

e 

ing 

Tiers 2 or 3 

icturP

Nam

Tier 1 only 

205% 

167% 

230% 

In the case of picture naming, children receiving the Tiers 2 and 3 interventions experienced 

significantly greater gains than non-intervention children (see Table 5 of Appendix 1 and Figure 

7 ), approaching a closing performance gap. Picture naming was the most repeated assessment 

for Tiers 2 or 3, and this finding suggests there may be a testing effect on performance in this 

area. 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 37 | P a g e 
 

 

 

 

 

Tier 1 only Tiers 2 & 3 

California Children and Families Foundation 

15AC170571

Picture Naming 

Figure 13: Comparison Tiers 2 or 3 and Tier 1-only average pre- and post-scores in Picture Naming 

20.5 

12.4 

16.9 

5.0 Average Pre Average Post 

Kindergarten Readiness Benchmarks 

Preschool teachers assessed 30% of the children in the program with the DRDP near the end of 

the school year. Both the mean and median DRDP scale scores were in the “in-progress” and

“proficient” ranges. This finding demonstrates strong school-readiness skills for children 

participating in Reading Corps. The evaluators do not have information on why only 30% of the 

children were assessed, but the preliminary finding is encouraging and bears further 

investigation with a larger sample. 

Self Care & Motor Skills 

Figure 14: Percent of Pre-Kindergarten Observation Form year-end assessments by readiness skills in 

Self-Care and Motor Skills for children in Reading Corps program. n=315. 

38 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 

 

 

 

Performs basic self-help/self-care tasks 

Has general coordination on playground 

Uses Small Manipulatives 

45% 54% 

28% 

20% 

72% 

80% 

Not Yet Beginning In Progress Proficient 
 

Self-Regulation 

 

 

Figure 15 Percent of Pre-Kindergarten Observation Form year-end assessments by readiness skills in 

Self-Regulation for children in Reading Corps program. n=199-318 

 

 

 

 

 

 

 

 

42% 

49% 

46% 

42% 

50% 

42% 

41% 

41% 

54% 

49% 

52% 

57% 

49% 

57% 

53% 

54% 

Comforts self, using adult guidance 

Stays focused/pay attention during activities 

Controls impulses and self-regulates 

Follows one-to two step directions 

Negotiates with peers to resolve social conflicts 

Works and plays cooperatively with peers 

Participates successfully in circle time 

Handles frustration well 

Not Yet Beginning In Progress Proficient 

Social Expression 

 

 

Figure 16: Percent of Pre-Kindergarten Observation Form year-end assessments by readiness skills in 

Social Expression for children in Reading Corps program. n=197-318 

 

 

 

Engages in symbolic/imaginative play w ...

Expresses curiosity and eagerness for learning 

Has expressive abilities 

Expresses empathy or caring 

Expresses needs and wants appropriately in...

Relates appropriately to other adults 

42% 56% 

46% 54% 

43% 56% 

44% 55% 

29% 67% 

40% 59% 

Not Yet Beginning In Progress Proficient 

California Children and Families Foundation 

15AC170571

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 39 | P a g e 
 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Kindergarten Academics 

Figure 17: Percent of Pre-Kindergarten Observation Form year-end assessments by readiness skills in 

Kindergarten Academics for children in Reading Corps program. n=205-318 

 

 

 

Recognizes primary shapes 

Recognizes basic colors 

Recognizes letters of the alphabet 

Counts 10 objects correctly 

Recognizes rhyming words 

Writes own first name 

Engages with books 

Not Yet Beginning In Progress Proficient 

57% 41% 

21% 79% 

58% 40% 

38% 60% 

67% 30% 

43% 56% 

53% 45% 

Correlations between Literacy Scores and Readiness 

Snapshot indicators of socio-emotional and academic readiness as reported by parents and 

teachers were examined for relationships with IGDIs/FAST early literacy scores. Significant 

associations were found between IGDIs/FAST literacy change scores and teacher’s assessment 

of kindergarten readiness (see Table 7 of Appendix 1) particularly with the academic readiness 

subscale. This relationship indicates that progress in early literacy skills (especially in rhyming, 

alliteration, letter sounds) predicted preschool teachers’ perception that the child was ready for

kindergarten, especially in the academic skills domain. This supports the idea that the Reading 

Corps literacy intervention indexed skills needed in kindergarten. 

Home Readiness Activities and Children’s Readiness 

The Snapshot parent data offered an opportunity to examine associations between reported 

parental behavior and children’s literacy and kindergarten readiness. There were several small 

but significant correlations between parent-reported behaviors and kindergarten readiness 

ratings from both the IGDIs/FAST and DRDP. Specifically: higher frequency of singing songs; 

drawing and solving puzzles; and eating meals together were linked to kindergarten readiness 

outcomes. 

Similarly, several of the reported parenting kindergarten preparation activities were 

significantly linked to one or more literacy change score measures. These included: seeking out 

registration information; seeking out information on becoming involved in school; participation 

in “Raising a Reader” book program; having help from extended family and neighbors; and 

being engaged with community resources such as health clinics, churches and non-profit 

agencies. 

40 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

 

California Children and Families Foundation 

15AC170571

Quasi-Experimental Comparison 

While pre/post comparisons revealed significant improvements in literacy scores for program 

participants over the course of the program, a quasi-experimental comparison provides a more 

rigorous test of program effectiveness. Ideally, Santa Cruz Reading Corps children would be 

compared to similar children who were in a similar program without the Reading Corps 

component. Since DRDP, Snapshot and IGDIs/FAST data were collected in SEEDS classrooms 

prior to the initiation of the Reading Corps program, data from these prior years were examined 

for the potential of a quasi experimental comparison. Investigation for comparability to a group 

with IGDIs/FAST and/or Snapshot data was attempted, but the comparison group did not 

qualify. They differed significantly in ethnicity and home language but the more critical 

challenge was insufficient post data collected to conduct a valid quasi-experimental 

comparison. (See Appendix 1 for additional description.) 

Descriptive Comparison of Readiness Scores 

While the investigated comparison group did not meet the specifications for quasi-experimental 

comparison, the DRDP scores were available for 48% of the children in the SEEDS classrooms 

that did not participate in Reading Corps (n=149). These school readiness scores were 

compared with scores available for 30% (n=318) of the children who were enrolled in preschool 

classrooms with Reading Corps. Descriptive comparison revealed significantly higher scores on 

all outcomes for children who had participated in Reading Corps compared to children who 

experienced a similar SEEDS program without the Reading Corps component. 

Figure 18: Comparison of DRDP Year-End Scores between SEEDS programs with and without
 

Reading Corps. Means are significantly different ** p<.01 ***p<.001
 

 

 

Overall Readiness***
 

Kinder Academics**
 

Self-Regulation**
 

Emotional Expression***
 

Social Skills***
 

Reading Corps Without Reading Corps 

3.2 3.25 3.3 3.35 3.4 3.45 3.5 3.55 3.6 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 41 | P a g e 
 

 

California Children and Families Foundation 

15AC170571

Reading Corps Tutors’ Perceptions 

Six of the 12 Reading Corps Tutors responded to an online survey about their program 

experience. They reported extensive training. All 6 felt they consistently received the support 

they needed to be effective throughout their service period and that it met or exceeded their 

hopes and expectations. They felt the Reading Corps curriculum was engaging to children. 

Tutors were attracted to the program to build experience, to serve the community, to see how 

they like teaching, and to help children be ready for kindergarten. Most reported that the 

program exceeded their hopes and expectations. 

Limitations 

The investigators were introduced to this project in the spring of the program year, and 

therefore reliant on existing data. The design assumed faithful implementation of three 

evidence-based programs (SEEDS, Reading Corps, and Raising a Reader) but did not assess 

fidelity. The design also assumes validity and reliability of assessment tools. 

42 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

 

California Children and Families Foundation 

15AC170571

Conclusions 

There is evidence of significant positive change in literacy scores for children in early learning 

programs that include the addition of the Reading Corps component with SEEDS Early Literacy 

and Raising a Reader programs. Early literacy skills, especially in rhyming, alliteration, and 

letter sounds, predict preschool teacher’s assessment of kindergarten readiness, especially in

academic skills. Children who enter preschool with the lowest scores in early literacy, and 

participate in Reading Corps, see similar improvements as those who enter preschool more 

ready. The family’s role in school readiness is also important. Families that report 

connectedness to community resources and engaging with their child through singing, 

drawing, puzzle-solving and family meals have children more likely to be ready for 

kindergarten. The coaching provided to Reading Corps Tutors throughout their 9 months of 

service helped them feel of service to children and the community, while building their own 

skills. An integrated approach of: teacher quality; supplemental skill-specific instruction within 

the preschool program; access to children’s literature at home; with the possible addition of 

explicit parent education; will increase early literacy skills for children preparing to enter 

kindergarten. 

Considerations for Future Investigation 

1.	 Measure fidelity to each of the three program component models. 

2.	 Consider methods to collect appropriate data from a comparable group for the 

development of quasi-experimental design.
 

3.	 Validate inter-rater reliability of the assessments for comparison group classrooms. 

4.	 Consider adding and evaluating a more rigorous parent education component on 

parent’s role, both at home and in the classroom. 

5.	 Work with program staff in advance to implement strategies for more complete data 

collection. 

6.	 Consider collecting data about amount and type of Tier 2 and 3 interventions. 

7.	 Collect attendance data to inform assessment of dosage effects. 

8.	 Discuss, coordinate and plan in advance for how the data will be matched and cleaned. 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 43 | P a g e 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix 1 

DATA ANALYSIS OVERVIEW 

Santa Cruz Reading Corps 2012-14 

SAMPLE 

The sample included a total of 1054 students enrolled in the Santa Cruz Reading Corps program 

for two academic year cohorts (2012-13 and 2013-14). The sample was primarily Latino/a, with 

a mean age at entry of 4.25 years (SD = .46). The sample included 530 children from ‘12-‘13 

program year and 528 from the ‘13-‘14 program year. Other demographic characteristics are 

summarized in Table 1. 

Table 1: Demographic Information for Total Sample (n=1054) 

Categorical 

Variable 

Ethnicity 

(n=791) 

6.7 % White/Caucasian 

91.3% Latino/a 

2.0% Other 

Gender 

(n=1028) 

47.7% Male 

51.3% Female 

Primary Language 

(n=812) 

53.8% Spanish 

30.2% English 

16.0% Bilingual/Other 

6.70% 

91.30% 

2% White 

Latino/a 

Other 

51.3% 47.7% 

Male 

Female 

53.8% 30.2% 

16.0% Spanish 

English 

Bilingual/Other 

MEASURES AND DATA COLLECTION 

Two assessment tools were administered to collect data on early literacy 

IGDIS/FAST 

The program developed an assessment drawn from two tools, the Individual Growth & 

Development Indicators (IGDIs) and the Formative Assessment System for Teachers (FAST). 

The IGDIs (Individual Growth & Development Indicators) is an early childhood assessments for 

monitoring the growth and development of preschool-aged children prior to Kindergarten. The 

assessments provide a mechanism for “identifying children who are experiencing difficulties 

acquiring fundamental skills necessary for later academic achievement”10 and can be used to 

target and adjust instruction for individual children. It is a useful tool for programs that utilize a 

10 http://www.myIGDIs.com 

44 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

Response to Intervention (RtI) approach, as does the Santa Cruz Reading Corps, where the 

IGDIs scores are used to identify children eligible for the Tier 3 (intensive one-on-one) and Tier 

2 (small group) interventions. The FAST is another “evidence-based assessment used to screen 

and monitor student progress.”11 These combined IGDIs/FAST literacy assessments were 

conducted for all enrolled children three times during the program (Fall, Winter, Spring). 

Children in Tier 2 or Tier 3 were progress monitored in the specific area of literacy (using one of 

the five assessments in the IGDIs or FAST) in which they are receiving an intervention once 

every four weeks to track growth. 

Five areas of literacy were assessed in the current sample: 

1. Picture Naming 

2. Rhyming 

3. Alliteration 

4. Letter Sounds 

5. Letter Naming 

Child Snapshot 

The Child Snapshot is a form completed by parents and the preschool teacher at the end of the 

preschool year. It includes parent reports of demographic information and home-based 

readiness activities, as well as teacher-rated readiness in social-emotional functioning and 

kindergarten academics. 

Table 2: Child Snapshot Parent-Reported Variables 

Variable Readiness Activities 

READBOOKS/FREQBOOKS Share or read books/Frequency book reading 

SINGSONGS/FREQSONG Tell stories or sing songs/Frequency 

MEALS/FREQMEAL Sit down to meals together?/Frequency 

DRAWS/FREQ DRAW Draw, color, paint, play games/puzzle/Frequency 

LIBRARY/FREQLIB Visit the library?/How many times per week 

BEDTIME Child’s usual bed time on a week night? 

CHORES Does child help with chores around the house? 

REGINFO Information about how and when to register child for school? 

MEETINFO Information on how to meet child's kindergarten teacher? 

INVOLINFO Information about how to get involved with the school/classroom 

NEEDSINFO Information about child's strengths, needs, and interests 

BOOKBAG Participation in Raising a Reader (home book bag program) 

PARENTED Parent education classes (positive discipline, etc.) 

PARSUPP Parent support groups (Together in the Part, etc.) 

FAMHELP Help from extended family or neighbors and/or friends 

COMMHELP Community resources (health clinics, churches, nonprofit) 

11 http://www.fastforteachers.info/ 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 45 | P a g e 
 

California Children and Families Foundation 

15AC170571

Appendix 1 

LITACTIVE Attended school literacy activities or parent nights 

ALLERGY Does child have any food or other allergies? 

HEALTHPROB Does child have any health problems (such as asthma)? 

GLASSES Does child wear glasses? 

LANGPROB Do child have any language/speech concerns about your child? 

SPECNEED Does child have a diagnosed special need, disability or health 

concern? 

IEP Does child have an IEP or an IFSP? 

VISION Health assessments or screenings-Vision? 

HEARING Health assessments or screenings-Hearing? 

MEDICAL Health assessments or screenings- Medical? ? 

DENTAL Health assessments or screenings-Dental? 

DEVELOP Health assessments or screenings-developmental? 

DOCTOR/DENTIST Does your child have a regular doctor/pediatrician?/Dentist? 

Additionally, preschool teachers reported on 2 sets of items. 

Teachers reported on the child’s language development, presence of IEP and general 

attendance, including: 

 progress in primary language; 

 English comprehension skills; 

 English speaking skills; 

 IEP plan and referral; and 

 absence rating. 

Desired Results Developmental Profile (DRDP) 

Preschool teachers also rated readiness items using Desired Results Developmental Profile 

(DRDP) items in the following domains: 

 self-care; 

 social expression; 

 self- regulation; 

 kindergarten academics; and
 

 overall readiness.
 

These five items were scored similarly to the DRDP (1=Not Yet, 2=Beginning, 3=In Progress, 

4=Proficient) and total scale scores represented a mean of scale items (thus, they also ranged 

from 1 to 4). 

ATTRITION AND MISSING DATA 

IGDIs/FAST pre- and post-test data were evaluated for missing values and data attrition. There 

were a total of 1062 cases available for the combined ‘12-‘13 and ‘13-‘14 Reading Corps program 

46 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

 

 

 

-

-

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

years. Eight students had IDs with no data attached and were dropped from the data set. The 

remaining 1054 cases were examined for (a) whether the pre-test score was missing (i.e., 

incomplete data) (b) whether the post-test score was missing (i.e., attrition) and (c) whether both 

scores were missing (i.e., the student was not assessed on that item). The results varied by 

outcome and are presented in Table 3. Attrition rates ranged from 9% to 17% and are all at 

acceptable levels for the WWC. Parent reports of home activities and demographics were 

available for 544 (52%) children, and early educator reports of readiness were available for 318 

(30%) children. 

Table 3: Frequencies for Missing Literacy Data and Student Attrition 

Outcome 

COMPLETE 

DATA 

Used in analyses 

INCOMPLETE 

PAIR 

Missing initial 

score 

MISSING 

PAIR 

Missing pre and 

post scores 

ATTRITION 

Missing final 

score 

Picture Naming 804 160 20 70 (9%) 

Rhyming 673 263 49 69 (10%) 

Alliteration 638 268 65 83 (13%) 

Letter Sounds 428 239 328 59 (14%) 

Letter Names 493 174 304 83 (17%) 

Missing data within cases were handled in one of two ways. First, total scale scores were 

created by taking a mean of scale items, thereby averaging across any missing items within 

scales. Second, missing data were excluded per each analysis, rather than “listwise”, i.e., all

available data for a given analysis was used, rather than excluding cases with any missing data 

for all analyses. 

PRE-POST COMPARISONS OF LITERACY OUTCOMES 

OUTCOME 1: “As a result of program participation, increased early literacy skills as 

measured by the IGDIS/FAST in: 

Pre (Fall) and Post (Spring) scores were available for the following early literacy outcomes: 

 Picture Naming (PNBM1 and PNBM3) 

 Rhyming (RBM1 and RBM3) 

 Alliteration (ALBM1 and ALBM3) 

 Letter Naming (LNBM1 and LNBM3) 

 Letter Sounds (LSBM1 and LSBM3) 

A series of paired-sample were conducted to examine observed differences from fall to spring 

across both program years. There were significant (p <.001) improvements in Pre- and PostF i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 47 | P a g e 
 

 

 

California Children and Families Foundation 

15AC170571

Appendix 1 

scores for all literacy measures such that children scored significantly higher on literacy at the 

end of the program, for both 2012-13 and 2013-14 program years (see Table 3). 

Table 4: Pre-post Comparisons for Literacy Outcomes 

Outcome Variable 

n Mean (std) Mean (std) Paired t* Time 1 (Fall) Time 2 (Spring) 

Picture Naming 804 10.45 (8.99) 19.53 (8.53) 37.77 

Letter Naming 516 5.41 (7.80) 16.35 (11.87) 26.53 

Rhyming 673 2.92 (3.90) 7.90 (6.35) 23.30 

Alliteration 638 1.88 (2.75) 5.16 (4.87) 18.52 

Letter Sounds 428 2.78 (4.44) 9.83 (7.89) 22.06 

*Note: All ts are significant at p<.001 

Comparing Tiers 2 & 3 with Tier 1 only Groups 

The Reading Corps program design utilizes what is currently being called a Response to 

Intervention (RtI) model by the education field. The Reading Corps program provides 

additional small group and individualized support for children with the lowest initial 

IGDIs/FAST assessments. Theoretically, such intensive intervention should raise mean scores 

for the total group, as well as create individual improvements for the children receiving 

intervention. We examined differences in performance for intervention and non-intervention 

students. 

A total of 229 (22%) children (89 in ‘12-‘13 and 140 in ‘13-‘14) received Tiers 2 or 3 intervention

based on early IGDIs scores. As may be expected, children receiving the more intensive Tiers 2 

or 3 interventions had significantly lower initial scores and final scores than children not 

receiving these supplemental interventions. However, children in the intervention group 

experienced similar significant gains in literacy as measured by change scores; their individual 

rates of change were similar to the non-intervention children. In one case (Picture Naming), 

children receiving the Tiers 2 or 3 interventions experienced significantly greater gains than nonintervention children (see Table 5). 

48 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

Table 5: Comparison of Standard Intervention and Tier 2 or 3 Intervention Groups on Literacy 

Outcomes 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

Tier 1 Only Tiers 2 or 3 t 

n Mean (SD) n Mean (SD) 

Average Initial Scores (Fall) 

Picture Naming 660 12.26 (9.08) 214 4.93 (6.13) 11.02*** 

Letter Naming 444 5.94 (8.29) 132 3.00 (4.71) 3.88*** 

Rhyming 562 3.21 (4.04) 180 1.99 (3.26) 3.67*** 

Alliteration 543 2.05 (2.86) 178 1.38 (2.28) 2.83** 

Letter Sounds 381 2.97 (4.52) 106 1.65 (2.99) 2.84** 

Average Final Scores (Spring) 

Picture Naming 742 19.39 (9.13) 222 16.77 (7.27) 3.92*** 

Letter Naming 533 15.72 (12.22) 177 12.12 (9.84) 3.56*** 

Rhyming 720 7.52 (6.22) 216 6.36 (5.61) 2.46** 

Alliteration 695 5.14 (4.95) 211 4.18 (3.98) 2.55** 

Letter Sounds 508 9.63 (8.01) 159 6.97 (6.04) 3.84*** 

Averaged Individual Change Scores (Spring minus Fall) 

Picture Naming 597 8.10 (6.65) 207 11.91 (6.49) -7.146*** 

Letter Naming 390 11.03 (9.36) 126 10.69 (9.44) .35 ns 

Rhyming 501 5.12 (5.52) 172 4.60 (5.65) 1.05 ns 

Alliteration 468 3.44 (4.66) 170 2.85 (3.94) 1.49 ns 

Letter Sounds 328 7.48 (6.71) 100 5.63(6.08) 2.49* 

*p<.05 **p<.01 ***p<.001 

A change score, representing each individual child’s change over the duration of the program, 

was created by subtracting that child’s Fall Score (pre) from the Spring (post) score. T-tests were 

conducted to compare Tier 1-only students and the Tiers 2 or 3 intervention groups on Fall (pre) 

scores, Spring (post) scores, and change scores (Spring minus Fall). 

As shown in Table 5, as a group, children with Tiers 2 or 3 interventions had significantly lower 

pretest and post-test scores than children in the Tier 1-only group for all literacy outcomes. This 

is to be expected because initial pretest scores were used to select children for the Tiers 2 or 3 

interventions. 

In the analysis of the change scores, however, children in the Tiers 2 or 3 group showed 

significantly greater change over the course of the program in picture naming (the area in which 

children received the most frequent progress monitoring assessments), and similar levels of 

change in alliteration, rhyming and letter naming. The only score in which Tiers 2 and 3 

children showed lower change than the other children in the classroom was in letter sounds. 

Because there was no comparable group of children identified as qualifying for Tiers 2 or 3 

interventions who did not receive the intervention, there is no statistical way to know whether 

these similar levels of change represent a benefit of the more intensive interventions. This 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 49 | P a g e 
 

California Children and Families Foundation 

15AC170571

Appendix 1 

analysis does not constitute such a comparison. It merely shows that these initially low scoring 

children are not continuing to score poorly on the literacy measures, and are instead improving 

at similar or greater rates to children in non-intervention groups. 

These results also suggest that a re-examination of the Tiers 2 or 3 intervention curriculum for 

letter sounds, where change rates are significantly lower for Tiers 2 or 3, may be helpful. 

KINDERGARTEN READINESS Benchmarks 

“OUTCOME 2: “As a result of program participation, there will be increased early socioemotional and academic skills as measured by (cross-walked) DRDP” 

DRDP Indication of Readiness 

Because there were no cross-walked DRDP pretest scores of kindergarten readiness, there was 

no way to measure changes in readiness over the course of program participation. Crosswalked DRDP items were available for 318 (30%) of the children. These subscales were scored 

on a 1-4 scale (1=not yet, 2=beginning, 3=in progress, 4=proficient). Table 6 shows means and 

medians for these subscales. While no conclusions can be drawn about program effectiveness 

from these data, they do show that a majority of children are considered “in-progress” (3) or

“proficient “ (4) in these domains at the end of the program.

Table 6: Means and Standard Deviations for Readiness Scores at End of Program (n=318) 

Readiness Scores Mean (SD) Median 

Social Skills 3.54 (.43) 3.60 

Emotion Expression 3.56 (.42) 3.67 

Self-Regulation 3.59 (.38) 3.67 

Academics 3.47 (.40) 3.43 

Total Readiness 3.53 (.36) 3.63 

Correlations between Literacy Scores and Readiness 

Parent and early educator reports of socio-emotional and academic readiness were also 

examined relative to program participation. Descriptive analyses were conducted, and links 

between (a) change scores in literacy assessments, (b) parent activities, and the cross-walked 

DRDP indicators of kindergarten readiness were examined, revealing significant associations 

between these variables. 

Pearson Correlations revealed significant associations between IGDIs/FAST literacy change 

scores and kindergarten readiness (see Table 7). There were also significant correlations 

between literacy change scores and teachers’ ratings of readiness, particularly with the 

academic readiness subscale. Thus, changes in literacy occurring in the program (especially in 

rhyming, alliteration, letter sounds) predicted kindergarten readiness scores (especially in the 

50 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

 

- - -

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

academic skills domain). This provides some support for the idea that literacy intervention 

provided in the program indexed skills needed in kindergarten. 

Table 7: Pearson Correlations between Literacy Change Scores and Kindergarten Readiness Scores 

Kindergarten Picture Naming Rhyming Alliteration Letter Sound Letter Name 

Readiness Change change Change Change Change 

Social 

Skills 

-.02 .11 .10 .18 .02 

Emotional 

Expression 

-.07 .10 .13 .17 .04 

Academic 

Skills 

-.03 .25*** .30*** .30*** .25*** 

Self 

Regulation 

-.04 .18* .16* .20* .13 

Overall 

Readiness 

-.04 .19** .21** .24** .12 

*p<.05 **p<.01 ***p<.001 

Home Readiness Activities and Children’s Readiness 

The Child Snapshot parent data offered an opportunity to examine associations between 

reported parental behavior and children’s literacy and kindergarten readiness. As shown in 

Table 8, there were several small but significant correlations between parent-reported behaviors 

and readiness. Specifically, higher frequency of singing songs, drawing and solving puzzles, 

and eating meals together was linked to kindergarten readiness outcomes. 

Similarly, several reported parenting preparation activities were significantly linked to one or 

more literacy change score measures seeking out registration information, seeking out 

information on becoming involved in school, participation in “Raising a Reader” book program, 

help from family of neighbors, and help from community resources. 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 51 | P a g e 
 

 

 

California Children and Families Foundation 

15AC170571

Appendix 1 

Table 8: Pearson Correlations between Parents’ Readiness Behaviors and Readiness 

Outcome 

PARENT READINESS BEHAVIORS 

Read 

Books 

Sing 

Songs 

Draw/ 

Puzzles 

Eat Meals 

Together 

Picture Naming -.03 -.09 -.07 -.03 

Rhyming .02 .02 .01 .00 

Alliteration .07 .13* .00 .05 

Letter Sounds .04 .04 .02 .06 

Letter Names .02 -.09 -.13* -.13* 

Social Skills .08 .15** .12* .12* 

Emotional Expression .09 .19** .13* .11 

Kinder Academics .10 .23*** .15** .15* 

Self Regulation .06 .14* .11 .07 

Overall Readiness .09 .19** .14* .12* 

*p<.05 **p<.01 ***p<.001 

Dosage Effects 

No attendance data were available in the current sample. The Child Snapshot data contained 

one question asking the teacher to rate frequency of absences from the program (1=never to 

5=frequently). There were no significant correlations between this rating and any of the literacy 

or readiness variables. 

52 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

 

-

- -

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix I 

QUASI-EXPERIMENTAL COMPARISON 

While pre/post comparisons revealed significant improvements in literacy scores for program 

participants over the course of the program, a quasi-experimental comparison provides a more 

rigorous test of program effectiveness. Ideally, Santa Cruz Reading Corps children would be 

compared to similar children who were in a similar program without the Reading Corps 

component. 

In this case, the same literacy measures were collected over previous program years in Santa 

Cruz County state preschool programs that did not have the Reading Corps component. While 

not ideal for many reasons (e.g., teachers gain in experience over time, characteristics of 

participants change over time and from year to year, etc.), the most recent cohorts (2012-13 and 

2013-14) represent children who experienced the Reading Corps component, compared to 

previous cohorts. Thus, we investigated whether data from these earlier cohorts provided an 

acceptable comparable comparison group for use in evaluating program effectiveness. 

A comparison group was identified representing a subsample of program sites from the ‘11-‘12 

program year where teachers were trained on IGDIs assessment protocols. This comparison 

group included 211 children over 10 sites whose IGDIs assessments were administered Fall, 

Winter, and Spring in picture-naming, alliteration, and rhyming. 

Comparability of Comparison Groups

The first task in such a quasi-experimental comparison is to establish, to whatever extent 

possible, that the groups are comparable. Child ethnicity, language and gender were available 

for nearly all of the children in the sample. However, family demographic data (e.g., family 

income, parent education) were not available for program participants, limiting any 

examination of comparability between the two groups. However, there is reason to believe that 

the participants in different years are similar in that all students were eligible for state 

preschool. 

Table 9: Demographic Characteristics of Comparison Groups 

Categorical 2012 13 and 2013 14 

Variable 2011 12 

Ethnicity 17.7 % White 6.7 % White/Caucasian 

74.7% Latino/a 91.3% Latino/a 

.079% Other 2.0% Other 

Gender 50.3% Male 47.7% Male 

49.7% Female 52.3% Female 

Primary language 60.8% Spanish 53.8% Spanish 

24.8% English 30.2% English 

.14% Bilingual/Other 16.0% Bilingual/Other 

Mean (Std) Age 4.35 (.40) 4.25 (.46) 

in years at program entry 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 53 | P a g e 
 

California Children and Families Foundation 

15AC170571

Appendix 1 

Language, gender, age and ethnicity were compared between the two groups. A summary of 

characteristics of the two comparison groups are shown below in Table 7. Analyses revealed 

significant differences between the comparison groups in ethnicity (χ2= 98.78, p<.001) and 

primary language (χ2= 13.44, p<.01), such that there were proportionally more Latinos and 

proportionally fewer Spanish speakers in the 2012-14 group than the 2011 group. Therefore, 

these two variables would need to be accounted for in subsequent analyses. There were no 

significant differences between the two groups in gender composition or age. 

Attrition/Differential Attrition Rates 

Attrition refers to missing post scores in a pre-post design. High rates of attrition can create a 

situation in which the post-score group is substantially different from the pre-score group. For 

example, children who drop out or who are not assessed at time 2 are also more likely to score 

lower on an initial assessment (low scorers are more likely to drop out). This can create a 

situation in which low scorers are dropped from change scores, thus inflating change scores. 

An additional problem occurs when the comparison groups have vastly different attrition rates

as this calls the groups’ comparability into question (one group may contain more low scorers 

than the other). 

Any quasi-experimental design must be evaluated for both overall attrition and differences in 

attrition rates across the comparison groups. What Works Clearinghouse (WWC) attrition rate 

standards specify with lower overall attrition rates, higher differential attrition rates are 

tolerated and vice versa. In the current comparison then, we needed to evaluate whether the 

attrition rates for literacy outcomes (picture naming, alliteration, rhyming) met these standards

While overall attrition rates for the 2012-13 group were acceptable at 9-13% (see Table 2), 

attrition rates for the 2011-12 comparison group (ranging from 40-48%) were significantly 

higher, leading to a large difference (30% or more) in the attrition rate between the groups. This 

level of differential attrition exceeds WWC standards and indicates that a comparison between 

these two groups using literacy change scores is not likely to be valid. 

, 

. 

Descriptive Comparison of Readiness Scores 

While it was not possible to conduct a valid quasi-experimental comparison of program 

effectives because of high differential attrition rates, early educator rated readiness, in the form 

of DRDP scores were available for 149 children (48%) in the 2011-12 comparison group (without 

Reading Corps) and 318 children (30%) in the 2012-14 sample (with Reading Corps). 

As shown in Table 10, a comparison of readiness scores revealed significantly higher scores on 

all outcomes for children who had participated in Reading Corps after the change compared to 

children who experienced a similar program without the Reading Corps component. 

54 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

Table 10: Mean DRDP Readiness Scores by Comparison Groups: Without Reading Corps (2011-12) vs. 

With Reading Corps (2012-14) 

Outcome n Mean(SD) 

Social Skills*** Before program change 149 3.35 (.38) 

After program change 318 3.54 (.43) 

Emotional Expression*** Before program change 149 3.38 (.41) 

After program change 318 3.56 (.42) 

Self-Regulation** Before program c hange 149 3.47 (.32) 

After program change 318 3.59 (.38) 

Kinder Academics** Before program change 149 3.34 (.37) 

After program change 318 3.47 (.40) 

Overall Readiness*** Before program change 149 3.39 (.30) 

After program change 318 3.53 (.36) 

California Children and Families Foundation 

15AC170571

Appendix I 

Note: Means are significantly different ** p<.01 ***p<.001 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 55 | P a g e 
 

 

 

 

 

 

 

 

 

 

 

California Children and Families Foundation 

15AC170571

Appendix IV 

Santa Cruz Reading Corps Logic Model 

Resources Activities Outputs Outcomes 

First 5 Staff 

 $285,000 total contribution (F5SC and 

District staff, 12 members living 

allowance, all trainings & materials, 

Minnesota Reading Corps fee of $27K. 

 .75 First 5 Santa Cruz Reading Corps 

Coordinator (Christine Sieburg) 

 .20 FTE SEEDS Master Coach (Irene 

Freiberg) 

F5SC: AmeriCorps member recruitment, screening, hiring. 

211 hours of Preschool Literacy Tutor Training from various providers: 

- 5-day AmeriCorps Summer Institute (Sept) & Active Citizenship Training (March) 

- 12 hours online AmeriCorps literacy and social emotional training 

- 3-day Santa Cruz Reading Corps Institute & supplemental trainings (Sept) 

- 8 hours PCA Leadership training 

- 4 hour “Challenging Behaviors” training by First 5 

- 79 hr. SEEDS of Early Literacy training (Sept-Nov.) 

- 2.5 hr. RAR Orientation (Sept) 

- 8 hr. Response to Intervention by SEEDS founder, Kate Horst 

12 trained AmeriCorps members 

each engaged in ~900 hours of 

reading tutor activities (Sept-May) 

# preschool programs with 

enriched literacy environments and 

activities demonstrated by ELLCO 

ratings on 19 measures 

~500 children per year receive 

Increased early literacy 

skills as measured by the 

IGDIs/FAST in: 

 Picture Naming 

 Rhyming 

 Alliteration 

 Letter Naming 

 Letter Sounds 

AmeriCorps 

12 AmeriCorps Reading Corps Tutors (30 - 16 hours Progress Monitoring (IGDIs) (Sept,Nov, Feb) high quality preschool 

hours/week - 2 hour/month “Monthly Meetups” with Reading Corps members (Oct-May) 

# Host Teachers- participate in orientation and ongoing coordination of Literacy Schedule with 

Tutors. Review and address Early Language and Literacy Classroom Observation Tool (ELLCO) 

standards. 

 attending at least 3 hours in 

literacy enriched program and 

engaged in Tier 1+2 instruction 

 assessed with Individual Growth 

& Development 

2 School District Partners 

 Pajaro Valley Unified School District, 14 

classrooms, ~300 students 

 Live Oak School District, 9 2 Internal District Coaches Indicators(IGDIs) assessments 

classrooms,~200 students  Participate in training, meet-ups, and data review 3x/year 

 SEEDS -trained PreK/TK teachers, all  Observe assigned classrooms 2x/mo.  At least 5 children per class 

with Site Supervision Permit.  Provide tutor coaching 2x/mo 

Master Coach 

 Provides SEEDS training 

 Facilitates data review with district coaches and tutors 3x/year for instructional decisions. 

 Observes and coaches district coaches and tutors 

Tutors assess students using IGDIs and FAST 3x/yr. Analyze data & progress monitor students 

identified for Tier 2 or 3 monthly. Enters progress into online data management tool (edSpring) 

Tutors provide daily SEEDS of Early Literacy activities for ~500 children in coordination with host 

teacher for: 

 Whole classroom (Tier 1) 

 Small groups (Tier 2) 5-10 minutes daily 

 One-on-one tutoring (Tier 3) 3-5 minutes daily 

Children receive Raising a Reader books to take home weekly. 

Reading Corps embedded in preschool programs which are 3-8 hours per day, 5 days per week. 

receive Tier 2&3 instruction daily 

(Note: hours of State Preschool 

programming varies from 4 to 9 

Evidence hours per day) -Based Programs 

 SEEDS of Early Literacy (& training by 

founder, Kate Horst) 

o Big 5 Early Literacy skills (conversation 

&oral language, alphabet knowledge, book 

& print skills, phonological awareness, 

vocabulary & background knowledge.) 

o Relationship-based (sensitive, 

encouraging, educational, doing, selfesteem) 

 Minnesota Reading Corps pre-k 

coaching & assessment model 

 Raising a Reader book bag program 

56 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
 

 

 

California Children and Families Foundation 

15AC170571

Appendix III
 

Santa Cruz Reading Corps Evaluation Plan 

Outcome Indicators Data Collection 

Methods & Tools 

Frequency/Schedule 

of Data Collection 

Sample 

Size 

Analytical 

Methods 

Increased Early

Literacy

6. Picture naming 

7. Rhyming 

8. Alliteration 

IGDIs All children 3x/year; Tiers 2 

or 3 children every 4 weeks. 

First assessment last week in 

Sept. or first week in Oct. 

~500 

(100% 

service 

population) 

Paired t-test 

(pre/post) 

Compare with PreReading Corps 

scores (FYs 10/11 

and 11/12) 

9. Letter naming 

10. Letter sounds FAST 

Child 

Snapshot 

Linked to DRDP 

100% Descriptive 

Linked 

AmeriCorps members contribute

to community and strengthen their

career path

 Self report of perceived contribution 

to community from not at all to 

great; narrative explanation 

 Self-report of perceived benefit for 

career path from not at all to great. 

Narrative explanation. 

AmeriCorps Survey 1x, in last month of service. 

All 

AmeriCorps 

serving TT 

Pre-K 

program 

Qualitative narrative, 

simple response 

rates. 

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 57 | P a g e 
 

 

 

California Children and Families Foundation 

15AC170571

Appendix IV 

AmeriCorps Member Perceptions 

All 12 Reading Corps Tutors were invited to participate in an anonymous online survey. Six 

responded. The six that did not responds received two subsequent requests. 

Four of the six (67%) reported that they were very well trained and two (33%) reported that they 

were pretty well trained to have a meaningful role implementing Reading Corps. Comments 

included “college-like training” in general and specifically, the excellent training by Kate Horst. 

One commented “I learn best by doing, so by the time it got towards the end, I think I could 

have used some refreshers on how to use different interventions. I became confused when I 

started to get check-ins with my internal coach on Visual Discrimination in Letter Naming, but 

my students were still not out of the red in Picture Naming. I had not realized that I could have 

been working on letter names earlier/ congruently, because I thought I needed to go in order 

and focus on picture naming first. And by the time I was using other interventions, it was 7 or 8 

months since our initial training on them.”

All 6 (100%) felt they consistently received the support they needed to be effective throughout 

their service period. The only comment suggesting room for improvement was “I was 

consistently supported by the AmeriCorps and Santa Cruz Reading Corps staff. However, I was not so 

supported by my site staff.” 

Most Tutors reported that Reading Corps curriculum was engaging. 

How well does Reading Corps engage the children? 

It can be challenging...

Somewhat, there are times that the children are...

Quite well, most of the children are readily ...

Very much so, all the children are engaged...

0% 

17% 

17% 

67% 

The Tutors reported various personal goals for signing up with Santa Cruz Reading Corps, 

88.3% reporting that the program exceeded their hopes and expectations. 

 My main goal was to spend time with children while getting scholarship money. 

 To aid in closing the achievement gap. To expand my job opportunities out of university. 

 I saw this program as a great "crash-course" in teaching, to see if I want to pursue it in the 

future. 

 To become a more involved member of my community. 

 To help children become kinder-ready 

 To give back to my community and help shape students' lives with meaningful education. 

58 | P a g e F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 
Appendix IV 

Five of the six responding tutors reported that the program exceeded their hopes and 

expectations, with the remaining tutor reporting that it met her hopes and expectations. 

 I came out with MANY more benefits that what I ever could have imagined. 

 I witnessed incredible English language growths both years I was involved in the program. It was 

amazing. 

 I definitely got an insight into what is involved in teaching and what my strengths are, as well as 

where my interests lie. 

When given the opportunity to report anything else about the program, the enthusiasm 

continued. 

 It is an amazing program. I believe that children coming out of the Reading Corps classrooms will 

have great advantages over other children. Not only academic advantages, but self esteem, 

perseverance and the knowledge that they can succeed. 

 I hope it is around for many, many years to come and I feel so lucky to have been a part of such an 

amazing team. 

 I had an overall great experience. I learned a lot about myself, teaching, working with others, and 

in my community. 

 It is one of the most, if not, the most amazing early education programs out there. Their goal is 

important and they know what teaching methods work with kids and their families. I am proud to 

have been part of this program. 

 

California Children and Families Foundation 

15AC170571

F i r s t 5 S e r v i c e C o r p s  S a n t a C r u z R e a d i n g C o r p s F i n d i n g s 59 | P a g e