An Evaluation of the

JUMPSTART 

CALIFORNIA 

PROGRAM

2017-2018

jstart.org

ccr-analytics.com

ccrcca.org

January 2019
Acknowledgements

The growth and resiliency of the children included in this evaluation are a source of 

inspiration. The evaluation team would like to thank these children and the families that 

support them. We are grateful to the many Jumpstart members who committed hours of 

their time to support California’s most vulnerable children and to the Jumpstart campus 

site staff who work tirelessly to support their members. We also want to thank the early 

education organizations that partner with Jumpstart. Their openness to partnering with 

Jumpstart is an indication of their commitment to serving children. Together, these early 

education organizations form one of California’s most vital pieces of infrastructure. Finally, 

we would like to thank Jumpstart’s staff Christine Patton, Senior Director of Research and 

Evaluation and Truyen Tran, Senior Director of Field Programs, Affiliate Partners. Their 

patience and direction allowed for the quick production of the evaluation.

Funding

This report is based on work supported by CaliforniaVolunteers, under grant no. 

15ACHCA0020017, and by the Corporation for National and Community Service (CNCS). 

Opinions or points of view expressed in this document are those of the authors and do not 

necessarily reflect either the official position of CaliforniaVolunteers or CNCS or positions 

that are endorsed by CaliforniaVolunteers or CNCS.

Evaluation Team

Peter Huffaker, CCR Analytics

April Morris, CCR Analytics

Brianna Briones, Child Care Resource Center

Olivia Pillado, Child Care Resource Center

Susan Savage, Child Care Resource Center

About CCR Analytics ccr-analytics.com

Since 2006, CCR Analytics has consistently raised the bar on the quality of analysis 

conducted on the operational data of early education agencies. We believe in providing 

the early education sector with the information and analysis they need to accomplish their 

mission. We do this by supporting data collection efforts, providing top-notch analysis, and 

delivering trainings to all levels of the organization. CCR works with over 100 early education 

agencies, including some of the largest (and smallest) non-profits and school districts in 

the country. Our goal is to provide high quality, cost effective insight and analysis to the 

early education sector. 

About Child Care Resource Center (CCRC) ccrcca.org

CCRC is a unique and distinctive leader in the early care and education industry whose 

mission is to cultivate child, family and community well-being. As a nonprofit organization, 

each month CCRC provides programs to almost 50,000 children and the adults who support 

them in the communities of the Antelope, San Fernando, and Santa Clarita Valleys as well as 

the entirety of San Bernardino County. This evaluation was supported by CCRC’s Research 

and Evaluation team. This team includes 16 professionals skilled in data science that is both 

grounded in the community and has policy-level impact intended to positively affect the lives 

of all communities. 

An Evaluation Of The Jumpstart California Program 2017-2018 i
Executive Summary

Key Findings

● Children participating in the Jumpstart program were found to be making strong 

progress toward school readiness.

● The differences in developmental growth between Jumpstart children and 

children in the comparison group were not statistically significant. 

● Given the Jumpstart-dosage target (100 hours in a program year), impact 

expectations should be low -- likely lower than 0.15. Finding effect sizes in 

that range requires finely tuned measurement tools and large sample sizes 

-- neither of which were available for this evaluation.

● A likely secondary outcome of Jumpstart is the support provided to teachers 

and other early education staff. The evaluation team recommends including 

data and analysis of those supports as an element of future evaluations.

Jumpstart is an early education organization with the vision that “every child in America 

enters kindergarten prepared to succeed.” Their mission is to provide language, literacy, and 

social-emotional programming for preschool children from under-resourced communities 

across the nation and promote quality early learning for all. Jumpstart hopes to achieve this 

mission by partnering with adults, known as Jumpstart Corps members, from local colleges 

and universities as well as other local partners, who are trained in the Jumpstart curriculum 

to engage with preschool children during classroom sessions and Child Centered Time. Given 

the documented link between low adult literacy level and ill social and economic outcomes, 

programs that impact children’s literacy development such as Jumpstart are critical to ensure 

children’s future success.

An evaluation was conducted of the 2017-2018 program year to examine the effect of 

Jumpstart on children’s language, literacy, and social-emotional development in preschool 

programs in California. The evaluation focused on two critical questions:

1. Are Jumpstart participants making developmental gains that will make them 

more school ready?

2. Do Jumpstart participants demonstrate greater gains than comparison 

children over the program year on a developmental continuum measure 

(the Desired Results Developmental Profile (DRDP) (2015): Preschool View)?

In answer to the first question, children participating in the Jumpstart program were found 

to be making strong progress toward school readiness. While few children were meeting age 

specific developmental expectations during the Pre-Assessment, the majority of children 

were meeting age expectations in each domain by the Post-Assessment. Jumpstart children 

are experiencing healthy growth and are either on-track to being school ready or are moving 

closer to that target.

An Evaluation Of The Jumpstart California Program 2017-2018 ii
Figure 2. Percentage of Jumpstart Participants Meeting Age Adjusted Developmental Expectations

Language 

Development

17%

59%

64%

21%

81%

41%

83%

48%

65%

23%

51%

14%

% of Children

Literacy 

Development

Social & 

Emotional 

Development

Approaches to 

Learning - 

Self-Regulation

Cognition -

Math

Cognition - 

Science

Pre Post

To answer the second question, a quasi-experimental design was used to examine whether 

Jumpstart participants demonstrated greater gains than a group of comparison participants 

using the DRDP (2015). The DRDP is widely recognized and used in California to measure 

children’s early learning and development. Children’s fall 2017 Pre-Assessment and spring 

2018 Post-Assessment observations on the DRDP domains and subdomains of Language 

Development, Literacy Development, Social and Emotional Development, Approaches 

to Learning-Self-Regulation, Cognition-Math, and Cognition-Science were analyzed. 

Preschool settings for which DRDP data was available were included in the evaluation. Special 

needs classifications, Pre-Assessment scores, and child demographic data including child 

gender, home language, and ethnicity were first used to match Jumpstart classrooms with 

non-Jumpstart classrooms to identify the comparison group. The result was a sample size 

of 683 children (294 children in the Jumpstart program and 389 children in the comparison 

group), all in Head Start programs with Jumpstart services provided by Jumpstart members 

from seven colleges and universities.

Differences in growth on the DRDP domains and subdomains between Jumpstart children 

and the comparison group ranged from 0.1 to 1.3 points. However, the gains for children 

in the Jumpstart group were not significantly greater than the gains for children in the 

comparison group. Regression models were built for each DRDP domain and subdomain 

controlling for child and program variables including child age (in months) at the time of 

Pre-Assessment, number of days between Pre- and Post-Assessment, and full- versus partday program participation among additional variables to test for effect of the intervention. 

While the pre-regression effect sizes for Language Development (0.11) and Cognition-Math 

(0.07) were at the expected level, the differences in growth between both groups of children 

were not statistically significant.

An Evaluation Of The Jumpstart California Program 2017-2018 iii
Figure 3. Side by Side Growth for Comparison and Jumpstart Groups

Language 

Development

24.0 1.3

25.3

26.6

25.9 26.2

25.9

27.3 27.4

25.5

24.7 24.8

24.1

Average Growth

Literacy 

Development

Social & 

Emotional 

Development

Approaches to 

Learning - 

Self-Regulation

Cognition - Math Cognition - 

Science

Comparison Group Jumpstart Group

0.7

0.7

0.8

0.3

0.1

It is important to set reasonable expectations in evaluating programs like Jumpstart. A typical 

part-day preschool program provides over 600 hours of services in a program year. A quality 

preschool program might have an effect size of 0.5. CCR Analytics’ 2014 analysis of nearly 

50,000 California Head Start children found effect sizes in the range of 0.39 to 0.48 using 

the DRDP (2010). The 2010 National Head Start Impact Study found effect sizes in the range 

of 0.09 to 0.26. Within this context, expectations for a program that provides 100 hours of 

service should be modest. We would be surprised to see Jumpstart effect sizes much larger 

than 0.1 to 0.15. Finding effect sizes in that range requires a finely tuned measurement tool and 

a large sample size (CCR Analytics, 2014; Westat, 2010).

This evaluation had several limitations including lack of available data from the complete 

sample of settings delivering the Jumpstart curriculum. Various obstacles, including 

logistical challenges in accessing and extracting early education programs’ DRDP data 

resulted in the inclusion of data for seven of the 14 campuses Jumpstart partnered with to 

deliver the curriculum. As a result, the data available was only for children in Head Start 

programs and was not inclusive of all early education settings such as state preschool 

programs also served by Jumpstart. Furthermore, lack of data on additional factors such as 

community context, familial income levels, and information on previous Jumpstart experience 

as well as a non-randomized study design and sample size made determining 

the intervention’s true effect difficult.

Recommendations for future evaluation activities include exploring and utilizing child 

development tools more closely aligned to the intervention for measuring child outcomes 

and expanding the evaluation to include outcomes related to the classroom teachers’ and 

Corp members’ experiences. Greater alignment between a measurement tool and the 

Jumpstart program may allow for more precise measurement of child outcomes related to 

the intervention. Notably, the Language Development subdomain measured by the DRDP 

revealed the highest effect size. With language environment as a main focus of the Jumpstart 

curriculum, this may suggest a need for further exploration with this area of development. 

Examining the effects of Jumpstart on teacher outcomes such as well-being and Corp 

members’ knowledge and work experience would contribute to a more holistic view of the 

impact of Jumpstart.

An Evaluation Of The Jumpstart California Program 2017-2018 iv
TABLE OF CONTENTS

Introduction.....................................................................................................................................................1

Jumpstart Sessions...................................................................................................................................1

Evaluation Plan............................................................................................................................................. 2

Evaluation Questions.................................................................................................................................2

Measure ..................................................................................................................................................2

 DRDP Domains ....................................................................................................................................3

 DRDP Ratings - Developmental Levels ....................................................................................................4

 DRDP Domain Scoring ..........................................................................................................................4

Data Collection.........................................................................................................................................5

 Table 1. Jumpstart Higher Education Partners .........................................................................................6

Participants ............................................................................................................................................. 7

 Table 2. Demographic Characteristics ....................................................................................................8

 Group Equivalence...............................................................................................................................9

 Table 3. Average Pre-Assessment Domain Scores .....................................................................................9

Findings .................................................................................................................................................... 10

Data Quality............................................................................................................................................ 10

 Figure 1. Number of Classes by Percentage of Suspect Ratings.................................................................. 10

Evaluation Questions.................................................................................................................................11

 Figure 2. Percentage of Jumpstart Participants Meeting Age Adjusted Developmental Expectations ...............11

 Figure 3. Side by Side Growth for Comparison and Jumpstart Groups ........................................................ 12

Regression Analysis.................................................................................................................................. 12

Language Development - Table 4. Predicting Growth in Language Development Subdomain ............................. 13

Literacy Development - Table 5. Predicting Growth in Literacy Development Subdomain................................... 13

Social & Emotional Development - Table 6. Predicting Growth in Social & Emotional Development Domain ......... 13

Approaches to Learning - Self-Regulation - Table 7. Predicting Growth in Approaches to Learning - .....................

Self-Regulation Domain............................................................................................................................ 14

Cognition - Math - Table 8. Predicting Growth in Cognition-Math Subdomain .................................................. 14

Cognition - Science - Table 9. Predicting Growth in Cognition-Science Subdomain ........................................... 14

Discussion................................................................................................................................................... 15

Limitations ............................................................................................................................................. 16

Recommendations................................................................................................................................... 18

References .................................................................................................................................................. 19

Appendix A: DRDP Domain Scores and Growth .............................................................................................. 20

Appendix B: Full Regression Models ..........................................................................................................22

An Evaluation Of The Jumpstart California Program 2017-2018 v
An Evaluation Of The Jumpstart California Program 2017-2018 1

INTRODUCTION 

Jumpstart is an early education organization with the vision that “every child in America enters 

kindergarten prepared to succeed.” Their mission is to provide language, literacy, and socialemotional programming for preschool children from under-resourced communities across 

the nation and promote quality early learning for all. Jumpstart aims to achieve this mission 

by partnering with Jumpstart Corps members, adult volunteers from local colleges and 

universities as well as other local partners. Jumpstart Corps members are trained in the 

Jumpstart curriculum to engage with preschool children during classroom sessions.

Jumpstart Sessions

To provide consistent delivery of the Jumpstart curriculum through purposeful learning 

activities, after extensive pre-service training, Corps members conduct two-hour sessions 

twice a week during the school year. During these sessions, each Corps member partners with 

a small group of three to five children to directly support them during the session. They also 

work with other Corps members and classroom teachers to support the entire classroom and 

to ensure collaboration in delivering the Jumpstart curriculum to children. At the time of this 

study, sessions supported children’s language, literacy, and social-emotional development by 

building children’s skills in three areas:

Oral Language with an emphasis on building vocabulary and comprehension through 

children learning new words and talking about the books during reading activities.

Book and Print with an emphasis on building alphabet knowledge and the meaning 

and use of print through children practicing letters and sounds as well as writing.

Phonological or Sound Awareness with an emphasis on phonemic and rhyme awareness through 

children listening to different words during circle time and reciting poems with Corps members.

Jumpstart utilizes a set of 20 children’s books that have been selected based on their ability 

to support vocabulary and comprehension during sessions to aid in development of these 

skill areas. A Jumpstart classroom session typically follows this routine: 

● Welcome – Transition into Jumpstart session from a previous activity.

● Reading – Corps members lead children in a reading activity and discussion 

of the story, prompting the learning of vocabulary words and comprehension. 

● Circle Time – Corps members and children sing songs, play word and letter 

games, and read poems.

● Center Time – Activity centers are set up to support language and literacy 

development, depending on the type of story that was read with the aim of 

deepening children’s understanding of the story. 

● Let’s Find Out About It – This activity is geared toward children’s ability 

to learn new things and ask about how they work, with a focus on building 

vocabulary and concept knowledge.

● Sharing and Goodbye – In a larger group, Corps members and children talk 

about their favorite parts of the session.
An Evaluation Of The Jumpstart California Program 2017-2018 2

In addition to the Jumpstart session, each Corps member provides 2-4 hours per week 

of additional service to children through Child Centered Time. This allows for additional 

support in strengthening skills that are specific needs for those individual children. Through 

these efforts, Jumpstart provides consistent programming with low adult-child ratios that 

elicit meaningful interactions in the hopes of positively influencing children’s language and 

literacy development. 

EVALUATION PLAN

Evaluation Questions

As a means of assessing the Jumpstart program’s progress toward achieving its mission 

and in order to build upon past findings, an evaluation of the 2017-2018 program year was 

conducted. A child-level outcome tied to Jumpstart’s theory of change was the focus of this 

evaluation - children from under-resourced communities are on-target with early literacy 

skills as an element of kindergarten readiness. The evaluation aimed to assess progress 

toward this outcome by answering the following questions:

1. Are Jumpstart participants making developmental gains that will make them 

more school ready?

2. Do Jumpstart participants demonstrate greater gains than comparison 

children over the program year on a developmental continuum measure 

(the Desired Results Developmental Profile (DRDP) (2015): Preschool View)?

In order to answer these questions, the evaluation made use of demographic data to first 

match Jumpstart classrooms with non-Jumpstart classrooms for comparison (matching 

procedures are described below). Classroom scores on the DRDP from the Pre-Assessment 

(conducted in the fall of 2017) and the Post-Assessment (conducted in the spring of 2018) 

were used to investigate differences in score gains between these groups and the effect of 

the Jumpstart intervention. The DRDP tool was used to build upon findings from previous 

evaluations of Jumpstart, as this tool is widely used in California to assess children’s early 

learning and development during the years leading up to kindergarten transition.

Measure

The Desired Results Developmental Profile (DRDP) 2015, an observation-based assessment 

tool developed by the California Department of Education is used to document 

developmental milestones over time using written observations, photographs and samples 

of the child’s work. The DRDP (2015) Preschool View is made up of eight domains of 

developmental constructs. However, only the four in bold were analyzed for this evaluation: 

Approaches to Learning–Self-Regulation (ATL-REG), Social and Emotional Development 

(SED), Language and Literacy Development (LLD), English-Language Development (ELD),

Cognition, Including Math and Science (COG), Physical Development–Health (PD-HLTH), 

History-Social Science (HSS), Visual and Performing Arts (VPA). Each domain is composed 

of four to eleven measures. In total, the DRDP (2015) Preschool tool contains 56 measures. 

Teachers collect observations for participating children into portfolios that teachers then 

use to assess children on each measure individually. The DRDP assessment is completed 3 

times a year (at pre, mid, and post) in Head Start programs and 2 times a year for California 

state funded programs. Only the Pre- and Post-Assessments conducted in the fall and spring, 

respectively, were used in the evaluation.
An Evaluation Of The Jumpstart California Program 2017-2018 3

DRDP Domains

Language and Literacy Development - This domain assesses children’s development in 

foundational language and literacy in any language or mode of communication. Language skills 

assessed in this domain include understanding of language (receptive), responsiveness to 

language, communication and use of language (expressive), and reciprocal communication and 

conversation. Literacy skills assessed in this domain include interest in literacy, comprehension of 

age-appropriate text, concepts about print, phonological awareness, letter and word knowledge, 

and emergent writing. The two subdomains, Language and Literacy, were analyzed separately as 

part of the evaluation.

Social and Emotional Development - This domain assesses children’s ability to understand and 

interact with others as well as their ability to form relationships. Skill areas assessed in this domain 

include social and emotional understanding, identity of self in relation to others, relationships and 

social interactions with familiar adults, relationships and interactions with peers, and symbolic and 

sociodramatic play. 

Approaches to Learning-Self-Regulation - This domain assesses children’s learning and selfregulation as both are important to school readiness and success. Items that assess approaches to 

learning include curiosity and initiative, attention maintenance, engagement and persistence. Selfregulation skills measured include self-control of feelings and behavior, self-comforting, imitation, 

and shared use of space and materials.

Cognition, Including Math and Science - This domain assesses children’s skills in observation, 

exploration of people and objects, and investigation of objects and concepts. Math skills 

assessed include spatial relationships, cause and effect, classification, number sense of 

quantity, number sense of math operations, measurement, patterning, and shapes. Science 

skills assessed include inquiry through observation and investigation, documentation and 

communication of inquiry, and knowledge of the natural world. The two subdomains Math and 

Science were analyzed separately as part of the evaluation.
An Evaluation Of The Jumpstart California Program 2017-2018 4

DRDP Ratings - Developmental Levels

Children are rated on a developmental spectrum covering early infancy to kindergarten with 

up to nine developmental levels for each measure. The number of developmental levels in 

a measure varies depending on the competencies that are appropriate for that measure’s 

developmental continuum. The levels are organized under four categories: Responding, 

Exploring, Building, and Integrating. Children are rated a particular developmental level once 

they demonstrate the skills, behaviors, and knowledge defined by that level consistently over 

time and in different situations or settings.

Responding (Earlier, Later) - Children rated as “responding” exhibit knowledge, skills, or behaviors 

that develop from basic responses (e.g., using senses) to differentiated responses. These responses 

are generally observed in back-and-forth interactions between children and familiar adults as well as 

through children using nonverbal messages. 

Exploring (Earlier, Middle, Later) - Children rated as “exploring” exhibit knowledge, skills, 

or behaviors that include active exploration, such as manipulation of objects, and purposeful 

movements and communication. Typically, children move from nonverbal to verbal communication 

at this level. 

Building (Earlier, Middle, Later) - Children rated as “building” exhibit knowledge, skills, or 

behaviors indicating their understanding of how things work, how people and objects relate to one 

another, and how to investigate ideas. Children rated at this level are using verbal communication to 

express thoughts and feelings, are developing literacy and numeracy skills, and are participating in 

small group interactions. 

Integrating (Earlier) - Children rated as “integrating” exhibit knowledge, skills, or behaviors 

that show their ability to express complex thoughts and feelings, problem solve, and participate 

in a wide range of activities. Children rated at this level demonstrate their ability to engage in 

mutually supportive interactions.

DRDP Domain Scoring

Domain scores are based on a psychometric scoring model that takes into consideration 

the difficulty levels of different steps in the developmental continuum. The domain 

scores are centered on 200 and range between 100 (expected in early infancy) and 300

(expected at Kindergarten). In order to reach 300 at the age of five, a growth of 20 points 

every 6 month period of time would be expected between early infancy and reaching 

the top of the developmental spectrum on the tool by the age of five. The average time 

between the Pre- and Post-Assessments included in this evaluation is just over six 

months. As a result, children would be expected to increase in their developmental level 

by approximately 20 points between Pre- and Post-Assessment. 

In order to meaningfully interpret the assessment results, CCR Analytics aligned the 

DRDP (2015) Preschool tool to the California Preschool Learning Foundations. The 

alignment allows us to understand a child's development within an age appropriate 

context and to meaningfully compare results across the DRDP domains and subdomains. 

The specifics of the alignment are available at ccr-analytics.com/presentations/ “Linking 

DRDP to Foundations.” The California Early Learning Foundations are established for 

5 different ages (8 months, 18 months, 36 months, 48 months, and 60 months). After 

CCR Analytics established the alignment between those ages and the DRDP (2015) 

Preschool, expectations were estimated for ages in between those age groups. This 

allows one to assess whether a child who is 54 months old is on track to meet the 60 
An Evaluation Of The Jumpstart California Program 2017-2018 5

month age expectations. This step is important because of the rapid growth typical of 

young children. Six months can make a big difference in development. You would not 

use a 54 month benchmark to assess a child 42 months old. As a result, as children 

age, the development expectations increase in the analysis reporting. The results are 

reported as the percentage of children at or above age expectations. Increases in group 

scores between two assessment periods indicate that children are developing faster than 

developmental expectations.

Data Collection

Jumpstart partners with 14 higher education universities and colleges across California 

to train Corp members and deliver the Jumpstart curriculum in early education 

settings. Data from seven (shown in bold in Table 1) of the 14 campuses and the early 

education programs in which the curriculum was delivered were available for inclusion 

in this evaluation. The early education programs are run by a variety of organizations -- 

nonprofits, school districts, county offices of education, and local governments. Teachers 

and staff from each early education program collect the DRDP data in one of five ways:

1. DRDPtech -- an online software developed by the California Department of 

Education, WestEd, and Berkeley’s BEAR Center.

2. Learning Genie -- an application developed to help programs collect child 

portfolios and DRDP data.

3. CCR Scanning Services -- a service provided by CCR Analytics which collects data 

from DRDP rating records.

4. COCOA -- an online software to track attendance and child reporting, developed 

by the San Francisco Office of Early Care and Education. 

5. ChildPlus Software-- software used by multiple Head Start programs to manage 

their child, family, and administrative data.

In late summer 2018, the California Department of Education unexpectedly shutdown 

DRDPtech forcing users to transition to a new software, DRDP Online. Unfortunately, 

many components of DRDP Online were not yet available. This resulted in early education 

programs that relied solely on DRDPtech to no longer have access to their DRDP data and 

could not make it available for this evaluation. The data available for inclusion consisted 

of DRDPtech users who had previously downloaded their data for external evaluation 

and programs that used Learning Genie, CCR Scanning Services, ChildPlus Software, or 

COCOA and still had access to their program’s DRDP data at the time of the evaluation.
An Evaluation Of The Jumpstart California Program 2017-2018 6

Table 1. Jumpstart Higher Education Partners

University/College # of Classes Supported # of Children Supported

California State University, Dominguez Hills 7 125

California State University, Fresno 15 313

California State University, Fullerton 7 142

California State University, Los Angeles 3 55

California State University, Northridge 7 114

Pepperdine University 11 234

Pitzer College 7 114

San Francisco State University 32 601

St. Mary's College 7 154

University of California, Berkeley 7 167

University of California, Irvine 14 314

University of California, Los Angeles 13 274

University of Southern California 10 173

Whittier College 7 125

TOTAL 147 2905
An Evaluation Of The Jumpstart California Program 2017-2018 7

Participants

Data was available from 34 of the 147 classes supported by Jumpstart with 456 children who 

had both a Pre- and a Post-assessment. Data for children who did not receive a minimum 

of 100 hours of Jumpstart services were excluded from the evaluation. As a result, data 

from 162 children and 8 classes were dropped from the analysis. The final data set included 

data on 294 children receiving Jumpstart services1 in 26 classes. All of the classes were in 

Head Start programs, programs which support children’s growth and development through 

comprehensive services including services targeted towards early learning, health, and family 

well-being. In a majority of cases, children’s families’ income must fall below the poverty 

guidelines to qualify and receive Head Start services. 

Comparison group classrooms were chosen one by one at each program by selecting a nonJumpstart classroom with similar demographic characteristics to a Jumpstart classroom 

at that program (in one case, a match could not be found within the same program and a 

comparison group class was chosen from another program). The evaluation team matched 

classes first on Pre-assessment Language and Literacy DRDP domain scores, then by age, 

followed by gender, ethnicity and language demographic characteristics. The 26 classes in the 

matched comparison group contained 389 children with both Pre- and Post-Assessment data.

 As Table 2 shows, the demographic characteristics of the Jumpstart and comparison group 

children indicate that they were roughly equivalent, although there were some difficulties 

in matching the groups. About half of the children in each group were female and about 

half were male. Both groups had a large percentage of Latino children. However, there 

were differences in the percentage of English- and Spanish-speakers. Due to the Jumpstart 

group having more children with their ethnicity and/or language being unknown (7%), 

the comparison group had a larger percentage of Latino Spanish speakers (51%) than the 

Jumpstart group (41%). Additionally, there was a rather large difference in the percentage of 

Asian children who speak an Asian language between the two groups. This difference is due 

to one agency having a few Jumpstart classrooms being comprised of predominantly Chinese 

children, whereas the classrooms available to match were not of the same ethnic makeup. 

All other ethnicity-language categories had roughly equivalent percentages of children 

represented in each group.

1 Children considered as 

having fully received the 

Jumpstart intervention 

received 100 mentoring 

hours throughout the 

program year, including 

both session hours and 

Child Centered Time.
An Evaluation Of The Jumpstart California Program 2017-2018 8

Table 2. Demographic Characteristics

Jumpstart Children

(n=294)

Comparison Group 

Children (n=389)

Total

(n=683)

Characteristics Percent N Percent N Percent N

Gender

 Male 46% 135 49% 191 48% 326

 Female 54% 159 51% 198 52% 357

Ethnicity and Language

 Latino, English Speakers 22% 66 22% 87 22% 153

 Latino, Spanish Speakers 41% 122 51% 200 47% 322

African-American,

English Speakers 11% 33 16% 62 14% 95

 Asian, English Speakers 3% 10 2% 7 3% 17

 Asian, Asian Language 

Speakers 10% 30 4% 15 7% 45

 Caucasian, English 

Speakers 4% 12 3% 10 3% 22

 Other 7% 21 2% 8 4% 29

Other Child Characteristics

Children with an 

Individualized Education 

Program (IEP)

10% 29 16% 61 13% 90

Children enrolled in Head 

Start in ‘16-‘17 43% 126 39% 150 40% 276

Children enrolled in Full 

Day Classes 53% 156 41% 154 45% 310

Child Age

Jumpstart Children Comparison Group Children

Average age in months at 

Pre-Assessment 48.4 48.3
An Evaluation Of The Jumpstart California Program 2017-2018 9

Group Equivalence

Displayed in Table 3 are both Jumpstart and comparison group Pre-Assessment 

DRDP scores by domain, indicating that the groups were roughly similar in their 

Pre-Assessment scores. 

Table 3. Average Pre-Assessment Domain Scores

Domain Jumpstart Children Comparison Group Children

Score N Score N

Language Development 204.4 294 207.2 388

Literacy Development 207.4 280 207.8 383

Social and Emotional 

Development 207.6 294 209.7 389

Approaches to Learning - 

Self-Regulation 205.8 293 207.6 389

Cognition - Math 206.7 294 208.9 385

Cognition - Science 203.7 287 204.6 324

Note: Sample sizes differ by domain due to some classrooms not conducting every item in the domain.
An Evaluation Of The Jumpstart California Program 2017-2018 10

Findings

Data Quality

A data check to assess the validity of the DRDP data collected was conducted first. 

A common issue with the DRDP is that children receive unrealistically low assessment 

scores on the Pre-Assessment -- 3 and 4 year-olds being assessed at levels typical of infants 

or young toddlers. Due to this concern, the evaluation team flagged any rating that was 

three or more developmental levels lower than the age adjusted expectations based on an 

alignment of the DRDP with the California Early Learning Foundations. Classes with a high 

percentage of ‘suspect’ ratings generally indicates that the teacher is not conducting a 

reliable assessment using DRDP criteria -- often due to inexperience or insufficient training 

on the DRDP. Figure 1 shows the number of classes by the percentage of suspect ratings in 

each class. To account for this issue in the evaluation, the percentage of suspect ratings 

in a class was included as a variable in the regression models. Models were also built 

excluding classes with greater than 10% suspect ratings - although the results were the 

same and these secondary models are not included in the final report.

Figure 1. Number of Classes by Percentage of Suspect Ratings

< 1%

4 4

8

1

5

6 6

5 5

2

3 3

# of Classes

% of Suspect Ratings

1% < 2.5% 2.5% < 5% 5% < 7.5% 7.5% < 10% > 10%

Comparison Classes Jumpstart Classes
An Evaluation Of The Jumpstart California Program 2017-2018 11

Evaluation Questions

1. Are Jumpstart participants making developmental gains that will make them more 

school ready?

In each domain, Jumpstart children showed meaningful improvements on their path to school 

readiness. In Language Development, only 17% of children met California Early Learning 

Foundation Age Expectations at the Pre-Assessment. By the Post-Assessment, 59% of 

children met age expectations in Language Development. Keep in mind that in this analysis, 

the developmental expectations increased with the children’s age. Meaning that children 

grew faster than developmental expectations. To be concrete about the developmental 

expectations in Language Development, a 60 month old child is expected to “understand 

and use both simple and complex words that describe the relations between objects.” For 

example, “After reading a story about the zoo, the teacher asks, ‘What animals are smaller 

than an elephant?’ The child correctly identifies a lion, a tiger, a bear, and a zebra.” Of course, 

not all of the children were 60 months-old at the time of the Post-Assessment, but the 

analysis indicates that 59% of children were on-track to meet those age expectations by the 

time they reach 60 months (California Department of Education, 2008).

For Literacy Development, the percentage of children meeting California Early Learning 

Foundation Age Expectations increased from 21% to 64% between the Pre- and PostAssessments. For Social and Emotional Development, the numbers increased from 41% 

to 81%. For Approaches to Learning-Self-Regulation, the numbers increased from 48% to 

83%. For Cognition - Math, the numbers increased from 23% to 65%. And in Cognition - 

Science, the numbers increased from 14% to 51%.

The analysis clearly shows that Jumpstart children are experiencing healthy growth and are 

either on-track to being school ready or are moving closer to that target.

Figure 2. Percentage of Jumpstart Participants Meeting Age Adjusted Developmental Expectations

Language 

Development

17%

59%

64%

21%

81%

41%

83%

48%

65%

23%

51%

14%

% of Children

Literacy 

Development

Social & 

Emotional 

Development

Approaches to 

Learning - 

Self-Regulation

Cognition -

Math

Cognition - 

Science

Pre Post
An Evaluation Of The Jumpstart California Program 2017-2018 12

2. Do Jumpstart participants demonstrate greater gains than comparison 

children over the program year on a developmental continuum measure 

(the Desired Results Developmental Profile (DRDP) (2015): Preschool View)?

Findings from initial analyses indicate that Jumpstart children did not demonstrate 

statistically significant greater growth over the program year in the DRDP domains than 

the children in the comparison group. Straight average differences in growth (prior to any 

regression analyses) among the domains and subdomains between Jumpstart children 

and the comparison group ranged from 0.1 to 1.3 points (Figure 2). At first glance this looks 

promising, with Jumpstart children growing at modestly higher rates. The corresponding 

effect sizes2

 for Language Development (0.11) and Cognition - Math (0.07) were at the level 

we hypothesized to be most likely for a program with Jumpstart’s dosage (100 hours over 

several months). However, none of the differences in growth were statistically significant.

Figure 3. Side by Side Growth for Comparison and Jumpstart Groups

2 Effect sizes were 

calculated by dividing each 

difference in growth by the 

standard deviation of the 

comparison group. See 

Appendix A.

Language 

Development

24.0 1.3

25.3

26.6

25.9 26.2

25.9

27.3 27.4

25.5

24.7 24.8

24.1

Average Growth

Literacy 

Development

Social & 

Emotional 

Development

Approaches to 

Learning - 

Self-Regulation

Cognition - Math Cognition - 

Science

Comparison Group Jumpstart Group

0.7

0.7

0.8

0.3

0.1

Regression Analysis

To further investigate whether the differences in domain score growth between Jumpstart 

and comparison group children were significant, a regression model was built for each DRDP 

domain and subdomain as reported in Tables 4-9. For each model, the growth in the domain 

or subdomain was used as the dependent variable. Each model also contained a binary 

variable “Jumpstart vs. Comparison Group” that was one for the Jumpstart group and zero 

for the comparison group. The coefficient and statistical significance of the Jumpstart vs. 

Comparison Group variable indicate the size and impact of the Jumpstart program controlling 

for the other variables in the model. If the p-value is less than or equal to 0.05, then the 

difference between the intervention and comparison is considered statistically significant and 

the coefficient shows how much higher the growth was for Jumpstart children -- adjusted 

for the differences between the two groups. If the p-value is greater than 0.05 then the 

coefficient (the adjusted difference in growth) is considered indistinguishable from zero. 

The Jumpstart vs. Comparison Group variable was not found to be statistically significant 

in any of the models.
An Evaluation Of The Jumpstart California Program 2017-2018 13

The other variables controlled for in the regression models included age (in months) at the 

time of the Pre-assessment, the number of days between the Pre- and Post-assessments, 

the Pre-assessment domain score, gender, child ethnicity and language, which program the 

child attended, whether the child was enrolled in the program during the 2016-2017 school 

year, whether the child was enrolled in a full-day or part-day program, whether the child had 

an Individualized Education Plan (indicating a special need), and the percentage of suspect 

ratings for the class. Clustered Robust Standard Errors were calculated, clustering on the 

class for each child. This step is important as the observations are not fully independent. 

The practical effect of using Clustered Robust Standard Errors is that the reported p-values 

are higher, while the coefficients are unchanged. Full results for each regression model can 

be found in Appendix B.

Language Development

The regression model for the Language Development subdomain explains 43.81% of 

the variance in the difference in growth scores in the Language subdomain. Jumpstart 

participation did not significantly predict a larger growth in the Language Development 

subdomain score (t = -0.42, p = .68).

Table 4. Predicting Growth in Language Development Subdomain (N = 682)

Predictor Coefficient Robust Standard 

Error

t p 95% CI

Jumpstart vs.

Comparison Group -0.63 1.49 -0.42 0.68 [-3.61, 2.36]

Literacy Development

The regression model for the Literacy Development subdomain explains 41.35% of the 

variance in growth, and Jumpstart participation did not significantly predict greater growth 

in the Literacy Development subdomain score (t = -0.4, p = .69).

Table 5. Predicting Growth in Literacy Development Subdomain (N = 663)

Predictor Coefficient Robust Standard 

Error

t p 95% CI

Jumpstart vs. 

Comparison Group -0.67 1.68 -0.4 0.69 [-4.05, 2.70]

Social and Emotional Development

The regression model for the Social and Emotional Development domain explains 44.13% of 

the variance in growth, and Jumpstart participation did not significantly predict growth in 

the Social and Emotional Development domain (t = -1.15, p = .26). 

Table 6. Predicting Growth in Social and Emotional Development Domain (N = 683)

Predictor Coefficient Robust Standard 

Error

t p 95% CI

Jumpstart vs. 

Comparison Group -1.71 1.49 -1.15 0.26 [-4.70, 1.28]
An Evaluation Of The Jumpstart California Program 2017-2018 14

Approaches to Learning - Self-Regulation

The regression model for the Approaches to Learning-Self-Regulation domain explains 

44.03% of the variance in growth, and Jumpstart participation did not significantly predict 

larger growth in the Approaches to Learning-Self-Regulation domain (t = -1.15, p = .25).

Table 7. Predicting Growth in Approaches to Learning-Self-Regulation Domain (N = 682)

Predictor Coefficient Robust Standard 

Error

t p 95% CI

Jumpstart vs. 

Comparison Group -1.90 1.64 -1.15 0.25 [-5.20, 1.40]

Cognition - Math

The regression model for the Cognition-Math subdomain explains 41.58% of the variance 

in growth, and Jumpstart participation did not significantly predict greater growth in the 

Cognition-Math subdomain (t = -0.64, p = 0.53). 

Table 8. Predicting Growth in Cognition-Math Subdomain (N = 679)

Predictor Coefficient Robust Standard 

Error

t p 95% CI

Jumpstart vs. 

Comparison Group -1.05 1.66 -0.64 0.53 [-4.38, 2.27]

Cognition - Science

The regression model for the Cognition-Science subdomain explains 39.64% of the variance 

in growth, and Jumpstart participation did not significantly predict greater growth in the 

Cognition-Science subdomain (t = -0.31, p = 0.76). 

Table 9. Predicting Growth in Cognition-Science Subdomain (N = 611)

Predictor Coefficient Robust Standard 

Error

t p 95% CI

Jumpstart vs. 

Comparison Group -0.53 1.70 -0.31 0.76 [-3.94, 2.89]
An Evaluation Of The Jumpstart California Program 2017-2018 15

Discussion

Reading and writing skills are fundamental to functioning as a productive adult in current 

American society. Research shows that adults with low levels of literacy are plagued 

by many social and economic ills including being paid lower wages, lower levels of 

engagement in employment, reduced likelihood of voting, being less informed about civic 

matters, reduced likelihood of being able to meet their family’s health care needs, and 

being more likely to have an interaction with law enforcement, or other social challenges 

such as drug use, teen pregnancy, and violence (Bennett, Brown, Boyle, Racine, & Offord, 

2003; Haigler, Harlow, O’Connor, & Campbell, 1994; Kellam & Anthony, 1998; Kirsch, 

Jungeblut, Jenkins, & Kolstad , 2002; Matson & Haglund, 2000; Sum, 1999). Programs like 

Jumpstart that set children on a better path to literacy development are critical to ensure 

future success.

Findings from this evaluation suggest that although developmental change scores were higher 

for Jumpstart children than comparison group children, Jumpstart participation did not result 

in statistically significantly greater growth than the comparison group between Pre- and PostAssessments on the DRDP domains and subdomains analyzed. However, evidence suggests 

some modest effect of the program. The fact that the Language domain measured by the 

DRDP revealed the highest effect size and the language environment is a main focus of the 

Jumpstart curriculum may suggest a need for further exploration with this area 

of development.

This leads to opportunity for further investigation into Jumpstart’s effects on both children 

and the adults who support them during Jumpstart sessions. By definition, all Jumpstart 

participants are already participating in an early education program. The Jumpstart 

intervention lowers the child to adult ratio and introduces the Jumpstart curriculum into 

supported classes. An intervention that seeks to enhance an existing early education 

environment is likely to have a lower impact than an intervention that moves a child into 

an early education environment. Jumpstart’s supports are also likely to have positive 

effects for the staff, the impacts of which may not immediately accrue to the children. 

For example, higher teacher satisfaction may result in higher retention over time. The 

implications of these insights for understanding Jumpstart’s impact shape the discussion 

and recommendations. 
An Evaluation Of The Jumpstart California Program 2017-2018 16

It is important to set reasonable expectations in evaluating programs like Jumpstart. 

A typical part-day preschool program provides over 600 hours of services. A quality 

preschool program might have an effect size of 0.5. CCR Analytics’ 2014 analysis of nearly 

50,000 California Head Start children found effect sizes in the range of 0.39 to 0.48 using 

the DRDP (2010). The 2010, national Head Start Impact Study found effect sizes in the range 

of 0.09 to 0.26. Within this context, expectations for a program that provides 100 hours of 

service should be modest. It would be surprising to see Jumpstart effect sizes much larger 

than 0.10 to 0.15. Finding effect sizes in that range requires a finely tuned measurement tool 

and a large sample size (CCR Analytics, 2014; Westat, 2010).

Understanding the uses and limitations of the DRDP also helps to put these findings 

into perspective. The DRDP tool has a variety of important uses. As a developmental 

assessment, it is an authentic measure of children’s development (in natural settings and 

activities and over time). The tool is aligned with the California Early Learning Foundations, 

the Head Start Early Learning Outcomes Framework, and the Office of Special Education 

Programs Child Outcomes and is used by many early childhood programs throughout 

California. The tool allows for programs to report to their funders on how children are 

developing in their programs. Additionally, the tool provides for evidence-based planning 

for individual children as well as groups of children as educators plan their lessons and 

activities. Finally, the tool fosters parent engagement when used to provide parents with 

feedback on their child’s development and activities are provided for the parent to engage 

their children in learning activities at home. 

Any measure used to evaluate the effectiveness of an intervention should have a sensitivity 

level for measuring change that is well matched to the intervention. Developmental 

measures are generally expected to have scores that follow a child’s developmental 

level, which is expected to increase over time. There is not sufficient research on the 

DRDP to know if it has the sensitivity level adequate to differentiate between intervention 

and comparison groups in the same way that more standardized and norm-referenced 

outcomes measures do. As a result, an intervention may need to have a very high level of 

dosage or intensity for a greater period of time in order for a developmental measure to 

pick up the differences between an intervention and comparison group.

Additionally, alignment between the domains of the DRDP and the key domains of the 

Jumpstart curriculum should be explored. Domains that are not aligned well may not need 

to be evaluated in the future. Alignment between the DRDP and the Jumpstart checklist 

that has been used in prior evaluations could also be considered. If domains on these 

two measures do not align well, this could explain differences between these results and 

those in prior evaluations. Initial review of these three resources (DRDP with Jumpstart 

curriculum and Jumpstart checklist) suggest that there may be challenges with alignment.

Limitations

This evaluation had several limitations, mostly due to lack of data. First, there was a limited 

amount of demographic and DRDP score data available. Although Jumpstart partners with 

about 14 universities and colleges in California, the evaluation team only had data from seven 

of those campuses to analyze, making it difficult to generalize findings to all Jumpstart efforts 

in California. The fact that the partnering early care and education programs employed 

different data systems for collecting their DRDP posed a challenge, particularly when DRDP 

data became unexpectedly unavailable to programs using DRDPtech to collect their data. 
An Evaluation Of The Jumpstart California Program 2017-2018 17

Additionally, data that was available was only for Head Start centers, not including the 

other early childhood education settings that Jumpstart serves. Therefore, findings can 

only speak to the effect of Jumpstart on Corps members from those campuses and children 

from the Head Start centers they supported. Based on eligibility for the program, most 

families enrolled in Head Start have incomes below the Federal Poverty Level or qualify for 

other serious criteria such as homelessness or status as a foster child. Many of the families 

in both the intervention and the comparison groups are likely to be living in poverty and 

from predominantly disadvantaged backgrounds. However, certain factors that could have 

helped explain the effectiveness of Jumpstart on this sample were not available, e.g., 

community context and familial income levels. 

Another factor that could have potentially influenced the effect of the intervention and 

should be considered in future evaluations is prior exposure to the Jumpstart curriculum. 

Although previous Head Start experience was included in the regression models, having 

data on previous Jumpstart experience would have further contributed to knowledge of 

how the Jumpstart intervention from the program year 2017-2018 affected language and 

literacy development on sample children. 

Lastly, as with any non-randomized study, it is difficult to determine the intervention’s 

true effect on participants’ gains because we can’t know if the findings are due to the 

intervention or other factors that contributed to whether children participated in Jumpstart 

or not. Limitations on available data result in challenges with conducting specific types of 

analyses as well, such as nested design analyses. These generally require larger datasets

and the ability to nest within different levels.
An Evaluation Of The Jumpstart California Program 2017-2018 18

Recommendations

Recommendations for future evaluation activities include making use of other child 

development tools to measure developmental change, or evaluating the alignment between 

domains of the Jumpstart curriculum, Jumpstart checklist, and the DRDP as well as 

investigating the classroom teachers’ experiences. If certain measures within the DRDP 

are not well-aligned with the Jumpstart curriculum or the Jumpstart checklist that 

resulted in positive results, there may be reason to focus on fewer measures within the 

DRDP that have optimal alignment with the curriculum. 

Furthermore, Jumpstart may have had a positive impact on teachers. However, impact on the 

workforce was not measured as part of the current evaluation. Research Connections (2017) 

has compiled an extensive literature review showing that workplace stressors in the ECE 

workforce negatively affect the teacher-child relationship. Research shows that relationships 

between adults and children form the bases for learning in early childhood (Espinosa, 2002; 

Hamre & Pianta, 2005; National Research Council and Institute of Medicine 2000; Pianta, 

1999). Given the ever-increasing demands on the ECE workforce, having additional assistance 

in the classroom may result in factors such as general stress reduction, release time for lesson 

preparation, etc. These aspects of overall stress and general classroom climate may be worth 

measuring as impacted by the Jumpstart program. 

Finally, an additional measure of success of the Jumpstart program may be the impact that 

participation in the program has for the Corp members. Specifically, does participation in 

Jumpstart serve as a pipeline to the ECE workforce for the Corps members? And what kinds 

of gains in their knowledge and work experience do Corp members experience as a result of 

their participation in Jumpstart? From teachers, assistants, home visitors, to para-educators, 

the Corps members may be helping to build the future ECE workforce.
An Evaluation Of The Jumpstart California Program 2017-2018 19

References

Bennett, K.J., Brown, K.S., Boyle, M., Racine, Y., & Offord, D. (2003). Does low reading achievement at school entry cause 

conduct problems? Social Science and Medicine, 56 (12): 2443-2448.

California Department of Education (2008). California Preschool Learning Foundations Volume 1 

(Sacramento, CA: CDE Press) 60.

CCR Analytics (2014). California Head Start Child Outcomes Bulletin 2014. 

http://caheadstart.org/pdfs/bulletin/ChildOutcomes2014.pdf

Espinosa, L. (2002). The connections between social-emotional development and early literacy, Set for Success: Building 

a Strong Foundation for School Readiness Based on the Social-Emotional Development of Young Children (Kansas City, MO: 

The Ewing Marion Kauffman Foundation) 30-44.

Haigler, K.O., Harlow, C., O’Connor, P., & Campbell, A. (1994). Literacy behind prison walls: Profiles of the prison population 

from the National Adult Literacy Survey. National Center for Educational Statistics. https://nces.ed.gov/pubs94/94102.pdf

Hamre, B & Pianta, R.C. (2005). Can instructional and emotional support in the first grade classroom make a difference for 

children at risk of school failure? Child Development, 76, 949-967.

Kellam, S. G., & Anthony, J. C. (1998). Targeting early antecedents to prevent tobacco smoking: Findings from an 

epidemiologically based randomized field trial. American Journal of Public Health, 88(10), 1490-1495. 

http://dx.doi.org/10.2105/AJPH.88.10.1490

Kirsch, I.S., Jungeblut, A., Jenkins, L., & Kolstad, A. (2002). Adult Literacy in America: A first look at the findings of the 

National Adult Literacy Survey. National Center for Education Statistics. https://nces.ed.gov/pubs93/93275.pdf

Matson, S.C., & Haglund, K.A. (2000). Relationship between scholastic and health behaviors and reading level in 

adolescent females. Clinical Pediatrics, 39 (5): 275-280.

National Research Council and Institute of Medicine (2000). From Neurons to Neighborhoods: The Science of Early 

Childhood Development, Committee on Integrating the Science of Early Childhood Development, eds. Jack P. Sonkoff and 

Deborah A. Phillips (Washington, DC: National Academy Press).

Pianta, R. (1999). Enhancing Relationships Between Children and Teachers 

(Washington, DC: American Psychological Association).

Research Connections (2017). Early Care and Education Workplace Conditions and Teacher Stress: Research-to-Policy 

Resources. https://www.researchconnections.org/childcare/resources/35052/pdf

Sum, A. (1999). Literacy in the labor force: Results from the National Adult Literacy Survey. National Center for Education 

Statistics. https://nces.ed.gov/pubs99/1999470.pdf

Westat (2010). Head Start Impact Study Final Report Executive Summary. 

https://www.acf.hhs.gov/sites/default/files/opre/executive_summary_final.pdf
An Evaluation Of The Jumpstart California Program 2017-2018 20

Appendix A. DRDP Domain Scores and Growth

Table 10. DRDP Domain Scores and Growth

DOMAIN: LANGUAGE DEVELOPMENT

Jumpstart

Group

Comparison

Group

Difference in 

Growth

Unadjusted Effect 

Size

Sample Size 294 388 -- --

Average Pre-Assessment Score 204.4 207.2 -- --

Average Post-Assessment Score 229.7 231.1 -- --

Average Growth 25.3 24.0 1.3 0.11

StdDev Growth 11.6 12.4 -- --

DOMAIN: LITERACY DEVELOPMENT

Jumpstart

Group

Comparison

Group

Difference in 

Growth

Unadjusted Effect 

Size

Sample Size 280 383 -- --

Average Pre-Assessment Score 207.4 207.8 -- --

Average Post-Assessment Score 232.5 233.8 -- --

Average Growth 26.6 25.9 0.7 0.05

StdDev Growth 12.4 12.9 -- --

DOMAIN: SOCIAL & EMOTIONAL DEVELOPMENT

Jumpstart

Group

Comparison

Group

Difference in 

Growth

Unadjusted Effect 

Size

Sample Size 294 389 -- --

Average Pre-Assessment Score 207.6 209.7 -- --

Average Post-Assessment Score 233.8 235.5 -- --

Average Growth 26.2 25.9 0.3 0.03

StdDev Growth 12.4 12.7 -- --
An Evaluation Of The Jumpstart California Program 2017-2018 21

Appendix A. DRDP Domain Scores and Growth

Table 10. DRDP Domain Scores and Growth

DOMAIN: APPROACHES TO LEARNING -SELF-REGULATION

Jumpstart

Group

Comparison

Group

Difference in 

Growth

Unadjusted Effect 

Size

Sample Size 293 389 -- --

Average Pre-Assessment Score 205.8 207.6 -- --

Average Post-Assessment Score 233.2 234.9 -- --

Average Growth 27.4 27.3 0.1 0.00

StdDev Growth 12.9 13.1 -- --

DOMAIN: COGNITION - MATH

Jumpstart

Group

Comparison

Group

Difference in 

Growth

Unadjusted Effect 

Size

Sample Size 294 385 -- --

Average Pre-Assessment Score 206.7 208.9 -- --

Average Post-Assessment Score 232.2 233.6 -- --

Average Growth 25.5 24.7 0.8 0.07

StdDev Growth 12.0 12.0 -- --

DOMAIN: COGNITION - SCIENCE

Jumpstart

Group

Comparison

Group

Difference in 

Growth

Unadjusted Effect 

Size

Sample Size 287 324 -- --

Average Pre-Assessment Score 203.7 204.6 -- --

Average Post-Assessment Score 228.4 229.0 -- --

Average Growth 24.8 24.1 0.7 0.06

StdDev Growth 11.5 12.1 -- --
An Evaluation Of The Jumpstart California Program 2017-2018 22

Appendix B. Full Regression Models

The following tables contain results of the full Regression model for each domain 

or subdomain of the DRDP. The following variables were included in each domain 

and subdomain model: 

● Child participation in Jumpstart or non-participation (Comparison Group)

● Child age at the time of assessment

● Child gender

● Full day or part day attendance in the Head Start program

● Child ethnicity and home language

● Whether the child was enrolled in Head Start during the year prior to the intervention 

(2016-2017 program year)

● Whether the child has an Individualized Education Plan (IEP); indicates a special need

● Percentage of suspect ratings in the classroom

● The agency the child is enrolled (“agency”)

● Pre-assessment domain or subdomain score

● The number of days between Pre- and Post-Assessment 
An Evaluation Of The Jumpstart California Program 2017-2018 23

Table 11. Full Regression model predicting growth in Language Development subdomain (N = 682)

Predictor Coefficient Robust Standard Error t p 95 % Cl

Jumpstart vs.

Comparison Group -0.63 1.49 -0.42 0.68 [-3.61, 2.36]

*Age at Assessment 0.61 0.12 5.29 0.00 [0.38, 0.84]

Female vs. Male 1.48 0.80 1.84 0.07 [-0.13, 3.08]

Full-day vs. Part-day Program 0.87 1.88 0.46 0.65 [-2.90, 4.64]

*Latino-Spanish Language 1.86 0.88 2.12 0.04 [0.10, 3.62]

African-American-English

Language

0.03 1.28 0.02 0.98 [-2.55, 2.61]

 Asian-Asian Language 1.47 1.78 0.83 0.41 [-2.11, 5.05]

*Asian-English Language 4.20 2.01 2.09 0.04 [0.16, 8.23]

White-English Language 0.09 1.65 0.05 0.96 [-3.22, 3.40]

Ethnicity-Language

Other or Missing -0.58 1.39 -0.41 0.68 [-3.37, 2.22]

Enrolled in Prior Year vs. Not 

Enrolled in Prior Year 1.67 1.28 1.3 0.20 [-0.91, 4.24]

*Special Needs - Has an IEP vs. 

Does not have an IEP -4.40 1.12 -3.92 0.00 [-6.66, -2.15]

Data Quality Issue -6.39 7.65 -0.84 0.41 [-21.74, 8.95]

Agency 2 5.96 3.32 1.8 0.08 [-0.70, 12.63]

*Agency 3 16.92 3.67 4.61 0.00 [9.55, 24.29]

*Agency 4 7.73 3.15 2.45 0.02 [1.40, 14.05]

*Agency 5 7.96 2.36 3.38 0.00 [3.23, 12.69]

Agency 6 8.64 4.55 1.9 0.06 [-0.50, 17.78]

*Agency 7 7.26 2.87 2.53 0.02 [1.49, 13.02]

*Agency 8 7.50 2.42 3.1 0.00 [2.64, 12.37]

*Pre-Assessment Language 

Subdomain -0.33 0.05 -6.88 0.00 [-0.43, -0.23]

Days between Preand Post-Assessment 0.04 0.05 0.68 0.50 [-0.07, 0.15]

Constant 44.56 12.58 3.54 0.00 [19.31, 69.80]

*Individual predictors significant at p < .05 in the model.
An Evaluation Of The Jumpstart California Program 2017-2018 24

Table 12. Full Regression model predicting growth in Literacy Development subdomain (N = 663)

Predictor Coefficient Robust Standard Error t p 95 % Cl

Jumpstart vs.

Comparison Group -0.67 1.68 -0.4 0.69 [-4.05, 2.70]

*Age at Assessment 0.65 0.13 5.17 0.00 [0.40, 0.91]

Female vs. Male 1.21 0.89 1.35 0.18 [-0.59, 3.00]

Full-day vs. Part-day Program 0.96 2.09 0.46 0.65 [-3.24, 5.15]

Latino-Spanish Language 1.61 0.87 1.86 0.07 [-0.13, 3.36]

African-American-English

Language

-0.05 1.45 -0.03 0.97 [-2.97, 2.87]

 Asian-Asian Language 2.77 1.64 1.69 0.10 [-0.53, 6.06]

Asian-English Language 4.67 2.83 1.65 0.11 [-1.00, 10.34]

White-English Language -1.13 2.32 -0.49 0.63 [-5.79, 3.53]

Ethnicity-Language

Other or Missing -0.72 1.45 -0.5 0.62 [-3.63, 2.19]

Enrolled in Prior Year vs. Not 

Enrolled in Prior Year 1.43 1.46 0.98 0.33 [-1.50, 4.37]

*Special Needs - Has an IEP vs. 

Does not have an IEP -3.90 1.16 -3.38 0.00 [-6.22, -1.58]

Data Quality Issue -1.56 11.28 -0.14 0.89 [-24.21, 21.09]

Agency 2 5.81 3.34 1.74 0.09 [-0.89, 12.52]

*Agency 3 16.81 4.26 3.94 0.00 [8.25, 25.37]

Agency 4 5.43 3.16 1.72 0.09 [-0.92, 11.77]

*Agency 5 7.77 2.25 3.46 0.00 [3.26, 12.28]

Agency 6 8.76 4.57 1.92 0.06 [-0.42, 17.93]

*Agency 7 7.30 2.72 2.68 0.01 [1.84, 12.76]

*Agency 8 7.65 2.48 3.08 0.00 [2.66, 12.63]

*Pre-Assessment Literacy 

Development Subdomain -0.35 0.05 -6.32 0.00 [-0.45, -0.24]

Days between Preand Post-Assessment 0.03 0.06 0.54 0.59 [-0.09, 0.15]

Constant 49.30 14.67 3.36 0.00 [19.85, 78.75]

*Individual predictors significant at p < .05 in the model.
An Evaluation Of The Jumpstart California Program 2017-2018 25

Table 13. Full Regression model predicting growth in Social & Emotional Development Domain (N = 683)

Predictor Coefficient Robust Standard Error t p 95 % Cl

Jumpstart vs.

Comparison Group -1.71 1.49 -1.15 0.26 [-4.70, 1.28]

*Age at Assessment 0.54 0.10 5.61 0.00 [0.35, 0.74]

Female vs. Male 1.49 0.78 1.9 0.06 [-0.08, 3.06]

Full-day vs. Part-day Program 0.26 1.87 0.14 0.89 [-3.50, 4.02]

*Latino-Spanish Language 2.50 0.84 2.99 0.00 [0.82, 4.19]

African-American-English

Language

-0.57 1.21 -0.47 0.64 [-3.01, 1.87]

 Asian-Asian Language 1.24 1.82 0.68 0.50 [-2.41, 4.89]

Asian-English Language 2.78 2.61 1.06 0.29 [-2.47, 8.03]

White-English Language -2.04 2.33 -0.88 0.39 [-6.72, 2.64]

Ethnicity-Language

Other or Missing 0.95 1.70 0.56 0.58 [-2.46, 4.36]

Enrolled in Prior Year vs. Not 

Enrolled in Prior Year 0.85 1.50 0.57 0.57 [-2.17, 3.87]

*Special Needs - Has an IEP vs. 

Does not have an IEP -3.75 1.28 -2.93 0.01 [-6.32, -1.18]

Data Quality Issue -10.07 6.82 -1.48 0.15 [-23.76, 3.61]

Agency 2 3.41 2.76 1.23 0.22 [-2.14, 8.95]

*Agency 3 13.26 3.52 3.77 0.00 [6.20, 20.32]

Agency 4 3.91 2.92 1.34 0.19 [-1.95, 9.77]

*Agency 5 5.30 2.28 2.33 0.02 [0.73, 9.87]

Agency 6 7.29 4.14 1.76 0.08 [-1.03, 15.60]

Agency 7 4.48 2.61 1.71 0.09 [-0.77, 9.73]

Agency 8 3.24 2.32 1.39 0.17 [-1.42, 7.90]

*Pre-Assessment Social & 

Emotional Development Domain -0.33 0.05 -6.87 0.00 [-0.43, -0.24]

*Days between Preand Post-Assessment 0.13 0.05 2.72 0.01 [0.03, 0.23]

Constant 37.31 12.49 2.99 0.00 [12.23, 62.38]

*Individual predictors significant at p < .05 in the model.
An Evaluation Of The Jumpstart California Program 2017-2018 26

Table 14. Full Regression model predicting growth in Approaches to Learning-Self-Regulation Domain (N = 682)

Predictor Coefficient Robust Standard Error t p 95 % Cl

Jumpstart vs.

Comparison Group -1.90 1.64 -1.15 0.25 [-5.20, 1.40]

*Age at Assessment 0.59 0.11 5.57 0.00 [0.38, 0.81]

Female vs. Male 1.33 0.88 1.51 0.14 [-0.44, 3.10]

Full-day vs. Part-day Program -0.62 2.12 -0.29 0.77 [-4.88, 3.65]

*Latino-Spanish Language 2.01 0.79 2.54 0.01 [0.42, 3.60]

African-American-English

Language

-0.77 1.27 -0.61 0.55 [-3.31, 1.78]

 Asian-Asian Language 2.02 1.91 1.06 0.30 [-1.81, 5.85]

Asian-English Language 3.91 2.50 1.57 0.12 [-1.10, 8.92]

White-English Language -3.37 2.74 -1.23 0.22 [-8.86, 2.12]

Ethnicity-Language

Other or Missing 0.99 1.54 0.64 0.52 [-2.10, 4.07]

Enrolled in Prior Year vs. Not 

Enrolled in Prior Year 0.98 1.59 0.61 0.54 [-2.22, 4.17]

*Special Needs - Has an IEP vs. 

Does not have an IEP -3.98 1.35 -2.94 0.01 [-6.69, -1.26]

Data Quality Issue -11.14 7.10 -1.57 0.12 [-25.40, 3.12]

Agency 2 4.96 2.98 1.66 0.10 [-1.03, 10.95]

*Agency 3 14.22 4.02 3.54 0.00 [6.16, 22.29]

Agency 4 4.57 3.28 1.39 0.17 [-2.01, 11.14]

*Agency 5 6.55 2.67 2.45 0.02 [1.19, 11.92]

Agency 6 7.61 4.71 1.61 0.11 [-1.85, 17.07]

*Agency 7 5.79 2.77 2.09 0.04 [0.22, 11.36]

Agency 8 4.06 2.79 1.45 0.15 [-1.54, 9.66]

*Pre-Assessment Approaches 

to Learning-Self-Regulation 

Domain

-0.35 0.05 -7.36 0.00 [-0.45, -0.26]

*Days between Preand Post-Assessment 0.13 0.05 2.63 0.01 [0.03, 0.22]

Constant 40.39 12.60 3.21 0.00 [15.10, 65.68]

*Individual predictors significant at p < .05 in the model.
An Evaluation Of The Jumpstart California Program 2017-2018 27

Table 15. Full Regression model predicting growth in Cognition- Math subdomain (N = 679)

Predictor Coefficient Robust Standard Error t p 95 % Cl

Jumpstart vs.

Comparison Group -1.05 1.66 -0.64 0.53 [-4.38, 2.27]

*Age at Assessment 0.58 0.12 5.03 0.00 [0.35, 0.82]

Female vs. Male 0.96 0.79 1.23 0.23 [-0.62, 2.55]

Full-day vs. Part-day Program 0.58 1.86 0.31 0.76 [-3.16, 4.32]

Latino-Spanish Language 1.48 0.85 1.75 0.09 [-0.22, 3.18]

African-American-English

Language

-0.53 1.45 -0.37 0.72 [-3.45, 2.39]

 Asian-Asian Language 1.65 1.65 1.00 0.32 [-1.66, 4.97]

Asian-English Language 3.50 2.60 1.34 0.19 [-1.73, 8.73]

White-English Language -0.79 2.25 -0.35 0.73 [-5.31, 3.74]

Ethnicity-Language

Other or Missing 0.39 1.33 0.29 0.77 [-2.28, 3.05]

Enrolled in Prior Year vs. Not 

Enrolled in Prior Year 1.41 1.27 1.11 0.27 [-1.14, 3.96]

*Special Needs - Has an IEP vs. 

Does not have an IEP -4.26 1.12 -3.82 0.00 [-6.50, -2.02]

Data Quality Issue -8.36 10.86 -0.77 0.45 [-30.16, 13.44]

Agency 2 6.68 3.43 1.95 0.06 [-0.20, 13.56]

*Agency 3 16.04 4.11 3.90 0.00 [7.79, 24.29]

*Agency 4 6.24 2.98 2.09 0.04 [0.26, 12.22]

*Agency 5 8.30 2.10 3.95 0.00 [4.09, 12.52]

*Agency 6 10.68 4.34 2.46 0.02 [1.96, 19.40]

*Agency 7 8.12 2.82 2.88 0.01 [2.46, 13.79]

*Agency 8 7.74 2.23 3.47 0.00 [3.26, 12.21]

*Pre-Assessment CognitionMath Subdomain -0.34 0.05 -7.19 0.00 [-0.44, -0.25]

Days between Preand Post-Assessment 0.05 0.06 0.84 0.41 [-0.07, 0.17]

Constant 48.39 14.52 3.33 0.00 [19.24, 77.53]

*Individual predictors significant at p < .05 in the model.
An Evaluation Of The Jumpstart California Program 2017-2018 28

Table 16. Full Regression model predicting growth in Cognition-Science subdomain (N = 611)

Predictor Coefficient Robust Standard Error t p 95 % Cl

Jumpstart vs.

Comparison Group -0.53 1.70 -0.31 0.76 [-3.94, 2.89]

*Age at Assessment 0.47 0.12 3.78 0.00 [0.22, 0.72]

*Female vs. Male 1.62 0.73 2.22 0.03 [0.15, 3.09]

Full-day vs. Part-day Program 0.66 2.08 0.32 0.75 [-3.52, 4.84]

Latino-Spanish Language 1.20 0.96 1.25 0.22 [-0.73, 3.14]

African-American-English

Language

-1.09 1.63 -0.67 0.51 [-4.36, 2.19]

 Asian-Asian Language 0.95 1.58 0.6 0.55 [-2.23, 4.12]

Asian-English Language 3.19 2.50 1.28 0.21 [-1.84, 8.23]

White-English Language -1.05 1.87 -0.56 0.58 [-4.81, 2.72]

Ethnicity-Language

Other or Missing -0.70 1.50 -0.47 0.64 [-3.72, 2.31]

Enrolled in Prior Year vs. Not 

Enrolled in Prior Year 1.78 1.23 1.45 0.15 [-0.69, 4.25]

*Special Needs - Has an IEP vs. 

Does not have an IEP -4.35 1.16 -3.75 0.00 [-6.68, -2.02]

Data Quality Issue -7.68 10.49 -0.73 0.47 [-28.78, 13.42]

*Agency 2 7.57 3.41 2.22 0.03 [0.71, 14.43]

*Agency 3 16.89 3.93 4.3 0.00 [8.98, 24.79]

*Agency 4 8.00 2.98 2.68 0.01 [2.00, 14.01]

*Agency 5 9.22 2.37 3.9 0.00 [4.46, 13.98]

*Agency 6 11.16 4.64 2.41 0.02 [1.83, 20.50]

*Agency 7 9.67 3.05 3.17 0.00 [3.53, 15.80]

Agency 8 0.00 (OMITTED) [3.26, 12.21]

*Pre-Assessment CognitionScience Subdomain -0.33 0.06 -5.41 0.00 [-0.45, -0.21]

Days between Preand Post-Assessment 0.01 0.06 0.23 0.82 [-0.11, 0.14]

Constant 54.50 14.50 3.76 0.00 [25.33, 83.68]

*Individual predictors significant at p < .05 in the model.