An Evaluation of Jumpstart: Effects on 

Preschool-age Children in California

October 13, 2015

cultivating learning and positive change

www.magnoliaconsulting.org
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

ii

An Evaluation of Jumpstart 

This material is based upon work supported by the Corporation for National and Community 

Service (CNCS) under Grant No. 12ACHCA0010009. Opinions or points of view expressed in 

this document are those of the authors and do not necessarily reflect the official position of

CNCS nor a position that is endorsed by the CNCS Program.

About Magnolia Consulting 

Magnolia Consulting is a woman-owned small business offering comprehensive, customized 

evaluation and research services. We cultivate learning and positive change by meeting clients’ 

specific needs with timely, relevant, and useful information. With our expertise, we help clients 

achieve their goals in PK–20 education, adult education, organizational change and development, 

community- and school-based programs, and informal education and public outreach. For more 

information about Magnolia Consulting, please visit our website: www.magnoliaconsulting.org. 

Acknowledgments 

Magnolia Consulting would like to thank the children who participated in the study. Without 

their time, effort, and participation, this study would not be possible. We would also like to 

thank Jumpstart, and especially Christine Patton, Director of Research and Evaluation, for her 

collaborative efforts to complete this study and to demonstrate the outcomes of Jumpstart’s

program through independent, objective research. We would also like the thank Beth Peery for 

her work preparing databases and all of the members of the Magnolia Consulting team for their 

hard work and support of this study. 

The authors,

Billie-Jo Grant, Ph.D.

Lisa Shannon, Ph.D.

Magnolia Consulting, LLC

5135 Blenheim Rd. 

Charlottesville, VA 22902

(ph) 855.984.5540 (toll free)

http://www.magnoliaconsulting.org
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

iii

Key Findings 

The composition of Jumpstart and 

comparison groups was similar 

regarding gender, language, and 

pretest scores. 

There were statistically significant 

differences in the composition of 

Jumpstart and comparison groups

regarding ethnicity and age, as well 

as by the type of center from which 

children were drawn.

The average JSSC gain of children 

who participated in Jumpstart was

statistically significant and 

substantively important based on 

the What Works Clearinghouse 

(WWC) threshold of 0.25. 

The average JSSC gain of children 

who participated in Jumpstart were 

statistically significantly greater 

than the gains of children who did 

not participate in the program, and 

the effect size for this impact was 

substantively important based on 

the WWC threshold.

Executive Summary 

Jumpstart is a national early education 

organization that provides preparation for 

kindergarten for preschool children who are 

between 36 and 59 months of age and who 

are living in some of the most under

resourced neighborhoods across the 

country. Jumpstart recognizes the 

importance of gathering scientific research 

evidence to support decision-making about 

instructional programs. As such, Jumpstart 

contracted with Magnolia Consulting, LLC, 

an independent evaluation consulting firm, 

to conduct data analyses of student 

assessment data collected during the 2014–

2015 school year to support its evaluation 

needs for the Corporation for National and 

Community Service (CNCS) under Grant No. 

12ACHCA0010009. 

Study Design & Methods

The purpose of this study is to examine the 

impact of Jumpstart on improving

participants’ oral language and early literacy 

skills. This report focuses on answering the 

following primary evaluation questions:

What were the demographic 

characteristics of Jumpstart and 

comparison groups? Did these 

groups differ significantly based 

on their demographic 

characteristics? 

Did Jumpstart and comparison 

groups differ significantly based 

on their pretest scores?

Did Jumpstart children 

demonstrate statistically 

significant gains on JSSC 

scores? 

Did Jumpstart children

demonstrate statistically 

significant gains that were

greater than the gains of 

comparison-group children? 

To answer these evaluation questions, 

Jumpstart conducted a quasi-experimental 

study. The treatment group was comprised 

of children who participated in the 

Jumpstart program. A comparison group, 

comprised of children who did not use the 

Jumpstart program, was identified by site 

staff with guidance and support from

Jumpstart’s National Research & Evaluation 

team. 

The final analysis sample included 1,716 

Jumpstart children and 562 comparisongroup children in 162 classrooms across 78 

childhood centers in California. 
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

iv

Approximately half of the children included 

in the final sample were male (50.95%) and 

half were female (49.05%). Of these 

children, 56.80% were non-English 

speakers and nearly three-quarters of 

children were classified as Hispanic 

(71.41%). 

The Jumpstart School Success Checklist 

(JSSC) was the primary measure to gauge 

the impact of the Jumpstart program.

Administered at the start and at the 

completion of the school year, the JSSC 

measures children’s oral language and early 

literacy skills, with a possible score of one

(lowest) through five (highest) on a variety 

of measures, and these scores were

averaged to calculate a total scale score. 

Group Equivalence 

L

d

s

r

m

s

m

KEY FINDING:

There were no statistically significant 

differences between Jumpstart and 

comparison groups in regards to gender 

and language. There were statistically 

significant group differences for ethnicity 

and age.

Evaluators examined group equivalence

between the Jumpstart and comparison 

groups by conducting chi-square analyses 

and an independent samples t-test on 

demographic characteristics and by running 

multilevel modeling analyses on pretest 

scores.

Chi-square analyses showed no statistically 

significant differences between groups in 

regard to gender and language. There was a 

statistically significant difference for 

ethnicity, with the comparison group having

a greater percentage of Hispanic children 

(77.15%) than the Jumpstart group 

(69.57%). Evaluators used an independent 

samples t-test to determine if there was a 

difference between groups in regard to age. 

Results were statistically significant and 

showed that, on average, the Jumpstart 

group was slightly older (1.55 months) than 

the comparison group.

Student JSSC Scores 

Pretest to Posttest 

5 

4 

3 3.71 

2 

2.24 

1 

0 

Pretest Posttest 

Figure 1. Unadjusted pretest and posttest 

JSSC scores for Jumpstart participants. 

KEY FINDING:

Multilevel modeling analyses revealed no 

statistically significant differences 

between groups on mean pretest JSSC 

scores.

astly, evaluators ran multilevel modeling to

etermine if there were statistically 

ignificant differences between groups with

egard to pretest scores. Multilevel 

odeling analyses revealed no statistically 

ignificant differences between groups on 

ean JSSC pretest scores.

Learning Results

KEY FINDING:

On average, Jumpstart children gained 

1.47 points from pretest to posttest on the 

JSSC. This gain is statistically significant 

and corresponds to a substantively 

important effect size. 
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

v 

To address key evaluation questions

regarding child learning on the JSSC,

evaluators conducted a variety of analyses, 

including descriptive and inferential 

statistics as well as the calculation of effect 

sizes.

Results show that, on average, JSSC 

scores for Jumpstart children increased 

from pretest to posttest by 1.47 points (see 

Figure 1). This average gain was statistically 

significant (p = <.01) and corresponded to a 

substantively important effect size (1.87). 

KEY FINDING: 

The gains of students who participated in 

Jumpstart were statistically significantly 

greater than gains of students who did not 

participate in the program, and the effect 

size for this impact was substantively 

important based on the WWC threshold. 

Evaluators conducted descriptive analyses 

and multilevel modeling analyses to 

compare JSSC scores among children who 

participated in Jumpstart and comparisongroup children who did not participate in 

Jumpstart. On average, the Jumpstart 

program had a statistically significant impact 

on participants’ JSSC scores. More 

specifically, at the end of the study period, 

children who participated in Jumpstart

program scored an average of 0.45 points 

higher on the JSSC than comparison 

children (see Figure 2). The effect size for 

this impact was 0.62, which exceeds the 

WWC threshold of 0.25 for determining 

whether effect sizes are substantively 

important. The WWC improvement index 

corresponded to 23 percentile points.

Figure 2. Jumpstart and comparison-group

children’s JSSC unadjusted pretest and 

posttest scores.

Summary

Overall, group equivalence analyses 

revealed that Jumpstart and comparison 

groups were similar for gender, language, 

and pretest scores. The groups were 

statistically significantly different for 

ethnicity, age, and the type of center from 

which children were drawn. 

Analyses of JSSC scores revealed that on 

average, Jumpstart children demonstrated 

statistically significant increases in their 

scores from pretest to posttest, and results 

corresponded to a substantively important

effect size. Findings from analyses 

comparing the performance of Jumpstart

and comparison-group children indicated 

that Jumpstart had a statistically significant 

impact on JSSC gain scores when 

compared to comparison-group children 

who did not participate in the Jumpstart 

program. 

Recommendations for future studies are a) 

to include subgroup analyses to examine

the program’s impact for different 

demographics of children, b) to collect 

program implementation and perception 

data using teacher logs and surveys, 

interviews, and focus groups, permitting a 

Student JSSC Scores 

5 

3.71 

4 

3.43 

3 2.40 

2 

2.24 

1 

0 

Pretest Posttest 

Treatment Comparison 
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

vi

deeper understanding of the program, c) to 

collect and analyze student learning from 

multiple measures, and d) to conduct a 

randomized control trial study. 
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

vii

Table of Contents

Introduction................................................................................................................................... 1

Jumpstart Program Description .................................................................................................... 1

Evaluation Plan.............................................................................................................................. 5

Evaluation Questions ................................................................................................................................. 5

Methods..................................................................................................................................................... 5

Measures .................................................................................................................................................... 6

Data Collection.......................................................................................................................................... 6

Settings ...................................................................................................................................................... 7

Participants ................................................................................................................................................ 7

Child Analysis Sample............................................................................................................................................. 7

Childhood Centers.................................................................................................................................................. 8

Group Equivalence........................................................................................................................ 9

Demographic Characteristics...................................................................................................................... 9

Pretest Scores ........................................................................................................................................... 11

Summary.................................................................................................................................................. 12

Performance Results ..................................................................................................................12

Descriptive Examination of Jumpstart Participants’ JSSC Scores...........................................................................12

Multilevel Modeling Analyses Examining Jumpstart Children’s Score Gains.........................................................13

Analyses of Participants’ JSSC Scores by Treatment and Comparison Groups.......................................... 14

Descriptive Findings Comparing JSSC Scores by Study Condition .......................................................................14

Multilevel Modeling Analyses Comparing JSSC Scores by Study Condition .........................................................15

Summary...............................................................................................................................................................16

Summary and Discussion ...........................................................................................................17

References..................................................................................................................................19

Appendix A: Unadjusted Pretest and Posttest Means by Study Condition ................................21

Tables 

Table 1. Site Characteristics ........................................................................................................................... 7

Table 2. Percentage of Childhood Centers Affiliated with each University (n=78) ......................................... 8 

Table 3. Children Demographics by Group ................................................................................................... 8 

Table 4. Demographics of Affiliated Childhood Centers.............................................................................. 10

Table 5. Group Equivalnce Analysis for Pretest Scores ................................................................................. 11

Table 6. Jumpstart Participants Mean Pretest to Posttest JSSC Gains........................................................... 13

Table 7. Impact of Jumpstart on Participants’ Pretest-to-Posttest Gains....................................................... 16

Figures

Figure 1. Unadjusted pretest and posttest JSSC scores for Jumpstart participants...........................................iv

Figure 2. Jumpstart and comparison-group children’s JSSC unadjusted pretest and posttest scores. ................ v

Figure 3. Jumpstart core program components............................................................................................... 2
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

viii

Figure 4. Children’s ethnicity by condition. ................................................................................................. 10

Figure 5. Children’s average age (in months by condition). .......................................................................... 11

Figure 6. Percentage of children enrolled in each childhood center type. ...................................................... 11

Figure 7. Unadjusted pretest and posttest JSSC scores for Jumpstart participants......................................... 13

Figure 8. Unadjusted pretest and posttest JSSC scores for Jumpstart and comparison-group participants..... 14
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

1 

Introduction

U.S. and global educational policy continues to push literacy as a key priority because

children’s early language and literacy skills are a strong predictor of positive student 

achievement and progress through school. These skills are also essential for individuals to 

engage and contribute fully to society (Mackenzie & Hemmings, 2014; Rasinski et al., 2005; 

Wilson, Dickson, & Rowe, 2013). Adults with low literacy skills are more likely to have lower 

incomes and struggle for employment; they are less likely to vote and are more likely to have 

legal trouble (National Institute for Literacy, 2008). Thus, low literacy skills can have a significant 

impact on students’ educational careers and widespread implications for their economic 

livelihood and social and civic success (Lesnick, George, Smithgill, & Gwynn, 2010; NELPR, 

2008).

Children who start school academically behind their peers are more likely to stay behind,

with this gap only widening over time. Poverty is a risk factor for poor emergent literacy and 

language skills, (Snow, Burns, & Griffin, 1998) and children from low-income neighborhoods 

start kindergarten 60% behind their more affluent peers (Jumpstart, 2015). Furthermore, 

students who don’t read proficiently by third grade are four times more likely to leave school 

without a diploma than proficient readers (Hernandez, 2011).

Jumpstart recognizes the early years of children’s lives as a critical time for language 

and literacy development and has developed a comprehensive supplemental curriculum and 

support program for low-income preschool children. Jumpstart strongly believes in providing 

the highest quality programs for language and literacy development. As such, it contracted with 

Magnolia Consulting, LLC, an external, independent consulting firm specializing in research and 

evaluation, to examine the impact of Jumpstart on improving participants’ oral language and 

early literacy skills by conducting data analyses of student assessment data collected by 

Jumpstart during the 2014 – 15 school year. 

Jumpstart Program Description

Jumpstart is a national early education organization that provides preparation for 

kindergarten to preschool children who are between 36 and 59 months of age and who are 

living in some of the most under resourced neighborhoods across the country. The Jumpstart 

program’s mission is to ensure that every child in America enters kindergarten prepared to 

succeed. This encompasses improving academic gains and kindergarten readiness for young 

children, involving families, and fostering a pipeline of early childhood educators and champions

of early childhood education.

To reach preschool classrooms, Jumpstart engages with Jumpstart Corps members and 

college and university partners on a national level. Jumpstart seeks to partner with classrooms 

in which at least 75% of children qualify as low income by federal requirements for reduced 

cost meals.
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

2 

Jumpstart 

Family 

Involvement 

Corps 

Members 

Jumpstart 

Sessions 

Child 

Centered 

Time 

Volunteer 

Engagement 

Figure 3. Jumpstart core program components.

Jumpstart’s core program 

components include:

• Family Involvement

• Pipeline of Champions: 

Jumpstart Corps 

Members

• Jumpstart Sessions

• Child Centered Time 

• Volunteer Engagement

Family Involvement

Jumpstart recognizes that 

families are children’s first and 

most important teachers. 

Jumpstart’s family involvement 

approach is based on two key 

components: ongoing, consistent 

communication and the 

Jumpstart-home learning 

connection. Jumpstart uses ongoing, consistent communication to build families’ knowledge of

Jumpstart and to inform them of their children’s interests and accomplishments in Jumpstart 

sessions. This process of involving, informing, and educating families may happen face-to-face,

if families are available, but it also includes monthly communication sent home to families.

Jumpstart also strives to build connections between its sessions and children’s home learning

by providing families with resources and ideas to support their ongoing engagement in their 

children’s language and literacy development. Jumpstart’s family calendar and unit newsletters

include take-home activities that support the Jumpstart-home learning connection. When 

families are available and interested, Corps members welcome their participation in Jumpstart 

sessions and encourage conversations about children’s interests and activities at home.

Pipeline of Champions: Jumpstart Corps Members

Jumpstart engages dedicated, trained adults who are interested in providing language 

and literacy support to small groups of preschool children through yearlong relationships and 

through consistent delivery of Jumpstart sessions in program partner classrooms. The majority 

of Jumpstart Corp members nationwide 

are students at Jumpstart’s higher 

education partner campuses. Others are 

community members from a variety of 

sectors who participate in Jumpstart 

through city-based Community Corps 

programs. Jumpstart Corps members 

serve for a minimum of one academic 

year during which they gain experience 

in classrooms and communities, which 

increases their early childhood 

knowledge and builds their awareness of 

social issues that affect young children. 
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

3 

This program thus fosters a pipeline of future educators and champions of early childhood 

education.

Jumpstart Corps members participate in approximately 30 hours of training before they 

enter the classroom. During this training, Corps members complete hands-on activities that 

deepen their understanding of developmentally appropriate practice, children’s language and 

literacy development, active learning, classroom management, and family involvement. Corps 

members learn how to deliver Jumpstart’s curriculum during trainings that focus on

comprehending its target domains and skills, implementing its routines, and understanding and 

using its reading strategies. Corps members build upon and strengthen their skills through 

application and practical learning during in-service trainings held throughout the year. 

Each Jumpstart team is coordinated by a peer team leader. Team leaders participate in 

additional training related to leadership, classroom management, and best practices for 

implementing the portions of Jumpstart sessions that they lead. Team leaders also play a 

greater role in relationship building and communication with program partners, teachers, and 

staff, as directed by site managers. 

Team leaders receive support from Jumpstart site managers during weekly team leader 

meetings. These meetings include detailed review and practice of Jumpstart session plans, 

troubleshooting common challenges occurring across teams, and mini-trainings to support 

team leaders in overall management of their teams. Team leaders transfer this information to 

Corps members during team planning meetings, generally held just before or immediately after 

each team’s scheduled Jumpstart sessions. Throughout the year, Corps members practice 

upcoming session activities and discuss children’s interests and accomplishments in previous 

sessions in a consistent effort to improve session quality and increase children’s kindergarten 

readiness.

Jumpstart Sessions

Twice a week during the school year, teams of Jumpstart Corps members engage a 

classroom of preschool children in a two-hour Jumpstart session. During Jumpstart sessions, 

teams deliver a curriculum based on Jumpstart’s three language and literacy domains: oral 

language, books and print knowledge, and phonological awareness. Each Corps member is 

paired with approximately three 

children and supports these partner 

children directly during select session 

elements while sharing responsibility 

for supporting the entire classroom of 

children during other parts of the 

session. 

Classroom teachers are asked 

to participate in Jumpstart sessions by 

following the session plan and 

supporting children in specific 

activities. This involvement varies

based on the total number of children 

in the classroom, the size of the 

Jumpstart team serving the class, and 
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

4 

the specific needs of the children enrolled. Jumpstart site managers and team leaders are 

asked to work with teachers to ensure they are familiar with session plans and are treated as 

members of a collaborative team delivering Jumpstart’s curriculum to children.

Child Centered Time (CCT)

In addition to the Jumpstart session, each Corps member provides 2-4 hours per week 

of additional service to children through Child Centered Time (CCT). This allows Corps 

members sufficient time to focus on assisting individual children in strengthening language and 

literacy skills specific to that child

Volunteer Engagement

Jumpstart sites and regional offices also coordinate volunteer engagement activities. 

Through volunteer engagement, Jumpstart seeks to involve additional campus and community 

members in services to benefit the Jumpstart program, its program partners, and the children 

and families it serves. Jumpstart staff or Strengthening Communities Coordinators work with 

site managers and program partners to determine volunteer engagement projects that meet a 

specific program, center, or family need. Jumpstart staff or Strengthening Communities 

Coordinators often organize volunteers to:

• Create the resources needed for upcoming Jumpstart sessions, leaving teams to 

concentrate on practicing session elements and engaging children; 

• Refresh or revitalize indoor and outdoor areas of program partner facilities to allow 

children and families to use rooms or playgrounds they may not have had access to 

before; and

• Make packets of take-home learning resources for families in order to continue and 

increase children’s learning.
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

5 

Evaluation Plan

The purpose of this study was to a) examine the impact of Jumpstart on improving 

participants’ oral language and early literacy skills, and b) determine if Jumpstart children

demonstrated statistically significant gains that were greater than the gains of comparisongroup children who did not use Jumpstart. In addition, evaluators examined group equivalence 

between Jumpstart and comparison groups for demographic variables and pretest scores. This 

section of the report details information about the study’s methodology, including a discussion 

of the data sources and variables of interest, as well as provides a discussion of the methods 

evaluators used to clean and explore the data in preparation for the primary data analyses. This 

section also contains an overview of the study’s samples for analysis.

Evaluation Questions

This report focuses on answering the following primary evaluation questions:

1. What were the demographic characteristics of Jumpstart and comparison groups? Did

these groups differ significantly based on their demographic characteristics? 

2. Did Jumpstart and comparison groups differ significantly based on their pretest scores?

3. Did Jumpstart children demonstrate statistically significant gains on JSSC scores? 

4. Did Jumpstart children demonstrate statistically significant gains that are greater than 

the gains of comparison-group children? 

Methods

Per AmeriCorps’ regulations for grants over $500,000, this evaluation of the Jumpstart 

program used a quasi-experimental study design. The treatment group was comprised of 

children who participated in the Jumpstart program. The AmeriCorps’ National Research & 

Evaluation team helped Jumpstart regional and site staff identify and secure assessment data

for a comparison group of children. Essential attributes of comparison centers included the 

following: 

• served a low-income population; 

• enrolled children who were 36 to 59 months of age at the beginning of the school year; 

• were located in the same (or geographically comparable) neighborhoods as Jumpstart 

program partners; 

• enrolled children who were demographically comparable (age, language(s) spoken, 

gender, and race) to the children Jumpstart serves; 

• included classrooms where children do not receive early literacy interventions such as 

volunteer reading, mentoring, or tutoring services; and 

• were registered as 501(c)(3) non-profit organizations.
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

6 

Measures

The primary measure for this evaluation was the Jumpstart School Success Checklist 

(JSSC). 

Jumpstart School Success Checklist (JSSC) 

The JSSC is a central part of Jumpstart’s assessment strategy as its use allows 

Jumpstart to measure the impact of its program on participating children. The JSSC is derived 

from the High Scope Educational Research Foundation’s Preschool Child Observation Record 

(COR) (2012), a standardized teacher observation tool independently created and vetted by High 

Scope. Jumpstart chose the COR because it has been shown to have good test properties for a 

diverse group of children and to be helpful for engaging early childhood educators in program 

improvement.

The JSSC uses 15 items from the COR that are directly related to Jumpstart’s priority of 

promoting oral language and early literacy skills. The JSSC includes eight items from the COR’s 

Language and Literacy scale, four items from its Social Relations scale, and three items from its 

Initiative scale. Each child is rated based on his or her demonstrated level of ability for that skill, 

with a possible score of one (lowest) through five (highest), and these scores were averaged to 

calculate a total scale score. Based on recommendations from previous research studies using 

the JSSC, this evaluation used the total scale scores to measure preschool students’ literacy 

skills instead of distinct subscale scores (Immekus, 2011). External studies of the internal 

consistency and external validity of the JSSC indicated that its internal consistency is high 

(Cronbach’s Alpha of .95 and higher) (Immekus, 2011) and that it is moderately to strongly 

correlated with other prominent measures of language and literacy such as the Test of Early 

Reading Ability (Pearson Coefficients ranged from .47-.63 for subtest) (Meyers, et al., 2011), 

and the Get Ready to Read (Pearson Coefficient of .63) (Meyers, et al., 2011).

After receiving a standardized orientation and training led by Jumpstart, teachers 

completed a JSSC for all children with signed parental consents in their classrooms in the fall. 

These baseline assessments were also completed for children who enrolled late and were 

likely to attend for at least 120 calendar days. In the spring, teachers completed a JSSC for all 

children who had baseline JSSC data. The ideal timeframe for teachers to complete the JSSC is 

two or three weeks before Jumpstart sessions start in the fall and two or three weeks after 

Jumpstart sessions end in the spring. Administrators and teachers who were willing to 

participate in the JSSC assessment process also tested comparison-group students in the fall 

and spring. 

Data Collection

For this evaluation, the Jumpstart team was responsible for collecting and handling 

initial JSSC data preparation. In the summer of 2015, Jumpstart provided Magnolia Consulting 

evaluators with a clean database, which included JSSC pretest and posttest data, demographic 

data, and evaluation criteria information for Jumpstart and comparison-group children. 

Evaluators reviewed the data in depth and collaborated with the Jumpstart team to ensure its 

accuracy and minimize errors. Following this process, Magnolia Consulting evaluators 

completed final data preparation procedures for its analyses. 
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

7 

Settings

The analysis sample for this study included children from 78 childhood centers 

distributed across three market areas in the West Pacific region of the United States. San 

Francisco and Fresno were similar in size, while Los Angeles was considerably larger. Nearly a 

quarter of San Francisco students and over half of the student populations in both Los Angeles 

and Fresno were Hispanic or Latino. The three market areas were similar in regard to race 

except that nearly a third of students in San Francisco were Asian and an estimated third of 

students in both Los Angeles and Fresno were classified as some other race. Across all three 

market areas, about a quarter of students were classified as ELL and approximately 10% of 

students had IEPs.

Table 1. Site Characteristics

Los Angeles San Francisco Fresno

Geographic location and West Pacific West Pacific West Pacific 

city description* City: Large City: Large City: Large

Total Schools 1,008 127 108

Total Students 655,455 56,970 73,689

Student/Teacher ratio 23.55 19.83 24.32

Ethnicity

 Hispanic or Latino 65.13% 21.97% 52.60%

 Non-Hispanic or Latino 34.87% 78.03% 47.40%

Race

White 38.29% 30.92% 34.97%

Black or African American 11.53% 11.82% 9.67%

American Indian or Alaska Native 0.98% 0.44% 1.93%

Asian 6.23% 36.70% 14.49%

Hawaiian or other Pacific Islander 0.24% 1.07% 0.16%

Some other race 36.48% 10.91% 31.53%

Two or more races 6.25% 8.15% 7.26%

ELL Students 28.47% 24.92% 23.87%

Students with IEPs 12.57% 11.96% 9.83%

Participants

The final analysis sample included 1,716 Jumpstart children and 562 comparison-group

children in 162 classrooms across 78 childhood centers in California. 

Child Analysis Sample

To be included in the final analysis sample, children needed to meet a specific set of 

criteria. Children were removed from the evaluation sample if they did not have parental 
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

8

final analysis sample 2,278 children

No parental 

consent

Jumpstart

Comparison

Not high 

needs

Missing JSSC 

time point

Less than 

120 Days

-30 children

-0 children

-279 children

-82 children

-30 children

-54 children

-94 children

N/A

1,716

Jumpstart

562

comparison

consent (30 Jumpstart children), were not considered a high-need beneficiary at pretest (279 

Jumpstart and 82 comparison-group children), or did not have both fall and spring JSSC data (30 

Jumpstart and 54 comparison-group children). Additionally, Jumpstart children were removed 

from the sample if they were not enrolled in the Jumpstart program for more than 120 days (94 

Jumpstart children). Jumpstart’s National Research & Evaluation team uses 120 days as a 

standard cut-off for required dosage; the intervention literature suggests that an intervention 

program be at least 16, but preferably 20 weeks, to demonstrate programmatic impact (Deshler, 

Hock, Ihle, & Mark, 2011). Comparison-group children had no dosage requirement. Given these

criteria, a total of 569 children (433 Jumpstart and 136 comparison-group children) were 

removed from the sample, yielding a final analysis sample of 2,278 children (1,716 Jumpstart 

and 562 comparison-group children). 

Childhood Centers

A classroom or childhood center was included in the analysis sample if it had at least 

one child enrolled who met the sample criteria and was included in the final analysis sample. 

The 78 childhood centers included in the final analysis sample were affiliated with 12

universities (see Table 2) in three market areas, more than half of which were in Los Angeles 

(60.26%), a third in San Francisco (33.33%), and 6.41% in Fresno. Additionally, 43.59% of

these childhood centers were classified as Head Start centers, about a third as communitybased centers (30.77%), nearly a quarter as public centers (24.36%), and the remaining 1.28%

as private centers. 

Table 2. Percentage of Childhood Centers Affiliated with each University (n=78)

%

San Francisco State University 20.51%

Pepperdine University 14.10%

Comparison University (not identified) 10.26%

California State University, Northridge 8.97%

California State University, Fullerton 8.97%

University of California, Irvine 7.69%

California State University, Fresno 6.41%

Saint Mary's College of California 6.41%

University of California, Berkeley 6.41%

University of Southern California 3.85%

California State University, Dominguez Hills 3.85%

Pitzer College 2.56%
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

9 

Group Equivalence

In this section evaluators present findings from group equivalence analyses between 

the Jumpstart and comparison groups by conducting chi-square analyses and independent 

samples t-test on demographic characteristics and running multilevel modeling analyses on 

pretest scores. 

Demographic Characteristics 

What were the demographic characteristics of Jumpstart and comparison groups? Did Jumpstart 

and comparison groups differ significantly based on their demographic characteristics?

KEY FINDING:

There were no statistically significant differences between Jumpstart and comparison 

groups in regard to gender and language. There were statistically significant group 

differences for ethnicity and age.

 Evaluators examined several demographic characteristics of children in the analysis 

sample including gender, language, ethnicity, and age (Table 3). Approximately half of the 

children included in the final sample were male (50.95%) and half were female (49.05%). Of 

these children, 56.80% were non-English speakers and nearly three-quarters of children were 

classified as Hispanic (71.41%). Jumpstart children were an average of 48.83 months of age

and comparison-group children had a mean age of 47.28 months.

Table 3. Children Demographics by Group

Jumpstart Comparison-Group Children (n=1,716) Children (n=562) 

Characteristics Percent N Percent N 

Total 

(n=2,278) 

Percent N 

Chi-square 

Results

Sig. X2 (alpha Value =.05)

Gender

Male 50.41% 862 52.59% 294

Female 49.59% 848 47.41% 265

50.95% 1,156

49.05% 1,113

0.72 .40

Language

English 44.08% 730 40.42% 211

Non-English 55.92% 926 59.58% 311

43.20% 941

56.80% 1,237

2.02 .16

Ethnicity 

Hispanic 69.57% 1,184 77.15% 422 71.41% 1,606

Non- 30.43% 518 22.85% 125 Hispanic 28.59% 643

11.29 <.01
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

10

Evaluators conducted analyses to determine if these demographic characteristics 

differed significantly by group. Chi-square analyses showed no statistically significant 

differences between groups in regard to gender and language. There was a statistically 

significant difference for ethnicity. More specifically, the comparison group had a greater 

percentage of Hispanic children (77.15%) than the Jumpstart group (69.57%) as seen in Figure 

4. Additionally, evaluators used an independent samples t-test to determine if there was a 

difference between groups in regards to age. Results showed that there was a statistically 

significant difference, t(2262) = -4.80, p = <.01, as the Jumpstart group was slightly older (1.55 

months) than the comparison group (see Figure 5). 

Figure 4. Children’s ethnicity by condition.

77.15% 69.57% 

30.43% 22.85% 

Hispanic Non-Hispanic 

Jumpstart (n=1,702) Comparison (n=547) 

Figure 5. Children’s average age (in months) by 

condition.

48.83 47.28 

Age 

Jumpstart (n=1,709) Comparison (n=555) 

 

KEY FINDING:

There were statistically significant differences by group regarding the type of centers from 

which the Jumpstart and comparison group samples were drawn.

In addition to these demographics, evaluators also reviewed the type of centers where 

the children in the final analysis sample were enrolled (see Table 4). Nearly half of the children 

in the analysis sample were enrolled in Head Start centers (47.28%), a quarter of the children 

were in community-based centers (26.60%), another quarter of the children were enrolled in 

public centers (25.55%) and very few children were enrolled in private centers (0.57%).

Table 4. Demographics of Affiliated Childhood Centers

 

Characteristics

Jumpstart Children 

(n=1,716) 

Percent N 

Comparison-Group 

Children (n=562) 

Percent N 

Total 

(n=2,278) 

Percent N 

Chi-square Results

Sig. (alpha X2 Value =.05)

Communitybased 29.84% 512 16.73% 94 26.60% 606

Head Start 36.83% 632 79.18% 445 47.28% 1077 321.141 <.01

Private 0.76% 13 0.00% 0 0.57% 13

Public 32.58% 559 4.09% 23 25.55% 582

1 Due to a small sample enrolled in private centers, the chi-square analysis only included community-based, Head Start, 

and public centers.
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

11

Results of chi-square tests showed statistically significant differences by group 

regarding the type of centers from which the Jumpstart and comparison group samples were 

drawn. In regard to the type of the childhood center, the majority of comparison-group children 

were enrolled in Head Start centers (79.18%). Alternatively, Jumpstart children were evenly 

enrolled across Head Start (36.83%), public (32.58%), and community-based centers (29.84%)

(see Figure 6). Private centers were not included in this analysis due to a very small sample size.

79.18% 

36.83% 

29.84% 32.58% 

16.73% 

4.09% 

Community-based Head Start Public 

Jumpstart (n=1,716) Comparison (n=562) 

Figure 6. Percentage of children enrolled in each childhood 

center type.

Pretest Scores

Did Jumpstart and comparison groups differ significantly based on their pretest JSSC scores? 

KEY FINDING:

Multilevel modeling analyses revealed no statistically significant differences between 

Jumpstart and comparison groups on mean pretest JSSC scores.

After examining potential group differences by demographic characteristics, evaluators 

ran multilevel modeling analyses to determine if there were statistically significant differences 

between groups with regard to pretest scores. These analyses revealed no statistically 

significant differences between groups on mean pretest scores, suggesting that Jumpstart and 

comparison-group children were similar regarding the areas assessed by the JSSC at the 

beginning of the study. 

Table 5. Group Equivalnce Analysis for Pretest Scores

Outcome Variable Coefficient Standard 

Error

tvalueApprox. 

df p-value

Pretest Score -0.16 0.10 -1.60 83 .11

* Statistically significant at the 0.05 level.
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

12

Summary

Overall, Jumpstart and comparison groups were similar regarding gender, language, and 

pretest scores. There were statistically significant group differences by ethnicity and age, as 

well as by the type of center from which children were drawn. 

Performance Results

To address key evaluation questions concerning child learning on the JSSC during the 

2014–2015 school year, evaluators conducted a variety of analyses, including descriptive and 

inferential statistics as well as the calculation of effect sizes. In this study, children were nested 

in teachers’ classrooms, making multilevel modeling the most appropriate analytic technique 

for conducting inferential analyses with child outcome data. Evaluators also compared JSSC 

gain scores among Jumpstart children and comparison-group children who did not use the 

Jumpstart program.

Evaluators considered findings statistically significant using an alpha level of 0.05. When 

interpreting effect sizes, evaluators followed the What Works Clearinghouse (WWC) guidelines 

that consider effect sizes substantively important when they are greater than or equal to 0.25 

(2014). 

Descriptive Examination of Jumpstart Participants’ JSSC Scores

Did Jumpstart children demonstrate statistically significant gains on JSSC scores?

Before running multilevel modeling analyses to evaluate Jumpstart children’s JSSC 

gains, evaluators calculated changes in JSSC scores from pretest to posttest (see Figure 8). 

The unadjusted means were calculated for descriptive purposes to show trends and were not 

intended to determine if differences were statistically significant. These scores are considered 

“unadjusted” or original scores because they do not take into account the variance associated 

with child- and teacher-level factors.

For Jumpstart children in this study, descriptive statistics suggest that overall, JSSC

scores increased over the school year (pretest to posttest). Figure 7 presents the pretest-toposttest unadjusted scores for Jumpstart participants.
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

13

Student JSSC Scores 

Pretest to Posttest 

5 

4 

3 3.71 

2 

2.24 

1 

0 

Pretest Posttest 

Figure 7. Unadjusted pretest and posttest 

JSSC scores for Jumpstart participants

Multilevel Modeling Analyses Examining Jumpstart Children’s Score Gains

To determine whether Jumpstart children’s

JSSC gains were statistically significant, evaluators 

ran a three-level multilevel modeling analysis. This 

model accounted for the nesting of children in 

teachers’ classrooms as well as for the nesting of 

teachers in different centers. The outcome of 

interest in this analysis was the pretest-to-posttest 

gain score. To determine the magnitude of the 

average gain, evaluators calculated a standardized 

effect size by dividing the adjusted pretest-toposttest difference by the pretest standard deviation. 

As previously mentioned, effect sizes greater than 

+/- 0.25 were considered substantively important. 2

Results show that on average, JSSC scores increased from pretest to posttest by 1.47

points. This average gain was statistically significant (p = <.01) and corresponded to a 

substantively important effect size (1.87).

On average, 

Jumpstart 

children 

gained 1.47

points from 

pretest to 

posttest.

JSSC

Table 6. Jumpstart Participants Mean Pretest to Posttest JSSC Gains. 

Outcome Variable Coefficient Standard 

Error

tvalueApprox. 

df p-value Effect Size

Pretest to Posttest Gain 1.47 0.07 22.73 68 <0.001* 1.87**

* Statistically significant at the 0.05 level.

**Substantively important based on the WWC Standards. 

 

2 What Works Clearinghouse guidelines: 

http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook.pdf
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

14

Analyses of Participants’ JSSC Scores by Treatment and Comparison Groups 

Did the Jumpstart children demonstrate statistically significant gains that are greater than the gains 

of comparison-group children?

KEY FINDINGS: 

The average JSSC gain of children who participated in Jumpstart was statistically 

significantly greater than the average gain of children who did not participate in the program, 

and the effect size for this impact was substantively important based on the WWC 

threshold.

First, evaluators conducted descriptive analyses to compare JSSC scores between

children who participated in Jumpstart and comparison-group children who did not participate in

Jumpstart. Next, evaluators ran multilevel modeling analyses to examine the impact of 

Jumpstart on JSSC gains. Evaluators calculated an effect size and WWC improvement index

(2014) to help readers interpret the magnitude of the program impact.

Descriptive Findings Comparing JSSC Scores by Study Condition 

Before running multilevel modeling analyses, evaluators calculated means 

corresponding to pretest and posttest JSSC scores by study condition. Examining the means 

visually (see Figure 8) revealed that, on average, Jumpstart children gained .44 points more 

than children in the comparison group. Readers should note that evaluators calculated these 

means for descriptive purposes rather than to determine if differences in JSSC scores by study 

condition were statistically significant. 

Student JSSC Scores 

5 

3.71 

4 

3.43 

3 2.40 

2 

2.24 1 

0 

Pretest Posttest 

Jumpstart Comparison 

Figure 8. Unadjusted pretest and posttest 

JSSC scores for Jumpstart and comparisongroup participants
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

15

Multilevel Modeling Analyses Comparing JSSC Scores by Study Condition

After visually examining descriptive statistics, evaluators used multilevel modeling to 

establish whether the Jumpstart program had a statistically significant impact on JSSC scores

when compared to the comparison group program(s). For this analysis, the pretest-to-posttest 

gain score served as the outcome variable. The three-level model accounted for the nesting of 

children in teachers’ classrooms and the nesting of teachers in centers. The model included a 

study condition variable at the teacher level to enable evaluators to estimate the impact of 

Jumpstart on JSSC gains. Additionally, the model included child-level covariates for ethnicity 

and age to account for group differences identified in the group equivalence analyses. After 

running the model, evaluators calculated the standardized effect size3 to examine the 

magnitude of the program impact. After calculating the effect size, evaluators calculated a 

WWC improvement index, which reflected the change in an average comparison-group child’s

percentile rank that would be expected if that child had participated in the Jumpstart program

(2014).

4

 

Findings from the multilevel modeling 

analyses are displayed in Table 3 and show that the 

Jumpstart program had a statistically significant 

positive impact on children’s JSSC scores. More 

specifically, at the end of the study period, children

who participated in Jumpstart scored an average of 

0.45 points higher on the JSSC compared to 

children who did not participate in the program. 

Furthermore, the corresponding effect size of 0.62 

was substantively important based on the WWC 

guidelines. Finally, the WWC improvement index 

suggests that if comparison-group children had 

participated in Jumpstart, the percentile rank for 

their pretest-to-posttest gains would likely have 

increased by 23 percentile points (2014). 

On average, 

Jumpstart 

children

gained .45 points

more than 

comparison 

children on the 

JSSC from pretest 

to posttest 

 

3 Effect sizes were calculated by dividing the adjusted difference between treatment and comparison groups by the 

standard deviation of the comparison group

4 According to the WWC (2014), an improvement index of 10 percentile points (corresponding to an effect size of 0.25), 

would suggest that an intervention would likely yield a 10% increase in percentile rank if a typical comparison-group 

student were to participate in the program. Additionally, it would suggest that 60% of the treatment-group students scored 

higher than the mean for the comparison-group students.
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

16

Table 7. Impact of Jumpstart on Participants’ Pretest-to-Posttest Gains 

Outcome Variable Coefficient Standard 

Error

tvalueApprox. 

df p-value Effect Size

Improvement 

Index

Difference in 

Pretest-to-Posttest 

Gain by Study 0.45 0.11 3.95 83 <0.001* 0.62** 0.23

Condition

* Statistically significant at the 0.05 level.

**Substantively important based on the WWC Standards. 

Summary

Overall, results show that, on average, Jumpstart children demonstrated statistically 

significant and substantively important gains in their JSSC scores from pretest to posttest. 

Moreover, findings indicated that Jumpstart had a statistically significant positive impact on 

JSSC gain scores when compared to the comparison group program(s). The effect size 

corresponding to this impact was substantively important based on the WWC guidelines. 
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

17

Summary and Discussion

The purposes of this quasi-experimental study was to a) examine the impact of 

Jumpstart on improving participants’ oral language and early literacy skills, and b) determine if 

Jumpstart children demonstrated statistically significant gains that were greater than the gains 

of comparison-group children who did not use Jumpstart. In addition, evaluators examined 

group equivalence between Jumpstart and comparison groups for demographic variables and 

pretest scores.

The primary measure for this evaluation was the Jumpstart School Success Checklist 

(JSSC), an oral language and early literacy skills assessment. Jumpstart’s team was responsible 

for collecting and handling initial JSSC data preparation. In the summer of 2015, Jumpstart 

provided Magnolia Consulting evaluators with a clean database, which included JSSC pretest 

and posttest data, demographic data, and evaluation criteria information for Jumpstart and 

comparison-group children. Magnolia completed final data preparation procedures for its 

analyses. The final analysis sample included 1,716 Jumpstart children and 562 comparisongroup children in 162 classrooms across 78 childhood centers in California. 

Evaluators examined group equivalence between the Jumpstart and comparison groups 

by conducting chi-square analyses and an independent samples t-test on demographic 

characteristics and running multilevel modeling analyses on pretest scores. 

Key findings concerning group equivalence are: 

The composition of Jumpstart and comparison groups was similar regarding gender, 

language, and pretest scores. 

There were statistically significantly differences in the composition of Jumpstart and 

comparison groups regarding ethnicity and age, as well as by the type of center from 

which children were drawn. 

To address key evaluation questions regarding participant language and early literacy 

gains as assessed by the JSSC, evaluators conducted a variety of analyses, including 

descriptive statistics, multilevel modeling, and the calculation of effect sizes. Key findings 

concerning child learning and program impact include:

The average JSSC gain of children who participated in Jumpstart was statistically 

significant and substantively important based on the What Works Clearinghouse 

(WWC) threshold of 0.25. 

The average JSSC gain of children who participated in Jumpstart was statistically 

significantly greater than the gains of children who did not participate in the program, 

and the effect size for this impact was substantively important based on the WWC 

threshold. 
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

18

This evaluation yielded important, positive findings regarding the impact of the 

Jumpstart program. Additional topics are worth examining that were beyond the scope of this 

study, and evaluators recommend the following future evaluation activities:

• conduct subgroup analyses to help determine the program’s impact for different 

types of children; 

• collect implementation data collection, interviews, and focus groups to permit a 

deeper understanding of implementation and perceptions of the program; 

• use multiple measures of student outcomes to increase the validity of findings; and

• consider using a randomized control trial (RCT) design in future evaluations.
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

19

References

Deshler, D. D., Hock, M. F., Ihle, F. M., & Mark, C. A. (2011). Designing and conducting literacy 

intervention research. In Kamil, M. L., Pearson, P. D., Moje, E. B., & Afflerbach, P. P. 

(Eds.), Handbook of Reading Research, Vol. IV (pp. 66-83). New York: Routledge.

Erikson, F. (1986). Qualitative methods in research on teaching. In M. C. Wittrock (Ed.), 

Handbook of research on teaching (3rd ed., pp. 119–161). New York: MacMillan.

HighScope. (2012). Educational Research Foundation’s Preschool Child Observation Record. 

http://www.highscope.org/Content.asp?ContentId=78

Immekus, J. C. (2011). The psychometric properties of the Jumpstart School Success 

Checklist: Testing item- and score- level equivalency across sex and program samples. 

Unpublished manuscript, California State University, Fresno.

Jumpstart. America’s early education crisis. Retrieved September 28, 2015 from 

http://www.jstart.org/our-work/americas-early-education-crisis1

Lesnick, J., George, R., Smithgill, C, & Gwynn, J. (2010). Reading on grade level in third grade: 

How is it related to high school performance and college enrollment? Chicago, IL: 

Chapin Hall at the University of Chicago.

Mackenzie, N. & Hemmings, B. (2014) Predictors of success with writing in the first year of 

school. Issues in Educational Research 24(1), 41-54.

Meyers, A. B., Hoff, K. E., Pfister, K. M., Kreke, E. J., Colón, B. M., & Levine, R. A. (2011). 

Assessment of language, literacy, and socio-emotional development among children 

attending preschools served by Jumpstart. Unpublished manuscript, Illinois State 

University.

National Institute for Literacy (2008). Developing early literacy: A report of the National Early 

Literacy Panel. Retrieved from https://www.nichd.nih.gov/publications/pubs

/documents/NELPReport09.pdf

Rasinski, T., Padak, N., Mckeon, C., Wilfong, L., Friedauer, J., & Heim, P. (2005). Is reading 

fluency a key for successful high school reading? Journal of Adolescent & Adult Literacy,

49(1), 22-27.

Snow, C. E., Burns, M. S., & Griffin, P. (1998). Preventing reading difficulties in young children. 

Washington, DC: National Research Council, National Academy Press.

TESOL. Scenarios for ESL standards-based assessment. (2001). Alexandria, Virginia: 

TESOL. Available online http://www.idra.org/IDRA_Newsletter/ 

April_2002_Self_Renewing_Schools_Early_Childhood_Education/TESOL_Statement_on_

Language_and_Literacy_Development_for_Young_English_Language_Learners
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

20

Hernandez, D. (2011). Double jeopardy: how third grade reading skills and poverty influence 

 high school graduation. Baltimore, MD: The Annie E. Casey Foundation.. Retrieved from 

http://www.aecf.org/m/resourcedoc/AECF-DoubleJeopardy-2012-Full.pdf. 

What Works Clearinghouse, U.S. Department of Education, Institute of Education Sciences 

(2014). What Works Clearinghouse procedures and standards handbook (Version 3.0). 

Washington, DC: Author. Retrieved from: 

http://ies.ed.gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v3_0_standards_

handbook.pdf

Wilson, S., Dickinson, D., Rowe, D. 2013). Impact of an Early Reading First program on 

the language and literacy achievement of children from diverse language backgrounds. 

Early Childhood Research Quarterly 28, 578– 592. 
Appendix A: Unadjusted Pretest and Posttest Means by 

Study Condition

This table shows the unadjusted means 

condition.

for the JSSC assessment by variable, time point, and 

Jumpstart Child 

(n=1,716)

N Mean SD

Comparison-Group

Child (n=562)

N Mean SD

Total 

N 

(n=2,278)

Mean SD

Listening to and understanding 

speech

Fall 1,716

Spring 1,716

Using vocabulary

Fall 1,716

Spring 1,716

Using complex patterns of speech

Fall 1,716

Spring 1,716

Showing awareness of sounds in 

words

Fall 1,716

Spring 1,716

Demonstrating knowledge about 

books

Fall 1,716

Spring 1,716

Using letter names and sounds

Fall 1,716

Spring 1,716

Reading

Fall 1,716

Spring 1,716

Writing

Fall 1,716

Spring 1,716

Making choices and plans

Fall 1,716

Spring 1,716

Solving problems with materials

Fall 1,716

Spring 1,716

Initiating play

An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

2.20

3.77

2.37

3.80

2.41

3.81

1.85

3.26

2.42

3.73

1.83

3.50

2.10

3.44

1.89

3.43

2.48

3.85

2.20

3.65

1.16

1.15

1.07

0.99

1.08

1.04

0.82

1.14

0.97

0.91

0.98

1.20

0.82

0.93

0.97

1.08

0.93

0.95

0.88

1.03

562

562

562

562

562

562

561

562

562

562

562

562

562

562

562

562

562

562

562

562

2.38

3.49

2.48

3.54

2.49

3.53

2.09

3.01

2.54

3.56

2.03

3.23

2.20

3.10

2.06

3.12

2.56

3.49

2.29

3.36

1.11

1.20

1.07

1.12

1.11

1.15

0.89

1.09

0.94

0.96

0.99

1.20

0.87

0.96

1.02

1.04

1.00

1.03

0.91

1.03

2,278

2,278

2,278

2,278

2,278

2,278

2,277

2,278

2,278

2,278

2,278

2,278

2,278

2,278

2,278

2,278

2,278

2,278

2,278

2,278

2.24

3.70

2.40

3.74

2.43

3.74

1.91

3.20

2.45

3.69

1.88

3.43

2.13

3.36

1.93

3.35

2.50

3.76

2.22

3.58

1.15

1.17

1.07

1.03

1.09

1.07

0.84

1.13

0.97

0.92

0.99

1.20

0.83

0.95

0.99

1.08

0.95

0.98

0.89

1.03

21
An Evaluation of Jumpstart

Magnolia Consulting, LLC, October 13, 2015

22

Jumpstart Child 

(n=1,716)

N Mean SD

Comparison-Group

Child (n=562)

N Mean SD

Total 

N 

(n=2,278)

Mean SD

Fall 1,716 2.59 1.01 561 2.81 1.05 2,277 2.64 1.02

Spring 1,716 4.01 0.94 562 3.70 1.02 2,278 3.94 0.97

Resolving interpersonal conflict

Fall 1,714 2.12 0.86 562 2.30 0.90 2,276 2.16 0.87

Spring 1,716 3.51 1.12 562 3.27 1.06 2,278 3.45 1.11

Understanding and expressing 

feelings

Fall 1,716 2.17 1.06 562 2.40 1.06 2,278 2.23 1.06

Spring 1,716 3.76 1.16 562 3.47 1.15 2,278 3.69 1.16

Relating to adults

Fall 1,716 2.42 1.16 562 2.66 1.13 2,278 2.48 1.15

Spring 1,716 3.95 1.08 562 3.71 1.12 2,278 3.89 1.09

Relating to other children

Fall 1,716 2.55 1.13 562 2.73 1.09 2,278 2.59 1.12

Spring 1,716 4.11 1.01 562 3.81 1.07 2,278 4.03 1.03

Total scale 

Fall 1,716 2.24 0.78 562 2.40 0.83 2,278 2.28 0.80

Spring 1,716 3.71 0.86 562 3.43 0.90 2,278 3.64 0.88