Evaluation Report Brief

Jefferson Foundation: Jeffco Summer 

of Early Literacy (JSEL)

Office of Research and Evaluation, Corporation for National and Community Service May 2017

nationalservice.gov/research 1

What is the community challenge? 

Many research studies have underscored the importance of 

third-grade reading proficiency as it marks a pivotal point in a 

child’s education when children make the transition from 

learning to read to reading to learn. Research demonstrates 

that students lose, on average, about a month’s worth of 

academic progress in reading over the summer months. 

However, when these data are disaggregated by student 

income level, it is estimated that low-income students lose 

about two months of progress over the summer, while their 

middle-income peers actually increase slightly in their reading 

skills.

Program At-a-Glance

CNCS Program: Social Innovation Fund

Intervention: Jeffco Summer of Early Literacy 

(JSEL)

Subgrantee: Jeffco Schools Foundation

Intermediary: Mile High United Way (MHUW)

Focus Area: Education

Focus Population: At-risk elementary school 

students (grades K-6)

Community Served: Jefferson County, Colorado

What is the promising solution?

The Jeffco Summer of Early Literacy (JSEL) program is a six-week summer school program for elementary 

school students (K-6) in Jefferson County, Colorado. In the summer of 2014, JSEL was implemented at six 

schools and served 660 students, mainly K-3

rd graders. JSEL targets literacy instruction through smaller group

(no more than 15 to a classroom) and individual lessons, and uses Jeffco-produced Comprehensive Approach 

to Literacy Instruction (CALI) and Curriculum Alignment Project documents as guidelines. Teachers conduct 

assessments on a regular basis to tailor instruction. In addition, “Friday Folders” are sent home to keep 

families informed, provide academic tools, and gather feedback.

What was the purpose of evaluation?

The impact evaluation of the Jeffco Schools Foundation’s JSEL 

was conducted by Augenblick, Palaich and Associates (APA)

and completed in 2015. APA examined three years of data,

2012-2014, comparing K-3 students who participated in JSEL to 

similar students from the Jefferson County School District who 

did not participate. The study examined whether JSEL 

participants made greater progress in their reading proficiency 

than similar non-participants, and if there was a difference 

depending on the number of years students participated. APA 

compared scores on the DIBELS Next reading assessment that were collected both in the spring and fall prior 

to and following JSEL participation, and third-grade scores on the reading components of the Transitional 

Colorado Assessment Program (TCAP) standardized assessment. 

An implementation evaluation was also conducted in the summer of 2014 by Jeffco Public Schools to assess the 

fidelity of the program and identify areas for improvement. 
2

What did the evaluation find?

As a subgrantee of the Social Innovation Fund, the Jefferson 

Foundation engaged an independent evaluator to evaluate the 

impact of JSEL. 

•

•

•

•

•

Looking at data pooled from three years, JSEL 

participants in K-3 make greater progress in their 

reading proficiency than non-participants after one 

year of participation, with a difference equivalent to 

moving from the 50th to 54th percentile on the DIBELS 

Next assessment (effect size 0.10, p.<0.05).

The sample was not large enough to detect a 

significant difference between JSEL participants and 

non-participants who scored “proficient” on TCAP results (administered to 3rd graders only; schools 

also switched to PARCC instead of TCAP in the last year of the study).

There was also a statistically significant effect for participation in multiple years of JSEL, indicating 

increased participation was associated with higher test scores. For students who participated in three 

years of JSEL, this was the equivalent of moving from the 50th to the 61st percentile.

Based on focus groups, interviews, and survey responses, the implementation analysis found that:

Most teachers, principals, and parents valued JSEL for student academic opportunities and teacher

professional learning.

All six sites used CALI structures and practices on a regular basis, although there were some 

differences in teachers’ practices related to writing instructions and interventions. 

Evaluation At-a-Glance

Evaluation Design(s): Quasi-experimental using 

propensity score matching

Study Population: K-6 Grade students

Evaluator: Augenblick, Palaich and Associates 

(APA)

This Evaluation’s Level of Evidence*: Moderate

*SIF and AmeriCorps currently use different definitions of levels of evidence.

Notes on the evaluation

The originally envisioned five-year evaluation plan was truncated to three years. This resulted in a reduction 

in the number of JSEL students and comparison students included in the analysis, which reduced the statistical 

power and decreased the likelihood that it would detect a statistically significant effect of the JSEL program.

How is Jeffco using the evaluation findings to improve? 

The Jeffco Foundation has grown JSEL during the summer of 2015 to 

30 schools using support from Jeffco Public Schools. The evidence of 

program effectiveness was part of the information used by Jeffco 

Public Schools to support JSEL’s growth and expansion. Several 

changes were made to the 2015 programing in response to the 

implementation evaluation and changes in the amount of resources 

that program planners had at their disposal, including:

•

•

•

•

Utilizing a universal intervention (Leveled Literacy 

Intervention) and resource (Reading A to Z);

Adding the TRC/Reading 3D to assessments administered to participating students;

Adding a transportation system between schools and host site; and

Replacing field trips with on-site enrichment.

Jefferson County, Colorado

The content of this brief was drawn from the full evaluation report submitted to CNCS by the grantee/subgrantee. The section of the brief that 

discusses evaluation use includes contribution of the grantee/subgrantee. All original content from the report is attributable to its authors.

To access the full evaluation report and learn more about CNCS, please visit nationalservice.gov/research.

The Social Innovation Fund (SIF), a program of the Corporation for National and Community Service (CNCS), combines public and private resources to grow the impact of 

innovative, community-based solutions that have compelling evidence of improving the lives of people in low-income communities throughout the U.S. The SIF invests in three 

priority areas: economic opportunity, healthy futures, and youth development.

Office of Research and Evaluation, Corporation for National and Community Service May 2017 

nationalservice.gov/research