Evaluation Report Brief

Mile High Early Learning and 

Clayton Early Learning: Play and 

Learn Groups with LENA® Feedback

Office of Research and Evaluation, Corporation for National and Community Service August 2018

nationalservice.gov/research 1

What is the community challenge? 

A critical opportunity for developing fundamental 

early literacy skills is adult-child interaction with 

books and storytelling. Research indicates that there 

are disparities in children’s language and literacy 

development, with children and families from 

under-resourced communities receiving less early 

language and literacy support.

Program At-a-Glance 

CNCS Program: Social Innovation Fund

Intervention: Play and Learn Groups (PLG) with LENA®

(Language Environment Analysis System)

Grantee: Mile High United Way

Subgrantee: Mile High Early Learning and Clayton Early Learning

Focus Area(s): Youth Development

Focus Population(s): Low-income families with infants and 

toddlers ages birth to 30 months 

Community Served: Denver, CO

What is the promising solution?

Play and Learn Groups (PLG) are structured play groups for young children and their caregiver(s) led by a 

trained facilitator. Participants receive: training in Dialogic Reading (DR; an evidence-based early literacy 

program that focuses on interactive reading); parent meetings on developmental topics; and coaching in 

reading and parenting skills. The LENA® system uses an audio-recording device that children wear for 10 to 16 

hours to record the home literacy environment. Facilitators present LENA® results to parents and discuss ways 

to increase meaningful adult-child interactions that support the development of oral language and 

communication skills. The program seeks to increase language-rich interactions between children and parents, 

increase the quality and frequency of book reading, and increase children’s oral language and communication 

skills.

What was the purpose of evaluation?

The evaluation of the Play and Learn Groups’ use of LENA® began in 2012, with reporting completed in 2017

by the Butler Institute for Families at the University of Denver. The evaluation assessed whether families who 

received LENA Feedback in addition to DR and the other PLG components experienced greater gains in child 

oral and communication skills and greater frequency and quality of literacy activities in the home relative to 

those who did not receive LENA® Feedback. The evaluation employed a quasi-experimental design with sites 

randomly assigned to test the impact of the LENA® feedback system. There were three treatment sites and two

control sites. During the time of the study, 370 eligible children were enrolled in the centers; 132 children were 

in the treatment group (58% of eligible PLG participants from three sites) and 78 children were in the control 

group (55% of eligible PLG participants from two sites).
Office of Research and Evaluation, Corporation for National and Community Service August 2018

nationalservice.gov/research 2

What did the evaluation find?

Mile High Early Learning and Clayton Early Learning engaged in a rigorous evaluation of the impact on

parents and children receiving LENA® Feedback as part of the Play and Learn Groups program. The 

evaluation included an implementation study that focused on observing how the program was implemented 

and operated, and an impact study that sought to determine the extent to which the feedback influenced

participants’ language-rich interactions, book reading, and oral language and communication skills relative to 

those who did not receive feedback.

This five-year study demonstrated that:

• Parents in the intervention group (who received LENA® Feedback and DR) made statistically 

significant gains over time in support of their child(ren)’s language and literacy skills, while

comparison parents (in the DR only group) remained stable or showed more modest gains. These 

significant impacts are summarized below.

Outcome Evidence of Impact Effect Size

Reading Frequency 

•

•

Statistically significant difference between intervention and 

comparison groups’ growth over time.

Greatest gains among those who received 2 or more LENA®

Feedback sessions.

Small

Storytelling Frequency •

•

Intervention group: Statistically significant increase over time.

Comparison group: No significant increase over time. Medium

Interactive Reading 

Behaviors (parent report) 

•

•

Intervention group: Statistically significant increase over time.

Comparison group: No significant increase over time. Medium

Notes on the evaluation

Because literacy is a complex construct, this evaluation utilized multiple indicators of literacy, measured at 

either the child or parent/caregiver level (e.g., reading or storytelling frequency, conversational turns, 

interactive reading, vocabulary, words produced, verbal and nonverbal communication). By most indicators of 

literacy, parents who received LENA® Feedback did not make significantly greater gains than did parents who 

did not receive feedback (when compared directly to one another in the same analysis), possibly due to lack of 

impact or lack of power to detect an effect. Therefore, although families in the treatment condition experienced 

increases over time on most indicators, the evaluation cannot confidently attribute improvements in scores 

over time to the LENA® intervention. Also, baseline equivalence on race/ethnicity and language was not 

achieved between the unmatched treatment and comparison groups, suggesting the significant effects that did 

emerge in the evaluation may be due to preexisting group differences or demographics. On several of the 

measures, scores cannot be calculated if data on multiple items are missing (this was the case for only one 

participant on one measure), while on other measures, fewer than 2% of the responses on any given item were 

missing, thus the study did not adjust for missing data. The study did not adjust for multiple comparisons, nor 

did it account for nestedness of students within sites as site characteristics were confounded with 

race/ethnicity and language, which were controlled for where relevant. At the child level, vocabularies of both 

intervention and comparison children increased significantly, but due to low power and threats to validity, the 

evaluation could not establish with certainty that the LENA® Feedback intervention was responsible for these 

increases.
Office of Research and Evaluation, Corporation for National and Community Service August 2018

nationalservice.gov/research 3

How are Clayton Early Learning and Mile High Early 

Learning using the evaluation findings to improve? 

Based on the results of this study, it was determined 

that LENA® Feedback may be too expensive to continue 

to implement, considering its relatively small impact. 

Future opportunities to support families’ early 

language and literacy skills could include identifying 

other types of individualized data (e.g., parent-child 

observations) that are simpler and less expensive to 

collect and can be used during one-on-one coaching 

sessions with families.

Evaluation At-a-Glance

Evaluation Design: Quasi-experimental (QED) impact 

evaluation with five sites randomly assigned to either the 

experimental (LENA® Feedback) or comparison (PLGs “as 

usual”) conditions 

Study Population: Infants and toddlers ages birth to 30 

months

Independent Evaluator: The Butler Institute for Families

This Evaluation’s Level of Evidence*: Preliminary

*SIF and AmeriCorps currently use different definitions of levels of evidence.

The content of this brief was drawn from the full evaluation report submitted to CNCS by the grantee/subgrantee. The section of the brief that discusses 

evaluation use includes contribution of the grantee/subgrantee. All original content from the report is attributable to its authors.

To access the full evaluation report and learn more about CNCS, please visit http://www.nationalservice.gov/research.

The Social Innovation Fund (SIF), a program of the Corporation for National and Community Service (CNCS), combines public and private resources to grow the impact of innovative, 

community-based solutions that have compelling evidence of improving the lives of people in low-income communities throughout the U.S. The SIF invests in three priority areas: 

economic opportunity, healthy futures, and youth development.