Evaluation Report Brief

Greater Twin Cities United Way: Great 

by Eight Program

Office of Research and Evaluation, Corporation for National and Community Service September 2018

nationalservice.gov/research 1

What is the community challenge? 

During the early years, the home is a child’s primary developmental 

context, and the processes that occur within this setting are important 

influences on children’s subsequent development. Children and 

families who experience high levels of poverty may not have access 

to the necessary community resources and information that

adequately supports families and their children’s overall growth and 

development

What is the promising solution?

The Great by Eight program is a parent-centric program that uses 

home visits to increase parent knowledge and engagement for their 

child’s development, which will lead to improved child success in 

school and life.

Program At-a-Glance

CNCS Program: Social Innovation Fund

Intervention: Great by Eight Program

Subgrantee: Way to Grow

Grantee: Greater Twin Cities United Way in 

Partnership with Generation Next

Focus Area: Youth Development.

Focus Population: Families and their 

children ages 3 to 8 who are most at risk for 

low academic achievement.

Community Served: Minneapolis 

Metropolitan Area

What was the purpose of evaluation?

The Center for Applied Research and Educational Improvement (CAREI) conducted the evaluation of the 

Great by Eight Program that included both an implementation and impact study. The final implementation 

study examined parent’s perceptions of program components and quality using a parent survey, standardized 

assessments, and program data. The final impact study examined the impact of home visits on Kindergarten 

Readiness and Third Grade School Competencies. Propensity score matching techniques were used to facilitate 

a multilevel logistic and multiple regression methodology. The evaluation included all Great by Eight 

students who entered kindergarten in the fall of 2017, and children who entered third grade in the fall of 2015 

or fall 2016who were enrolled at MPS. The total sample size of treatment and control members included in the 

evaluation for both kindergarten and third grade was 190 children.

What did the evaluation find?

As a grantee of CNCS Programs Greater Twin Cities United Way, in collaboration with subgrantee Way to 

Grow, engaged the independent evaluator CAREI to evaluate the Great by Eight Program. The evaluation 

found: 

•

•

•

•

Results from the parent survey indicate that parents had a high overall level of reported learning and 

perceived usefulness of the Great by Eight program and its components.

Great by Eight had a positive program impact (t=2.107; p < .037; d=.37) on third grade reading with its 

students scoring significantly higher than comparison group third grade students. 

Children whose parents were involved in Great by Eight achieved appropriate milestones.

Great by Eight four year olds had statistically significant increases in literacy skills on the Individual 

Growth and Development Indicators (IGDI) and Dynamic Indicators of Basic Early Literacy Skills 

(DIBELS) assessments, and met expectations for child development and growth milestones in 
Office of Research and Evaluation, Corporation for National and Community Service September 2018

nationalservice.gov/research 2

preparation for success in formal schooling at each fall to winter and winter to spring time point. The 

Great by Eight’s home visiting dosage had a positive effect on parents’ accessing health, medical, and 

educational resources, all of which has a positive impact on children’s physical, emotional, and 

cognitive development.

Notes on the evaluation

The Beginning Kindergarten Assessment (BKA) test scores were unavailable for about 50 percent of study 

participants, and those were probably not missing at random because the test was optional for schools and 

some chose not to administer. Thus, the null findings for Kindergarten Readiness are questionable. In 

measuring child developmental milestones, the evaluation was unable to get a sufficient data set of children in 

this age group who were also part of the random sample of 100 parents for the main impact study questions. 

As a result, they opened up the question to all four-year-olds in their program. It is unclear if or whether this 

larger sample of four-year-olds (n=98) differed from those in the other impact questions of this study.

How is Way to Grow using the evaluation findings to improve? 

The results of the evaluation generated several recommendations for how the Great 

by Eig

•

•

•

ht program might be strengthened. 

In addition to parent surveys and staff discussions, frequent in-home 

observation of educator-family interactions are necessary. The Home Visit 

Rating Scales (HOVRS) may or may not be the best instrument for 

supervisors to assess family educator fidelity of implementation efforts 

during home visits. 

To avoid relying on public school data for program evaluation Great by 

Eight should explore appropriate alternative assessments that can be 

administered internally for the children and families it serves to more 

reliably assess home visiting, early learning and elementary programing.

Assess the appropriateness of the program’s current outcome measurement 

instruments. The LSP scales may or may not be reliable indicators for 

assessing Way to Grow parents, and the Parent Engagement Code may not 

provide useful formative information for family educator support efforts for 

parents. 

Evaluation At-a-Glance

Evaluation Design:

Propensity Score Matching 

with multilevel logistic and 

multiple regression.

Study Population:

Kindergarten and third 

grade students in 

Minneapolis Public 

Schoolswho participated in

the Great by Eight 

Program.

Independent Evaluator: 

Center for Applied 

Research and Educational 

Improvement (CAREI)

This Evaluation’s Level of 

Evidence*: Moderate

*SIF and AmeriCorps currently use 

different definitions of levels of evidence.

The content of this brief was drawn from the full evaluation report submitted to CNCS by the grantee/subgrantee. The section of the brief that discusses 

evaluation use includes contribution of the grantee/subgrantee. All original content from the report is attributable to its authors.

To access the full evaluation report and learn more about CNCS, please visit http://www.nationalservice.gov/research.

The Social Innovation Fund (SIF), a program of the Corporation for National and Community Service (CNCS), combines public and private resources to grow the impact of innovative, 

community-based solutions that have compelling evidence of improving the lives of people in low-income communities throughout the U.S. The SIF invests in three priority areas: 

economic opportunity, healthy futures, and youth development.