Evaluation Report Brief

Greater Twin Cities United Way, Saint Paul 

Promise Neighborhood: NdCAD-Sankofa

Office of Research and Evaluation, Corporation for National and Community Service August 2018

nationalservice.gov/research 1 

Program At-a-Glance

CNCS Program: Social Innovation Fund (SIF)

Intervention: Network for the Development of Children 

of African Descent’s (NdCAD) Sankofa Program

Grantee: Greater Twin Cities United Way

Subgrantee: Saint Paul Promise Neighborhood (SPPN) 

Focus Area: Youth development

Focus Population: Kindergarten through third-grade

children of African-descent enrolled in elementary 

schools or living in the SPPN boundaries with low 

reading scores.

Community Served: Greater Twin Cities MinneapolisSaint Paul, MN

What is the community challenge?

The Saint Paul Promise Neighborhood (SPPN), a community 

initiative was awarded subgrantee funding from GTCUW and 

Generation Next to partner with The Network for the 

Development of Children of African Descent (NdCAD) to 

implement the Sankofa reading program at its partner schools

to close the achievement gap in reading skills among 

elementary-school students in African and African-descended 

communities in the SPPN boundaries. 

What is the promising solution?

Sankofa is a culturally-based literacy tutoring program for 

children of African descent designed to increase reading skills, 

increase independent guided reading levels, and improve 

academic performance. The program consists of 

comprehensive after-school tutoring with strong reading and 

cultural components. 

What was the purpose of evaluation?

This report presents an evaluation of Network for the Development of Children of African Descent (NdCAD)’s 

Sankofa program implemented in the Saint Paul Promise Neighborhood (SPPN) from spring 2013 through fall 2017, 

with a particular focus on the third and fourth years, school years 2015 – 2016 and 2016 – 2017, of the SIF. The 

Center for Applied Research and Educational Improvement (CAREI) at the University of Minnesota began the 

evaluation in 2013 and finished reporting in 2017. This evaluation included an implementation evaluation and an 

impact evaluation of preliminary and moderate evidence. Assessment of implementation fidelity was carried out to 

ensure that the core components of Sankofa took place. The preliminary evidence impact evaluation utilized a onegroup pre-post design, and descriptive analyses were carried out on pre- and post-program performance of 

participants on a standardized reading assessment. The designs for moderate evidence utilized an interrupted time 

series (ITS) design and propensity score matched (PSM) comparison to assess the effects of Sankofa on reading 

ability using progress monitoring assessments developed by FastBridge Learning (FBL) and a statewide 

standardized reading assessment. Research questions examined potential effects of the program on several external 

reading outcomes, including scores on standardized assessments, soft grade-level reading targets, and other reading

indicators. Parents also reported on changes in their children’s behaviors and attitudes on reading and school. 

What did the evaluation find?

As a subgrantee of Greater Twin Cities United Way, Saint Paul Promise Neighborhood was assigned an 

independent evaluator, CAREI, to evaluate Sankofa. 

• The preliminary impact evaluation design found 90 percent of Sankofa participants increased their reading

performance on the DRA II by at least one reading level over the course of the program. The percent of

students below the soft-target grade-appropriate reading level likewise decreased for all grades (K-3). These
2 

findings from the preliminary evidence design, lacked a comparison group and suffered from limited 

internal validity (as the increase could have been caused by external factors or normative growth). 

• The moderate evidence impact evaluation design found a significant effect of Sankofa on growth in reading 

for the Decodable Words assessment using an ITS design (moderate level of evidence design), but not for 

the Letter Sounds or Sight Word assessments. However, this effect for Decodable Words was not significant 

after correcting for multiple comparisons (suggesting this finding could have been obtained by chance) and 

it was not corroborated by the multiple imputation pooled analysis.

• For the exploratory matched-comparison design, in Years 1 and 2, there were no effects of Sankofa on MCAReading scores. In Years 3 and 4, a significant difference was found. On average, Sankofa participants were 

expected to score 7.25 points lower on the MCA than the non-Sankofa SPPS comparison. 

Notes on the evaluation

The evaluation was not able to fully address all major threats to internal validity. As noted in the report, missing 

data was the largest threat to the evaluation’s internal validity. Although multiple imputation was used to address 

missingness, because the variables included in the multiple imputation model were driven by availability and not 

by theory, analyses may have had limited statistical validity. For the interrupted time series analysis, it was difficult 

to identify prospective participants far enough in advance to assess them six weeks prior to the program and track 

them for six weeks post-Sankofa, resulting in incomplete data for every participant. Likewise, the evaluation could 

not compare Sankofa students to similar students who did not participate in Sankofa for most of the outcome 

variables due to difficulties identifying an appropriate comparison sample or obtaining outcome data from 

comparison group students or their schools. 

 

How is GTCUW using the evaluation findings to improve?

The results of the evaluation generated several recommendations for how the 

program might be strengthened to improve recruitment, enrollment and attendance 

procedures, as well as data collection strategies. 

• As the number of homeless or highly mobile Sankofa participants increases, it 

might be useful to develop a tutor training module focused on supporting these 

students. 

• Consider adding home visits and developing strategies for former 

students/graduates and their families to remain connected to the program.

• Consider additional tutor trainings that focus on working with readers with 

varied skills given the range of reading abilities of participants entering Sankofa. 

• Continue to develop an evaluation framework that identifies different types of 

outcome measures suited for tracking growth and individual student’s progress 

that can validly measure program impact, and are practically feasible to 

implement. The framework should use data collected by the program or the 

evaluation to reduce reliance on external sources such as schools. If an ITS 

design is not feasible, alternative quasi-experimental designs such as regression 

discontinuity could be considered. 

Evaluation At-a-Glance

Evaluation Design(s): Several 

designs used including PSM 

and ITS quasi-experimental 

designs

Study Population: Kindergarten 

through third-grade children of 

African-descent enrolled in 

elementary schools or living in 

the SPPN boundaries. 

Independent Evaluator: Center 

for Applied Research and 

Educational Improvement

(CAREI) at the University of 

Minnesota 

This Evaluation’s Level of 

Evidence*: Preliminary 

*SIF and AmeriCorps currently use different 

definitions of levels of evidence.

The content of this brief was drawn from the full evaluation report submitted to CNCS by the grantee/subgrantee. The section of the brief that discusses 

evaluation use includes contribution of the grantee/subgrantee. All original content from the report is attributable to its authors. To access the full 

evaluation report and learn more about CNCS, please visit http://www.nationalservice.gov/research. 

The Social Innovation Fund (SIF), a program of the Corporation for National and Community Service (CNCS), combines public and private resources to grow the impact of innovative, 

community-based solutions that have compelling evidence of improving the lives of people in low-income communities throughout the U.S. The SIF invests in three priority areas: 

economic opportunity, healthy futures, and youth development.

Office of Research and Evaluation, Corporation for National and Community Service August 2018

nationalservice.gov/research