Executive Summary

USING INTEGRATED 

STUDENT SUPPORTS TO 

KEEP KIDS IN SCHOOL

A QUASI-EXPERIMENTAL EVALUATION OF 

COMMUNITIES IN SCHOOLS

———

Marie-Andrée Somers 

Zeest Haider 

APRIL 2017

Executive Summary

Using Integrated Student Supports to 

Keep Kids in School 

A Quasi-Experimental Evaluation of 

Communities In Schools

Marie-Andrée Somers

Zeest Haider 

 

 

April 2017
This report is funded by the Edna McConnell Clark Foundation’s Social Innovation Fund, a White 

House initiative and program of the Corporation for National and Community Service (CNCS). The 

Edna McConnell Clark Foundation’s Social Innovation Fund includes support from CNCS and 15 

private coinvestors: The Edna McConnell Clark Foundation, The Annie E. Casey Foundation, The 

Duke Endowment, The William and Flora Hewlett Foundation, The JPB Foundation, George Kaiser 

Family Foundation, The Kresge Foundation, Open Society Foundations, The Penzance Foundation, 

The Samberg Family Foundation, The Charles and Lynn Schusterman Family Foundation, The Starr 

Foundation, Tipping Point Community, The Wallace Foundation, and the Weingart Foundation. 

Dissemination of MDRC publications is supported by the following funders that help finance 

MDRC’s public policy outreach and expanding efforts to communicate the results and implications of our work to policymakers, practitioners, and others: The Annie E. Casey Foundation, 

Charles and Lynn Schusterman Family Foundation, The Edna McConnell Clark Foundation, Ford 

Foundation, The George Gund Foundation, Daniel and Corinne Goldman, The Harry and Jeanette 

Weinberg Foundation, Inc., The JBP Foundation, The Joyce Foundation, The Kresge Foundation, 

Laura and John Arnold Foundation, Sandler Foundation, and The Starr Foundation.

 

In addition, earnings from the MDRC Endowment help sustain our dissemination efforts. Contributors to the MDRC Endowment include Alcoa Foundation, The Ambrose Monell Foundation, 

Anheuser-Busch Foundation, Bristol-Myers Squibb Foundation, Charles Stewart Mott Foundation, Ford Foundation, The George Gund Foundation, The Grable Foundation, The Lizabeth and 

Frank Newman Charitable Foundation, The New York Times Company Foundation, Jan Nicholson, Paul H. O’Neill Charitable Foundation, John S. Reed, Sandler Foundation, and The Stupski 

Family Fund, as well as other individual contributors.

Opinions or points of view expressed in this document are those of the authors and do not necessarily reflect the official position of — or a position that is endorsed by — CNCS, the Social 

Innovation Fund program, or other MDRC funders. 

For information about MDRC and copies of our publications, see our website: www.mdrc.org. 

Copyright © 2017 by MDRC®. All rights reserved.
iii

Overview 

The Communities In Schools (CIS) Model of Integrated Student Supports aims to reduce dropout 

rates by providing students with integrated and tiered support services based on their levels of need. 

The model includes preventive services that are available to all students (Level 1 services) as well as 

intensive, targeted, and sustained services provided through case management (Level 2 services) for 

the 5 percent to 10 percent of students who display significant risk factors for dropping out, such as 

poor academic performance, high absentee rates, or behavioral problems. The CIS model posits that 

these tiered, integrated services will give students the skills and resources they need to succeed, 

which will lead to improvements in their outcomes. 

In elementary schools, the CIS model focuses on improving attendance rates by engaging parents. In

middle schools, the model begins to emphasize helping students improve their behavior. In high 

schools, the model focuses on services specifically intended to prevent students from dropping out, 

to help them progress through school, and to make sure they graduate.

This study, which is based on a quasi-experimental research design, examines the CIS model’s 

effect on students’ outcomes in elementary schools, middle schools, and high schools. The sample 

for this study includes 53 CIS schools in Texas and North Carolina (14 high schools, 15 middle 

schools, and 24 elementary schools) that started implementing the CIS model between 2005 and 

2008. The study compares these CIS schools with 78 matched comparison schools (18 high schools, 

24 middle schools, and 36 elementary schools). It is funded by the Edna McConnell Clark Foundation’s Social Innovation Fund. 

For the high schools, the main finding is that on-time graduation rates increased — and dropout rates 

decreased — in the study schools after the CIS model was launched. Graduation and dropout rates 

also improved in the comparison schools, so it is unclear whether the CIS model was more effective 

than the strategies used by the comparison schools. The findings do suggest that the CIS model may 

be at least as effective as these other approaches. In elementary schools, attendance rates (a central

outcome the CIS model aims to effect in the elementary grades) improved in schools implementing 

the CIS model more than they did in a group of similar, comparison schools. There was no effect on 

attendance in middle and high schools. In middle schools, English/language arts test scores did not 

improve in schools implementing the CIS model, whereas they did improve in a group of similar, 

comparison middle schools. There was no effect on test scores in elementary and high schools. (It is 

important to note, however, that the CIS model does not attempt to improve state test scores or the 

quality of instruction provided during regular school hours.) Unfortunately, it was not possible to 

evaluate whether the CIS model improved middle school students’ behavioral outcomes, which is 

the model’s primary goal in those grades. 

v 

Preface

Although many services and other forms of support are available to students at risk of dropping 

out, far too many students still fail to complete high school. The problem may lie in the fact that 

services for at-risk students are spread across many different government agencies and nonprofit

organizations, which makes it more challenging for schools to identify the services available to 

students and for students themselves to use them.

This report examines an integrated model of student support created by Communities In 

Schools (CIS), which is now working in about 2,400 schools and 360 school districts. The CIS 

model provides comprehensive and integrated services to students in different areas (academics, 

behavior, social skills and life skills, family outreach, health and wellness, etc.), delivered with 

varying intensity and duration based on students’ level of need. Level 1 short-term, preventive 

services are broadly available to all students at a school, whereas Level 2 intensive, longer-term, 

targeted services are for students at higher risk of dropping out.

The schools in this study, located in Texas and North Carolina, started implementing 

the CIS model between 2005 and 2008. Using a quasi-experimental design, the study found 

mixed but promising results. In elementary schools, it appears that the CIS model may have 

improved attendance rates, which is consistent with the findings of other studies of CIS (including a randomized experiment in K-8 schools in Chicago). In middle schools, the CIS model 

does not appear to have improved any of the outcomes that could be measured in this study 

(attendance rates and test scores). In high schools, graduation rates improved after the CIS 

model was implemented, but it is not clear whether CIS caused these improvements.

Thus far, two quasi-experimental studies — the present one by MDRC and an earlier 

one by ICF International — have found that high schools implementing the CIS model have 

increased their graduation rates. In both studies, however, these findings have been inconclusive 

because of limitations related to finding a group of credible comparison schools. Therefore, a 

useful next step for CIS would be an evaluation based on a school-level randomized experiment. Such an experiment would provide the most rigorous evidence of the model’s effects, and 

ultimately the most useful information for policymakers, districts, and schools. 

Gordon L. Berlin 

President, MDRC

vii

Acknowledgments

This report is based on work supported by the Social Innovation Fund, a White House initiative 

and program of the Corporation for National and Community Service (CNCS). The Social 

Innovation Fund combines public and private resources with the goal of increasing the impact 

of innovative, community-based solutions that have compelling evidence of improving the lives 

of people in low-income communities throughout the United States.

The Edna McConnell Clark Foundation’s Social Innovation Fund includes support 

from CNCS and 15 private coinvestors: The Edna McConnell Clark Foundation, The Annie E. 

Casey Foundation, The Duke Endowment, The William and Flora Hewlett Foundation, The 

JPB Foundation, George Kaiser Family Foundation, The Kresge Foundation, Open Society 

Foundations, Penzance Foundation, The Samberg Family Foundation, The Charles and Lynn 

Schusterman Family Foundation, The Starr Foundation, Tipping Point Community, The 

Wallace Foundation, and Weingart Foundation. This report would not have been possible 

without these organizations’ support and commitment to the well-being of young people in lowincome communities in the United States.

We owe special thanks to Communities In Schools (CIS) national and affiliate staff 

members for their support and cooperation throughout this study. At the national level, Heather 

Clawson provided important information about the organization. She also coordinated with 

other national and affiliate staff members to obtain data about the schools in the study, and we 

are grateful to them for providing this information.

We wish to thank several individuals outside of MDRC who gave us useful criticisms

and encouragement throughout the study, and who provided insightful comments on earlier 

drafts of this report: Heather Clawson and Kevin Leary at CIS National; Gabriel Rhoads, Kelly 

Fitzsimmons, and Partheev Shah at The Edna McConnell Clark Foundation; Hilary Rhodes and 

Dara Rose at The Wallace Foundation; Jason Snipes and Bob Granger, who reviewed the report 

as members of the Evaluation Advisory Board for the Edna McConnell Clark Foundation’s 

Social Innovation Fund; and the reviewers at CNCS. 

At MDRC, William Corrin and Leigh Parise — the project director and manager for the 

study — provided extremely helpful suggestions, support, and guidance at critical times during 

the analysis of the data and the writing of this report. Kateryna Lashko processed and cleaned 

the many data files used for the analysis. Kelly Quinn and Daphne Chen contributed to report 

production, spending countless hours formatting tables and checking the accuracy of the 

information in the report, among other tasks. Kelly Granito helped our team organize its work 

on this report, with support from Kate Gualtieri. Kate was also a valuable link between the 
viii

project team and the broader Edna McConnell Clark Foundation Social Innovation Fund

community of program organizations and evaluators.

Gordon Berlin, Fred Doolittle, Howard Bloom, Robin Jacob, Rob Ivry, and Joshua 

Malbin carefully reviewed earlier drafts of the report, offered helpful ideas about how to present 

the findings, and provided valuable advice on conducting additional analyses. We also want to 

extend our thanks to MDRC’s board members, who offered insightful critiques on how to 

interpret the findings from the study. Joshua Malbin edited the full report, and Carolyn Thomas

and Ann Kottner prepared it for publication.

The Authors
1

Executive Summary

Every day more than 7,000 students drop out of school.1 One-fifth of students who enter high 

school do not graduate within four years,2 and more than two-fifths of Latino and AfricanAmerican boys drop out.3 Many students at risk of dropping out need academic and social

services and other forms of support to make it through high school. However, these services are 

scattered across numerous government agencies and nonprofit organizations, which limits their 

potential to change the path of an at-risk student. Integrating student support services and 

connecting them with schools is viewed as a promising approach to assist school staff members

and help students stay on track to graduate.4

This report presents the findings from a quasi-experimental study of the Communities 

In Schools (CIS) Model of Integrated Student Supports (referred to in this report as the “CIS 

model”), a promising whole-school approach that aims to reduce dropout rates by providing 

students with integrated and tiered support services based on their needs. Because some of the 

factors that put students on the path toward dropping out of high school are established well 

before ninth grade, CIS works with elementary schools, middle schools, and high schools. This 

study of the CIS model, which is funded by the Edna McConnell Clark Foundation’s Social 

Innovation Fund, provides a unique opportunity to gain a better understanding of the potential 

effects of a tiered and integrated approach to dropout prevention that serves students of all 

ages.5

In high schools, the main finding from this study is that on-time graduation rates increased — and dropout rates decreased — in study schools after the CIS model was launched. 

However, it is not clear whether these improvements were caused by the CIS model. On the one 

 1

Christopher B. Swanson, “Progress Postponed,” Education Week 29, 34 (2010): 22-23. 2

Richard J. Murnane, “U.S. High School Graduation Rates: Patterns and Explanations,” Journal of 

Economic Literature 51, 2 (2013): 370-422; Marie C. Stetser and Robert Stillwell, Public High School FourYear On-Time Graduation Rates and Event Dropout Rates: School Years 2010-11 and 2011-12 (Washington, 

DC: National Center for Education Statistics, U.S. Department of Education, 2014). 3

Schott Foundation for Public Education, The Urgency of Now: The Schott 50 State Report on Public 

Education and Black Males (Cambridge, MA: Schott Foundation for Public Education, 2012). 4

Kristin A. Moore, Selma Caal, Rachel Carney, Laura Lippman, Weilin Li, Katherine Muenks, David 

Murphey, Dan Princiotta, Alysha Ramirez, Angela Rojas, Renee Ryberg, Hannah Schmitz, Brandon Stratford, 

and Mary Terzian, Making the Grade: Assessing Evidence for Integrated Student Supports (Bethesda, MD: 

Child Trends, 2014). 5

The Social Innovation Fund is a program of the Corporation for National and Community Service. The 

Social Innovation Fund combines public and private resources to increase the impact of innovative, communitybased solutions that have compelling evidence of improving the lives of people in low-income communities 

throughout the United States.
2 

hand, graduation and dropout rates improved by greater amounts in the CIS high schools than in 

a group of comparison high schools, which suggests that the CIS model may have improved 

these rates more than they would have improved otherwise. On the other hand, the comparison 

schools and CIS schools had different graduation and dropout rates before the CIS model was 

implemented, so the comparison schools may not provide a credible reference point. For this 

reason, it is not possible to determine whether the CIS model was more effective than the 

strategies used by the comparison schools, although the study’s findings suggest that the CIS 

model may be at least as effective as these other approaches.

In elementary schools, attendance rates (a central outcome measure CIS aims to affect 

in the elementary grades) improved in schools implementing the CIS model more than they did 

in a group of similar comparison schools. In middle schools, English/language arts (ELA) test 

scores did not improve in schools implementing the CIS model, whereas they did improve in a 

group of similar, comparison middle schools. Unfortunately, it was not possible to evaluate 

whether the CIS model improved middle school students’ behavioral outcomes, which is the 

model’s primary goal in those grades. 

What Is the CIS Model?

Communities In Schools, which was founded in 1977 by children’s advocate Bill Milliken, 

works with low-income K-12 students who are at risk of failing or dropping out of the nation’s 

poorest-performing schools. CIS seeks to reduce dropout rates by integrating preventive 

services available to the entire school with intensive, targeted, and sustained services for the 5 

percent to 10 percent of students who display significant risk factors for dropping out, such as 

poor academic performance, high absentee rates, or behavioral problems. CIS now serves 1.5 

million students and their families in 25 states and the District of Columbia. It is active in 

approximately 2,300 schools and 360 school districts.6

CIS provides services and support to students in 10 different areas: academics, behavior, social skills and life skills, basic needs and resources, college and career preparation, 

enrichment and motivation, family outreach and engagement, health and physical wellness, 

community service, and mental health. These 10 categories of services are provided at two 

levels of intensity and duration, depending on students’ needs:

• Level 1 school-wide and preventive services: Level 1 services are broadly 

available to all students at the school and are usually short-term, lowintensity activities or services (for example, making clothing or school sup-

 6

Communities In Schools, 2015 Annual Report (Arlington, VA: Communities In Schools, 2015). 
3 

plies available to students, organizing a school-wide career fair, or hosting a 

financial aid workshop for twelfth-graders).

• Level 2 targeted services: In contrast, Level 2 services are intensive, often 

long-term, and targeted forms of support that are delivered to students who 

are displaying one or more significant risk factors for dropping out, such as 

poor academic performance, a high absentee rate, or behavioral problems. 

Level 2 services include forms of support such as individual and group counseling, tutoring, and after-school programs. 

The CIS model’s goals are different in high schools, middle schools, and elementary 

schools. In all three, the CIS model provides support in each of the 10 service categories, but the 

emphasis in each case is calibrated to the model’s core goals for that type of school. In elementary schools, the model focuses on improving attendance rates by reaching out to and engaging 

parents. In middle schools, the model begins to emphasize helping students improve their

behavior. In high schools, the model focuses on services specifically intended to prevent 

students from dropping out, help them progress through school, and make sure they graduate.

How Was the CIS Model Evaluated?

This study conducted by MDRC examines whether introducing the CIS model with all of its 

components improves schools’ graduation rates, dropout rates, attendance rates, and state test 

scores.

The effect of the CIS model is evaluated using a comparative interrupted time series 

(CITS) design.

7 Two groups of schools are studied: one group that implemented the CIS model 

(CIS schools) and another group that did not implement the CIS model but was free to adopt 

some other reform or initiative (comparison schools). The first step in a CITS design is to 

determine the trends in school outcomes for the CIS schools and the comparison schools during 

the years before the intervention was launched. These are called the “baseline trends.” The 

second step is to gauge how much the CIS and comparison schools “deviated” from their 

baseline trends after the intervention was launched. The outcomes of the two groups of schools 

are not compared directly; instead, the analysis compares the amount by which the two groups

deviated from their separate baseline trends. If the CIS model is more effective than other 

programs or reforms available to schools (those used by the comparison schools), then the CIS 

 7

For a discussion and history of CITS designs, see William R. Shadish, Thomas D. Cook, and Donald T. 

Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference (Boston: 

Houghton Mifflin, 2002). For a discussion of these designs in the context of education research, see Howard S. 

Bloom, “Using ‘Short’ Interrupted Time-Series Analysis to Measure the Impacts of Whole-School Reforms, 

with Applications to a Study of Accelerated Schools,” Evaluation Review 27, 3 (2003): 3-49. 
4

schools should experience improvements relative to their baseline trend that exceed the improvements found in comparison schools.

The comparison schools play an important role in this study design. Their trends over 

time reveal what happened to similar schools that did not implement the CIS model and thus are 

intended to show how much CIS schools’ outcomes would have improved had they not implemented it. For example, had they not implemented the CIS model, CIS schools’ outcomes may 

have improved because they chose to implement some other school reform model (instead of 

CIS), or they may have improved due to a district-wide or statewide policy change. The CITS 

design makes it possible to identify the CIS model’s effect over and above the effect of these 

alternative reforms and system-wide policy changes (that is, the model’s net effect).

This study examined 53 schools in Texas and North Carolina (14 high schools, 15 middle schools, and 24 elementary schools) that started implementing the CIS model from 2005 to 

2008. The study sample also includes 78 comparison schools (18 high schools, 24 middle 

schools, and 36 elementary schools). The comparison schools were chosen from non-CIS 

schools in counties in Texas and North Carolina where there is a CIS presence, ensuring that 

they are located in the kinds of districts where CIS typically operates. Matching methods were 

used to select comparison schools whose baseline characteristics and trends in school outcome

measures were as similar as possible to those of the CIS schools.

Overall, the CIS schools and the comparison schools have similar baseline characteristics and outcome measure values, with one exception: The comparison high schools had higher 

graduation rates than the CIS schools (by about 9 percentage points) and lower dropout rates 

(by about 1.5 percentage points). These differences are small enough to meet commonly used 

criteria for baseline equivalence, such as the one used by the What Works Clearinghouse.8

Substantively, however, these differences are large enough that the CIS schools may have had 

more incentive to turn themselves around. For example, had they not implemented the CIS 

model, the CIS schools may have chosen to implement different — but still intensive — wholeschool interventions to improve their low graduation rates. In contrast, the comparison schools 

may have felt less need or pressure than the CIS schools to initiate a turnaround, and accordingly, they may have adopted less intensive strategies to improve their graduation rates. In that 

case, the comparison schools’ deviations from trend would not provide the right information 

about what would have happened to the CIS schools had they not implemented the model. Any 

improvements in the comparison schools’ graduation rates would understate the amount CIS 

 8

The What Works Clearinghouse criterion is that differences in baseline characteristics should not exceed 

0.25 standard deviations. See What Works Clearinghouse, What Works Clearinghouse Procedures and 

Standards Handbook, Version 3.0 (Washington, DC: U.S. Department of Education, Institute of Education 

Sciences, 2014).
5

schools’ own rates would have improved without the CIS model; by extension, the findings 

from this study would overstate the true effect of the CIS model. Various sensitivity tests were 

used to explore this possibility as well as other factors that could affect the credibility of the 

comparison schools as a reference point.

Data for the study were obtained from existing, publicly available school-level historical databases.9 Using these data, this study is able to examine whether the CIS model was 

successful at meeting its main objectives: increasing on-time graduation rates and reducing high 

school dropout rates. The study can also examine whether the CIS model had a positive effect 

on attendance rates (the main outcome of interest in elementary schools) and performance on 

state tests. However, the study is not able to evaluate whether the CIS model improved students’ 

behavioral outcomes, which is one of the model’s main goals in middle schools. Another datarelated limitation is that information is not available on what kinds of services and support are 

offered to students in the comparison schools, and how these services differ from those included 

in the CIS model. This limitation makes it more challenging to interpret the study’s findings, 

because it is unclear what initiatives and services the CIS model is being compared with.

In addition to evaluating the effect of CIS’ whole-school model, MDRC is also evaluating the effect of one component of the CIS model — Level 2 services — using a student-level 

random assignment research design. This study is being conducted in 28 secondary schools in 

North Carolina and Texas. Eligible at-risk students were randomly assigned to receive Level 2 

case management and services or to continue with business as usual, with access to whatever 

other forms of support were available to them.

The first report from the random assignment study described the effect of Level 2 services on students’ behavioral and academic outcomes after one year.10 It found that Level 2 

services had a positive and statistically significant impact on students’ reports of having caring, 

supportive relationships with adults outside of home and school; the quality of their peer 

relationships; and their belief that education has positive value for their lives. But MDRC found 

no evidence that Level 2 services had improved students’ attendance, course performance, or 

behavior. However, it is too early to make any definitive conclusions, because one year of case 

management may not be sufficient to improve the outcomes of these students, all of whom face 

serious academic and personal challenges. Accordingly, the next (and final) report will examine 

the effect of Level 2 case management after two years.

 9

These databases include the Common Core of Data and data sets maintained by the Texas Education 

Agency, the North Carolina Research and Data Center at Duke University, and the North Carolina State 

Department of Education. 10William Corrin, Leigh Parise, Oscar Cerna, Zeest Haider, and Marie-Andrée Somers, Case Management 

for Students at Risk of Dropping Out: Implementation and Interim Impact Findings from the Communities In 

Schools Evaluation (New York: MDRC, 2015).
6 

Did the CIS Model Improve School Outcomes?

MDRC’s study of the CIS model examines the following two research questions:

• After launching the CIS model, did the CIS schools have better outcomes 

than predicted by their baseline trend? This question is answered by looking 

at the deviations from baseline trend for the CIS schools in the study.

• Did the CIS schools’ outcomes improve more than they would have in the 

absence of the CIS model? This question is answered by looking at the difference between the deviations from baseline trend for the CIS schools and 

the comparison schools. This difference represents the estimated net effect of 

the CIS model.

High School Graduation and Dropout Rates

• After three years of implementing the CIS model, on-time graduation 

rates and dropout rates improved by statistically significant amounts in 

the CIS high schools, relative to what would have been expected given 

their baseline trends.

• It is not possible to determine the extent to which these improvements 

can be attributed to the CIS model. Graduation and dropout rates improved more in the CIS schools than in the comparison high schools, but 

this pattern of results depends on the choice of comparison schools.

Figure ES.1 plots the trend in graduation rates for the CIS high schools (solid black 

line) and the comparison schools (solid gray line), during the school years before the CIS model 

was launched, and also shows graduation rates after the model was launched. Before the CIS 

model was implemented, graduation rates had been steadily declining for both groups of 

schools. After the model was launched, graduation rates in the CIS high schools were higher 

than expected relative to these schools’ baseline trend. Graduation rates were also higher than 

predicted in the comparison schools, perhaps because they chose to use a strategy other than the 

CIS model to turn themselves around. However, graduation rates improved more in the CIS 

schools than in the comparison schools, and therefore the estimated net effect of the CIS model 

is positive.

Table ES.1 summarizes the findings. As shown in the first panel, the estimated effect of 

the CIS model on graduation rates is consistently positive during the first three years of 
7

Figure ES.1

Baseline Trends and Deviations from Trend in High School Graduation Rates

40

50

60

70

80

90

100

-3 -2 -1 0 1 2 3

Relative year (CIS introduced at Year 1)

Comparison schools' outcomes CIS schools' outcomes

Comparison

+8.08

CIS 

+15.58

CIS predicted trend

Comparison 

predicted trend

Estimated effect 

in Year 3 = 7.50

(p-value = 0.088)

Graduation rate (%)

CIS baseline trend

Comparison 

baseline trend

SOURCES: MDRC calculations based on school-level data on student outcomes from the Texas Education Agency.
8 

Table ES.1

Estimated Effects on Selected Outcomes

Deviation from Baseline Trend P-Value for

Outcome and Follow-Up Year

CIS Comparison

Schools Schools

Estimated

Effect

Lower Upper

90% CI 90% CI

Estimated

Effect

High schools

Graduation rate (%)

Year 1 2.71 † -1.74 4.44 -0.48 9.37 0.137

Year 2 6.19 ††† 0.69 5.51 -0.77 11.78 0.148

Year 3 15.58 ††† 8.08 †† 7.50 * 0.27 14.73 0.088

Number of schools 8 11

Dropout rate (%)

Year 1 -0.75 0.30 -1.06 -2.40 0.29 0.197

Year 2 -1.85 †† -0.65 -1.19 -2.75 0.36 0.207

Year 3 -3.8 ††† -2.3 ††† -1.5 -3.4 0.5 0.211

Number of schools 14 18

Middle schools

ELA state test scores (z-scores)a

Year 1 0.01 0.05 † -0.05 -0.11 0.02 0.252

Year 2 0.02 0.10 ††† -0.08 -0.16 0.00 0.110

Year 3 0.00 0.11 ††† -0.11 * -0.20 -0.01 0.061

Number of schools 8 15

Elementary schools

Attendance rate (%)

Year 1 0.30 ††† 0.14 † 0.16 -0.07 0.39 0.257

Year 2 0.44 ††† 0.17 0.27 -0.01 0.54 0.111

Year 3 0.61 ††† 0.20 † 0.41 ** 0.10 0.72 0.030

Number of schools 24 36

(continued)
9 

Table ES.1 (continued)

SOURCES: MDRC calculations based on school-level data on student outcomes from state websites (the Texas 

Education Agency and the North Carolina State Department of Education) and from the North Carolina Education 

Research Data Center at Duke University.

NOTES: The values in the "CIS Schools" and "Comparison Schools" columns are the estimated deviations from 

baseline trend for each group of schools. The values in the "Estimated Effect" column are the differences between 

CIS schools and comparison schools with respect to their deviations from baseline trend. The values in the "Lower 

90% CI" and "Upper 90% CI" columns are the 90 percent confidence intervals for the estimated effects. Rounding 

may cause slight discrepancies in calculating sums and differences. 

A two-tailed test was applied to estimated deviations and estimated differences between CIS schools and 

comparison schools. The statistical significance of estimated deviations is indicated as: ††† = 1 percent; †† = 5 

percent; † = 10 percent. The statistical significance of estimated effects is indicated as: *** = 1 percent; ** = 5 

percent; * = 10 percent. aState test scores were converted to z-scores based on the estimated student-level mean and standard deviation 

in test scores for the state.

implementation, and it becomes larger over time, as one would expect.11 By the third year, the 

estimated effect is 7.5 percentage points, which represents an 11 percent increase in graduation 

rates relative to the last baseline year. An effect of this size means that in the third year of 

implementation, 55 additional students in each study school graduated from high school on 

time. Though the CIS model’s estimated effect on dropout rates is not statistically significant, 

those results follow a similar pattern (as shown in the second panel of Table ES.1).

As explained earlier, a central assumption of the CITS design is that the comparison 

schools’ deviations from their baseline trend represents what would have happened to the CIS 

schools in the absence of the intervention. As shown in Figure ES.1 and discussed earlier, the 

comparison schools had substantially higher baseline graduation rates (and lower dropout rates) 

than the CIS schools, so they may have had less incentive to turn their graduation rates around. 

This difference places an additional burden on the study to demonstrate that the comparison 

schools can still represent what would have happened to CIS schools in the absence of the CIS 

model. Various sensitivity tests were conducted to examine the credibility of this assumption. 

Some of these sensitivity tests suggest that the comparison schools do provide a good point of 

reference for the CIS schools, but others suggest that they may not, and that the net effects 

shown in Table ES.1 may therefore be overstated. 

Given these results, it is not possible to determine whether the CIS model improved 

graduation and dropout rates more than they might have improved otherwise, or if it did, by 

 11Graduation rates are only available for 8 of the 14 CIS high schools, because data on this outcome are 

only available for high schools in Texas.
10

how much. However, the study does show that the average graduation and dropout rates for the 

CIS high schools improved relative what would have been expected given their prior trends. In 

addition, although it is not possible to determine whether the CIS model was more effective 

than the strategies used by the comparison schools, the study’s findings suggest that the CIS 

model may be at least as effective.

Elementary School Attendance Rates

• After three years, the average attendance rate increased by a statistically 

significant amount in the CIS elementary schools, relative to what would 

have been expected given their baseline trend.

• The improvement in attendance rates for these CIS schools was larger 

than the improvement in comparison schools by a statistically significant 

amount. The CIS model appears to have improved these schools’ attendance rates more than they would have improved otherwise. 

Attendance is the main student outcome CIS aims to affect in elementary schools. After 

the CIS model was launched, attendance rates in the CIS elementary schools were higher than 

predicted by these schools’ baseline trends by a statistically significant amount (as shown in the 

last panel of Table ES.1). Attendance rates in the comparison schools were also higher than 

predicted, perhaps because these schools were using other strategies to improve their attendance 

rates. However, in all three follow-up years, the CIS elementary schools deviated from their 

baseline trend by a greater amount than in the comparison schools, and that difference becomes

statistically significant in the third year of implementation. This pattern of results holds across 

all of the sensitivity tests that were conducted to examine the credibility of the comparison 

schools as a reference point. Therefore, the CIS model may have improved attendance rates in 

the study’s elementary schools more than the strategies being used by the comparison schools.

On its face, the CIS model’s estimated effect on attendance rates may seem small: By 

the third year, the estimated effect is 0.4 percentage points, which represents an extra 0.7 days 

of school.12 However, it is important to note that attendance rates were already high before the 

CIS model was launched (96.1 percent), so an effect of 0.4 percentage points gets schools 10 

percent closer to perfect attendance.13 In addition, average daily attendance rates can mask high 

rates of chronic absenteeism: 11 percent of elementary school students are chronically absent 

 12This number assumes a school year of 180 days. 13The gap between baseline attendance and perfect attendance is 3.9 percentage points (100 - 96.1). An 

effect of 0.4 percentage points is 10 percent of that gap (0.4 ÷ 3.9).
11

(that is, they are absent 15 or more days during the school year).14 A small effect on overall 

attendance rates may represent a larger effect on chronic absenteeism.

Middle and High School Attendance Rates

In CIS middle schools, attendance rates did not improve by a statistically significant 

amount. In CIS high schools, attendance rates did improve by a statistically significant amount, 

but not by a greater amount than they would have improved otherwise. These results are 

consistent across the sensitivity tests that were conducted to examine the credibility of the 

comparison schools.

Middle School State Test Scores

• After three years of implementation, state test scores did not improve in 

the CIS middle schools, relative to what would have been expected given 

their baseline trends.

• During the same period, state test scores did improve by a statistically 

significant amount in the comparison schools. CIS middle schools’ test 

scores appear to be lower than they would have been otherwise. 

After the CIS model was launched, state test scores in the CIS middle schools were not 

better than predicted by their baseline trends. In contrast, the comparison schools performed 

better on state tests than predicted, perhaps because they implemented other reforms or initiatives to improve their students’ performance. As a result, the CIS model’s estimated effect on 

middle school students’ ELA state test scores is consistently negative and it becomes more 

negative over time (as shown in the third panel of table ES.1).15 By the third year, the estimated 

effect size is -0.11 standard deviations, which translates into about 14 weeks of learning.16

These results hold across all of the sensitivity analyses that were conducted to examine the 

credibility of the comparison schools as a reference point. This result suggests that the CIS 

model may have been less successful at improving state test scores than the strategies used by 

 14Office of Civil Rights, 2013-2014 Civil Rights Data Collection, a First Look: Key Data Highlights on 

Equity and Opportunity Gaps in our Nation’s Public Schools (Washington, DC: U.S. Department of 

Education, Office of Civil Rights, 2016). 15The number of CIS and comparison schools for this analysis is less than the total number of middle 

schools in the study, because time-series data on state test scores are not available for all schools. 16The conversion from effect size to weeks of learning is based on data in Carolyn J. Hill, Howard S. 

Bloom, Alison Rebeck Black, and Mark W. Lipsey, “Empirical Benchmarks for Interpreting Effect Sizes in 

Research,” Child Development Perspectives 2, 3 (2008):172-177. Middle school students make gains of about 

0.008 standard deviations per week in reading. Therefore, an effect size of -0.11 standard deviations is 

equivalent to 14 weeks of learning (0.11 ÷ 0.008).
12

the comparison schools. It is important to note, however, that the CIS model does not attempt to 

improve state test scores or the quality of instruction provided during regular school hours.

Elementary School and High School State Test Scores

In CIS elementary schools, ELA state test scores improved by a statistically significant 

amount, but not by more than they would have improved otherwise. These results are consistent 

across the sensitivity tests that were conducted to examine the credibility of the comparison 

schools. In CIS high schools, ELA state test scores also improved by a statistically significant 

amount. However, it is not possible to determine the extent to which these improvements can be 

attributed to the CIS model due to the limitations in identifying a group of credible comparison 

schools described above. 

Discussion

Because this study is based on a small number of purposefully selected schools that started 

implementing the CIS model a decade ago, its findings may not represent the effect of the CIS 

model nationally as it exists today. Therefore, the results from this study should be considered 

alongside the results of other evaluations of the CIS model, two of which are worth noting. The 

first is a national quasi-experimental study of the CIS model conducted by ICF International.17

The second study is an (as yet) unpublished school-level random assignment evaluation of the 

CIS model in Chicago K-8 schools.18

All three studies of the CIS model conducted thus far have found positive effects on attendance rates for younger students. With respect to state test scores, the effect of the CIS model 

appears to depend on the local context: This study finds negative effects in middle schools, 

whereas the Chicago study finds positive effects. This difference suggests that in some settings, 

the CIS model can have positive effects on students’ test scores and be more effective than the 

other strategies available.

The findings in high schools are promising but more difficult to interpret. The ICF 

study finds an improvement of 1.7 percentage points in ninth-grade students’ probability of 

graduating from high school, but this effect is not statistically significant. In the present study, 

graduation rates improved for the CIS schools after they launched the model, but it is unclear to 

 17ICF International, Communities In Schools National Evaluation Volume 1: School-Level Report. Results 

from the Quasi-Experimental Study, Natural Variation Study, and Typology Study (Fairfax, VA: ICF 

International, 2008). 18David N. Figlio, “Experimental Evidence of the Effects of the Communities In Schools of Chicago 

Partnership Program on Student Achievement,” Northwestern University Working Paper (Evanston, IL: 

Northwestern University, 2015). 
13

what extent graduation rates would have improved had these schools not implemented the CIS 

model.

Therefore, it would be a useful next step for CIS to undertake a rigorous and large-scale 

study of its model in high schools in particular, based on a randomized experiment and supplemented by a cost study. A school-level random assignment research design would provide the 

best evidence of the CIS model’s effect on student outcomes relative to that of other programs 

and strategies. By collecting cost data on the implementation of the CIS model as well as cost 

data on the strategies and interventions used by the control schools, one could also determine 

the CIS model’s relative cost-effectiveness, which would ultimately provide the most useful 

information for school districts.

15 

Earlier MDRC Publications on Communities In Schools

Case Management for Students at Risk of Dropping Out

Implementation and Interim Impact Findings from the Communities In Schools Evaluation

2015. William Corrin, Leigh Parise, Oscar Cerna, Zeest Haider, and Marie-Andrée Somers. 

About MDRC

MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 

to learning what works to improve the well-being of low-income people. Through its research 

and the active communication of its findings, MDRC seeks to enhance the effectiveness of social and education policies and programs.

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 

for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 

Its projects are a mix of demonstrations (field tests of promising new program approaches) and 

evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 

combination of research and organizational experience to their work, providing expertise on the 

latest in qualitative and quantitative methods and on program design, development, implementation, and management. MDRC seeks to learn not just whether a program is effective but also 

how and why the program’s effects occur. In addition, it tries to place each project’s findings in 

the broader context of related research — in order to build knowledge about what works across 

the social and education policy fields. MDRC’s findings, lessons, and best practices are proactively shared with a broad audience in the policy and practitioner community as well as with the 

general public and the media.

Over the years, MDRC has brought its unique approach to an ever-growing range of policy areas and target populations. Once known primarily for evaluations of state welfare-to-work programs, today MDRC is also studying public school reforms, employment programs for exoffenders and people with disabilities, and programs to help low-income students succeed in 

college. MDRC’s projects are organized into five areas:

• Promoting Family Well-Being and Children’s Development

• Improving Public Education

• Raising Academic Achievement and Persistence in College

• Supporting Low-Wage Workers and Communities

• Overcoming Barriers to Employment

Working in almost every state, all of the nation’s largest cities, and Canada and the United 

Kingdom, MDRC conducts its projects in partnership with national, state, and local governments, public school systems, community organizations, and numerous private philanthropies. 
New York

16 East 34th Street

New York, NY 10016

Tel: 212 532 3200

California

475 14th Street

Oakland, CA 94612

Tel: 510 663 6372

www.mdrc.org