GOING AWAY 

TO SCHOOL

An Evaluation of SEED DC

Rebecca Unterman

Dan Bloom

D. Crystal Byndloss

Emily Terwelp

 June 2016

Going Away to School

An Evaluation of SEED DC

Rebecca Unterman

Dan Bloom

D. Crystal Byndloss

Emily Terwelp

June 2016 
This material is based upon work supported by the Social Innovation Fund (SIF), a program of the 

Corporation for National and Community Service (CNCS). The Edna McConnell Clark Foundation

(EMCF) SIF initiative includes support from CNCS and 15 private co-investors: EMCF, The Annie 

E. Casey Foundation, The Duke Endowment, The William and Flora Hewlett Foundation, The JPB 

Foundation, George Kaiser Family Foundation, The Kresge Foundation, Open Society Foundations, 

The Penzance Foundation, The Samberg Family Foundation, The Charles and Lynn Schusterman 

Family Foundation, The Starr Foundation, Tipping Point Community, The Wallace Foundation, and 

the Weingart Foundation.

Dissemination of MDRC publications is supported by the following funders that help finance 

MDRC’s public policy outreach and expanding efforts to communicate the results and implications of our work to policymakers, practitioners, and others: The Annie E. Casey Foundation, 

Charles and Lynn Schusterman Family Foundation, EMCF, Ford Foundation, The George Gund 

Foundation, Daniel and Corinne Goldman, The Harry and Jeanette Weinberg Foundation, Inc., 

The JBP Foundation, The Joyce Foundation, The Kresge Foundation, Laura and John Arnold 

Foundation, Sandler Foundation, and The Starr Foundation.

 

In addition, earnings from the MDRC Endowment help sustain our dissemination efforts. Contributors to the MDRC Endowment include Alcoa Foundation, The Ambrose Monell Foundation, 

Anheuser-Busch Foundation, Bristol-Myers Squibb Foundation, Charles Stewart Mott Foundation, Ford Foundation, The George Gund Foundation, The Grable Foundation, The Lizabeth and 

Frank Newman Charitable Foundation, The New York Times Company Foundation, Jan Nicholson, Paul H. O’Neill Charitable Foundation, John S. Reed, Sandler Foundation, and The Stupski 

Family Fund, as well as other individual contributors.

The findings and conclusions in this report do not necessarily represent the official positions or 

policies of the funders. 

For information about MDRC and copies of our publications, see our website: www.mdrc.org. 

Copyright © 2016 by MDRC®. All rights reserved.
iii

Overview

The prospect of a well-paying job for a worker without a college education has significantly dimmed

in the past three decades, in the wake of sweeping changes in the U.S. economy and labor market. 

The effects of these changes are particularly devastating for young people from disadvantaged urban

communities. In response to this issue, the SEED Foundation, founded in 1997 by former management consultants Rajiv Vinnakota and Eric Adler, opened the first public, urban, college-preparatory 

boarding school in the country. The primary mission of the SEED School of Washington, DC 

(SEED DC), is to provide an intensive education program that prepares students from low-income 

and underserved communities for college enrollment and success.

The SEED school, located in a residential section of southeast Washington, serves approximately 

320 sixth- through twelfth-graders. Students attend school on campus five days a week, arriving on 

Sunday evening and going home on Friday afternoon. The SEED model posits that an alternative 

urban academic environment that puts a high priority on academic excellence and personal development will allow students to succeed through high school and in college. An important facet of the 

SEED model is to surround students with a cadre of adults to support them in preparing for college

success — including their teachers, school administrators, and the residence hall staff.

Using the random assignment inherent in the school admissions lottery, this report presents results 

from a six-year evaluation of SEED DC, including both an implementation study — to understand

how the school operates in practice — and an examination of the impacts of winning admission to 

SEED DC on a broad range of student outcomes.

Key Findings

•

•

•

•

SEED DC creates a highly supportive environment for its students, with a network of caring 

adults. The school provides students with a wide array of services, ranging from academic support to emotional support and relationship-building activities. 

The school produced significant, positive impacts on students’ standardized test scores and proficiency levels — particularly in math — in comparison with outcomes among students who did 

not win admission to SEED. 

For students in the earliest cohorts, who can be followed through high school, SEED DC did not 

increase the proportion who graduated from high school in four years. 

Although SEED DC showed a couple of positive behavioral effects, it did not show an impact 

on the key nonacademic outcomes, such as teen pregnancy or interaction with the criminal justice system, that could justify its higher cost. 

It is important to note that SEED DC operates in a district with many innovative alternatives and, 

owing to local rules, is unable to target the most underserved students. Thus the evaluation cannot 

speak to the question of whether SEED’s unusual boarding school model could produce larger effects in different environments, serving students who face more serious obstacles to success.

v 

Contents

Overview iii

List of Exhibits vii

Preface ix

Acknowledgments xi 

Executive Summary ES-1

Chapter

1 Introduction 1 

About SEED 2 

The SEED Evaluation 3 

2 SEED DC in Operation 7

General Structure of SEED DC 7 

Academics 8 

Student Life 14 

Student Support Services 18 

College Counseling 19 

A Network of Supportive Adults and Peers 20 

3 The Academic and Behavioral Impacts of SEED 23

Random Assignment and the Study Sample 23 

The Three Analytic Samples 25 

Short-Term Effects of SEED 28 

Longer-Term Effects of SEED 34 

4 Conclusion 43

Some Factors to Consider 43 

Summary 45 

Appendixes

A School Program Improvements, As Described by SEED 47 

B Implementation Research Data Collection and Analysis 53

C The SEED DC Lottery Process 91

D The Analytic Approach 97 

E Estimated Effects of Enrolling in SEED: A “Treatment-on-the-Treated” 

 Analysis 105

F The SEED Student Survey 109

References 119

vii

List of Exhibits

Table

2.1 Middle School Courses at SEED DC 10

2.2 High School Courses at SEED DC 10

3.1 Baseline Characteristics of SEED Lottery Participants: Full Sample 24

3.2 Follow-Up Periods by Cohort 29

3.3 Estimated Effects of Being Offered the Opportunity to Attend SEED, 

Seventh-Grade Entrant Sample 31 

3.4 Estimated Effects of Being Offered the Opportunity to Attend SEED, 

Sixth-Grade Entrant Sample 33 

3.5 Estimated Differences in Student Experience, Cohorts 1 and 2 36

3.6 Estimated Effects of Being Offered the Opportunity to Attend SEED on 

Academic Achievement, Cohorts 1 and 2 38 

3.7 Estimated Effects of Being Offered the Opportunity to Attend SEED on 

Behavior Outcomes, Cohorts 1 and 2 40 

D.1 Baseline Characteristics of SEED Lottery Participants, Cohorts 1 and 2 101

D.2 Baseline Characteristics of SEED Lottery Participants in Cohorts 1 and 2

for Whom Graduation Data Are Available 102 

E.1 Program Group Enrollment in SEED by Cohort and Year 106

E.2 Estimated Effects of Enrolling in SEED, Cohorts 1 and 2 107

F.1 Cronbach Alpha Values for Survey Constructs 112

 

Figure

1.1 SEED DC Logic Model 4

2.1 SEED DC Organizational Chart 9

2.2 Sample Student Life Schedule: Sixth Grade 15

C.1 Flow of Study Sample Through SEED Application Process 95

Box

2.1 Sample Character Education and HALLS Lessons 17

3.1 Definitions of Study Outcomes 27

ix

Preface

Of the 24 million adolescents in the United States today, 41 percent live in low-income families 

and 19 percent, or 4.7 million, live in families with incomes falling below the federal poverty 

threshold. These young people are more likely to move multiple times, to face food insecurity, 

and to live in neighborhoods characterized by crime and a lack of resources. While educational 

attainment is often heralded as a pathway out of poverty, many of these young people live in 

neighborhoods with failing schools and come from families without a history of high school 

completion or college enrollment. 

Education reformers working in low-income areas have tried for decades to improve 

students’ school options and break the cycle of poverty. While there have been some successes, 

for some students, the compounding effects of multiple disadvantages are often too disruptive. 

In response the founders of SEED boarding schools sought to create a holistic intervention that 

provides students with a constant, safe place to live; regular healthy meals; and the kinds of resources — such as a library and a peaceful outdoor area — found in middle- and high-income 

communities. Within this context SEED strives to provide its students with a rigorous, collegeready academic program and supplements it with a youth development-focused life skills curriculum after school hours.

The SEED Foundation created its first school, SEED DC, in 1997. It currently serves 

320 students in grades 6 through 12; students live on campus Monday through Friday and go 

home on the weekends. The evaluation described in this report, funded under the Social Innovation Fund, takes advantage of lotteries within the SEED DC admissions process to assess 

SEED’s effects on students throughout middle and high school. The study focuses mainly on 

students’ academic outcomes but also uses a survey to measure whether SEED students are 

more likely to be engaged in school and have positive plans for the future. 

Notably, SEED improved students’ scores on standardized tests, but to date a partial 

sample of students who won a lottery to attend SEED were no more likely to graduate in four 

years than students who lost the lottery and attended other schools. There is no evidence that 

SEED students were less likely to engage in risky behaviors. Finally, many students left SEED 

after middle school. Still, it will be important to understand whether the gains in academic performance translate into additional high school graduates and higher rates of college enrollment 

in later years. The results raise the question of whether the intensive SEED approach would be 

more effective if aimed more narrowly at students facing very serious obstacles to success. 

Gordon L. Berlin

President, MDRC

xi

Acknowledgments

This report would not have been possible without the support of many individuals and organizations. In particular, the SEED Evaluation and the production of this report were funded by the 

Corporation for National and Community Service and the Edna McConnell Clark Foundation, 

in partnership with the Social Innovation Fund.

We are tremendously grateful to several individuals on the staff at the SEED Foundation and the SEED School of Washington, DC. We could not have produced this report without 

the cooperation of SEED Foundation founders Rajiv Vinnakota and Eric Adler, as well as 

members of their staff, including Shane Mulhern, Mary Lease, Corin Collier, Lesley Poole, and 

Vincena Allen. We are equally grateful for the dedicated collaboration of so many school administrators and educators at the SEED DC school, including Carmen Johnson (Managing Director), Charles Adams (Head of School), Mecha Inman (Director of Admissions), Erika 

Asikoye (Director of Student Support Services), Jon Tucker (Director of Student Life), Kara 

Stacks (Principal), Kerry Richardson (Director of College Counseling), Matthew Carothers (Director of Middle School), Stacey Pearl (Special Education Coordinator), and Melissa Freeman 

(College Success Manager).

We could not have learned about the implementation of the SEED DC school model 

without the help of many other staff members at SEED DC, including middle school, ninthgrade, and high school teachers, resident advisers, academic intervention and life skills counselors, health counselors, and all other members of the SEED DC community, who are deeply 

committed to creating a caring and supportive space for the young people there.

At MDRC, Howard Bloom, William Corrin, Kate Gualtieri, and Janet Quint provided 

thoughtful comments on several drafts of this report. Karla Mendez and Danielle Craig coordinated the production of the report. Jennie Kaufman edited the report, and Stephanie Cowell prepared it for publication.

We are especially grateful to the SEED DC students, who graciously and enthusiastically welcomed us into their school. They participated in surveys and focus group discussions and 

allowed us to partake in and observe their daily lives. Many of the students were excited about 

our research and the opportunity to develop more knowledge about urban public boarding 

schools and the services that they can provide for young people.

The Authors

ES-1 

Executive Summary

Sweeping changes in the U.S. economy and labor market over the past three decades have dramatically reduced the availability of well-paying jobs for workers without postsecondary education. Yet one in five high school freshmen nationwide do not graduate in four years, and many 

who do complete school are not ready to perform college-level work.1These patterns are particularly pronounced in urban areas, and among students from low-income and underserved families. 

In recent years, charter schools, which receive public funding but operate independently 

of local school districts, have increased in number and popularity, in part due to their flexible 

governance structure, which allows them to implement innovative new education models. One 

such model belongs to the SEED School of Washington, DC (SEED DC), the nation’s first urban, public, college-prep boarding school. The school provides students with an intensive, fully

integrated academic and boarding school program, including scheduled study time, constant 

access to positive role models, and life skills training. SEED’s model is based on the assumption that, for certain disadvantaged students who face overwhelming barriers to success at home 

and in the community, piecemeal reform efforts will not be sufficient.

This report presents the findings from a rigorous evaluation of SEED DC, which was

supported by the Social Innovation Fund (SIF), a program of the Corporation for National and 

Community Service (CNCS). The Edna McConnell Clark Foundation (EMCF) is leading a SIF 

project that includes support from CNCS and 15 private co-investors. EMCF’s SIF project included an investment in the SEED Foundation, the national nonprofit organization that oversees 

SEED DC and the two other SEED schools (in Baltimore and Miami) currently in operation.

SEED has been the subject of previous studies, including an impact study that found 

that SEED led to significant gains in standardized test scores in seventh and eighth grade.

2 The 

authors of that study questioned whether these increases were large enough to justify the high 

cost of the boarding school model. If SEED affects nonacademic outcomes such as teen pregnancy or crime involvement, which trigger very high social costs, the program could turn out to 

be a worthwhile investment of public funds.

The study uses SEED DC’s annual admissions lottery to identify two comparable 

groups of students: those who applied to SEED and were selected, at random, to be offered a 

 1

Grace Kena, Susan Aud, Frank Johnson, Xiaolei Wang, Jijun Zhang, Amy Rathbun, Sidney WilkinsonFlicker, and Paul Kristapovich, The Condition of Education 2014, NCES 2014-083 (Washington, DC: National 

Center for Education Statistics, 2014). 2

Vilsa E. Curto and Roland G. Fryer Jr., “Estimating the Returns to Urban Boarding Schools: Evidence 

from SEED,” National Bureau of Economic Research Working Paper No. 16746 (2011). 
ES-2 

slot in the school, and those who applied to SEED and were not offered a slot. By following 

those two groups of students over time, the study can estimate the impacts of SEED DC on 

standardized test scores, high school graduation rates, and other nonacademic outcomes. This 

study focuses only on SEED DC, which is by far the most mature of the existing SEED schools.

The evaluation examined both the implementation of SEED DC and its impact on student outcomes. The study’s two overarching research questions are as follows: 

•

•

How is SEED DC structured and how does it operate in practice? Using 

interviews with staff members and students, observation, and other methods, 

the evaluation team set out to understand the on-the-ground reality of SEED 

DC and how the school is experienced by students. This information not only 

helps in interpreting the impact findings, but also may help SEED improve 

service delivery over the long run.

What is the effect of being offered an opportunity to attend SEED DC on 

student outcomes? Academic outcomes include standardized test scores and 

high school graduation. Nonacademic outcomes include both attitudes (for 

example, college aspirations) and behaviors (positive ones like homework

completion and risky ones like alcohol and drug use).

The evaluation focuses on 766 students who “won” or “lost” the SEED lottery as fifth- or sixthgraders between 2006 and 2011. The study followed those students through the 2013-2014 academic year, which means that only a small number of them could have graduated from high 

school or enrolled in college during the study period. Thus, while improving students’ performance in college is a key goal of SEED’s, it is too soon to assess whether SEED improves students’ postsecondary outcomes.

SEED DC in Operation

Located in a residential section of southeast Washington, SEED DC serves approximately 340 

sixth- through twelfth-grade students. Students in the study sample were primarily AfricanAmerican and were economically and academically disadvantaged. Of those who won the 

SEED lottery (the SEED group), four out of five qualified for free or reduced-price lunch. In the 

year they applied to SEED, 14 percent of the SEED group students qualified for special education services and just under 50 percent scored at or above proficient on the district-wide reading 

and math exams. 

Students attend school on campus five days a week, arriving on Sunday evening and returning home on Friday afternoon. During the days that students reside on campus, they have

access to quiet places to study and sleep, nutritious meals, academic resources, and spaces to 
ES-3 

engage in extracurricular activities. By design, students are also surrounded by a cadre of caring 

adults who support them in preparing for success in college. In both academic and residential 

life programs, the school uses a grade-based cohort system, meaning that distinct goals, expectations, and approaches are set for middle school, ninth-grade, and high school students.

Academic Curriculum

The school philosophy is that all students have the same inherent potential for academic 

success, and thus all students are expected to excel at SEED. The academic department strongly 

believes in using data to guide and inform instruction, so all students take interim assessments in 

English and math four times per year. After each assessment, academic and Student Life staff

members meet in teams to discuss the results and identify priority issues. Teachers then use the 

assessment results to develop lesson plans to “reteach” skills that students have been unable to 

master.

SEED students are expected to attend college following high school graduation. The 

College Counseling department is responsible for providing support to students in the college 

search, application, and selection process. From sixth grade on, students are encouraged to visit 

the College Café, a colorful and inviting space stocked with information and decorated with 

memorabilia from many of the nation’s colleges. In middle school, students engage in discussions about the value of enrolling in college, visit a college campus, and participate in activities 

to strengthen their academic habits. Starting in ninth grade, students practice taking college entrance exams (the PSAT) so that they become familiar with the test and can improve their 

scores. High school students have access to test preparation materials during after-school hours. 

Eleventh-graders receive college advising focused on finding the “right-fit” college — the one

that is the best academic match and also meets their financial, social, and personal needs — and

students in the twelfth grade are actively engaged in the college search, application, and choice 

process.

A unique feature of the SEED student experience is the support that SEED students and 

graduates receive from the College Transition and Success (CTS) team, a unit within the SEED 

Foundation. Working in collaboration with the College Counseling Department, the CTS team 

holds a series of college transition workshops for seniors and their parents (including a financial 

literacy workshop) and helps students finish required college enrollment paperwork. The CTS 

Team also monitors and supports SEED graduates as they make the transition to college and 

maintains contact with SEED graduates who are enrolled in college. 

Student Life Curriculum

Perhaps the most distinctive aspect of SEED’s learning environment is the time that 

adults spend with students after school and through the evening. The Student Life Department is 
ES-4 

responsible for developing and coordinating residential life programming and managing students’ time outside of the traditional academic day. Students are organized into houses (or 

groups) within the dorms, each of which is led by a resident adviser. Each house is named after 

a college or university and decorated with its pennants, pillows, and the like. 

The Student Life Department aims to develop students’ behavioral, social, and life 

skills while reinforcing what they are learning in the classroom. Middle school programming is 

intended to develop and refine social skills that are connected to meeting behavior expectations 

and routines, such as following instructions the first time they are given, adhering to the school 

dress code, and learning how to disagree appropriately. Ninth-grade Student Life activities aim 

to develop and reinforce the skills and habits necessary for success in high school, such as planning ahead, using anger control strategies, and building strong self-esteem. High school programming focuses on the transition to college. To accomplish these goals, the majority of Student Life time is structured, especially for the middle school grades.

The SEED-created Habits for Achieving Life-Long Success (HALLS) program teaches 

students social and basic life skills, such as decision making and communication strategies, and 

the importance of taking responsibility for oneself and others. HALLS activities focus on a variety of topics, such as bullying, dating relationships, and appropriate dining etiquette. 

Students described the social and life skills they are being taught as an important step in 

their preparation for college and beyond. They told of instructional and noninstructional staff 

members routinely discussing the personal habits and skills that students need to succeed in college, such as self-motivation, discipline, independence, strong time management skills (a recurring theme in all interviews and focus groups), leadership qualities, and other personal characteristics. Middle school students characterized their schedules as being regimented and related

an overall sense of being overscheduled, yet many also reported taking advantage of the various 

extracurricular activities offered at SEED. For example, some participated on the track team and 

some in student government.

Impacts on Academic Achievement and Behavior

At the beginning of the study period, students entered SEED as seventh-graders, but two years 

later SEED changed its model and began to enroll sixth-graders as well. In 2010-2011, SEED 

fully transitioned to its new model and admitted students only as sixth-graders. Analyses drew 

on separate samples for sixth-grade and seventh-grade entrants to estimate the short-term effects 

of SEED and drew on the sample of students from the first two years of the study (cohorts 1 and 

2 of the seventh-graders) to estimate the longer-term effects. 
ES-5 

Washington, DC, is an especially charter-rich environment, and many of the students 

who applied to SEED but did not win the lottery actively sought out other innovative school 

options. Specifically, of the SEED lottery losers, roughly half the students enrolled in charter 

schools and half enrolled in traditional District of Columbia middle schools. Thus, in this study 

the SEED student experience was compared with a diverse set of other school experiences that 

may not exist in other contexts.

Short-Term Effects of SEED

About 20 percent of the students who won the SEED lottery (the SEED group) did not 

enroll in SEED the following fall. Among lottery winners who did not enroll in SEED, a little 

more than half enrolled in other charter schools and the remainder enrolled in traditional District 

of Columbia middle schools.

Short-term academic effects were measured by standardized test scores in the first two 

years of follow-up. Findings for the Seventh-Grade Entrant Sample are as follows: 

•

•

On average, being offered the opportunity to attend SEED increased students’ 

academic achievement in math. In the first year, SEED group scores were 

higher than non-SEED group scores by 0.24 standard deviation, which is 

roughly equivalent to a 76 percent improvement on top of the typical annual 

gains for this age group. In the second year, the SEED effect in math was

equivalent to one and a half years of typical growth.

3

Students in the SEED group did not perform better than students in the nonSEED group in their first year of follow-up on the standard reading exam, but

in the second year, the SEED group’s test scores exceeded those of the nonSEED group by the equivalent of one year of typical growth in reading.

4

Students in the Sixth-Grade Entrant Sample experienced a similar, though slightly 

weaker, pattern of positive SEED effects. 

Longer-Term Effects of SEED

In their third follow-up year, students in the first two cohorts made the transition into 

high school.5SEED’s high school model is different from the middle school model in a few key 

ways — boys and girls are taught in the same classroom, students’ time after school is less 

structured, and students begin more rigorous college-preparatory activities. Between the second 

 3

Carolyn J. Hill, Howard S. Bloom, Alison Rebeck Black, and Mark W. Lipsey, “Empirical Benchmarks 

for Interpreting Effect Sizes in Research,” Child Development Perspectives 2, 3 (2008): 172-177. 4

Hill, Bloom, Black, and Lipsey (2008).

5

Ten percent of the students were retained in grade.
ES-6 

and third follow-up years, roughly 20 percent of the SEED lottery winners in cohorts 1 and 2

who were enrolled in SEED chose to leave and attend a different high school.

In the fall of 2013 the research team surveyed lottery winners and lottery losers in cohorts 1 and 2 and asked them a variety of questions about their experiences in school. At that 

point, most of the respondents were high school seniors or had recently graduated. Overall, students in the SEED group reported experiencing a more rigorous and supportive academic environment. 

•

•

Students in the SEED group took fewer Advanced Placement courses but a 

larger number of foreign language, advanced science, and advanced math 

courses; participated in more extracurricular activities; received more academic support from teachers and adults; and participated in more college-prep 

and work-readiness activities. 

SEED group students reported having more orderly classrooms and more academically motivated peers. Students did not report a difference in the calmness and order of weekday living environments or the frequency of meals, 

though students in the SEED group did report that they slept less and exercised more during the week. 

The effects on high school graduation for the earliest cohorts were limited: 

• Being offered the opportunity to attend SEED did not have an effect on students’ probability of four-year high school graduation.

In addition to positively affecting students’ academic achievement, SEED aims to promote positive behaviors like rigorous study habits and self-control, while discouraging “risky” 

ones like alcohol use and unplanned pregnancy. SEED may have had small effects on some

measures of student behavior, but only four are statistically significant: 

•

•

•

While the SEED group did not report having developed more rigorous study 

habits or organizational skills than the non-SEED group, they did report 

spending four more hours a week doing homework than the non-SEED group. 

Students in the SEED group reported slightly lower tobacco use in the past 30 

days than non-SEED group students.

Students in the SEED group indicated slightly more frequent risky behavior 

(for example, skipping school, arguing with parents, or hitting someone) in 

the three months before they were surveyed, compared with non-SEED group 

students.
ES-7 

• SEED group students reported slightly lower levels than non-SEED group 

students on scales designed to measure “grit” or perseverance.

Conclusions

Several factors are critical to the interpretation of the findings to date from this evaluation:

•

•

•

•

Cost. Owing to its boarding school model, SEED costs at least twice as much 

per student as a traditional nonresidential school. If SEED has impacts on 

four-year high school graduation and nonacademic outcomes such as teenage 

childbearing and justice involvement, it would have the potential to produce 

large societal benefits that would offset its cost. At this point, there is little evidence that SEED DC has affected either graduation or the nonacademic outcomes, though it is important to note that the sample size for measuring longer-term impacts (about 200 students) is quite small.

Enrollment targeting. SEED’s founders argue that the intensive, holistic, 

boarding-school model is needed for some students who face very serious obstacles to school success in their homes and communities. And, indeed, both 

the Maryland and Florida SEED schools are open only to students who meet 

certain specific criteria signifying severe disadvantage. In contrast, owing to 

local rules, the DC school, the focus of this evaluation, is open to any student 

who resides in Washington. It is possible that SEED’s model would produce 

larger impacts for students facing more serious obstacles to success.

Local educational context. Washington, DC, has many innovative charter 

and magnet schools, and it appears that a large proportion of the students who 

lost the SEED lottery enrolled in these schools, particularly by the time they 

reached high school, when two-thirds of the non-SEED group were attending 

charter or magnet schools. It is possible that SEED would make a bigger difference in a context with fewer innovative alternatives. 

Length of stay at SEED. SEED DC’s impacts on academic proficiency are 

substantial, particularly in the middle school years, and other literature suggests that middle school test scores are highly predictive of high school graduation. Yet there is no evidence to date that SEED has increased high school 

graduation rates. This fact, coupled with data showing that less than half the 

SEED group was still at the school in twelfth grade, raises the question of 

whether SEED could have larger, more sustained impacts if more students 
ES-8 

remained in the school longer and received a larger “dose” of SEED — in 

other words, whether the school needs to focus more on promoting retention. 

• Implementation quality. While the implementation study noted many positive qualities of SEED DC’s operation, it also raised questions about the 

quality of instruction in SEED DC’s classrooms — particularly at the high 

school level — and noted that many students seemed to be struggling with the 

transition from eighth grade to ninth grade, even while remaining at the same 

school. The SEED Foundation has recently announced a renewed and intensified focus on the quality of instruction, leadership, and services at its schools.

For example, the foundation reports that SEED DC has identified new curriculum resources to strengthen the middle school math program and is seeking 

to increase student engagement through interactive learning technologies. 

Given these and other ongoing changes, it is possible that impacts on student 

outcomes — and, perhaps, retention rates — will be stronger in the future.
1 

Chapter 1

Introduction

Sweeping changes in the U.S. economy and labor market over the past three decades have dramatically reduced the availability of well-paying jobs for workers without postsecondary education. Yet one in five high school freshmen nationwide do not graduate in four years, and many 

who do complete school are not ready to perform college-level work.1These patterns are particularly pronounced in urban areas, and among students from low-income and underserved families. 

With these trends in mind, policymakers, practitioners, and researchers have developed 

and promoted a variety of approaches to improving students’ high school success. Some interventions are implemented within existing schools, such as Talent Development, which bolsters 

the support students receive while transitioning from eighth to ninth grade.2 Others restructure 

schools to provide personalized learning environments that foster relationships between teachers 

and students and closely track students’ academic progress (Ninth-Grade Academies) and others create new district schools altogether (New York City’s Small Schools of Choice).3 In recent 

years, charter schools, which receive public funding but operate independently of local school 

districts, have increased in number and popularity, in part due to their flexible governance structure, which allows them to implement innovative new education models. 

Rigorous studies have found positive results for some of these new approaches, but 

much remains to be done. For example, in a 2010 report on the impacts of 36 charter middle 

schools across 15 states, on average, students who applied to and won the lotteries for oversubscribed charter schools did not perform higher on follow-up reading and mathematics exams 

than students who did not win the lotteries.

4 More encouraging were findings from the researchers’ secondary analysis: Among schools serving more low-income or low-achieving students, 

charter schools did have positive effects on students’ math scores, although not on students’ reading scores. Findings from MDRC’s study of New York City’s Small Schools of Choice also indicate a need for more intensive interventions for the most disadvantaged students. Specifically, 

while enrolling in one of these schools substantially increased graduation rates for the lowestachieving students, less than half the study students in this group graduated within four years.5

 

 1

Kena et al. (2014). 2

Kemple (2008); Kemple, Herlihy, and Smith (2005); Snipes, Holton, Doolittle, and Sztejnberg (2006). 3

Bloom and Unterman (2014). 4

Gleason et al. (2010).

5

Bloom and Unterman (2014). 
2 

This report presents the findings from a rigorous evaluation of the SEED School of Washington, DC (SEED DC), the nation’s first urban, public, college-preparatory boarding school.

SEED’s boarding school model is predicated on the assumption that, for certain disadvantaged 

students who face overwhelming barriers to success at home and in the community, school reforms such as those described above, and other enhancements such as after-school programs, 

will not be sufficient. SEED’s leaders believe that, for these students, achieving success in high 

school and beyond requires a fully integrated academic and boarding program that also provides 

scheduled study time, constant access to positive role models, and life skills training. 

This report is based upon work supported by the Social Innovation Fund (SIF), a program of the Corporation for National and Community Service (CNCS). The Edna McConnell 

Clark Foundation (EMCF) is leading a SIF project that includes support from CNCS and 15 

private co-investors. EMCF’s SIF project included an investment in the SEED Foundation, the 

national nonprofit organization that oversees SEED DC and the two other SEED schools (in 

Baltimore, Maryland, and Miami, Florida) currently in operation. The evaluation is being conducted by MDRC, a nonprofit, nonpartisan education and social policy research organization. It 

uses SEED DC’s annual admissions lottery to identify two comparable groups of students:

those who applied to SEED and were selected, at random, to be offered a slot in the school, and 

those who applied to SEED and were not offered a slot. By following those two groups of students over time, the study can estimate the impacts of SEED DC on standardized test scores and

high school graduation rates, as well as nonacademic outcomes. This study focuses only on 

SEED DC, which is by far the most mature of the existing SEED schools.

In sum, the study found that SEED DC creates a highly supportive environment for its 

students. The program has produced notable, positive increases in students’ standardized test 

scores and proficiency levels. But, for the portion of students it was able to follow through all 

four years of high school, the study did not find an impact on the proportion of students who 

graduated from high school in four years or on key nonacademic outcomes.

About SEED

The SEED Foundation was created in 1997 by Rajiv Vinnakota and Eric Adler, two former 

management consultants who left their jobs to create a public boarding school for low-income, 

disadvantaged students. The SEED School of Washington, DC, opened in 1998 as a public 

charter school and currently serves about 320 students in grades 6 through 12. 

As the first SEED school, SEED DC has received widespread national attention. In 

2005, SEED received the Innovations in American Government award from Harvard’s John F. 

Kennedy School of Government. In 2009, President Obama visited the DC school to sign the 
3 

Edward M. Kennedy Serve America Act. In 2010, SEED was featured on 60 Minutes and in the 

documentary film Waiting for “Superman.” 

Ten years after SEED DC opened, a second SEED school was created in Baltimore, 

serving students throughout the state of Maryland. SEED Maryland serves about 400 students in 

grades 6 through 12. The third school, in Miami, admitted its first group of 60 sixth-graders in 

2014. SEED DC is open to anyone who lives in the District of Columbia, while the Maryland 

and Florida schools target students who meet specific criteria signifying serious disadvantage. 

In late 2014 the SEED Foundation announced a comprehensive effort to improve the quality of 

academic and student support programming at all its schools. The SEED Foundation reports 

that the new approach is being implemented first in the new Miami school. For more information about that school and its early success, see Appendix A. 

The SEED Evaluation

SEED has been the subject of several qualitative studies and one impact study, conducted by 

Vilsa Curto and Roland Fryer Jr. of Harvard University.6 Like the current evaluation, Curto and 

Fryer’s study built on the admissions lottery for SEED DC. It found that SEED led to significant gains in standardized test scores in seventh and eighth grade, but the authors questioned 

whether these increases were large enough to justify SEED’s very high cost — more than 

$35,000 per year per student. They noted that other charter schools have produced similar results without the boarding component, which accounts for a large proportion of SEED’s cost. 

The MDRC evaluation builds on the Curto and Fryer analysis by following a larger 

number of students for a longer period, conducting an implementation study to more fully describe how SEED operates, and obtaining data on a broader range of outcomes from school records and a student survey. This latter point is critical, because if SEED affects nonacademic 

outcomes such as teen pregnancy or crime involvement, which trigger very high social costs, 

the program could turn out to be a worthwhile investment of public funds.

The design for MDRC’s evaluation of SEED DC was shaped by the logic model shown 

in Figure 1.1. Starting at the far left, the model’s inputs include the students, teachers and staff, 

the boarding school infrastructure, and support from the SEED Foundation, among others. The 

inputs lead to a set of specific activities and program components: the academic program, the 

Student Life program, College Counseling, and Student Support Services. The model hypothesizes that these inputs and program components, if implemented according to plan, will lead to a

 6

Curto and Fryer (2011).
4 

SEED program model 

and philosophy

Teachers and staff

Community partners

Students and families

School leaders and 

trustees

Support from the 

SEED Foundation

Donors and investors

Well-Being Outcomes

Positive school climate

High amount of family involvement

Improved mental health

Low frequency of disciplinary actions

High student and staff satisfaction

Social Outcomes

Increased “college knowledge”

Reduced teen pregnancy

Lower involvement in juvenile justice 

system

Increased knowledge of life skills

Academic Outcomes

Improved standardized test scores

Improved GPA

Increased middle school reading levels

High AP participation and scores

College admission

High school 

graduation 

and college 

matriculation

z College 

graduation

SEED Evaluation Final Report

Figure 1.1

SEED DC Logic Model 

Academic program

Student Life program

College Counseling

Student Support 

Services

Inputs Program Components Intermediate Outcomes Long-Term Outcomes

SOURCE: Adapted from SEED Foundation.

NOTE: GPA = grade point average; AP = Advanced Placement.

Community characteristics
5

different type of school environment that is orderly and highly supportive, and to the transmission of specific knowledge and values to students. This result, in turn, should produce both intermediate student achievement outcomes — such as better grades and test scores — and wellbeing outcomes, as well as longer-term student outcomes such as high school graduation, and 

ultimately, college success.

In line with this logic model, the evaluation studied both the implementation of SEED 

DC and its impact on student outcomes. Owing to its timing, however, the evaluation cannot 

assess whether SEED is achieving its longer-term goals, such as helping students succeed in 

college. The evaluation’s two overarching research questions are: 

•

•

How is SEED DC structured and how does it operate in practice? Using 

interviews with staff members and students, observation, and other methods, 

the evaluation team set out to understand the on-the-ground reality of SEED 

DC and how the school is experienced by students. This information not only 

will help in interpreting the impact findings, but also may help SEED improve service delivery over the long run.

What is the effect of being offered an opportunity to attend SEED DC 

on student outcomes? The primary outcomes in the study are academic:

standardized test scores and high school graduation. Secondary, nonacademic outcomes include both student attitudes (for example, college aspirations) 

and student behaviors (for example, positive ones such as homework completion and risky ones such as interactions with the criminal justice system or 

unplanned pregnancy). Because some lottery winners did not enroll in SEED 

DC, a secondary analysis explores how enrolling in the school affects students’ short- and longer-term outcomes. 

The evaluation focuses on 766 students who “won” or “lost” the SEED lottery as fifthor sixth-graders between the 2005-2006 and 2010-2011 school years. The study followed those 

students through the 2013-2014 academic year, which means that only a small number of them 

could have graduated from high school or enrolled in college during the study period. Thus, 

while improving students’ performance in college is a key goal of SEED’s, it is too soon to assess whether SEED improves students’ postsecondary outcomes. 

The remaining chapters of this report describe the findings from the SEED evaluation.

Chapter 2 describes how SEED DC is staffed and organized and how it operated in practice 

over the course of three school years (starting in fall 2011). This chapter draws primarily from a 

series of site visits to SEED, which included interviews with staff members and students and 

observations of classes and other school activities. The research team also conducted interviews 
6

with officials from the SEED Foundation. In essence, these chapters focus on the left side of the 

logic model, describing both the SEED model and its implementation.

Chapter 3 addresses the second research question. It focuses both on the “treatment contrast” — how the experiences of lottery winners and lottery losers differed — and on the outcomes that students achieved. The analysis uses data from school records and a student survey 

to assess both academic and nonacademic outcomes. Chapter 4 summarizes the main conclusions and discusses their implications.
7 

Chapter 2

SEED DC in Operation

As described in Chapter 1, the SEED model posits that a different type of school environment 

can produce positive outcomes for students. In order to provide context for the impacts that will 

be discussed in the following chapters, it is necessary to understand what this different learning 

environment looks like on the ground. This chapter describes the SEED model in practice based 

on interview, focus group, and observational data collected in May 2012, November 2012, and 

November 2013. The study does not focus on fidelity of implementation — that is, whether 

SEED DC programming was implemented according to the SEED model’s design — but rather 

describes the school as it appeared in operation. The analytic approach and data collection activities are described in greater detail in Appendix B.

General Structure of SEED DC

Located in a residential section of southeast Washington, DC, in Ward 7, SEED DC is a college-preparatory public charter boarding school that serves approximately 320 sixth- through 

twelfth-grade students. Students attend school on campus five days a week, arriving on Sunday 

evening and returning home on Friday afternoon. The school’s physical facilities include an 

academic center, where classes are held; an administrative building; a library; a gymnasium; a 

college counseling office (the College Café); and two single-gender dormitories. Each floor of 

the dormitories is equipped with a common room, bathroom facilities, a small reading room, 

and small bedrooms that are furnished with beds, closets, desks, and chairs to accommodate two 

students each. During the five days that students reside on campus, they have 24-hour access to 

quiet places to study and sleep, nutritious meals, academic resources, and spaces to engage in 

extracurricular activities. By design, students are also surrounded by a cadre of caring adults 

who will support them in preparing for success in college. 

SEED DC is led by a head of school responsible for keeping the school’s operation true 

to its mission. While this chapter concentrates on the rest of the staff responsible for student 

programming, it is important to note that the school is supported by a number of other administrative staff, including a managing director and directors of finance, human resources, development, and campus operations. 

A distinguishing feature of SEED DC programming is that the school’s goals, expectations, and curricula are designed specifically for three groups of students: middle school students, ninth-graders, and tenth- through twelfth-graders. While ninth grade is technically part of 

SEED DC’s high school, the model treats this year as a uniquely important transitional year 
8 

warranting its own specific focus. A second distinguishing feature of the SEED model is how 

school programming is organized in terms of content area: Besides academics, there are departments of Student Life, Student Support Services, and College Counseling (see Figure 1.1).

While none of the four departments functions independently of the others, each has its own specific role and responsibilities. More detail on the staffing structures of each department is included in the following sections, as well as in the organizational chart shown in Figure 2.1. The 

rest of the chapter presents information organized into these four areas, as well as a brief section 

explaining how they interact.

Academics

The school principal is the leader of the academic department, supported by teachers and middle 

managers focused on middle school- and high school-specific curricula, special education, and 

evening programming. For more detail on staffing in SEED DC’s academic department, see 

Figure 2.1. SEED DC’s school day mirrors the schedule of a typical school day: seven to eight 

hours of instruction divided into subject-specific class periods. Tables 2.1 and 2.2 list the middle 

and high school courses offered by grade. 

The school philosophy is that all students have the same inherent potential for academic 

success, and thus all students are expected to excel at SEED. The academic department strongly 

believes in using data to guide and inform instruction, so all students take interim assessments in 

English and math four times per year. After each assessment, academic and Student Life staff 

members meet in teams to discuss the results and identify priority issues. Teachers then use the 

assessment results to develop lesson plans to “reteach” skills that students have been unable to 

master.

The SEED DC academic program follows the school’s grade-based cohort system, 

meaning that distinct goals, expectations, and approaches are set for middle school, ninth-grade, 

and high school students. More detail on SEED’s approach to middle school and high school 

(including ninth grade) academics is included in the following sections.

Middle School

Middle school administrators and faculty and staff members reported that many sixthgrade students enter SEED DC reading at one or more levels below grade level. We present baseline test score data later in the report. They also suggested that over time, SEED has observed an 

increase in the number of incoming students eligible for special education. To respond to students’ needs, the middle school program is intentionally designed to remediate gaps and promote growth. Diagnostic testing is administered on a frequent basis so that data can inform individualized instruction. An administrator explained that the school “make[s] sure that kids are
9 

Principal

Head of School

Overnight 

and Morning 

Supervisors

Student Life 

Coordinator: MS 

programming

Student Life 

Coordinator: HS 

programming

Student Life 

Coordinator: 9thgrade programming

Assistant Student 

Life Coordinators

MS LSCs

HS LSCs

HS RAs 9th-grade RAs

Director of Student 

Life

9th-grade LSCs

SEED Evaluation Final Report

Figure 2.1

SEED DC Organizational Chart 

Director of Enrollment 

Management

Director of College 

Counseling

Athletic 

Director

Mental Health 

Director

Counselors Assistant 

Deans

Nurses External Dean

Opportunities 

Coordinator

MS Program 

Director

 Evening 

Academic 

Intervention 

Coordinator

Special Education 

Coordinator

9th-grade 

Teachers

HS Program 

Director

HS 

Teachers

MS 

Teachers

Director of Student 

Support Services

MS RAs

SOURCE: SEED Foundation.

NOTES: HS = high school; MS = middle school; LSC = life skills counselor; RA = resident adviser. Organizational chart omits administrative staff not involved with student 

programming.
10

Middle School Courses at SEED DC

Table 2.1

SEED Evaluation Final Report

Subject 6th grade 7th grade 8th grade

English and language arts Reading and Writing Reading and Writing Reading and Writing 

Math Foundations of Math Prealgebra Algebra

Science Earth Science Life Science Physical Science

Social studies U.S. Geography Civics World Geography

Foreign language Introduction to Spanish

SOURCE: Interviews with SEED DC administrators.

Table 2.2

High School Courses at SEED DC

SEED Evaluation Final Report

Subject 9th grade 10th grade 11th grade 12th grade

English and Reading and Reading and AP English or AP Literature or

language arts Writing Writing English II 12th-grade English

Math Geometry Algebra II Pre-Calculus or AP Calculus or

Probability and Calculus or a Statistics b Precalculus

Science Conceptual Biology Chemistry AP Biology or 

Physics Anatomy and 

Physiology

Social studies World History I World History II AP U.S. History or DC History and

U.S. History either AP U.S. 

Government or

U.S. Governmentc

Foreign language Spanish I Spanish II Spanish III

SOURCE: Interviews with SEED DC administrators.

NOTES: AP = Advanced Placement. aProbability and Statistics is not always offered.

bThose who advance from Probability and Statistics take Precalculus.

c

Students take one semester of each history course.
11

being given the chance to all get to the same endpoint, but on a path that allows them to grow 

from where they’re starting.” An example of this diagnostic approach is the school’s practice of 

making students aware of their Fountas and Pinnell reading levels — a system that supports 

students in level-appropriate guided reading — so that they can work on advancing during the 

independent reading time that is built into the daily student schedule.1

In line with the philosophy that all students can succeed, the school offers a variety of 

interventions to help struggling students master foundational skills. All middle school students 

participate in a daily class period of “targeted instruction,” an opportunity for remediation in 

math or reading, if needed, or for acceleration. 

A distinguishing feature of the middle school is the delivery of instruction in singlegender classrooms. Studying the effectiveness of the single-gender approach was not part of the 

evaluation. However, administrators and faculty members indicated in interviews that the 

school chose to implement single-gender classrooms in part because of distractions that can occur when middle school boys and girls are placed in the same classrooms. Middle school students interviewed for this study agreed that single-gender classrooms were less distracting than 

coed classrooms. 

Observation findings. The evaluation team conducted a small number of informal observations of middle school classrooms to gain a general understanding of how the middle 

school classes are structured, how teachers and students interact, and the types of instructional 

practices being used at the middle school level. The evaluators noted numerous instances of independent work being done quietly and effectively. Evaluators also observed positive examples 

of teaching strategies being used. In one instance a teacher encouraged students to use higherorder thinking skills by looking at a picture, drawing conclusions based on their observations, 

and defending their reasons for drawing those conclusions. In another instance, evaluators saw 

evidence of coordination among middle school teachers when the U.S. Geography teacher introduced a new topic, coal, by linking it to a related science lesson on minerals in Earth Science. 

Evaluators also observed a few instances when students were confused by material covered in previous classes. For example, in sixth-grade General Foundations of Math, students 

struggled to identify which of two fractions was larger when practicing multiplying and dividing with improper fractions. In a middle school English class, students were left alone to work 

on a research project and seemed confused about how to differentiate between a main topic and 

subtopics. Unaware of the misunderstanding, the teacher did not address it.

In focus groups, most middle school students said they were able to keep up with the 

academic course work and that teachers were willing to offer step-by-step instructions to help 

 1

Fountas and Pinnell (2014).
12

them succeed in the classroom. A few students explained that before enrolling at SEED DC, 

they often felt bored in classes when they were capable of completing advanced course work 

but were not allowed to move ahead. At SEED DC, these students feel more engaged in their 

course work. When discussing their reading levels, two students described the Fountas and Pinnell reading system (of levels A to Z+) as “awesome.” Another said it gave students “something 

to look up to.” 

High School

Ninth grade. When SEED students make the transition to high school, they take classes in coed settings and are expected to demonstrate mastery in all subject areas. In contrast 

with the middle school focus on remediation, students are expected to be at grade level in basic 

reading, writing, and mathematics so that they can build critical reading, analytic thinking, and 

problem-solving skills. 

Because of higher expectations in and outside the classroom and an increase in academic rigor in ninth grade, SEED DC views this as a very important transitional year for students. 

To prepare students for the change, the school provides a two- to three-day freshman bridge 

program for students in the summer between eighth and ninth grades. In ninth grade, students

take all their classes together and are treated as their own grade-specific cohort, separate from 

middle and high school. But students who begin ninth grade below grade level often struggle 

with understanding advanced texts and concepts. Ninth-grade teachers suggested that the transition between eighth and ninth grades causes some struggling students to seriously consider 

whether they can succeed at SEED. One ninth-grade teacher offered the following analysis:

I think our students start to kind of realize, “... I can either make it here or I need 

to go. Maybe I’m not going to be able to rise to the occasion in ninth and tenth 

grade.” And I think that’s why we tend to lose more kids around those grades. So 

the ninth grade is very important here ..., but elsewhere too. The kids start to realize, “OK, school is for me or school is not for me.” ... I think some of them 

just have trouble making a distinction, “Oh, high school’s not just a continuation 

of middle school, it’s actually a separate entity.” 

Ninth-grade teachers also indicated a need to align the eighth- and ninth-grade curricula 

so that students are better prepared for the transition. Ninth-grade teachers reported working 

with the middle school staff to begin to introduce ninth-grade reading content in the eighthgrade curriculum, but they found that the reading gap was too large. Another staff member on 

the academic team reported high failure rates for the ninth-grade cohort in English, writing, geometry, and science, and identified the ninth-grade cohort as the cohort most often enrolled in 

on-site summer school. 
13

Grades 10 through 12. After ninth grade, the high school curriculum focuses on preparing students for success in college and beyond by further developing their higher-order 

thinking and problem-solving skills. SEED DC follows the Washington school district’s requirements for high school graduation and also requires students to earn additional credits. For 

example, SEED DC students must earn three credits of foreign language whereas the district 

requires only two, and all SEED DC science classes are laboratory sciences, whereas the district 

requires only two lab sciences. The school also provides six Advanced Placement classes for 

eligible students: eleventh-grade U.S. History and English Language and twelfth-grade Biology, 

Calculus, Government, and Literature. In addition, there are a few opportunities for eligible students to enroll in college-level courses at a local college.

Observation findings. The evaluation team visited a small number of high school 

classrooms to obtain a general understanding of the instructional practices being used at the 

high school level and to understand how students are being prepared for college-level work. The 

evaluation team found high school classrooms to be orderly, and most students participated in 

classroom assignments. However, evaluators did not observe many examples of students being 

prepared for challenging college course work that would qualify them for admission to competitive colleges. For example, the evaluation team did not see significant evidence of students being asked to use higher-order thinking skills, such as critical, logical, or reflective thinking, 

when completing classroom assignments. The team also observed several missed opportunities 

for redirecting students or correcting misinformation. For example, during individual student 

presentations in a government class, incorrect information shared during a student presentation 

about a legislative bill was not corrected, even when the activity included structured space for 

feedback and corrections. In Spanish II, at no point were students asked to speak or write a full 

sentence in the language. At one point, the teacher chose not to explain a rule, telling the students it was too difficult for them to understand. In addition, while student attention varied from 

class to class, in some instances students appeared to struggle with concentrating on tasks (for 

example, by talking with peers, asking to leave the room, or refusing to participate when called 

upon). 

While the evaluation team found reason to question whether students are being prepared for higher-tier colleges, high school students reported that they found the curriculum challenging. All students interviewed described being overwhelmed by their course loads and concluded that they were being prepared for college-level work (a few seniors in the focus group 

were enrolled in Advanced Placement courses). These students also cited positive competition 

among students as a factor that motivated them to succeed academically. They considered the 

school’s culture of making sure that all students understand the academic material to be a positive feature of their SEED experience.
14

Student Life

Perhaps the most distinctive aspect of SEED’s learning environment is the time that adults 

spend with students after school and through the evening. The Student Life department is responsible for this time, developing and coordinating residential life programming and managing 

students’ time outside of the traditional academic day. 

Led by a director, the department includes middle management in the form of Student 

Life coordinators, who oversee and supervise life skills counselors (LSCs), responsible for developing Student Life programming, and resident advisers (RAs), responsible for supervising 

students in the dormitories. Within the dorms, students are organized into houses (or groups), 

each of which is led by an RA. Each house is named after a college or university and decorated 

with that institution’s memorabilia (pennants, pillows, and so on, prominently displayed in the 

house’s common spaces). LSCs work with the same grade level each year, while RAs remain 

with the same group of students throughout the students’ time at SEED.

The Student Life department aims to develop students’ behavioral, social, and life skills 

while also reinforcing what they are learning in the classroom. In line with the school’s overall 

grade-specific cohort approach, there are specific goals for middle school, ninth-grade, and high 

school students. Middle school programming is intended to develop and refine social skills connected to meeting behavior expectations and routines, such as following instructions the first 

time they are given, adhering to the school dress code, and learning how to disagree appropriately. Ninth-grade Student Life activities aim to develop and reinforce the skills and habits necessary for success in high school, such as planning ahead, using anger control strategies, and 

building strong self-esteem. High school programming focuses on the transition to college, including preparation for the SAT, ACT, and Accuplacer entrance exams.

2 To accomplish these 

goals, the majority of Student Life time is structured, especially for the middle school grades. 

Figure 2.2 presents an example of a Student Life schedule, showing how sixth-graders spend 

their time outside of the school day. The following sections describe the wide variety of Student 

Life activities available to students.

Students described the social and life skills they were being taught as an important step 

in their preparation for college and beyond. Students told of instructional and noninstructional

staff members routinely discussing the personal habits and skills that students need to succeed 

in college, such as self-motivation, discipline, independence, strong time management skills (a 

recurring theme in all interviews and focus groups), leadership qualities, and other personal 

characteristics. Middle school students characterized their schedules as being overly reliant on

 2

The Accuplacer is a tool used by many colleges to assess students’ math, reading, and writing knowledge. 
15

SEED Evaluation Final Report

Figure 2.2

Sample Student Life Schedule: Sixth Grade

SOURCE: SEED DC.

Time Monday Tuesday Wednesday Thursday

5:30 am Wake up, make bed, 

prepare for the day

Wake up, make bed, 

room inspection Wake up, make bed, prepare for the day

6:00 am Uniform and room 

inspection Sparks (athletics, clubs, 

Uniform and room inspection

etc.)

6:15 am Breakfast Breakfast

6:45 am Morning house meeting Breakfast and uniform 

inspection Morning house meeting

7:45 am Resident adviser (RA) checks backpacks and school materials

7:55 am Transition to school

Academic school day

3:30 pm Study Zone (Study Hall)

5:00 pm Drop Everything and Learn (specified academic Prepare for dinner interventions based on needs)

5:30 pm Dinner

6:15 pm HALLS lesson (life skills Sparks (athletics, clubs, Character Education Sparks (athletics, clubs, 

education) etc.) lesson etc.)

7:05 pm Drop Everything and Read (independent reading time)

7:45 pm House programming (team-building, educational activities, games, etc.)

8:30 pm Quiet house, showers, preparation for the next day

9:15 pm Lights out and in bed
16

routines and related an overall sense of being overscheduled. Yet many reported taking advantage of the various extracurricular activities offered at SEED. For example, some participated on the track team and some in student government, and a few served as school ambassadors.3

 

Behavioral, Social, and Life Skill Development 

Student Life staff members described social skill development and life skills education 

as critical features of the SEED model. Students primarily engaged with these topics during 

SEED’s signature HALLS program, Character Education (at the middle school level), House 

Programming, and other grade-specific activities. 

The SEED-created Habits for Achieving Life-Long Success (HALLS) program teaches 

students social and basic life skills, such as decision making and communication strategies, and 

the importance of taking responsibility for oneself and others. HALLS activities focus on a variety of topics, such as bullying, dating relationships, and appropriate dining etiquette. During 

Character Education, middle school students learn about positive character development 

through activities such as watching movies and discussing various decisions and situations 

faced by the films’ characters. Box 2.1 describes HALLS and Character Education lessons that 

were observed during a site visit. House Programming provides time for discussions about 

weekly expectations, team-building activities, house celebrations, and participation in field trips. 

SEED DC implements other grade-specific activities when Student Life staff members deem 

necessary — for example, the Success Highways curriculum for eighth-grade students, a program that focuses on building resiliency, and the Search Institute’s I-Time social and emotional 

development curriculum for ninth-graders.4

Student Life staff members reported that they often chose topics for activities such as

HALLS and Character Education based on their own assessment of students’ needs. They reported using established curricula and activities, developing their own lesson plans, or referring 

to lesson plans developed by staff in the past. Based on interviews with staff members and a few 

observations of Student Life activities, it was not clear to the evaluation team how student needs 

were identified, what specific outcomes Student Life programming was intended to affect, or

how specific programming choices were expected to lead to desired outcomes. However, an 

administrator indicated that the school was planning to begin collecting annual survey data 

about student behavior and attitudes to understand areas where students may be demonstrating 

growth and where additional support is needed. 

 3

School ambassadors are students who are chosen to represent the school, for example, by giving tours to 

campus visitors. 4

For information on these curricula, see ScholarCentric (2015) and Search Institute (2016). 
17

Academic Supports and Activities

Student Life time accommodates a set of academic supports and activities designed to 

help students excel. Each day all students must participate in Drop Everything and Read 

(DEAR), a 45-minute period reserved for independent and silent reading. In addition, eligible 

middle school students participate in Drop Everything and Learn (DEAL) twice per week, an 

Box 2.1

Sample Character Education and HALLS Lessons

Character Education Lesson for Sixth-Grade Girls

The 12 sixth-grade girls of Oxford House gathered for their weekly Character Education 

lesson. They sat in a circle in the common area, a bright and comfortable space decorated 

with colorful pictures and posters of the girls’ choosing and furnished with sofas and 

chairs. It took a few minutes for the girls to settle, but once they did, the resident adviser 

(RA) explained that the evening’s lesson would focus on expressing one’s opinion in a 

respectful manner. Then she passed around a hat containing topics written on small pieces of paper. Each girl selected a topic from the hat and was tasked with talking about the 

issue for a full minute. The topics were wide-ranging, including abuse, teenage pregnancy, basketball, homework, and rap music, among others. One girl, sitting at a desk, served 

as the timer. Some girls struggled to speak for the full 60 seconds, but the RA encouraged 

them to try. After each girl had finished her one-minute speech, the RA invited others to 

share their opinions about the topic as well. Many of the girls challenged their peers and 

expressed different opinions. The RA used these opportunities to correct behaviors such 

as interrupting others and devaluing their perspectives and to model respectful disagreement and listening skills. 

HALLS Lesson for Ninth-Graders

Later in the evening, 12 ninth-grade girls gathered with their RA and their life skills 

counselor (LSC) for a HALLS lesson. Soon after gathering, the girls became quiet and 

listened for instructions. The RA asked the girls to reflect on a film they had watched and 

to consider one positive and one negative leadership quality that had been depicted in the 

film. The RA then distributed two Post-It notes to each girl and instructed them to record 

the negative and positive qualities on the notes and place them under the appropriate 

sheets that had been taped to the wall. After all the notes had been posted, the RA asked 

each girl to state a quality she had identified and to explain why it was either positive or 

negative. Halfway through the exercise, a student proposed a new activity. She asked the 

group to reflect on a different, more personal question: Which student in this house do 

you look up to and why? Many students acknowledged their admiration for girls in the 

house who demonstrated academic focus, were encouraging of others, or offered help to 

others with challenging homework assignments. 
18

activity during which individually tailored interventions are used to address educational gaps in 

a variety of subjects. The school also offers students support through the SEED Tutoring Enrichment Program. The tutoring program is led by the evening academic intervention coordinator and is staffed by SEED-trained volunteer tutors and SEED students who serve as peer tutors.

Youth Development Activities and Recreational Time

As a part of the Student Life curriculum, twice per week students are required to participate in a “spark,” an extracurricular activity or club (such as intramural volleyball, chess, yoga, 

or cosmetology) designed to help students explore their interests and develop motivation and 

initiative.5 Besides helping students discover, nurture, and demonstrate their skills, sparks are 

designed to allow students to build healthy relationships with the adults who lead the activities.

As explained below, athletic teams are a unit of Student Support Services, but students participate in these activities before and after school during Student Life time.

Student Support Services

As a unit, Student Support Services is responsible for meeting the mental health, physical 

health, and social and emotional needs of students and addressing barriers that might prevent 

students from achieving their full potential. This department is led by a director who oversees 

five smaller departments: Mental Health, Health and Wellness, Athletics, External Opportunities, and Dean of Students. These smaller units function as a combined approach to support students outside the classroom. 

SEED DC provides mental and physical health services that strongly resemble those at 

traditional schools, but because of the boarding setting, the staff must be available to provide 

assistance 24 hours a day. The Mental Health department is staffed by a director, two special 

education counselors, and two general education counselors who provide counseling services at 

the individual and group levels. The team offers one-on-one counseling for students who request it and holds small group discussions for students on topics such as dealing with loss or 

body image issues. 

The Health and Wellness department consists of two nurses who address students’ 

medical needs, administer medicine, and lead workshops with Student Life staff members on 

topics such as personal hygiene, community living, and sex education. 

The director of Athletics manages the school’s athletic programming and the interscholastic sports program. Physical education and athletics include soccer, volleyball, golf, lacrosse, 

 5

SEED worked directly with the Search Institute and used its research when designing this part of the Student Life program. 
19

flag football, basketball, cheerleading, baseball, and track. Some athletic activities are held on 

campus; others, such as track, are held in offsite facilities. Through SEED DC’s membership in 

an independent school league, students are able to compete with students from other schools. 

SEED DC administrators described exposure to positive educational experiences and 

enrichment opportunities as a critical part of their model, citing the importance of developing 

well-rounded students who have the social skills to operate outside the SEED environment. 

With this goal in mind, Student Support Services includes a small External Opportunities department whose coordinator is charged with actively increasing this exposure. Opportunities can 

take the form of students working with a local nonprofit organization to learn about efforts to 

stem the spread of HIV, or participating in international travel. To encourage students to make 

the most of the school’s external opportunities, all high school students must complete a summer opportunity and all seniors must complete a semester-long internship before graduating. In 

focus groups, several students said they had wanted to attend SEED DC specifically because of 

the external opportunities offered. Some students who were initially lukewarm about enrolling 

in a boarding school reported that the opportunity to travel abroad had convinced them to enroll.

SEED DC uses a skills-based approach to behavior management and discipline, believing that students must develop various social skills in order to meet behavior expectations. 

Building on a Boys Town curriculum, SEED created the “Model of Care” as a school-wide system for this approach to skill development and behavior management. Though all staff members are responsible for implementing this system, the Dean of Students department is charged 

with overseeing its implementation. When students are unable to meet expectations, the dean

and assistant deans are also involved in mediations and decision making about suspensions and 

expulsions. The Model of Care is discussed further below. 

College Counseling

SEED students are expected to attend college following high school graduation. The College 

Counseling department, led by a director, is responsible for supporting students in the college 

search, application, and selection process. Early college awareness begins in the sixth grade, 

with students being encouraged to visit the College Café (the College Counseling department), 

a colorful and inviting space that houses informational resources and is decorated with memorabilia from many of the nation’s colleges. In middle school, students are engaged in discussions about the value of enrolling in college and participate in a college campus visit as well as

activities to help develop academic habits that will prepare them for postsecondary success. 

Starting in ninth grade, students practice taking college entrance exams (the PSAT) so that they 

become familiar with the test and can improve their scores. High school students have access to 

ACT, SAT, and Accuplacer preparation materials during after-school hours. Students in eleventh grade enroll in Junior College Seminar, in which advisers help them find the “right-fit” 
20

college — the college that is the best academic match and that also meets their financial, social, 

and personal needs. In Senior College Seminar, students in the twelfth grade are actively engaged in the college search, application, and choice process.

A distinctive feature of the SEED student experience is the support that SEED students 

and graduates receive from the College Transition and Success (CTS) team, a unit within the 

SEED Foundation. Working in collaboration with the College Counseling department, the CTS 

team holds a series of college transition workshops for seniors and their parents (including a 

financial literacy workshop) and helps students finish required college enrollment paperwork. 

The CTS team also monitors and supports SEED graduates as they begin college and maintains 

contact with them while they are enrolled in college, functioning as an important bridge between SEED graduates’ high school and college experiences. The CTS team also performs an 

important data collection function. It collects and tracks college enrollment, withdrawal, and 

graduation data for SEED graduates, allowing the foundation to assess its graduates’ postsecondary progress.6

 

During focus groups, high school seniors said they felt supported by the College Counseling staff. They named staff members in the College Café and other “senior supporters” (staff 

members assigned to seniors to support them in other ways through their final year at SEED) as 

providing a great deal of encouragement during the college search and application process.

In addition to preparing students for college, the department is charged with developing 

success plans for students who opt not to enroll in college after high school. Depending on the 

case, such postsecondary planning may include helping students prepare for a “gap year” or find 

employment that is aligned to their skill set.

A Network of Supportive Adults and Peers

SEED DC’s five-day-a-week residential model is designed to provide a supportive and integrated learning environment for students. Therefore, it is essential that the four core programming 

departments coordinate, both in ways that students can directly see and experience and in more 

behind-the-scenes ways related to student needs and progress. 

School-Wide Systems and Climate

To create consistency and provide an integrated learning environment, SEED uses a variety of school-wide systems that span the four programmatic departments. These practices in-

 6

An analysis of postsecondary retention and graduation data led the CTS Team to develop the “right fit” 

college criteria that the college counseling office uses in the advising process. 
21

clude the previously mentioned Model of Care, a set of core values, and school-wide projects 

such as Presentations of Learning, discussed below. 

The Model of Care is at its simplest a behavior management system, overseen by SEED 

DC’s dean of students, but it plays a significant role in defining SEED culture and climate. All 

adults at SEED are expected to provide consistent messaging around behavior and expectations 

using this model. As previously described, the Model of Care defines a set of age-appropriate 

social skills that all students should possess. Staff members frequently name these skills when 

redirecting or praising students, especially in the middle school grades. Middle school students 

even carry around a “SEED note” which serves as a record of their behavior throughout each 

day. They receive “skills strengths” when exhibiting social skills defined in the Model of Care, 

and “improvement areas” when they misbehave. Points are awarded for skills strengths, and 

such points can be exchanged for prizes and other rewards. 

SEED DC also uses five core values to guide everything that happens at the school, as 

well as to set priorities for students. These values — responsibility, respect, self-discipline, 

compassion, and integrity — are displayed prominently throughout the school and are integrated in annual student presentations called “Presentations of Learning,” or POLs. During the 

POL, a student gives a 30-minute presentation showcasing how he or she has applied the core 

values in academic and Student Life work and how he or she has grown over the year. The 

evaluation team observed two middle school student presentations. Unfortunately, neither of the 

students who were observed received high scores from the teachers who graded them. It was not 

clear to the evaluation team how much support the students had been given in preparation for 

their presentation.

Communication and Relationships Among the Staff and Students

With about 40 faculty and staff members in the academic department and about 50 staff 

members in the Student Life department, each student has access to a large network of adults 

when needed. Administrators, teachers, and students report that through formal and informal 

structures adults are actively involved in SEED students’ lives. For example, as previously mentioned, both academic and Student Life staff members review interim assessment data to keep 

track of student progress and brainstorm ways to better support them. This monitoring means 

that the adults who are responsible for supervising students in the evening are intimately involved in their academic progress. In addition, teachers e-mail middle school and ninth-grade 

students’ homework assignments to their resident advisers, along with updates on student behavior. In the morning, Student Life staff members brief the faculty about issues that may have 

surfaced in the dorms overnight. Student Life staff members even sometimes sit in on classes so 

that they can understand what the students are learning and be better prepared to assist them.
22

Faculty and staff members agreed that ongoing communication among the staff is critically important in supporting students’ success — and this communication appears to make it 

very difficult for students to “get lost” at the school. Student Life staff members reported that 

among students who shoulder adult responsibilities and pressures at home, the school is a safe 

haven; not only do students have access to nurturing and positive adult role models and champions focused on helping them succeed, but according to an RA, at SEED DC students can feel 

“I’m free to be me — a child.”

Weekly Transitions to and from SEED

Unlike at traditional boarding schools, SEED DC students leave campus every Friday 

afternoon and return on Sunday evening. Faculty and Student Life staff members reported that 

the weekend transition could be smooth for some students but challenging for others. For example, middle school students who may not have an enforced bedtime at home may struggle to 

adhere to the school’s bedtime curfew when they return to campus. Students who are not encouraged to do their homework at home or do not have someone there to help them may return 

to campus without having completed assignments. To help students ease back into the SEED 

DC structure when they return, some RAs hold a Sunday evening check-in to learn about what 

may have transpired over the weekend. Staff members reported that these sessions allow students to “vent” and offer useful information that often explains their behavior. These sessions 

also give staff members an opportunity to find ways to help students who may need support to 

meet SEED expectations when they return to campus. A Student Life staff member stated, 

“[Y]ou’re able to bridge that gap. You’re able to find out okay, well, this weekend they went 

through X, Y, and Z so they’re having a difficult day so now what can we do? You’re able to be 

more solution-oriented.” While the transition can be tough for some students, Student Life staff 

members explained that students view their peers as family, and when they return to campus 

they are often excited about returning and are eager to share updates about their weekend experience. 

In focus groups, student participants explained that the transition could be challenging 

in other ways. All students reported that they do not have enough time on the weekends to 

spend it as they like, and a few mentioned not having enough time to devote to their relationships at home. Two ninth-grade students explained that after spending so many years at SEED 

DC, they noticed that they had “separated” or become distant from family because they were 

not home during the week to interact with their relatives.7 High school students also cited different school schedules as a factor preventing them from spending as much time with friends at 

home as they would like.

 7

One student suggested that the school hold a workshop to help students handle these changes in their personal relationships.
23

Chapter 3

The Academic and Behavioral Impacts of SEED

This chapter focuses on the academic and behavioral impacts of SEED on its students. It begins 

by briefly describing the study’s random assignment process and the characteristics of the study 

sample as a whole and explains the construction of the three analytic samples used in this research. The chapter then describes the short- and long-term impacts of SEED on students’ 

school experiences and academic and behavioral outcomes. SEED’s effectiveness is primarily 

measured using students’ scores on state reading and math exams and four-year high school 

graduation rates. Students’ survey responses regarding their school experiences and behaviors 

are secondary outcomes in this evaluation and are presented as a means of putting the pattern of 

academic effects in context. 

Random Assignment and the Study Sample

As mentioned in Chapter 1, the research team took advantage of the public SEED charter school 

admissions process to identify eight cohorts of students who had competed in a lottery for the 

opportunity to attend SEED.1 The students who won the lottery are known in experimental research as the “treatment group” and are referred to in this text as the SEED group; the students 

who lost the lottery are known as the “control group,” or the non-SEED group. Because a lottery is a form of random assignment, the two groups should be comparable, and therefore any 

difference in study outcomes between the two groups, on average, can be attributed to SEED.

The research team’s analytic approach is described in greater detail in Appendix D. 

Table 3.1 indicates that students in the study sample were primarily African-American 

and were economically and academically disadvantaged. Of the SEED group, roughly 46 percent of the students had direct certification status, meaning they automatically qualified for a 

free lunch because they were in foster care, were homeless, or were living in a household that 

received income or nutritional assistance from certain government programs.

2 An additional 33 

percent of the SEED group students qualified for free or reduced-price lunch on the basis of

household income level; in sum, 79 percent of the students qualified for free or reduced-price 

lunch. In terms of their academic characteristics, in the year they applied to SEED 14 percent of 

the SEED group students qualified for special education services, 46 percent scored at or above

 1

The SEED admissions process is described in greater detail in Appendix C.

2

Benefits that qualify a household for direct certification are the Supplemental Nutrition Assistance Program (SNAP), the Food Distribution Program on Indian Reservations (FDPIR), and Temporary Assistance for 

Needy Families (TANF).
24

SEED Evaluation Final Report

Table 3.1

Baseline Characteristics of SEED Lottery Participants: Full Sample

Characteristic

SEED

Group

Non-SEED Estimated

Group Difference

P-Value for

Estimated

Difference

Race (%)

Black 98.8 98.4 0.4 0.695

Female (%) 49.1 49.1 0.0 1.000

Economic indicator (%)

Eligible for free/reduced-price lunch

a Direct Certification status

78.5

45.9

87.4

45.1

-8.9 **

0.8

0.017

0.891

Special education status (%) 13.6 16.9 -3.3 0.360

Standardized math score 2.1 -11.0 13.1 0.194

At or above proficiency level in math (%) 46.1 39.4 6.7 0.287

Standardized reading score 6.6 -8.8 15.4 0.133

At or above proficiency level in reading (%) 47.0 40.7 6.4 0.309

Sample size 499 267

SOURCES: MDRC’s calculations using Office of the State Superintendent of Education (OSSE) 

student enrollment data and state test scores from the 2006-2007 to 2011-2012 school years.

NOTES: Values for SEED group members are the simple means or percentages. Values for the 

difference between SEED group members and non-SEED group members are obtained from a 

regression of a given baseline characteristic on a series of indicator variables that identify each lottery, 

a covariate indicating the probability that a student would win a spot on lottery day, and an indicator 

variable that equals 1 for lottery winners and 0 for lottery losers. The coefficient on the latter 

indicator variable equals the difference in the baseline characteristic between SEED group members 

and non-SEED group members. The value for non-SEED group members equals the corresponding 

value for SEED group members minus the estimated difference between SEED group members and 

non-SEED group members.

Data on students' income, proficiency in math, and proficiency in reading are not available for 

students in Cohorts 1 and 2.

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 

indicated as: ** = 5 percent; * = 10 percent. a

Direct Certification is automatic qualification for free lunch because the individual is in foster 

care, is homeless, or is living in a household receiving benefits from the Supplemental Nutrition 

Assistance Program (SNAP), the Food Distribution Program on Indian Reservations (FDPIR), or 

Temporary Assistance for Needy Families (TANF).
25

proficient on the state math exam, and 47 percent scored at or above proficient on the state reading exam. A comparison with the non-SEED group characteristics shows that the SEED group 

students were 8.9 percentage points less likely to have been eligible for free or reduced-price 

lunch (a statistically significant difference) and roughly 6 percentage points more likely to have 

scored at or above proficiency on their reading and math exams. Although random assignment 

is expected to produce two groups that are equivalent on all measurable characteristics, with 

small samples it is common to see differences such as these.3 Appendix D discusses this issue in 

greater detail. 

At baseline the academic characteristics of the SEED study sample appear similar to 

those of all students in District of Columbia Public Schools (DCPS), but the SEED study sample is slightly more economically disadvantaged. During the study period, on average, 75 percent of the students in DCPS were African-American and 43 percent of the students in DCPS

scored at or above proficient on their sixth-grade math exam, compared with 46 percent of the 

students in the SEED study sample. While sixth-grade-specific data on DCPS students’ family 

income levels are unavailable, during this time period roughly 71 percent of all the students in 

DCPS were eligible for free or reduced-price lunch, compared with 79 percent of the SEED 

study sample. It is important to note that in other cities SEED strives to work with an even 

greater proportion of students who are economically disadvantaged and gives these students 

preference during the school enrollment process. SEED DC is unable to do so given the District 

of Columbia’s strict charter school enrollment guidelines. 

The Three Analytic Samples

Originally students entered SEED as seventh-graders, but in the spring of 2009 SEED changed 

its model and began to enroll sixth-graders as well. For the next two years, students could enter 

SEED as either sixth- or seventh-graders. Then, in spring 2011, SEED fully transitioned to its 

new model and admitted students only as sixth-graders. As a result, the study sample consists of 

eight cohorts of students who competed in a lottery to attend SEED across six school years. Cohorts 1-4 contain students who competed in a lottery to begin SEED as seventh-graders in 2006-

2007 through 2009-2010. Together, these four cohorts are referred to as the Seventh-Grade Entrant Sample. Cohorts 5-8 contain students who competed in a lottery to begin SEED as sixthgraders from 2008-2009 through 2011-2012. These cohorts are referred to as the Sixth-Grade 

Entrant Sample. The third analytic sample, consisting of cohorts 1 and 2, is described below.

Students in the Seventh-Grade Entrant Sample were analyzed separately from those in 

the Sixth-Grade Entrant Sample for a few reasons. First, since traditional middle schools begin 

 3

In accordance with What Works Clearinghouse (2014) recommendations, these characteristics are included as covariates in the impact model. 
26

in sixth grade, students who wanted to leave their middle school and enter SEED as seventhgraders may be a self-selected sample of students who were particularly dissatisfied with their 

current middle school. Second, as students are changing rapidly during early adolescence, there 

may have been developmental differences between those who entered SEED as sixth-graders 

and those who entered as seventh-graders, and those differences may have affected how they 

responded to SEED’s unique environment and programming. Finally, since SEED changed its 

model when it began admitting sixth-graders, the two different samples may have experienced 

slightly different versions of the SEED program. 

The research team collected school records data from the District of Columbia’s Office 

of the State Superintendent of Education (OSSE).4 These data consisted of statewide student 

enrollment records, state exam performance data (each year the District of Columbia administers statewide reading and math exams to sixth-, seventh-, eighth-, and tenth-graders), student 

demographic data, and information on whether and when a student graduated from high school. 

Box 3.1 offers more information on these and other outcomes included in our analysis. These 

data were available for all DC students attending public schools — District of Columbia Public 

Schools and charter schools. The team collected these data for the 2006-2007 school year 

through the 2013-2014 school year. 

Table 3.2 shows the OSSE school records data follow-up period, by cohort, for the Seventh-Grade and Sixth-Grade Entrant Samples; the years in which students took the OSSE state 

exam are in bold. Three years of OSSE test score data are available for all four cohorts of the 

Seventh-Grade Entrant Sample and for three of the four cohorts in the Sixth-Grade Entrant 

Sample. The research team was able to follow a subsample of the Seventh-Grade Entrant Sample, students in cohorts 1 and 2, for six years, through their anticipated four-year high school 

graduation; this group is referred to as the Cohorts 1 and 2 Sample. Analyses in this chapter 

draw on the full Seventh- and Sixth-Grade Entrant Samples to estimate the “short-term” effects 

of SEED and the Cohorts 1 and 2 Sample to estimate the “long-term” effects of SEED.5

 Because of their different sizes, these samples have differing levels of statistical power (the ability 

to detect a true impact with statistical significance). Specifically, the full Seventh- and SixthGrade Entrant Samples are larger, enabling the study to detect impacts of roughly 0.14 standard 

deviation at a 95 percent probability level, while the Cohorts 1 and 2 Sample requires an impact 

of 12.7 percentage points on four-year high school graduation to show statistical significance. 

Although these minimum detectable effects would be characterized as moderate to large by education researchers, they are, arguably, not unreasonable for such an intensive intervention. Because of these already small sample sizes we are unable to explore whether the effects of SEED 

differ across student subgroups. 

 4

“State” refers to the District of Columbia.

5

The baseline characteristics of the Cohorts 1 and 2 Sample are presented in Appendix D; they appear similar to those of the full sample presented in Table 3.1. 
27

Box 3.1

Definitions of Study Outcomes

Academic Achievement Outcomes

•

•

•

Standardized DC CAS scaled scores. During the evaluation period, the District of 

Columbia used the DC Comprehensive Assessment System (DC CAS) to measure 

students’ mastery of the DC Content Standards. Students took the DC CAS math test 

in grades 3 through 8 and grade 10, and reading tests were administered in grades 2

through 10. (In grades 2 and 9, scores were reported only internally.) Students in 

grades 5 and 8, as well as those enrolled in Biology, also took the science test. Students in grades 4, 7, and 10 took the composition test. Finally, the health test was administered in grades 5 and 8 and during the high school year in which students take 

health class. This outcome measure reports student performance using their standardized scaled scores.

At or above DC CAS proficiency level. Student performance on the DC CAS is also 

reported using proficiency levels determined by standardized scaled scores: advanced 

(exceeding state standard), proficient (meeting state standard), basic (not meeting state 

standard), and below basic (not meeting state standard). 

Four-year high school graduation. This measure indicates whether a student graduated from high school within four years.

Student Experience Outcomes

• Courses taken. The outcome includes the number of Advanced Placement or honors 

courses students took, as well as an indicator of whether they took other important 

high school courses, such as Calculus and Statistics.

•

•

•

•

Extracurricular activities. Students were asked whether they participated in athletics, 

clubs, and so on, during the 2012-2013 school year or the last year that they attended 

school.

Academic support. Students were asked whether they received support such as tutoring and personalized academic counseling during the 2012-2013 school year or the 

last year that they attended school.

College preparatory activities. Students were asked whether they participated in activities to prepare them for college, such as visiting college campuses and learning 

about the admissions process, during the 2012-2013 school year or the last year that 

they attended school.

Work-readiness activities. Students were asked whether they participated in activities 

related to career exploration or work readiness, such as working as an intern or shadowing someone on a job, during the 2012-2013 school year or the last year that they

attended school.

(continued)
28

Box 3.1 (continued)

• School atmosphere. Students were asked about their perceptions of their school during the 2012-2013 school year or the last year they attended school — for example, 

whether they had orderly classrooms and whether they had a caring adult at school. 

• Living environment during the week. Students were asked about their perceptions 

of the place that they lived during the week in the 2012-2013 school year or the last 

year that they attended school. For example, they were asked whether they felt that 

they had adult support during the week and whether they ate three meals per day.

Behavioral Outcomes 

•

•

•

•

•

School behaviors. Students were asked about certain behaviors related to academic 

success, including study habits and organizational skills. 

Time use. Students were asked how many hours they spent doing various activities 

during the week, such as homework, reading, and video games.

Likelihood of attending college. Students were asked how likely they were to attend 

college.

Risky behaviors. Students were asked how many times they drank alcohol or skipped

school, as well as their overall engagement in risky behaviors — such as having unprotected sex or getting involved with the criminal justice system — in the years since 

random assignment.

Metacognitive skills. Students’ responses to questions about their personal characteristics and outlook were scored on scales used to measure “grit” and self-control.

Short-Term Effects of SEED

Difference in Student Experience

The District of Columbia is an especially charter-rich environment, and many of the 

students who applied to SEED but did not win the lottery actively sought out other innovative 

school options. Specifically, of the SEED lottery losers, 43 percent of the students enrolled in 

charter schools and a little more than half enrolled in traditional DC middle schools. Thus in this
29

SEED Evaluation Final Report

Table 3.2

Follow-Up Periods by Cohort

Cohort Lottery Year 2006-2007 2007-2008 2008-2009 2009-2010 2010-2011 2011-2012 2012-2013 2013-2014

7th-grade entrants

1 2007 Lottery year year 1 year 2 year 3 year 4 year 5 year 6

(6th grade) (7th grade) (8th grade) (9th grade) (10th grade) (11th grade) (12th grade)

2 2008 Lottery year year 1 year 2 year 3 year 4 year 5 year 6

(6th grade) (7th grade) (8th grade) (9th grade) (10th grade) (11th grade) (12th grade)

3 2009 Lottery year year 1 year 2 year 3 year 4 year 5

(6th grade) (7th grade) (8th grade) (9th grade) (10th grade) (11th grade)

4 2010 Lottery year year 1 year 2 year 3 year 4

(6th grade) (7th grade) (8th grade) (9th grade) (10th grade)

6th-grade entrants

5 2009 Lottery year year 1 year 2 year 3 year 4 year 5

(5th grade) (6th grade) (7th grade) (8th grade) (9th grade) (10th grade)

6 2010 Lottery year year 1 year 2 year 3 year 4 

(5th grade) (6th grade) (7th grade) (8th grade) (9th grade)

7 2011 Lottery year year 1 year 2 year 3

(5th grade) (6th grade) (7th grade) (8th grade)

8 2012 Lottery year year 1 year 2

(5th grade) (6th grade) (7th grade)

NOTE: The follow-up periods that coincide with years in which students took the Office of the State Superintendent of Education (OSSE) 

state exam are in bold.
30

study the SEED student experience was compared with a diverse set of other school experiences 

that may not exist in other contexts. 

Not all the students who won the SEED lottery enrolled in SEED the following fall. 

Specifically, 18 percent of the Seventh-Grade Entrant Sample’s lottery winners and 21 percent 

of the Sixth-Grade Entrant Sample’s lottery winners did not enroll in SEED. Sixty-three percent 

of these students enrolled in other charter schools, and the remainder enrolled in traditional DC 

middle schools. Even though some of the SEED lottery winners did not enroll in SEED, this 

chapter presents findings from an analysis of the difference in outcomes between all the SEED 

lottery winners and all the SEED lottery losers. Known as an “intent-to-treat” approach, this 

analysis is described as the impact of being offered the opportunity to enroll in SEED. Appendix E presents findings from an exploratory analysis of the effect of enrolling in SEED, known 

as “treatment on the treated.” Generally, estimates of the effect of enrolling in SEED are about 

15 percent to 20 percent larger in magnitude than the estimated intent-to-treat effects reported in 

the remainder of this chapter, but the overall pattern of effects and statistical significance is the 

same.

Effects on Academic Achievement

In each follow-up year, students’ academic achievement was measured by their performance on the District of Columbia’s statewide reading and math exams. After the first followup year some students progressed into the next grade while others did not. In order to keep the 

analysis sample intact, the research team reports each student’s exam score as a z-score, which 

is standardized on the sample mean and standard deviation of the student’s grade level; proficiency levels are also based on the exams students actually took. (To be consistent over time, 

students’ scores in the first follow-up year are standardized in this manner as well.) The estimated difference in standardized math scores is referred to as the “effect size.”

Findings for the Seventh-Grade Entrant Sample in Table 3.3 indicate that on average, in 

the first two years of follow-up, being offered the opportunity to attend SEED increased students’ academic achievement in math. The first two rows in the table present findings from an 

analysis of students’ performance on the OSSE math exam in the first follow-up year. On average, students in the SEED group scored 0.14 standard deviation above the sample mean, while 

students in the non-SEED group scored 0.11 standard deviation below the sample mean, resulting in an estimated difference of 0.24 standard deviation. The typical annual gain for seventhgraders in math is 0.32 standard deviation.6 Thus, the SEED effect of 0.24 standard deviation is 

roughly equivalent in magnitude to 75 percent of the typical annual gains for this age group. 

Said differently, at the end of the first follow-up year, students in the SEED group were roughly

 6

Hill, Bloom, Black, and Lipsey (2008). 
31

SEED Evaluation Final Report

Table 3.3

Estimated Effects of Being Offered the Opportunity to Attend SEED,

Seventh-Grade Entrant Sample

P-Value for

SEED Non-SEED Estimated Estimated

Outcome Group Group Difference Difference

a First year

Standardized math score 0.14 -0.11 0.24 ** 0.027

At or above proficiency level in math (%) 66.3 48.8 17.5 ** 0.035

Standardized reading score 0.05 0.04 0.01 0.920

At or above proficiency level in reading (%) 44.9 49.8 -4.9 0.532

Second year

Standardized math score 0.14 -0.18 0.33 ** 0.012

At or above proficiency level in math (%) 67.4 46.2 21.2 *** 0.001

Standardized reading score 0.08 -0.15 0.23 * 0.059

At or above proficiency level in reading (%) 55.6 37.9 17.6 *** 0.007

Sample size 184 165

SOURCES: MDRC’s calculations using Office of the State Superintendent of Education (OSSE) state

test scores from the 2007-2008 to 2011-2012 school years.

NOTES: The Seventh-Grade Entrant Sample consists of 349 students who participated in a lottery for 

SEED in the spring of 2007, 2008, 2009, or 2010.

Values for SEED group members are the simple means or percentages. Values for the difference 

between SEED group members and non-SEED group members are obtained from a regression of a 

given outcome on a series of indicator variables that identify each lottery, a student's baseline reading 

score, a student's baseline math score, indicators of family income, and an indicator variable that 

equals 1 for lottery winners and 0 for lottery losers. The coefficient on the latter indicator variable 

equals the difference in outcome between SEED group members and non-SEED group members. The 

value for non-SEED group members equals the corresponding value for SEED group members minus 

the estimated difference between SEED group members and non-SEED group members.

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 

indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. a

First-year DC Comprehensive Assessment System (CAS) data are only available for students who 

participated in a lottery for SEED in the spring of 2008, 2009, or 2010.
32

three-quarters of a year ahead of students in the non-SEED group in math. According to 

OSSE’s proficiency benchmark, this increase in SEED group students’ math scores translated 

into an 18 percentage point increase in the proportion of students scoring at or above proficient. 

Both of these effects are statistically significant. 

While students in the SEED group did not perform better than students in the non-SEED 

group in their first year of follow-up on the OSSE reading exam, they performed higher on both 

their math and reading exams in the second follow-up year. The second panel in Table 3.3 indicates that in the second follow-up year, being offered the opportunity to attend SEED increased 

students’ test scores over those of the non-SEED group by the equivalent of one and a half years 

of typical growth in math and one year of typical growth in reading (effect sizes = 0.33 and 0.23, 

respectively).

7 According to the OSSE benchmarks, SEED group students were 21 percentage 

points more likely to be at or above proficient in math and 18 percentage points more likely to be 

at or above proficient in reading. Both of these effects are statistically significant.

Findings in Table 3.4 indicate that students in the Sixth-Grade Entrant Sample experienced a similar, though slightly weaker, pattern of positive SEED effects. Specifically, similar to

the seventh-grade entrants, in the first follow-up year students in the SEED group scored 0.18 

standard deviation higher on their math exam than students in the non-SEED group, an effect 

equivalent in magnitude to 60 percent of the typical annual gains for this age group.

8 According 

to OSSE’s proficiency benchmark, this increase in SEED group students’ math scores translated 

into an 19.6 percentage point increase in the proportion of students scoring at or above proficient. 

The effect on students’ standardized math scores does not quite meet this report’s threshold for 

statistical significance, but the effect on students’ proficiency levels is statistically significant. 

Like the seventh-grade entrants, SEED group students in the Sixth-Grade Entrant Sample did not perform better than students in the non-SEED group in their first year of follow-up 

on the OSSE reading exam, but in the second follow-up year they performed higher on both 

their math and reading exams. The second panel in Table 3.4 indicates that in the second follow-up year, being offered the opportunity to attend SEED increased students’ test scores by the 

equivalent of roughly 1.2 years of growth in math (effect size = 0.39) and 81 percent of the typical gains for this age group in reading (effect size = 0.21).9 Unlike the Seventh-Grade Entrant

Sample findings, these test score gains did not translate into a large increase in the proportion of 

students passing the OSSE proficiency benchmark. SEED group students were only 6.3 percentage points more likely to be at or above proficient in math and 4.6 percentage points more 

likely to be at or above proficient in reading. Neither of these effects is statistically significant.

 7

Hill, Bloom, Black, and Lipsey (2008).

8

Hill, Bloom, Black, and Lipsey (2008).

9

Hill, Bloom, Black, and Lipsey (2008).
33

SEED Evaluation Final Report

Table 3.4

Estimated Effects of Being Offered the Opportunity to Attend SEED,

Sixth-Grade Entrant Sample

Outcome

SEED

Group

Non-SEED

Group

Estimated

Difference

P-Value for

Estimated

Difference

First year

Standardized math score 0.06 -0.11 0.18 0.116

At or above proficiency level in math (%) 49.8 30.2 19.6 *** 0.002

Standardized reading score 0.06 -0.01 0.07 0.539

At or above proficiency level in reading (%) 34.2 34.2 0.0 0.998

Second year

Standardized math score 0.13 -0.27 0.39 *** 0.001

At or above proficiency level in math (%) 63.4 57.1 6.3 0.304

Standardized reading score

At or above proficiency level in reading (%)

0.09

48.6

-0.12

44.0

0.21 *

4.6

0.072

0.478

a Third year

Standardized math score 0.06 -0.13 0.19 0.181

At or above proficiency level in math (%) 70.6 60.1 10.5 0.138

Standardized reading score -0.02 0.01 -0.03 0.789

At or above proficiency level in reading (%) 51.3 55.3 -4.0 0.570

Sample size 315 102

SOURCES: MDRC’s calculations using Office of the State Superintendent of Education (OSSE) state 

test scores from the 2009-2010 to 2013-2014 school years.

NOTES: The Sixth-Grade Entrant Sample consists of 417 students who participated in a lottery for 

SEED in the spring of 2009, 2010, 2011, or 2012.

Values for SEED group members are the simple means or percentages. Values for the difference 

between SEED group members and non-SEED group members are obtained from a regression of a 

given outcome on a series of indicator variables that identify each lottery, a student's baseline reading 

score, a student's baseline math score, indicators of family income, and an indicator variable that 

equals 1 for lottery winners and 0 for lottery losers. The coefficient on the latter indicator variable 

equals the difference in outcome between SEED group members and non-SEED group members. The 

value for non-SEED group members equals the corresponding value for SEED group members minus 

the estimated difference between SEED group members and non-SEED group members.

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 

indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. a

Third-year DC Comprehensive Assessment System (CAS) scores are only available for students 

who participated in a lottery for SEED in the spring of 2009, 2010, or 2011.
34

Surprisingly, when the Sixth-Grade Entrant Sample is followed into its third year, the 

second year’s positive effects in math return to levels slightly lower than those seen in the first 

year and the positive effects in reading disappear.10 The third panel in Table 3.4 indicates that in 

the third follow-up year, being offered the opportunity to attend SEED increased students’ test 

scores by the equivalent of roughly 86 percent of the typical gains for this age group (effect size 

= 0.19).

11 This increase in SEED group students’ math scores translated into a 10.5 percentage 

point increase in the proportion of students scoring at or above proficient. Neither of these effects is statistically significant, though the p-value of the effect on the percentage of students 

scoring at or above proficient in math is just above this report’s 0.10 benchmark for significance. The effect of SEED on students’ reading scores in the third year is unclear. Although the 

estimated difference between the SEED group and the non-SEED group appears negative on 

both reading measures, there is a high probability that the apparent effects are due to chance. 

Thus, it is difficult to determine whether the SEED effect on student’s reading scores persists, 

disappears, or changes direction in students’ third year.

To conclude, for students in both samples, being offered the opportunity to enroll in 

SEED led to a pattern of positive short-term effects. Students in each follow-up year experienced positive effects in math, while the effect on reading was most prominent in students’ second year. Although there is limited statistical power to explore any differences between cohorts, 

it appears that students in the Seventh-Grade Entrant Sample may have experienced slightly 

stronger SEED effects than students in the Sixth-Grade Entrant Sample. This may mean that 

SEED is more effective when students are offered the opportunity to enroll as seventh-graders, 

or it may mean that SEED became slightly less effective over time. 

Longer-Term Effects of SEED

The research team was able to follow a subsample of the Seventh-Grade Entrant Sample, students in cohorts 1 and 2, for six years, through their anticipated four-year high school graduation. When analyzed separately, students in these two cohorts experienced effects similar in 

magnitude and statistical significance to those experienced by all four cohorts of the SeventhGrade Entrant Sample in their first two years in math. Students in cohorts 1 and 2 experienced a 

somewhat smaller effect in reading in their second year than did the full Seventh-Grade Entrant 

Sample. 

 10Only cohorts 5, 6, and 7 can be followed into a third year. 11Hill, Bloom, Black, and Lipsey (2008).
35

Difference in Student Experience

In their third follow-up year students in cohorts 1 and 2 transitioned into high school.12 

As described in Chapter 2, SEED’s high school model is different from the middle school model in a few key ways — boys and girls are taught in the same classrooms, students’ time after 

school is less structured, and students begin more rigorous college preparatory activities. Between the second and third follow-up years, roughly 20 percent of the SEED lottery winners 

enrolled in SEED chose to leave and attend a different high school. Of this 20 percent, two out 

of five enrolled in a DCPS high school and the rest enrolled in another charter school. The students who chose to leave SEED at this point had, on average, seventh- and eighth-grade state 

test scores within 0.03 standard deviation of those who stayed at SEED, so there is no evidence 

that these students were academically struggling at SEED before their departure. As discussed 

in Appendix E, the proportion of SEED group students enrolled in SEED decreased over time. 

By twelfth grade, 44 percent of cohort 1 SEED group students and 63 percent of cohort 2 SEED 

group students were actually enrolled at the school. This issue is addressed further in Chapter 4. 

In the fall of 2013 the research team surveyed lottery winners and lottery losers in cohorts 1 and 2 and asked them a variety of questions about their experiences in school. Box 3.1 

provides more detail about the types of information captured by these questions and the relevant 

outcomes included in the analysis. At that point, most of the students in those cohorts were high 

school seniors or had recently graduated. Although the survey response rate was reasonably high 

(73 percent overall), the resulting small sample size decreases the study’s already limited statistical power. (The construction and administration of this survey is described in Appendix F.)

Overall, students in the SEED group reported experiencing a more rigorous and supportive academic environment. Specifically, while only a fraction of the measures reveal statistically significant differences, Table 3.5 shows that students in the SEED group took fewer Advanced Placement courses but a larger number of foreign language, advanced science, and advanced math 

courses; participated in more extracurricular activities; received more academic support from 

teachers and adults; and participated in more college preparatory and work-readiness activities. 

In addition, SEED group students reported having more orderly classrooms and more academically motivated peers. Students did not report a difference in weekday living environments. 

Effects on Academic Achievement

Findings for cohorts 1 and 2 in Table 3.6 indicate that on average, in the fourth followup year (tenth grade for most students), being offered the opportunity to attend SEED appears to 

have increased students’ academic achievement in math but not in reading. Specifically, in the 

fourth follow-up year students in the SEED group scored 0.20 standard deviation higher on their 

 12In the Cohorts 1 and 2 Sample, 10 percent of the students were retained in grade.
36

P-Value for

SEED Non-SEED Estimated Estimated

Outcome Group Group Difference Difference

Courses taken

Number of Advanced Placement/honors courses 1.50 1.90 -0.40 0.252

taken in high school (0-8)a

Number of years of foreign language taken (0-4)b 2.35 2.01 0.34 ** 0.050

Took physics in high school (%) 72.2 55.5 16.7 * 0.077

Took calculus in high school (%) 31.9 20.9 10.9 0.204

Took statistics in high school (%) 44.0 35.0 8.9 0.372

Activities at school

Extracurricular activities (1-5)c

Went on field trips 2.75 2.18 0.57 *** 0.006

Participated in sports/athletic teams at school 3.15 3.02 0.13 0.704

Participated in arts, music group, literary clubs 2.71 2.60 0.11 0.730

Participated in student leadership groups 2.54 1.98 0.57 * 0.074

or student government

Participated in community service activities 3.28 3.06 0.22 0.375

Academic support

Received tutoring/extra help from a teacher (1-5)c 3.19 2.86 0.33 0.205

Received tutoring/extra help from another adult (1-5)c 3.18 2.59 0.58 ** 0.034

Received personalized academic counseling (%) 80.2 78.0 2.3 0.784

Received help preparing for DC-CAS exam (1-5)c 2.73 2.54 0.18 0.506

College preparatory activities

Score on access to college preparatory 

activities scale (1-3)d 2.50 2.49 0.01 0.936

Received help preparing for ACT, SAT, PSAT,

or other college entrance exams (1-5)c 2.89 2.79 0.11 0.672

Went on college visits in the DC area (1-5)c 2.53 2.22 0.32 0.167

Went on college visits in another state (1-5)c 2.24 2.01 0.23 0.242

Work-readiness activities (1-5)c

Had an internship 2.35 1.80 0.56 ** 0.040

Participated in job shadowing or visits to 

workplaces 1.84 1.68 0.16 0.498

(continued)

SEED Evaluation Final Report

Table 3.5

Estimated Differences in Student Experience,

Cohorts 1 and 2
37

Outcome

SEED

Group

Non-SEED

Group

Estimated

Difference

P-Value for

Estimated

Difference

Table 3.5 (continued)

School atmosphere (1-4)

d Score on school comfort scale 3.06 3.11 -0.04 0.691

d Score on classroom order scale 2.83 2.59 0.25 * 0.063

d Score on peers' academic motivation scale

d Score on caring adult at school scale

d Score on school-wide future orientation scale

3.37

3.50

3.32

2.96

3.46

3.31

0.41 ***

0.04

0.01

0.003

0.716

0.929

Weekday home living environment

Score on order of home during the week scale (1-4)d

Score on caregiver support during the week scale (1-4)d

2.94

3.39

3.18

3.40

-0.24

-0.01

0.246

0.936

Days in last week ate breakfast (0-7)e 4.08 3.76 0.32 0.510

Days per week ate three meals per day (0-7)e 4.51 4.68 -0.18 0.704

Days per week engaged in physical exercise (0-7)e

Score on average weeknight sleep scale (1-7)f

4.45

3.69

3.98

3.91

0.46

-0.22

0.292

0.399

Sample size 121 93

SOURCE: MDRC calculations based on data collected in student survey.

NOTES: Cohorts 1 and 2 consist of 214 students who were sixth-graders in the springs of 2007 and 

2008, respectively.

Values for SEED group members are the simple means or percentages. Values for the difference 

between SEED group members and non-SEED group members are obtained from a regression of a 

given outcome on a series of indicator variables that identify each lottery, a student's baseline reading 

score, a student's baseline math score, indicators of family income, and an indicator variable that 

equals 1 for lottery winners and 0 for lottery losers. The coefficient on the latter indicator variable 

equals the difference in outcome between SEED group members and non-SEED group members. The 

value for non-SEED group members equals the corresponding value for SEED group members minus 

the estimated difference between SEED group members and non-SEED group members.

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are indicated 

as: *** = 1 percent; ** = 5 percent; * = 10 percent. a

This item is based on the following range: 0-8 courses.

bThis item is based on the following range: 0-4 years. c

These items are based on the following scale: 1: Never; 2: Once a term/semester; 3: Monthly; 4: 

Weekly; 5: Almost every day. Therefore, a higher score indicates a higher frequency.

dSee Appendix F for definitions of the scales used in this report.

e

This item is based on the following range: 0-7 days.

f

This item is based on the following scale: 1: 4 hours or less; 2: 5 hours; 3: 6 hours; 4: 7 hours; 5: 8 

hours; 6: 9 hours; 7: 10 hours or more.
38

math exam than students in the non-SEED group, an effect equivalent in magnitude to 1.4 times 

the typical annual gains for this age group.

13 According to OSSE’s proficiency benchmark, this 

increase in SEED group students’ math scores translated into a 13.8 percentage point increase in 

the proportion of students scoring at or above proficient. While the p-values of both these effects on students’ standardized math scores are just above this report’s 0.10 benchmark, they 

 13Hill, Bloom, Black, and Lipsey (2008).

P-Value for

SEED Non-SEED Estimated Estimated

Outcome Group Group Difference Difference

Fourth year

Standardized math score 0.39 0.19 0.20 0.257

At or above proficiency level in math (%) 66.7 52.9 13.8 0.168

Standardized reading score 0.18 0.24 -0.06 0.711

At or above proficiency level in reading (%) 61.1 60.1 1.0 0.914

Four-year high school graduation (%) 69.3 74.1 -4.8 0.592

Sample size 121 93

SEED Evaluation Final Report

Table 3.6

Estimated Effects of Being Offered the Opportunity to Attend SEED on

 Academic Achievement, Cohorts 1 and 2

SOURCES: MDRC’s calculations using OSSE state test scores from the 2010-2011 and 2011-

2012 school years.

NOTES: Cohorts 1 and 2 consist of 214 students who were sixth-graders in the springs of 2007 

and 2008, respectively.

Values for SEED group members are the simple means or percentages. Values for the 

difference between SEED group members and non-SEED group members are obtained from a 

regression of a given outcome on a series of indicator variables that identify each lottery, a 

student's baseline reading score, a student's baseline math score, indicators of family income, and 

an indicator variable that equals 1 for lottery winners and 0 for lottery losers. The coefficient on 

the latter indicator variable equals the difference in outcome between SEED group members and 

non-SEED group members. The value for non-SEED group members equals the corresponding 

value for SEED group members minus the estimated difference between SEED group members 

and non-SEED group members.

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 

indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent.
39

should not be discounted, as they are part of a pattern of positive math effects experienced by 

SEED group students. The effect of SEED on students’ reading scores in the fourth year is less 

clear, as the probability is high that the estimated effects are due to chance. 

The final row of Table 3.6 indicates that being offered the opportunity to attend SEED 

did not have an effect on students’ probability of four-year high school graduation. Specifically, 

69.3 percent of the students who were offered the opportunity to attend SEED as seventhgraders graduated from high school after four years (six years after the SEED lottery), while 

74.1 percent of the students in the non-SEED group did so.14 The difference between these two 

groups, 5 percentage points, is not statistically significant.

Effects on Student Behavior and Attitudes 

In addition to positively affecting students’ academic achievement, SEED aims to promote positive behaviors such as rigorous study habits and self-control while discouraging “risky” 

behaviors such as alcohol use and unplanned pregnancy (see Box 3.1 for more detail). Table 3.7 

indicates that SEED may have had small effects on some measures of student behavior, but only 

four are statistically significant. While SEED group students did not report having developed 

more rigorous study habits or organizational skills than non-SEED group students, they did report spending four more hours a week doing homework than non-SEED group students. Table 

3.7 also indicates that at the time of survey administration SEED group students may have used 

less tobacco in the past 30 days. At the same time, Table 3.7 also presents students’ scores on the 

recent risky behavior scale, which asked about specific behaviors, such as skipping school without permission, getting into a fight or argument with parents, or hitting someone, in the past three 

months. Here there is a statistically significant positive effect, indicating more frequent risky behavior among SEED group students. There also seems to be a negative effect of SEED on student reports of “grit.” This measure seeks to gauge students’ ability to stick with things over the 

long term by asking how often new ideas distract from previous ones, whether the student identifies as being a hard worker, and whether the student finishes what he or she begins and is diligent. This measure has been found to be a predictor of students’ future academic success.15 

 14Note that the four-year high school graduation rate computed for the SEED group student sample is 

based on students who entered a lottery for SEED in sixth grade. This graduation rate is calculated differently 

from the four-year high school graduation rate computed by OSSE based on SEED’s entering ninth-grade cohort; that rate was 89 percent for the cohort of 2009-2010 ninth-graders and was unknown for the 2010-2011 

ninth-graders, because the cohort size fell below OSSE’s sample size threshold of 25. 

15Duckworth, Peterson, Matthews, and Kelly (2007). 
40

Table 3.7

Estimated Effects of Being Offered the Opportunity to Attend SEED on

Behavior Outcomes, Cohorts 1 and 2

Outcome

SEED

Group

SEED Estimated

Group Difference

P-Value for

Estimated

Difference

School behaviors (1-4)a

Score on study skills scale

Score on organizational skills scale

b Time use

Hours spent doing homework per week

Hours spent reading for personal pleasure per week

Hours spent watching television per week

Hours spent playing video/computer games per week

Likelihood of attending college (1-4)c

a Risky behaviors

Score on recent frequency of risky behavior (1-5)

Practiced risky behavior (%)d

Had a baby/fathered a baby (%)

Alcohol use in past 30 days (1-7)e

Tobacco use in past 30 days (1-7)e

Alcohol use in lifetime (1-7)f

Marijuana use in lifetime (1-7)f

Metacognitive skills (1-5)a

Score on self-control scale

Score on grit scale

2.98

3.03

9.14

5.96

10.62

4.45

3.74

1.72

15.1

26.7

1.96

1.87

2.18

2.75

3.55

3.50

2.95

3.01

5.12

2.97

8.58

4.68

3.66

1.33

18.7

30.9

2.28

2.55

2.59

3.25

3.62

3.73

0.02

0.01

4.02 *

2.98

2.04

-0.23

0.08

0.39 **

-3.6

-4.2

-0.32

-0.67 *

-0.40

-0.50

-0.08

-0.23 *

0.800

0.917

0.051

0.280

0.495

0.894

0.471

0.030

0.515

0.857

0.291

0.100

0.214

0.277

0.553

0.080

Sample size 121 93

(continued)

SEED Evaluation Final Report
41

In conclusion, winning the lottery for SEED had a positive long-term effect on students’ 

math achievement but no effect on students’ long-term reading achievement or probability of 

four-year high school graduation. In addition, students who won the SEED lottery reported 

spending more time doing homework during the week but also engaging in slightly more risky 

behaviors, and they scored lower than students who lost the SEED lottery on a measure of grit. 

Table 3.7 (continued)

SOURCE: MDRC calculations based on data collected in student survey.

NOTES: Cohorts 1 and 2 consist of 214 students who were sixth-graders in the springs of 2007 and 2008, 

respectively.

Values for SEED group members are the simple means or percentages. Values for the difference between 

SEED group members and non-SEED group members are obtained from a regression of a given outcome on a 

series of indicator variables that identify each lottery, a student's baseline reading score, a student's baseline 

math score, indicators of family income, and an indicator variable that equals 1 for lottery winners and 0 for 

lottery losers. The coefficient on the latter indicator variable equals the difference in outcome between SEED 

group members and non-SEED group members. The value for non-SEED group members equals the 

corresponding value for SEED group members minus the estimated difference between SEED group members 

and non-SEED group members.

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are indicated as: ** 

= 5 percent; * = 10 percent. a

See Appendix F for definitions of the scales used in this report.

bThis item is based on the following range: 0-168 hours per week. cThis item is based on the following scale: 1: Definitely not; 2: Not very likely; 3: Sort of likely; 4: Very 

likely.

dPercentage represents those who answered “yes” to one of four questions on the overall engagement in 

risky behavior scale. See Appendix F for details. e

This item is based on the following scale: 1: 0 days; 2: 1 or 2 days; 3: 3 to 5 days; 4: 6 to 9 days; 5: 10 to 

19 days; 6: 20 to 29 days; 7: All 30 days.

f

This item is based on the following scale: 1: 0 days; 2: 1 or 2 days; 3: 3 to 9 days; 4: 10 to 19 days; 5: 20 to 

39 days; 6: 40 to 99 days; 7: 100 or more days.

43

Chapter 4

Conclusion

The preceding chapters present a mixed picture of the quality of SEED DC’s implementation 

and its impacts on student outcomes. For example, the evaluation team found that SEED DC 

creates a highly supportive environment for its students, with many opportunities for students to 

interact with caring adults. But classroom observations, while limited, found few examples of 

innovative, motivating instruction, particularly at the high school level.

Similarly, while the evaluation found that SEED DC led to significant, notable increases in standardized test scores and proficiency levels — particularly in math — the available evidence suggests that, among the subset of students whom researchers could follow through four 

years of high school, SEED DC did not increase the proportion who graduated on time. (This 

study’s follow-up period is too short to measure SEED’s impact on postsecondary outcomes.) 

There is also no strong evidence from the student survey that SEED DC affected key nonacademic outcomes such as the prevalence of risky behaviors. 

Some Factors to Consider

Several factors are critical to the interpretation of these findings: the program’s cost, the characteristics of students targeted for enrollment, the local educational context, the number of years 

spent in the school, and quality of the program’s implementation.

Cost

Owing to its boarding school model, SEED costs at least twice as much per student as a 

traditional nonresidential school. The earlier study of SEED DC by Curto and Fryer also found 

that SEED significantly increased students’ test scores but questioned whether those impacts 

were large enough to justify its very high cost.1 This evaluation does not include a benefit-cost 

analysis, but it does measure SEED’s impacts on four-year high school graduation and nonacademic outcomes such as teenage pregnancy and involvement in the justice system. These additional outcomes, which were not measured in the earlier study, have the potential to produce 

large societal benefits that could offset SEED’s cost. At this point, however, there is little evidence that SEED DC has affected either graduation or nonacademic outcomes, though it is important to note that the sample size for measuring these outcomes (including only about 200 

students in cohorts 1 and 2) is quite small. In addition, it is possible that longer follow-up would 

reveal additional impacts. For example, while SEED students may not be more likely to gradu-

 1

Curto and Fryer (2011).
44

ate high school, they may be better prepared for success in college. Nonetheless, based on the 

data available to date, the cost-effectiveness question raised by Curto and Fryer still seems highly relevant. 

Enrollment Targeting

SEED’s founders argue that the intensive, holistic boarding school model is needed for 

some students who face very serious obstacles to school success in their homes and communities. And, indeed, both the Maryland and Florida SEED schools are open only to students who 

meet certain specific criteria signifying severe disadvantage. In contrast, the DC school, the focus of this evaluation, is open to any student who resides in the District of Columbia. In general, 

the baseline characteristics of students in the study sample resemble those of the DC public 

school population as a whole during the study period — most students are African-American 

and low-income, and just under half were proficient in reading and math. In addition, test score 

data for the fourth year of follow-up (generally tenth grade) show that reading proficiency rates 

for the non-SEED group were roughly 17 percentage points higher and mathematics proficiency 

rates roughly 12 percentage points higher than for DC students overall. In other words, it does 

not appear that SEED DC is serving a particularly disadvantaged segment of the DC school 

population. It is possible that SEED’s model would produce larger impacts for students facing 

more serious obstacles to success.

Local Educational Context

In any rigorous evaluation it is critical to examine the “counterfactual,” or what would 

have occurred in the absence of the program, as illustrated by the services received by the control group. The District of Columbia has many innovative charter and magnet schools, and it 

appears that a large proportion of the students who lost the SEED lottery enrolled in these 

schools, particularly by the time they reached high school, when two-thirds of the non-SEED 

group were attending charter or magnet schools. It is possible that SEED would make a bigger 

difference in a context with fewer innovative alternatives. This point may also be related to the 

previous point: SEED may be serving a relatively motivated group of families who are likely to 

seek out other innovative options if they lose the lottery. Indeed, while one might assume that 

parents or guardians who are willing to send their child to a boarding school in sixth or seventh 

grade might be particularly concerned about the quality of their home life or neighborhood, it is 

equally likely that these parents are extremely motivated to ensure that their child has access to 

the best, most supportive school available to them. 
45

Length of Stay at SEED

SEED DC’s impacts on academic proficiency are substantial, particularly in the middle 

school years, and other literature suggests that middle school test scores are highly predictive of 

high school graduation. And yet there is no evidence to date that SEED has increased high 

school graduation rates. This fact, coupled with data showing that a sizable portion of students 

left SEED before twelfth grade, raises the question of whether SEED could have larger, more 

sustained impacts if more students remained in the school longer and received a larger “dose” of 

SEED — in other words, whether the school needs to focus mainly on promoting retention.

This question is a difficult one to answer. We do not know exactly why lottery winners left 

SEED, or how they would have performed if they had stayed. For example, some of those who 

left may have decided that they wanted a more traditional high school environment and would 

have seen their performance decline if they had stayed at SEED. On the other hand, since SEED 

had a large effect on students in their middle school years, if students had stayed they might 

have experienced continued positive effects.

Implementation Quality

Impacts are driven not only by a program’s model but also by how well that model is 

implemented in reality. While the implementation study noted many positive qualities of SEED 

DC’s operation, it also raised questions about the quality of instruction in SEED DC’s classrooms — particularly at the high school level — and noted that many students seemed to be 

struggling with the transition from eighth grade to ninth grade, even though they remained at the 

same school. The SEED Foundation has recently announced a renewed and intensified focus on 

the quality of instruction, leadership, and services at its schools. For example, the foundation 

reports that SEED DC has identified new curriculum resources to strengthen the middle school 

math program and is seeking to increase student engagement through interactive learning technologies. Given these and other ongoing changes, it is possible that impacts on student outcomes — and, perhaps, retention rates — will be stronger in the future. The SEED Foundation 

reports that its new approach is being implemented successfully in its new Miami school. For 

more information about that school and its early experiences, see Appendix A. 

Summary

SEED DC’s significant, positive effects on students’ academic achievement are notable. But 

given the lack of strong evidence of impacts on high school graduation or key nonacademic 

outcomes, the question about SEED’s cost-effectiveness, raised in earlier literature, remains 

relevant. 
46

Similarly, the evaluation leaves open the difficult question of whether a boarding environment is appropriate or necessary for disadvantaged students to succeed. Some have criticized 

this approach for its implicit assumption that certain low-income parents are not able or willing 

to provide a sufficiently supportive environment for their children, and have suggested that removing children from their home may cause stress to families. Others note that low-income 

children should have access to the same kinds of intensive, round-the-clock supports that upperincome children can obtain at private boarding schools, and that in some cases having an adolescent child live away during the week may reduce stress in the home. 

With all that said, it is important to note that these results pertain only to a specific period in the life of SEED DC, which operates in a particular context and, owing to local rules, is 

unable to give priority to the most disadvantaged students. Thus, the evaluation does not speak 

to the question of whether SEED’s unusual boarding school model can produce larger effects in 

different environments, with a more targeted group of students.
Appendix A 

School Program Improvements, As Described by SEED

49

1776 Massachusetts Avenue, N.W., Suite 600 | Washington, DC 20036 | 202.785.4123 | seedfoundation.com 

SEED School Growth

In the past year, SEED expanded beyond Washington, D.C., and Maryland and opened its third school in 

Miami, Florida. Many lessons learned from SEED’s work with students in D.C. and Maryland were 

applied to the program at SEED Miami. 

On August 17, 2014, SEED Miami opened with its first class of incoming sixth graders. The demographic 

breakdown of the founding sixth grade class included African American, Hispanic, and Haitian students. 

The majority of students in this group were from Miami-Dade County, with a select few from Broward 

County. Of the 62 entering students in fall of 2014, 56 remained enrolled for the entire school year. 

SEED Miami recruits students specifically from South Florida’s low-income underserved communities. 

Student eligibility criteria defined by Florida statute requires: residency of Florida and eligibility to 

attend school in participating school districts; eligibility for sixth grade during upcoming school year and 

born on or after February 1, 2003; family gross income at or below 200% federal poverty line; and 

eligibility for benefits or services funded by Temporary Assistance for Needy Families (TANF) or Title IV-E 

of the Social Security Act. 

In addition to the eligibility criteria, students must also meet at least one of the following at-risk factors: 

coverage under the terms of the state’s Child Welfare Waiver Demonstration project; in foster care or 

declared an adjudicated dependent; current head of household is not the student’s custodial parent; 

resident in a household receiving a housing voucher or eligible for public housing; or an immediate 

family member has been incarcerated. Furthermore, a third of the open slots at SEED Miami are 

reserved for children who have received or are receiving services from the child welfare system. 

Organizational Design 

Customizing its support to better serve SEED students, SEED Miami has a unique organizational 

structure. Unlike SEED DC and SEED MD, SEED Miami has a president who focuses on all operations and 

external relations functions such as board management, facilities, legislative advocacy, communications, 

and fundraising. This structure expands the capacity of the head of school to stay focused on the 

programmatic needs and outcomes of SEED’s 24-hour learning model. 

In addition to this strategic shift in organizational structure, the general process of hiring programmatic 

leaders for new schools has also been systematized. First, the head of school is hired approximately 14 

months prior to the opening of the school. Then, academic and student life directors are hired nine 

months before the schools opens. While a thorough search is conducted for leadership roles, SEED 

recognizes the importance of maintaining institutional knowledge and fidelity to the SEED way. Thus, 

SEED has nurtured internal talent to develop strong candidates for future leadership roles; the heads of 
50

1776 Massachusetts Avenue, N.W., Suite 600 | Washington, DC 20036 | 202.785.4123 | seedfoundation.com

school at SEED Miami and SEED MD were long-time leaders in SEED schools before transitioning to the 

head of school role. In addition to establishing a thorough process for identifying strong leadership, 

SEED also recognizes the importance of incorporating a year-long planning process prior to opening a 

new SEED school. This process enables school leaders to thoughtfully prepare to recruit students from 

underrepresented communities, implement SEED’s 24-hour learning program, and establish a 

performance plan to meet SEED’s network-wide goals and its school-specific goals. 

SEED Miami has also been deliberate in its approach to hiring to ensure that strong candidates are 

clearly aligned with the SEED model; in fact, mission-fit is a key criterion that is assessed at each stage in 

the application process. For the 2015-2016 SY, 95% of SEED Miami faculty who were asked to return 

have pledged to do so next year. 

Program Improvements

Based on academic lessons learned from SEED’s work with students in D.C. and Maryland, SEED Miami is 

directly addressing the need for writing support by deliberately separating the reading and writing 

curriculum. At SEED Miami, students are required to take three writing interim assessments prior to the 

administration of the Florida Standards Assessment (FSA).

1 The writing interim assessments cover the 

following three categories: (1) purpose, focus and organization, (2) evidence and elaboration, and (3) 

writing conventions. Each writing assessment requires students to respond to prompts using textual 

evidence from 2-3 provided supporting documents. These interim assessments are modeled after the 

FSA to provide staff with more information on which areas challenge students the most. These interim 

results inform instructional practices and form the basis of customized lesson plans. The trends in the 

fall and winter assessment scores this year indicate that students’ performance increased overall in all 

three writing categories. Further evidence of student improvement due to writing curriculum changes 

are reflected in the SEED Miami Spring 2015 ACT Aspire Summative writing assessment results. ACT 

Aspire Summative assessments are annual exams that measure a student’s proficiency in subject 

content to determine if a student is on track to be college- and career- ready. Compared to the sixth 

grade national average of 24% proficiency in writing, 40% of sixth grade SEED Miami students are 

proficient in writing according to ACT Aspire.

In addition to the curriculum improvements, other program updates are being implemented at SEED 

Miami. Notably, a six week onboarding process for all new programmatic employees offers orientation 

to the SEED mission and belief statements, Common Core and State Standards training, data analysis 

and action planning along with developmental assets overview. Also, regular home visits are conducted 

to promote the partnership between SEED and students’ families. Moreover, data is used intensively on 

a new platform to share students’ progress and, when necessary, to develop interventions accessible to 

families, students and SEED staff. 

1

The Florida Standards Assessments is Florida’s K-12 assessment system which measures student achievement of the state’s 

academic standards. These standards were developed to ensure students are prepared for success in college, career and life. 

Florida Department of Education, “Florida Standards Assessment” (2014), http://www.fldoe.org/accountability/assessments/k12-student-assessment/fsa.stml.
51

1776 Massachusetts Avenue, N.W., Suite 600 | Washington, DC 20036 | 202.785.4123 | seedfoundation.com

First-Year Outcomes

Within the reading curriculum, SEED Miami students are assessed on multiple factors, including reading 

level growth and English Language Learners’ (ELL) proficiency in English. Students’ reading level growth 

is assessed at the school level using the Fountas and Pinnell (F&P) reading system. SEED Miami assesses 

student reading levels at least four times during the school year; this is in addition to the New Student 

Orientation baseline assessment administered in July 2014. Despite reading three years below grade 

level, on average, at the time of enrollment, students at SEED Miami improved by one reading level in 

their first three months at SEED. Furthermore, all students grew two or more reading levels during their 

first year as depicted in Figure 1.2

 

8 

7 

6 

10 10

4 

9 

2 

6 

1 

0

2

4

6

8

10

12

# of 6th Graders

Growth in Reading Levels

The SEED School of Miami

F&P Reading Level Growth During 2014-2015 SY

Figure 1. According to the Fountas and Pinnell model, the pace and amount of growth in a single year 

depends on the starting reading level for each student and the teaching the student receives. Each 

successive level of the gradient makes greater and more varied demands on the reading process. 

Students typically advance through at least a few levels each year, but as the levels increase, so do texts’ 

difficulty; advancement through the higher levels sometimes takes longer than through the lower levels.3

In addition to F&P assessments, SEED Miami students also take the Comprehensive English Language 

Learning Assessment (CELLA). CELLA is a tool that the state of Florida utilizes to measure the progress of 

English Language Learners’ proficiency in English. This district-mandated, standardized assessment is 

administered every spring to ELL students to measure performance in reading, listening, and writing.

Seven SEED Miami sixth graders took the CELLA exam in spring 2015. Figure 2 (below) compares 

2

As cited in the SEED Miami Board of Trustees Report for 2014-2015 SY, Quarter 4. This chart totals 63 students which includes 

all children who were assessed more than one time (to include 1 withdrawn student).

3

Source: http://www.heinemann.com/fountasandpinnell/pdfs/whitepapertextgrad.pdf. 
52

1776 Massachusetts Avenue, N.W., Suite 600 | Washington, DC 20036 | 202.785.4123 | seedfoundation.com

students’ fifth grade scores prior to enrolling in SEED from spring 2014 to their sixth grade scores from 

spring 2015.

2126

2195

2287

2214

74

2211

85

2292

2263

23

2220

2224

2277

2250

2300

2350

ore

21

2128

21

2000

2050

2100

2150

2200

1 2 3 4 5 6 7

CELLA Sc

ELL Student

TOTAL SCALE 201

TOTAL SCALE 201

2400

37

The SEED School of Miami

2014 vs. 2015 CELLA Score Comparison

4

5

Figure 2. Overall, these seven SEED Miami sixth graders experienced overall gains in reading and 

individual growth in writing proficiency. Moreover, the seven sixth graders that took CELLA in spring 

2015 scored, on average, within the high intermediate range for reading and writing, meaning students 

are reading in English at grade level with minimal support. 
Appendix B

Implementation Research Data Collection and Analysis

55

Data Sources

To understand how SEED DC is structured and how it operated in practice during the three-year 

implementation study period, MDRC researchers conducted two-day site visits to SEED DC in 

spring and fall 2012 and fall 2013. During these site visits the research team requested and was 

granted interviews with staff members who had leadership and supervisory responsibilities for 

the various departments and programming, including the head of school, principal, director of 

Student Life, director of Student Support Services, director of College Counseling, middle and 

high school directors, and evening academic intervention coordinator, among others. MDRC 

also requested and was granted access to a cross section of the faculty, representing different 

grade levels and subject areas, recruited by SEED for the interviews. Finally, MDRC requested 

that SEED select a cross section of middle school, ninth-grade, and high school students to participate in grade-level focus groups. In some cases interviews were conducted by phone. These 

data collection activities were supplemented by interviews and focus groups with SEED Foundation staff members. Before these visits, researchers reviewed a variety of documents from the 

SEED Foundation and SEED DC, but the bulk of data collection occurred on site.1

In total, the team conducted interviews with 10 members of the SEED Foundation staff, 

22 members of the school’s administration and staff, and 13 members of the school’s faculty. 

(Over the course of the study, some individuals were interviewed more than once.) The team 

also “shadowed” three middle school students, one ninth-grade student, and three high school 

students, which involved accompanying the students to class and other activities, touring their 

dorm rooms, and eating lunch with the students and their friends. 

To break the process down by site visit, in spring 2012 the team conducted six interviews with SEED administrators and staff members and one focus group with two teachers. The 

team conducted observations of eight academic classes as well as other activities, such as the 

admissions lottery, student Presentations of Learning, Targeted Instruction, a junior-to-senior 

transition ceremony, a scholarship ceremony, and physical education. Four student shadows 

were also conducted.

During the fall 2012 visit, the team interviewed six SEED administrators and staff 

members and conducted one focus group with five Student Life staff members and one with

four middle school students. The students had enrolled in SEED in sixth grade and had been 

participating in the program for one to two years. The visit also included 10 classroom observations; an observation of Targeted Instruction; observation of three Student Life activities; one 

observation of outdoor, experiential education; and three student shadows. 

1

Following each site visit, MDRC provided the SEED Foundation with feedback memos that described 

the research team’s early impressions.
56

In fall 2013, the team conducted 10 phone interviews with SEED administrators and 

staff members. The fall 2013 school site visit included several focus groups: one with three 

middle school teachers, one with three ninth-grade teachers, one with three high school teachers, one with four instructional coaches, and one with five Student Life staff. Three student focus groups (one with four middle school students, one with six ninth-grade students, and one 

with six high school students) were also conducted. Most student participants had enrolled in 

the school in sixth grade, so middle school focus group participants had been at SEED for one 

to two years, ninth-grade students had been at SEED for three years, and high school students 

had been at SEED for four to six years. The site visit also included two observations of Student 

Life activities.

Data Collection and Analysis

As detailed above, MDRC researchers used semistructured interview and focus group protocols

to collect qualitative data from SEED DC administrators, staff members, and faculty members

and from SEED Foundation staff members. The purpose was to understand the school context, 

school structure, and adult perspectives on the key program components (academics, student 

life, support services, and college counseling) and to learn how these components interact. 

(Sample interview protocols appear at the end of this appendix.) These data also captured implementation changes from year to year. 

Similarly, student focus groups were conducted to gain an understanding of student 

perceptions about SEED DC and to learn more about students’ experiences at the school. The 

research team used focus group protocols to ask students about their decision to attend the 

school and the characteristics they found attractive about the school, their views on singlegender classes and the rigor of the curriculum, their experiences transitioning between middle 

school and high school, the role of Student Life and Student Support Services, their exposure to 

external opportunities, how they believed SEED prepared students for college, and the weekly 

transition to and from home on the weekends. (A sample high school focus group protocol follows the interview protocols at the end of this appendix.) As the students were chosen by 

SEED, the research team cannot know whether they were representative of the population as a 

whole. The research team did not compensate students, staff members, or faculty members for 

participating in the focus groups and interviews.

Informal classroom observations were conducted to gain a general understanding of 

how classes are structured, how teachers and students interact, and the types of instructional 

practices being used. In most cases, classes were observed for the full class period, although 

some classes were observed for less than the full period. In one instance a pair of researchers 

observed the same classroom, but in most instances each classroom was observed by one researcher. A formal observation rubric was not used. Instead, the researchers recorded their ob-
57

servations using handwritten notes that captured information about the structure of the classroom, the content covered, and the nature of teacher and student engagement, including classroom management practices tied to the use of the “SEED note” (in middle school classes) and 

the SEED core values, as described in Chapter 2. Observers also took notes on the kinds of instructional practices being used in the classroom. Following the observations, the team discussed what had been observed, identified themes across the set of observations, and identified 

clarifying questions for the SEED Foundation staff and SEED DC administrators and faculty. 

While the observations may be considered subjective and are based on a small number of middle school and high school classrooms, they are included to provide a general set of impressions 

about the SEED DC classroom experience on the days when classes were observed. Observations of academic classes were conducted in 10 middle school classrooms and 12 high school 

classrooms. 

In addition to conducting interviews, focus groups, and observations, researchers reviewed school and foundation documents. In the early stages of data collection, these data 

served primarily as background information on the SEED model and assisted the team in designing implementation data collection tools for site visits. The documents included an organizational chart, descriptions of SEED DC’s organizational departments, descriptions of staff positions, SEED’s belief statements, two research studies commissioned by the SEED Foundation 

on college success and boarding school models, and an overview of the SEED blueprint document process. Further along in data collection, the team requested concrete examples of student 

schedules, in particular to understand the Student Life component. Staff members at SEED DC 

provided these documents following interviews. Similarly, SEED Foundation staff members

provided documentation about SEED’s college matching approach after a phone interview on 

the topic.

Team members took notes during all data collection activities, and focus groups and interviews that were conducted in fall 2013 were audio-recorded and transcribed. The team did 

not use a qualitative analysis software program to analyze these data; instead, they were sorted, 

coded, and analyzed manually. The process for coding the interviews and focus groups involved 

multiple steps: First, all documents were reviewed: interview and focus group transcripts, which 

ranged from 15 to 40 pages, and interview and observation notes, which were less than 10 pages 

per session. Second, notes were taken on each document to highlight ideas and topics covered. 

This post-data collection documentation served as a general outline of relevant themes mentioned in each document and a useful reference guide to identify common themes across multiple data sources that could be coded together. The guide also helped researchers easily navigate 

the various data sources. Third, general codes were developed and assigned to specific text in 

each document. General codes reflected general themes outlined in the protocols (for example, 

academic support) and relevant themes that emerged from the data. Fourth, the general codes 

were reviewed and refined to represent subtopics within the larger theme (for example, academ-
58

ic support for struggling students). On occasion, these subtopics were further refined (for example, tutoring). Themes that emerged across more than one data source were combined, allowing

researchers to analyze the data from the perspective of multiple informants (such as faculty and 

students) and to conduct an analysis using multiple data sources (such as interviews and observations). This multistage process of sorting and coding the data informed the analysis presented 

in this report.

Examples of Interview and Focus Group Protocols

The following pages present interview protocols for the SEED DC head of school, principal, 

director of Student Life, high school program director, and director of College Counseling, and 

the high school student focus group protocol.
59

SEED Evaluation 

Interview Protocol for SEED DC Head of School 

Introduction

•

•

•

•

•

Thank respondent for his/her time.

Interviewer(s) introduces self and study.

Describe purpose of interview. (Mention follow-up to May 2012 interview.)

Confirm time allotted for interview.

Ask for permission to record the interview.

Roles and duties

1) We want to make sure we understand all of your roles and responsibilities at the school.

a. Please describe your roles and responsibilities for us.

2) In terms of reporting structures:

a. Who is your immediate supervisor?

b. Which staff members report to you?

3) Please describe the recent and impending changes to the DC School’s leadership.

a. What precipitated these changes?

b. Is there a transition plan in place to replace the principal?

c. What is the role of the Managing Director?

d. Are these changes reflected in an existing organizational chart that can be

shared with us?

Programmatic Leadership Team (PLT) structure

4) We want to make sure that we understand the structure of SEED’s programmatic leadership.

a. Please describe the composition of the Programmatic Leadership Team (PLT).

i. How often does the PLT meet?

ii. Who is responsible for leading the meeting?

iii. What topics or issues are discussed in the meeting?

Programmatic content 

5) Does SEED have a specific philosophy about programming?

6) What are the goals of the school’s student programming?

a. What are the specific goals of the model of care?

7) In your opinion, is the SEED programmatic experience different than that of other

schools in the area?

a. How does it differ?
60

8) How does the SEED programmatic approach support a “college prep” curriculum?

a. Probe:

i. In terms of programmatic activities.

ii. In terms of promoting a college-going culture.

Student Life

9) Please describe the role of the following in the SEED model:

a. Student Life programming

b. Gender-specific activities

10) Can you please explain the founders’ desire to make SEED a boarding/residential

school?

a. Do you believe it is effective?

i. If yes, why

ii. If no, why not

11) Have you been satisfied with the Student Life restructure implemented last year?

a. What has worked well?

b. Do area for improvement still exist?

Student Support Services 

12) How do you think the Student Support Services component of affects students’ experience at SEED?

Goals for SEED students’ academic and postsecondary preparation and transition 

13) Does SEED have a specific philosophy about postsecondary readiness, access and success?

14) What are the school’s postsecondary goals for students?

a. Is there an expectation that all students will attend college? Why or why not?

b. Are there specific types of colleges that SEED targets for its students? Why?

15) Are Student Life staff expected or instructed to talk about college and college-going in

a certain way?

16) How many students take the PSAT and/or the SAT/ACT in preparation for applying to

college?

17) Please describe the college matching process at SEED.

a. Who developed the college-matching process at SEED?

18) In general, do you feel that SEED is successful in its college-preparatory mission?

a. Are there areas in which you’d like the school to improve?
61

19) How do SEED students fare in college? 

a. What do the data suggest about postsecondary retention and graduation? 

b. Are you satisfied with the data?

i. If no, what steps is the school taking to improve upon the data? 

20) What specific supports are in place for students who do not plan to attend college? 

Students

Let’s discuss the average SEED student.

21) How would you describe the SEED student population? 

a. Probe: 

i. In terms of demographics (race, gender, SES) 

ii. Do they represent a specific geographic location?

22) Does the school target students during recruitment? 

a. Are students recruited from a specific set of elementary schools?

b. Does SEED make special efforts to enroll “at-risk” students? 

i. If yes, how do you define “at-risk”?

23) How would you describe the rate of enrollment from winning the lottery? 

a. When someone applies and is offered a spot, do they generally end up attending? 

b. How do parents respond to the boarding component? 

i. Do they view it as an asset? A deterrent?

24) In your opinion, what makes SEED attractive to parents? 

25) Once enrolled in SEED, are there a set of challenges that SEED students often or typically face?

a. Probe: 

i. In terms of academics

ii. Boarding feature 

iii. Lack of exposure to more diverse student populations

1. Can you tell us more about the school’s efforts to promote exposure to other students and experiences outside of SEED? 

26) What do you think is(are) the biggest challenge(s) facing SEED students?

a. Probe: 

i. How does the school attempt to address these issues?
62

27) What proportion of an entering cohort of 6th graders remains through graduation?

a. Probe:

i. For what reasons do students typically leave?

ii. What proportion of students is asked to leave the school?

Family engagement 

28) What are parents’/guardians’ and families’ expectations for their child at SEED?

a. Why do you think they choose SEED?

29) How involved are most parents throughout the year?

30) How does SEED communicate with parents about students’ progress?

31) Does SEED monitor students and parents’ usage of Power School to keep track of student progress?

a. Do you think parents are using this opportunity?

Foundation relationship

Let’s discuss your working relationship with the Foundation. 

32) How often are you in communication with Foundation staff?

a. Who is your main contact(s)?

b. In what specific ways does the Foundation support your work as Head of

School?

c. Are you working with Mary Lease at the Foundation in the development of the

SEED Blueprint?

i. Please tell us about this process.

Conclusion

33) What are your biggest challenges as Head of School?

a. Probe

i. Related to students’ academic success.

ii. Related to leading a boarding school.

iii. Related to leading a charter school.

34) What are the key challenges you expect the school to face in the future?

35) What have been your biggest successes/accomplishments to date as Head of School?

36) Is there anything we haven’t talked about that would be important for us to know about

SEED and its progress to date?

Thank you for your time. 
63

SEED Evaluation 

Interview Protocol for SEED DC Principal 

Introduction

•

•

•

•

•

Thank respondent for his/her time.

Interviewer(s) introduces self and study.

Describe purpose of interview. (Mention follow-up to May 2012 interview.)

Confirm time allotted for interview.

Ask for permission to record the interview.

Roles and duties

1) We want to make sure we understand all of your roles and responsibilities at the school.

a. Please describe your roles and responsibilities for us.

2) In terms of reporting structures:

a. Who is your immediate supervisor?

b. Which staff members report to you?

Academic staffing structure

3) We want to make sure that we understand the structure of SEED’s academic leadership.

a. Please describe the composition of the Academic Leadership Team (ALT).

b. Do the instructional coaches participate?

c. Do the program directors participate?

4) Please explain the roles of the instructional coaches.

a. How many are there?

b. How are these individuals chosen?

c. In what specific ways do you support their work?

5) Please explain the roles of the two program directors (Jessica and Matthew).

a. How are these individuals chosen?

b. In what specific ways do you support their work?

6) In what specific ways do the program directors interact with the instructional coaches?

And with the teachers?

a. Probe: Are there formal structures in place to facilitate collaboration?

i. If yes, describe.

7) We’d like to understand the nature of the ALT meetings.

a. How often does the ALT meet?

b. Who is responsible for leading the meeting?
64

c. What topics or issues are discussed in the meeting?

d. Since ALT participants have different expertise and grade level focus:

i. Does this present any challenges? Please describe.

ii. Does convening a mix of staff promote success? How?

8) Does SEED have a special education coordinator?

a. Does this person participate in the ALT?

b. What are her/his roles and responsibilities?

c. How does she/he coordinate services with the teachers?

Academic content 

9) Does SEED have a specific philosophy about academics or a specific academic focus?

10) What are the goals of the academic program at SEED?

a. Probe: What are the goals for students at each grade?

b. What are the goals for students at the school level (DC-CAS scores, etc.)?

11) What academic courses are students required to take in order to graduate?

12) Does SEED offer a differentiated curriculum?

a. AP or Honors classes?

b. Developmental education (remedial)?

c. Basic skills?

13) Are students able to participate in dual-enrollment programs or take college courses

while still in high school?

a. If yes, at what postsecondary institution?

b. If yes, what courses do they take/programs do they participate in?

14) We’d like to learn about the content that students are covering in each grade level:

6th grade 7th grade 8th grade 9th grade 10th grade 11th grade 12th grade

Math

Language 

Arts/English

Social Studies

Science
65

15) How do teachers and/or instructional coaches develop their curriculums?

16) Are you involved in curriculum development?

a. In what specific ways?

17) Has the Common Core been integrated into the SEED curriculum?

18) Are you working with Mary Lease at the Foundation in the development of the SEED

Blueprint?

a. Please tell us about this process.

19) Does the Foundation direct the curriculum or academic goals of the school?

a. In what specific ways

20) In your opinion, is the SEED academic experience different than that of other schools in

the area?

a. How does it differ?

21) How does the SEED approach support a “college prep” curriculum?

a. Probe:

i. In terms of specific course work.

ii. In terms of promoting a college-going culture.

Academic supports for students

22) What academic supports are available to all SEED students?

23) What supports are available to students who may be struggling academically?

24) Please describe the tutoring support provided.

a. How does this work?

25) Please describe the TAP system.

a. How does this work?

Student Life and academics

26) How do you think the Student Life component of SEED affects students’ academic experience?

a. Probe:

i. Directly (ex. tutoring time)

ii. Indirectly (ex. developing social skills)
66

27) Please describe the role of the following in the SEED model:

a. Structured homework time in the evenings

b. Structured opportunities for study/reading time

c. Gender-specific activities

28) How much does Student Life staff know about the academic life of students?

a. Probe:

i. Their academic subjects

ii. Their assignments

iii. How do they learn this information?

29) Do you think the boarding school model is effective?

a. Why? If yes, in what ways?

b. If no, why not?

Student support services and academics

30) How do you think the Student Support Services component of SEED affects students’

academic experience?

Goals for SEED students’ academic and postsecondary preparation and transition 

31) Does SEED have a specific philosophy about postsecondary readiness, access and success?

32) What are the school’s postsecondary goals for students?

a. Is there an expectation that all students will attend college? Why or why not?

33) Are staff (teachers) expected or instructed to talk about college and college-going in a

certain way?

34) How many students take the PSAT and/or the SAT/ACT in preparation for applying to

college?

35) Are there specific types of colleges that SEED targets for its students? Why?

36) Please describe the college matching process at SEED.

a. Who developed the college-matching process at SEED?

37) In general, do you feel that SEED is successful in its college-preparatory mission?

a. Are there areas in which you’d like the school to improve?

38) How do SEED students’ fare in college?

a. What do the data suggest about postsecondary retention and graduation?
67

b. Are you satisfied with the data?

i. If no, what steps is the school taking to improve upon the data?

39) What specific supports are in place for students who do not plan to attend college?

Students

Let’s discuss the average SEED student.

40) How would you describe the SEED student population?

a. Probe:

i. In terms of demographics (race, gender, SES)

ii. Do they represent a specific geographic location?

41) Are students drawn from a specific set of elementary schools?

42) How would you describe the rate of enrollment from winning the lottery?

a. When someone applies and is offered a spot, do they generally end up attending?

b. How do parents respond to the boarding component?

i. Do they view it as an asset? A deterrent?

43) In your opinion, what makes SEED attractive to parents?

44) Once enrolled in SEED, are there a set of challenges that SEED students often or typically face?

a. Probe:

i. In terms of academics

ii. Boarding feature

iii. Lack of exposure to more diverse student populations

45) During our last interview you mentioned a desire to increase SEED students’ exposure

to people from different backgrounds. Can you tell us more about the school’s efforts to

promote exposure to other students and experiences outside of SEED?

46) What do you think is the biggest challenge facing SEED students?

47) How does the school attempt to address these issues?

Family engagement 

48) What are parents’/guardians’ and families’ expectations for their child at SEED?

a. Why do you think they choose SEED?
68

49) How involved are most parents throughout the year?

50) How does SEED communicate with parents about students’ progress?

51) Does SEED monitor students and parents’ usage of Power School to keep track of student progress?

a. Do you think parents are using this opportunity?

Foundation relationship

Let’s discuss your working relationship with the Foundation. 

52) How often are you in communication with Foundation staff?

a. Who is your main contact(s)?

b. In what specific ways does the Foundation support your work as principal?

Conclusion

53) What are your biggest challenges as principal?

a. Probe

i. Related to students’ academic success.

ii. Related to leading a boarding school.

iii. Related to leading a charter school.

54) What have been your biggest successes to date as principal?

55) Is there anything we haven’t talked about that would be important for us to know about

SEED and its progress to date?

Thank you for your time. 
69

SEED Evaluation 

Interview Protocol for SEED DC Director of Student Life

Introduction

• Thank respondent for his time.

• Interviewer(s) introduces self and study.

• Describe purpose of interview.

• Confirm time allotted for interview.

• Ask for permission to record the interview.

Roles and duties

1) We want to make sure we understand all of your roles and responsibilities at the school.

a. Please describe your roles and responsibilities for us.

2) In terms of reporting structures:

a. Who is your immediate supervisor?

b. Which staff members report to you?

i. Probe: Anyone in addition to the LSCs and RAs?

1. Tutors?

2. Evening Academic Intervention Coordinator?

Student Life (general)

3) What are the main goals of the Student Life department?

a. Probe:

i. Are these different for the various age groups?

ii. Specify.

4) What specific skills do you hope students will acquire from the Student Life component?

a. Probe:

i. Are these different for the various age groups?

ii. Specify.

5) In your opinion, why is the Student Life component of SEED necessary for students?

6) In your opinion, what do SEED students gain during Student Life programming that a

student at a traditional public school would not?

7) Does SEED have a specific philosophy about youth development that informs the way

Student Life operates?

a. What is this philosophy?
70

8) How does Student Life approach facilitating relationships between students and adults? 

a. Probe: 

i. Do students have formal mentors?

ii. Informal mentors? 

9) In your opinion, how does the Student Life component prepare students for college? 

a. How does Student Life programming promote a college-going culture? 

Staffing

10) How many LSCs are there per cohort?

11) What are the main responsibilities of the LSCs?

a. Since several LSCs are responsible for each cohort, how do they divide responsibilities?

12) How many RAs are at the school?

13) For how many students are the RAS responsible? 

14) What are the main responsibilities of the RAs?

15) How do you coordinate with the Evening Academic Intervention Coordinator? 

16) Does the entire Student Life department meet on a regular basis?

a. How often?

Content of Student Life

17) Please describe all of the activities, lessons, services that comprise the Student Life 

component of SEED. We would also like to know when these activities take place, 

how often students participate, how many students participate (is it required or optional), and what the main goals of the activities are.
71

When does this 

happen during 

the day?

How often do 

students participate?

How many students participate 

(is it required or 

optional)? 

What are the 

main goals of 

the activity?

HALLS lessons

Drop Everything 

and Learn 

Drop Everything 

and Read

Sparks

Tutoring

Boys and Girls 

Clubs 

Study Zone

18) How are HALLS lessons developed?

a. What staff is involved?

b. Are the HALLS lessons developed at the beginning of the year or throughout

the years?

19) Are there specific content areas that are covered every year for each age cohort?

a. If so, what are they?

b. How was it decided that these were the necessary topics?

20) Please tell us more about HALLS lessons.

a. What do HALLS lessons look like?

i. How long do they last?

ii. What types of activities are usually included?

1. Probe:

a. Discussion?

b. Projects?

c. Lecture?

iii. Who leads them?
72

21) What happens during Drop Everything and Learn time?

22) What happens during Drop Everything and Read time?

23) Please tell us more about Sparks.

a. What are the Sparks that are available to students?

b. Are they specific to age cohorts?

c. How does a student learn about what Sparks are available to them?

d. Who leads the Sparks?

24) Please tell us more about tutoring at SEED.

a. How is it decided that a student should attend tutoring?

b. Who are the tutors?

25) Please tell us more about the Boys and Girls Clubs.

a. What happens during the time that they meet?

b. Who leads these clubs?

26) What happens during Study Zone?

27) Does the Student Life component involve activities or lessons about college/collegegoing?

a. If so, where does this happen?

b. What topics are covered?

c. How do you coordinate with the Director of College Counseling?

d. Are Student Life staff members instructed to talk about college in any certain

way?

28) Does SEED have a student government?

29) Are there opportunities for SEED students to participate in community service projects?

a. If so, what types?

30) Does SEED have a band or orchestra?

31) Are there any other clubs or activities available to students?
73

Student Life and academics

32) How do you think the Student Life component of SEED affects students’ academic experience? 

a. Probe: 

i. Directly (ex. tutoring time) 

ii. Indirectly (ex. developing social skills)

33) How much does Student Life staff know about the academic life of students? 

a. Probe: 

i. Their academic subjects 

ii. Their assignments 

iii. How do they learn this information? 

 

34) Do you think the boarding school model is effective? 

a. Why? If yes, in what ways?

b. If no, why not? 

Non-academic outcomes

35) What non-academic outcomes does the Student Life component try to affect?

a. Probe? 

i. Self-esteem

ii. Grit

iii. Self-control 

b. How does it try to affect them?

c. Why do SEED students need activities that focus on these areas? 

36) How do you measure these non-academic outcomes?

Conclusion

37) What are your biggest challenges as Director of Student Life? 

a. Probe 

i. Related to students’ academic success.

ii. Related to leading a boarding school. 

38) What have been your biggest successes to date as Director of Student Life? 
74

SEED Evaluation 

Interview Protocol for SEED DC High School Program Director 

Introduction

•

•

•

•

•

Thank respondent for her time.

Interviewer(s) introduces self and study.

Describe purpose of interview.

Confirm time allotted for interview.

Ask for permission to record the interview.

Roles and duties

1) We want to make sure we understand all of your roles and responsibilities at the school.

a. Please describe your roles and responsibilities for us.

2) How long have you been working at SEED?

3) In terms of reporting structures:

a. Who is your immediate supervisor?

b. Which staff members report to you?

Academic Leadership

4) Please describe your role on the Academic Leadership Team (ALT).

a. Probe:

i. Are there benefits to this structure?

ii. Any “cons” to this structure.

5) How does the principal support you in your work?

6) How do you work with the Instructional Coaches?

7) In what ways do you work with the Middle School Program Director?

a. Probe:

i. Do you meet regularly with him?

ii. Do you coordinate tasks or activities?

1. If yes, specify.

Academic content and structure

8) Does SEED have a specific philosophy about academics or a specific academic focus?

a. Probe:

i. Specifically for high school?

ii. How were you introduced to this philosophy?
75

9) What are the goals of the academic program at SEED?

a. Specifically for the high school?

b. Probe:

i. What are the goals for students at the student level (grades)?

ii. What are the goals for students at the school level (DC-CAS scores,

etc.)?

10) We understand that middle school classes are separated by gender.

a. Probe:

i. In 9th grade, how do students cope with being in coed classes?

11) How do teachers and/or instructional coaches develop their curriculums?

12) Are you involved in curriculum development?

13) Has the Common Core been integrated into the SEED curriculum?

14) Are there specific curriculums that your teachers are using? In...

a. ELA?

b. Math?

c. Science?

d. Social studies?

15) What are the main skills that high schoolers should have before transitioning to high

school? In...

a. ELA?

b. Math?

c. Science?

d. Social studies?

16) What are the main skills that high schoolers should have before transitioning to college?

In...

a. ELA?

b. Math?

c. Science?

d. Social studies?

17) How is college discussed in the high school grades?

a. Probe:

i. Are teachers instructed to talk about it in a certain way?

ii. Is it a formal part of students’ high school experience?

1. If so, how?
76

18) In your opinion, are students adequately prepared to attend a rigorous college at the end

of their senior year?

a. Probe:

i. Academically prepared?

ii. Socially prepared?

iii. Emotionally prepared?

Instruction and observation 

19) Are you responsible for observing teachers?

a. Probe:

i. If so, how many times per year?

ii. How do you assess their instruction?

iii. What are you looking for during your observations?

iv. How do you provide feedback to the teachers?

20) Are there specific instructional approaches that your teachers are using?

a. ELA?

b. Math?

c. Science?

d. Social studies?

21) What is your involvement in the development and implementation of interim assessments?

a. Probe:

i. Do you participate in Data Days?

22) How else do you support teachers?

23) Are you responsible for supervising teachers’ implementation of the Model of Care?

a. If so, how?

SEED Students

24) We learned that many middle school students arrive at SEED below grade level.

a. Are they performing on grade level by the time they reach 9th grade?

b. Do many students transfer into SEED at 9th grade?

25) What are SEED high school students’ main academic challenges?

26) How does the staff address these challenges?
77

27) Do you work closely with the Special Education Coordinator?

a. If so, how?

Teachers

28) How do you ensure adequate planning time for teachers?

29) What are the main challenges faced by your teachers?

a. How and when do you learn about these challenges?

b. How are these challenges addressed?

30) Are you involved in determining or developing professional development opportunities

for teachers?

a. If so, what do you focus on?

b. From what types of training do you think they benefit most?

c. In what areas might more training be helpful?

Coordination with other SEED components

31) How do you interact with the other departments at SEED?

a. Student Life

b. Student Support Services

c. College Counseling

32) Are you involved in students’ evening academic programming?

a. If yes, how?

33) In your opinion, how does the boarding environment affect students’ academic experience?

Conclusion

34) What are your biggest challenges as the High School Program Director?

35) What have been your biggest successes to date as the High School Program Director?

36) Is there anything you’d like to share about your role or SEED that we have not discussed so far?
78

SEED Evaluation 

Interview Protocol for SEED DC Director of College Counseling 

Introduction

•

•

•

•

•

Thank respondent for her time.

Interviewer(s) introduces self and study.

Describe purpose of interview. (Mention follow-up to November 2012 site visit/interview.)

Confirm time allotted for interview.

Ask for permission to record the interview.

Roles and duties

1) We want to make sure we understand all of your roles and responsibilities at the school.

a. Please describe your roles and responsibilities for us.

2) In terms of reporting structures:

a. Who is your immediate supervisor?

b. Which staff members report to you?

c. How large is the College Counseling department?

3) We’ve heard a little bit about the role of the Programmatic Leadership Team (PLT), but

we’d like to hear about it from your perspective.

a. How do you use the PTL to support your work?

b. How does the PLT support your work?

SEED’s postsecondary goals 

4) Does SEED have a specific philosophy about postsecondary readiness, access and success?

a. Please cite.

5) What are the school’s postsecondary goals for students?

a. Have the goals changed over time?

i. If yes, why and how?

6) Is there an expectation that all students will attend college?

a. Why or why not?

7) How many students take the PSAT and/or the SAT/ACT in preparation for applying to

college?
79

8) When does the college counseling discussion begin?

a. What types of college advising activities target:

i. Middle school students

1. Specify by grade

ii. High school students

1. Specify by grade

9) Do teaching staff allow the counseling staff to meet with students in their classes to discuss college preparation activities?

a. If yes:

i. How often do you meet with students in classroom settings?

ii. Which teachers (subject areas) have been most receptive to allowing

you to meet with students?

iii. What types of activities do you lead in the classroom setting?

10) Are Student Life staff expected or instructed to talk about college and college-going in

a certain way?

a. How does Student Life programming support SEED’s college-going goals?

i. At the middle school level

ii. At the high school level

College match approach

11) Please tell us about SEED’s philosophy and approach to college match.

a. Why focus on match?

b. Did specific data drive this decision?

12) How does the match focus influence the colleges that students apply to?

a. Please describe the development of the tiered list of match colleges approved

by SEED.

i. What criteria are used to develop the list?

ii. How do you work with the CTS team to develop the list?

iii. Does the tiered list overlap with the Barron’s selectivity categories?

1. If yes, how?

13) How has school staff reacted to the focus on match?

a. Has there been any opposition?

b. If yes, what does this look like?

14) Has the focus on match changed counselors’ advising practice?

a. How?
80

15) Describe how college counseling staff uses the tiered list to engage students in the college search process.

16) When discussing the concept of match with students...

a. Are there specific issues that you raise with students to help them think about

finding a college that is a good match for them?

b. Are there specific issues that students identify as limiting their college options?

17) Are there consequences associated with a student choosing to enroll in a college that is

not on the tiered match list?

a. How have students and families responded to these consequences?

18) When does the match-specific discussion begin with students?

a. In middle school

b. High school

c. During the application stage

The search and application process

19) Describe how you work with upperclassmen.

a. Juniors

b. Seniors

20) How would you describe seniors’ understanding of the college application process?

a. Probes:

i. Typically, what do they know?

ii. Typically, what don’t they know?

21) What types of activities do students request help with the most?

a. Probes:

i. Going on college tours

ii. Identifying match/best fit colleges

iii. Discussing a liberal arts education

iv. Essay writing

v. Assembling and posting college applications

vi. Identifying scholarships

vii. FAFSA assistance

viii. Decision-making about college offers

ix. Other
81

22) What types of activities do students request help with the least?

a. Probes:

i. Going on college tours

ii. Identifying match/best fit colleges

iii. Discussing a liberal arts education

iv. Essay writing

v. Assembling and posting college applications

vi. Identifying scholarships

vii. FAFSA assistance

viii. Decision-making about college offers

ix. Other

23) What selectivity tiers of colleges are SEED students applying to and enrolling in?

a. Probe:

i. Example: Using Barron’s selectivity tiers (most competitive, highly

competitive, etc.)

24) Typically, what factors have the most influence on where students choose to apply to

college?

a. Probes:

i. Financial aid package/scholarship support

ii. Academic reputation

iii. Availability of chosen major

iv. Closeness to home or distance from home

v. Family reasons/obligations (specify)

vi. Other

College acceptances and decision-making

25) Do students experience any pressure to accept offers of admission early (before the national May 1st deadline)?

a. What advice do you give them for how to address this?

b. Do you contact any colleges/financial aid offices directly, on behalf of students

on your match list?

26) What factors most influence match students’ decisions about which college to attend?

a. Financial aid package

b. Scholarship support

c. Academic reputation

d. Availability of chosen major

e. Closeness to home or distance from home

f. Family reasons (specify)

g. Other
82

Parent outreach

27) Let’s discuss your work with parents. 

a. In your opinion, what makes SEED attractive to parents? 

b. What does parent outreach from your office look like? 

c. What is the nature of your conversations with parents? 

i. What are they most interested in knowing? 

ii. Are there common questions that parents ask?

1. Specify.

d. Do you work with parents on completing the FAFSA? 

i. What are their concerns about completing the FAFSA and financial aid 

in general?

e. Do you work with any parents to help them evaluate their child’s offers of admission? 

i. What main things do you discuss with them? 

f. What’s been your biggest challenge working with parents? 

i. Have you figured out how to overcome it? How? 

College data

28) Let’s discuss your work with the College Transition and Support team at the Foundation. 

a. How would you describe the difference in the goals and the focus of the College Counseling Department and the CTS team?

i. Is the difference clear?

b. Do you meet regularly with the CTS staff?

i. If yes, how often? 

c. Are there activities that allow you to work closely together?

d. When you meet, what types of topics do you discuss? 

29) Let’s discuss the college application, enrollment and completion data that SEED collects.

a. What systems are in place at the school building level to track college application, enrollment and completion data? 

i. Describe in detail.

b. What systems are in place at the Foundation level to track college application, 

enrollment and completion data? 

c. How do these data drive changes in the school’s college advising practices?

Conclusion

30) What specific supports are in place for students who do not plan to attend college? 
83

31) In general, do you feel that SEED is successful in its college-preparatory mission?

a. Are there areas in which you’d like the school to improve?

32) What are your biggest challenges as Director of College Counseling?

a. What are the key challenges you expect the department to face in the future?

33) What have been your biggest successes/accomplishments to date as Director of College

Counseling?

34) Is there anything we haven’t talked about that would be important for us to know about

the College Counseling department and its progress to date?
84

SEED Evaluation 

Student Focus Group Protocol: High School

Introduction (read official introduction and assent language)

Application to SEED

1) Do you remember applying to SEED?

a. If so, was it your decision to apply to SEED or was it your parents’ or guardians’ decision?

i. How did you or your family hear about the school?

b. Do you remember why you applied to SEED?

i. If so, why?

ii. Were there other schools that you also applied to?

c. Did you want to come to the SEED school when you applied?

d. Why did you enroll?

Academics

2) What do you think of the classes at SEED?

a. Are they hard, in general?

b. Do teachers give you a lot of homework?

c. Do you like being in classes that are all girls or all boys?

i. Why or why not?

d. Are any of you taking AP classes?

i. Which classes are you taking?

ii. How/why did you decide to take these classes?

iii. Did you have to apply?

iv. Do you plan to take the AP test(s) next year?

3) When do you usually do your homework?

a. Do you feel like you have enough time to get all of your work done?

b. Are there adults that help you with your homework?

c. Do you work with your classmates on your homework?

4) Do any of you participate in tutoring?

a. How did you find out about tutoring?

b. What kinds of things do you do with your tutor?

c. Do you find tutoring helpful?

5) What do you think of the teachers at SEED, in general?

a. Do they explain things clearly?
85

b. Do you feel like you can ask them for help if you have a problem in school?

c. Can you go to them if you are having any personal problems or challenges?

d. Do you meet with them outside of class time?

e. Do they talk to you about your future goals?

College

6) Do you plan to go to college after high school?

a. How did you come to this decision?

7) How is SEED preparing you for college? Probe:

a. Academically?

b. Socially?

8) Are any of you applying now?

a. If not,

i. How do you think SEED will help you during this process?

ii. Has anyone started to talk to you about what this process involves?

iii. Do you ever go to the College Café? Probe:

1. Who do you work with when you go there?

2. What types of things do you do there?

b. If so, probe:

i. What schools are you applying to? Probe:

1. How did you decide this?

2. Have any SEED staff been involved in helping you make these

decisions?

a. If so, which ones?

ii. What other adults are working with you?

1. What are they helping you with?

2. When do they help you?

iii. Do you ever go to the College Café? Probe:

1. Who do you work with when you go there?

2. What types of things do you do there?

iv. When do you work on your applications?

v. Are there other events or workshops that SEED puts on for you or your

parents?

9) Did any of you participate in Junior Seminar last year?

a. If so, probe:

i. What types of things did you cover during this time?

ii. Did you find it helpful?
86

10) Are any of you participating in Senior Seminar right now?

a. If so, probe:

i. What types of things did are you covering?

ii. Are you finding it helpful?

11) How many of you have taken...

a. The PSAT?

i. How did SEED prepare you for this?

ii. Were you required to take it?

b. The SAT?

i. How did SEED prepare you for this?

ii. Were you required to take it?

c. The ACT?

i. How did SEED prepare you for this?

ii. Were you required to take it?

12) Do you feel like you will be ready for college when you graduate?

a. If so, probe:

i. What were the most helpful things that SEED did to prepare you?

b. If not, probe:

i. Why do you feel unprepared?

ii. How could SEED have done a better job preparing you?

13) Do you expect to stay in touch with SEED while you are in college?

a. Are you aware of the type of supports SEED offers its graduates?

Boarding experience

14) How you do you like living in the dorms? Probe:

a. What are the best parts of going to a boarding school?

b. What are the biggest challenges?

15) What is your relationship with your RA like? probe:

a. What kinds of things do you normally do with your RA?

b. Do they help you with your homework?

c. Do they teach you other things?

d. Do you feel like you can ask her/him for help if you have a problem in school?

e. Can you go to her/him if you are having any personal problems or challenges?

f. Do they talk to you about your future goals?
87

Student Life

16) How do you spend your time after school? Probe:

a. Do you study?

b. Do you participate in any Sparks?

c. Do you participate in any sports teams?

d. Do you participate in HALLS lessons?

i. How often do these happen?

ii. What things are usually covered during these lessons?

e. Do you ever participate in community service projects?

f. Do you belong to any other types of clubs or organizations?

g. What else do you do after school?

17) What is your relationship with the Life Skills Counselors (LSCs) like? Probe:

a. What kinds of things do you normally do with LSCs?

b. Do they help you with your homework?

c. Do they teach you other things?

d. Do you feel like you can ask them for help if you have a problem in school?

e. Can you go to them if you are having any personal problems or challenges?

f. Do they talk to you about your future goals?

18) Do you feel like SEED is teaching you useful life and social skills?

a. If so, probe:

i. What skills do they focus on?

ii. When do you learn these skills?

iii. How do you think these skills will help you in your future?

b. If not, probe:

i. Do you think they are trying to do this and are just not doing a good

job?

ii. If so, how could they do a better job?

External opportunities

19) Have you worked with the External Opportunities Office?

a. If so, how have you worked with them?

b. Have they helped you find any summer activities or internships?

i. If so, probe:

1. What have you done?

2. How did you choose this opportunity?

3. How was SEED involved in preparing you for this opportunity?
88

20) Have you gone on any field trips while you have been at SEED? Probe:

a. To colleges?

b. Where else?

Weekend transition

21) What is it like to go home on the weekends? Probe:

a. Do you feel like it is difficult transition?

i. Why or why not?

22) What do you do on the weekends?

23) What is it like coming back to SEED on Sundays?

Friends

24) Who are your closest friends? Probe:

a. Do they also go to SEED?

i. If not, probe:

1. What is it like seeing your friends on the weekends?

2. Do you talk about school with them?

a. If so, do you think you’re having a similar experience

to them?

i. Why or why not?

ii. If yes probe:

1. Do they live near you at home?

25) Do your closest friends want to go to college?

Other

26) Besides teachers, RAs, and LSCs, are you close with any other adults at SEED?

a. If so, probe:

i. Who are you close to?

ii. What kinds of things do you do with this person/these people?

iii. Do you feel like you can ask them for help if you have a problem in

school?

iv. Can you go to them if you are having any personal problems or challenges?

v. Do they talk to you about your future goals?

b. If not, probe:

i. Why not?
89

27) Do you have a lot of free time at SEED?

a. If so, probe:

i. When is this free time?

ii. What do you do during free time?

iii. Do you wish you had less free time?

b. If not, probe:

i. Do you feel like you ever have time to yourself?

ii. Would you want more free time?

iii. Do you ever feel too busy?

c. How do you know what activities are when?

d. Do you think SEED is doing a good job of teaching you how to manage your

time?

28) How is high school different than ninth grade? Probe:

a. Academically?

b. Socially?

Closing

29) What are your top three favorite things about SEED?

30) What are your three least favorite things about SEED?

31) How do you think SEED could do a better job

a. Helping you learn?

b. Providing you with activities after school?

c. Making the dorms a good place to live?

32) Is there anything else that we haven’t asked you about that we should know?

Appendix C 

The SEED DC Lottery Process

93

As is common among charter schools nationwide, SEED DC conducts a lottery each spring to 

determine which new students are offered the opportunity to enroll in the school the following 

fall. Entry is typically limited to one grade level, after which students already enrolled at SEED 

can freely progress into subsequent grades without having to “win” a seat. SEED’s grade of entry changed during the study period. Students who applied to start at SEED in the 2007-2008 

and 2008-2009 school years applied as sixth-graders, vying for seats in the seventh-grade class. 

SEED then decided to move toward a sixth-grade entry point in response to a growing trend 

among elementary schools of graduating students after fifth grade instead of sixth. To allow for 

a transitional period, students could apply for seats in both the sixth- and seventh-grade classes 

in the 2009-2010 and 2010-2011 school years. The 2011-2012 school year marked the first year 

in which students could begin at SEED only as sixth-graders. 

Having one point of entry means that open seats, even if they occur in the higher 

grades, are almost always filled by the incoming middle school applicants. For example, if an 

eleventh-grader left SEED during the 2014-2015 school year, the student’s “seat” would be 

filled by an incoming sixth-grader participating in the lottery for the 2015-2016 school year.1

Any District of Columbia resident who was born on or after February 1, 12 years before 

the lottery, is eligible to attend SEED. Families need only complete an application to be included in the lottery. In the past, families were required to attend the lottery in person, but in 2008 

SEED made their presence optional. 

SEED uses a bingo ball and cage system to select students. Students are assigned a random number that corresponds to one of the bingo balls, and the director of enrollment reads the 

numbers as they drop from the cage. Separate lotteries are held for boys and girls, and the number of seats being filled varies from year to year based on SEED’s enrollment needs and predicted attrition. Once available seats are filled, the order in which students’ numbers are chosen

determines their places on the waiting list. Students who apply to SEED after the deadline are 

placed at the end of the waiting list in the order in which their applications are received.

Students with siblings already enrolled in SEED receive preference, and a student with 

a sibling participating in the lottery in the same year is admitted if the sibling is offered a place. 

In the lotteries for the 2006-2007 through 2008-2009 school years, applicants with siblings already enrolled at SEED participated in the lottery but were later (nonrandomly) pushed to the 

top of the list. In the 2009-2010 lottery, students with SEED siblings no longer participated in 

the lottery; they were instead offered the opportunity to attend SEED. 

1

In some infrequent cases, when the entering grade, currently sixth grade, is at capacity in terms of staff, 

materials, class size, and so on, SEED enrolls students in later grades. If this occurs, students are still selected 

randomly based on the lottery in which they participated. 
94

Students selected in the lottery must accept or reject the offer by a specified deadline. 

SEED follows up with families who do not respond by the deadline, giving them two business 

days to respond to the reminder. After this follow-up, the director of enrollment management 

fills open seats by moving down the waiting list. Families are given the same two-business-day 

period to respond to the offer before SEED moves on to the next student. The director of enrollment will make very infrequent exceptions if the family has a compelling reason for failing 

to respond to the offer, for example, if the responding parent was incarcerated. SEED DC continues to fill seats until early October. 

All students who are selected in the lottery and are offered a seat at SEED on lottery 

day are invited to attend a weeklong new-student orientation during the summer preceding their 

enrollment at SEED. A select number of students from the waiting list are also invited in anticipation of a certain number of students changing their mind over the summer. Attendance at the 

orientation is not required but is strongly encouraged.

Figure C.1 shows how the study sample progressed through this lottery and enrollment 

process, beginning with 919 total applicants and 499 students being offered the opportunity to 

enroll at SEED. 
95

SEED Evaluation Final Report

Figure C.1

Flow of Study Sample Through SEED Application Process

Total number of applicants

919

Number of applicants 

automatically given the 

opportunity to enroll at 

SEED because of siblingsa

153

Number of applicants 

participating in lottery

766

Number of applicants given 

the opportunity to enroll at 

SEED from the waiting list

249

Total number of study 

sample applicants who were 

given the opportunity to 

enroll at SEED

499

SOURCES: MDRC’s calculations using SEED DC lottery data and Office of the State Superintendent of Education 

(OSSE) files on student enrollment and graduation.

NOTE: a

In the 2006-2007 through 2008-2009 lotteries, 39 siblings of students already enrolled in SEED were 

technically included in the lottery and then removed for automatic enrollment. These individuals are not counted as 

lottery participants; they are counted among the applicants automatically given the opportunity to enroll. Since their 

removal was based on a baseline characteristic, it does not affect the randomness of assignment of the remaining 

participants; that is, their removal does not disturb the internal validity of the remaining lottery sample. 

Number of applicants given 

the opportunity to enroll at 

SEED on lottery day

250

Number of applicants never 

given the opportunity to 

enroll at SEED

267

Appendix D 

The Analytic Approach

99

Impact Design

A student-level randomized experiment was used to evaluate SEED’s impact on the achievement 

and engagement outcomes of students offered the program compared with how they would have 

performed otherwise. During the study period, a greater number of rising sixth-grade students 

applied to SEED than could be served, so lotteries were conducted each year to determine who 

would be offered an opportunity to enroll.1 A subset of the applicants were randomly offered the 

opportunity to fill SEED’s approximately 60 school slots, constituting the SEED group. The remainder of the applicants were not offered admission to SEED and therefore constitute the control or “business as usual” condition — what would occur in the absence of SEED.

2 As described 

in Appendix C, students who were siblings of current SEED students did not participate in the 

lottery; they were automatically admitted to SEED and are not part of the study sample. Since 

random assignment creates comparable groups of students when samples are large,3 any difference in post-random assignment outcomes between those who were offered the opportunity to 

enroll in SEED and those who were not represents the impact of SEED. 

Statistical Analysis of Impacts

SEED’s impact was estimated by comparing the outcomes of students assigned to SEED and 

those of the non-SEED control group. The following statistical model was used:

YYii = �ααjj ∙ IIii

JJ

jj=1

+ ββTTii + � γγSS ∙ XXss

SS

jj=1

+ εεii

where:

Yi = outcome for student i 

Ti = indicator of treatment group membership, equal to 1 if student i was randomly assigned to 

SEED participation and 0 otherwise

1

To achieve equal numbers of males and females in their entering seventh-grade class, SEED DC conducts one lottery for boys and one lottery for girls. 2

Some of the students who are not admitted to SEED initially are placed on a waiting list. When students 

who win the lottery choose not to enroll in the school, students may be admitted to SEED from the waiting list.

MDRC researchers reviewed SEED waiting lists and confirmed that during the study period, students were 

admitted to SEED from the waiting list in the random order determined on lottery day. Accordingly, any student from the waiting list who was offered the opportunity to enroll in SEED before the fall of the following 

school year is considered a SEED group member. (This approach is similar to the one taken by Tuttle and colleagues in their 2013 report on KIPP charter schools.) 3

As shown in Table 3.1, students in the SEED group are more likely not to qualify for free or reducedprice lunch and have slightly higher test scores. In accordance with What Works Clearinghouse recommendations, these characteristics are included as covariates in the impact model. 
100

Iij = a set of j random assignment block indicators, equal to 1 if student i is in random assignment block j and 0 otherwise. A student’s block is defined based on her or his cohort (year of 

random assignment) and gender at the point of random assignment. These blocks were included 

in the model to capture a central feature of the research design (lotteries by school and gender 

within year).

Xsi = a set of s pre-random assignment characteristics for student i that are associated with student outcomes. These characteristics include standardized test scores from students’ baseline 

year and indicators of family income. 

εi = a student-level random error term.

In this model, β represents the estimated impact of SEED on the outcome of interest 

(Yi). Because the statistical analysis treats random assignment blocks as fixed effects, and β is

estimated as a fixed coefficient rather than a random one, β should be interpreted as the estimated impact of SEED for the average student in the set of schools where the study will take place. 

This approach is taken because the study sites are not a random sample of a larger population of 

schools or districts. Consequently, the evaluation results will not be generalizable to other 

schools or students. 

β is also an “intent-to-treat” estimate of the SEED school program impact. Some students assigned to SEED will not enroll. Thus, the main evaluation findings represent the estimated impact of offering students the opportunity to enroll in SEED (“intent to treat”), rather 

than the impact of SEED on students who actually attended (“treatment on the treated”). It was

expected that SEED participation among the program group would initially be high but would

decrease over time. Thus, in a quasi-experimental analysis, an instrumental variables approach 

was used to estimate the effects of actually enrolling in SEED DC (see Appendix E).

4

Sample Retention and Missing Data

Much of the impact study relies on existing administrative records from SEED, such as 

the lottery application and assignment offering, and from the Office of the State Superintendent 

of Education (OSSE), such as state standardized test scores and graduation records. After a 

small decrease (8 percent) in the sample based on the researchers’ inability to locate all lottery 

applicants in OSSE student records, there was a yearly attrition rate between 5 percent and 10 

percent. Despite the yearly attrition, the characteristics of the sample and its internal validity

appear to have been maintained. For example, Table D.1 presents the baseline characteristics of 

the first two cohorts, and Table D.2 shows the same characteristics for the sample members 

from these cohorts for whom graduation data were available six years later. 

4

See Schochet and Chiang (2009).
101

Table D.1

Baseline Characteristics of SEED Lottery Participants, Cohorts 1 and 2

Characteristic

SEED Non-SEED Estimated

Group Group Difference

P-Value for

Estimated

Difference

Race (%)

Black 100.0 100.0 0.0 1.000

Female (%) 43.8 43.8 0.0 1.000

Economic indicator (%)

Eligible for free/reduced-price lunch NA NA NA ---

a Direct Certification status NA NA NA ---

Special education status (%) 12.0 21.1 -9.1 0.120

Standardized math score 11.53 -5.52 17.05 0.202

At or above proficiency level in math (%) NA NA NA ---

Standardized reading score

At or above proficiency level in reading (%)

9.87

NA

-9.80

NA

19.67

NA

0.176

---

Sample size 121 93

SEED Evaluation Final Report

SOURCES: MDRC’s calculations using Office of the State Superintendent of Education (OSSE) 

files on student enrollment and state test scores from the 2006-2007 and 2007-2008 school years.

NOTES: NA = not available.

Cohorts 1 and 2 consist of 214 students in the study who were sixth-graders in the springs of 

2007 and 2008, respectively.

Values for SEED group members and non-SEED group members are the simple means or 

percentages. Values for the difference between SEED group members and non-SEED group 

members are obtained from a regression of a given baseline characteristic on a series of indicator 

variables that identify each lottery, a covariate indicating the probability that a student would win 

a spot on lottery day, and an indicator variable that equals 1 for lottery winners and 0 for lottery 

losers. The coefficient on the latter indicator variable equals the difference in the baseline 

characteristic between SEED group members and non-SEED group members. The value for nonSEED group members equals the corresponding value for SEED group members minus the 

estimated difference between SEED group members and non-SEED group members.

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 

indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. a

Direct Certification is automatic qualification for free lunch because the individual is in foster 

care, is homeless, or is living in a household receiving benefits from the Supplemental Nutrition 

Assistance Program (SNAP), the Food Distribution Program on Indian Reservations (FDPIR), or 

Temporary Assistance for Needy Families (TANF).
102

SEED Evaluation Final Report

Table D.2

Baseline Characteristics of SEED Lottery Participants in 

Cohorts 1 and 2 for Whom Graduation Data Are Available

Characteristic

SEED

Group

Non-SEED

Group

Estimated

Difference

P-Value for

Estimated

Difference

Race (%)

Black 100.0 100.0 0.0 1.000

Female (%) 46.7 46.7 0.0 1.000

Economic indicator (%)

Eligible for free/reduced-price lunch NA NA NA --- a Direct Certification status NA NA NA ---

Special education status (%) 10.8 13.6 -2.8 0.662

Standardized math score 15.98 3.07 12.90 0.545

At or above proficiency level in math (%) NA NA NA ---

Standardized reading score

At or above proficiency level in reading (%)

12.37

NA

-13.10

NA

25.47

NA

0.235

---

Sample size 75 58

SOURCES: MDRC’s calculations use SEED DC school application data for students in 2006-2007 

through 2011-2012 and Office of the State Superintendent of Educaton (OSSE) files on student 

enrollment and state test scores from the 2006-2007 to 2013-2014 school years.

NOTES: NA = not available.

Cohorts 1 and 2 consist of 133 students in the study who were sixth-graders in the springs of 2007 

and 2008, respectively.

Values for SEED group members and non-SEED group members are the simple means or 

percentages. Values for the difference between SEED group members and non-SEED group members 

are obtained from a regression of a given baseline characteristic on a series of indicator variables that 

identify each lottery, a covariate indicating the probability that a student would win a spot on lottery 

day, and an indicator variable that equals 1 for lottery winners and 0 for lottery losers. The coefficient 

on the latter indicator variable equals the difference in the baseline characteristic between SEED 

group members and non-SEED group members. The value for non-SEED group members equals the 

corresponding value for SEED group members minus the estimated difference between SEED group 

members and non-SEED group members.

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are 

indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. aDirect Certification is automatic qualification for free lunch because the individual is in foster 

care, is homeless, or is living in a household receiving benefits from the Supplemental Nutrition 

Assistance Program (SNAP), the Food Distribution Program on Indian Reservations (FDPIR), or 

Temporary Assistance for Needy Families (TANF).
Appendix E 

Estimated Effects of Enrolling in SEED: 

A “Treatment-on-the-Treated” Analysis

105

As described in Appendix D, the primary research design takes advantage of SEED lotteries to 

identify a set of students who randomly “won” the opportunity to attend SEED and a set of students who randomly “lost” and did not receive that opportunity. Because of this random assignment, any difference in students’ future outcomes can be attributed to the opportunity to 

attend SEED, otherwise known as the “intent-to-treat” effect of SEED. But receiving an offer to 

enroll in SEED is not the same as actually enrolling; some lottery winners decide to attend a 

different school. As shown in Table E.1, roughly three-quarters of SEED lottery winners initially enroll in SEED, and the proportion of this group who remain enrolled in SEED decreases 

each year. 

Because those who have the opportunity to attend SEED but choose not to are selfselected, rather than randomly selected, simply analyzing the outcomes among those who attend 

SEED and those who do not is subject to bias. Therefore, to estimate the effect of actually enrolling in SEED, this study employs what is known as a “treatment on the treated” approach using 

a standard application of instrumental variables (IV) analysis.

1 In any IV analysis, the key assumption involves the selection of an instrumental variable (for example, winning or losing the 

lottery) that is related to the outcome only through its relationship to the treatment variable (for 

example, enrolling in SEED). In this analysis the instruments include interactions between each 

lottery fixed effect (gender and year of random assignment) and the treatment dummy (winning 

or losing) for that lottery; these instruments are used to predict actual SEED enrollment. Because enrollment is a function of winning a lottery, which is a random process, the instrument 

fulfills the key assumption and is valid, and therefore allows the estimation for the full sample 

of the average treatment effect of actually enrolling in the SEED program. These effects of enrolling in SEED are roughly 20 percent larger than the intent-to-treat effects and are presented 

in Table E.2.

1

Gennetian, Morris, Bos, and Bloom (2005). The research team chose not to employ analytic models that 

have been employed in the past to estimate the yearly effects of receiving varying amounts of exposure to 

SEED (one year, two years, etc.). To be accurate these models require that researchers make the strong assumption of a linear relationship between years of exposure and impacts. Given that we are following students 

through middle and high school, a time of great, dynamic change in students, the assumption of linearity is not 

realistic.
106

Table E.1 

Program Group Enrollment in SEED by Cohort and Year

Percentage of SEED Group Enrolled in SEED

Year 1 Year 2 Year 3 Year 4 Year 5 Year 6

Cohort 1 73.9 69.4 NA 47.5 NA 44.0

Cohort 2 92.1 88.2 NA 75.9 NA 62.5

Cohort 3 68.4 72.2 NA 75.0 NA NA

Cohort 4 62.9 56.3 NA 27.6 NA NA

Cohort 5 83.8 80.6 77.4 NA 45.5 NA

Cohort 6 75.0 60.0 42.3 NA NA NA

Cohort 7 80.9 66.2 66.7 NA NA NA

Cohort 8 42.7 40.0 NA NA NA NA

SEED Evaluation Final Report

SOURCE: MDRC’s calculations using Office of the State Superintendent of Education (OSSE)

enrollment data, state test score data and state high school graduation data.

\ NOTE: NA = not available.

This table presents the proportion of students with OSSE follow-up data enrolled in SEED 

over time.
107

SEED Evaluation Final Report

Table E.2

Estimated Effects of Enrolling in SEED,

Cohorts 1 and 2

Target Non-SEED P-Value for

SEED Group Estimated Estimated

Characteristic Enrollees Counterparts Difference Difference

First year

Standardized math score 0.26 -0.10 0.36 * 0.060

Standardized reading score 0.03 -0.03 0.06 0.751

Second year

Standardized math score 0.32 -0.21 0.53 *** 0.006

Standardized reading score 0.07 -0.02 0.09 0.640

Fourth year

Standardized math score 0.51 0.26 0.26 0.877

Standardized reading score 0.24 0.32 -0.08 0.711

Four-year high school graduation (%) 69.3 74.1 -4.8 0.592

Sample size 121 93

SOURCES: MDRC’s calculations using Office of the State Superintendent of Education (OSSE) state 

test scores from the 2007-2008 to 2011-2012 school years.

NOTES: Cohorts 1 and 2 consist of 214 students in the study who were sixth-graders in the springs of 

2007 and 2008, respectively.

This table presents estimated effects for students who have follow-up data. Appendix E describes how 

values in the columns labeled "Target SEED Enrollees" and "Estimated Difference" are estimated. Each 

value for non-SEED group counterparts equals the corresponding value for target SEED enrollees minus 

the estimated difference.

A two-tailed t-test was applied to the estimated difference. Statistical significance levels are indicated 

as: *** = 1 percent; ** = 5 percent; * = 10 percent.

Appendix F 

The SEED Student Survey

111

Design and Fielding

In addition to assessing the impact of SEED DC on students’ academic outcomes, like District 

of Columbia Comprehensive Assessment System (DC CAS) test scores and high school graduation, the SEED evaluation aimed to understand its effects on a variety of nonacademic outcomes, such as metacognitive skills (self-control and “grit”), attitudes, health, and both positive 

and negative youth behaviors. SEED DC’s boarding school model positions it well to affect 

such outcomes not only because the school controls students’ environments for longer periods 

of time than traditional schools, but also because its Student Life programming includes explicit 

life skills instruction and character education. 

To capture SEED DC’s impact on these measures, the research team fielded a survey to 

both program group and non-program group students in cohorts 1 and 2. (The full survey appears at the end of this appendix.) These cohorts were selected because, if students in these cohorts progressed one grade level per school year, they would have been in their first post-high 

school year and seniors in high school, respectively, at the time of fielding. These older grades 

provided the best opportunity to understand students’ attitudes about attending college as well 

as the longer-term impact of SEED on both positive and negative youth behaviors.

Because the survey was fielded to both program group students and non-program group 

students, it also served as an important tool in understanding the differences in experiences between both research groups. Students were asked questions about their school climate, home 

environment during the week, courses and activities at school, and access to college preparatory 

activities. 

The survey was created using a variety of validated external scales to understand attitudes, behaviors, and opinions. Table F.1 provides the Cronbach’s alpha values for these scales 

(indicators of their reliability). Original questions and scales were also included to understand 

more about the courses, activities, and college preparation students experienced. More detailed 

information on the scales used and related outcome measures is provided below.

The research team partnered with Decision Information Resources to field the survey. 

Students had the opportunity to complete it by phone or on the Internet and were compensated

for their time with a small monetary reward. The full student survey in Computer-Assisted Telephone Interview (CATI) questionnaire format is available upon request.
112

Table F.1

Cronbach's Alpha Values for Survey Constructs

Scale Cronbach's Alpha

Study skills

Organizational skills 

Self-control

Grit

0.76

0.78

0.78

0.68

Recent frequency of risky behavior

Overall engagement in risky behavior

School comfort

Classroom order

Peers' academic motivation

0.82

0.79

0.79

0.81

0.86

Caring adult at school

School-wide future orientation

Access to college preparatory activities

Order of home during the week

Caregiver support during the week

0.90

0.90

0.94

0.78

0.87

SEED Evaluation Final Report

SOURCE: MDRC calculations based on survey data.

Detailed Information on Scales Used

Access to College Preparatory Activities Scale

The access to college preparatory activities scale consists of six survey questions from 

the University of Chicago Consortium on School Research (UChicago Consortium) 2012 My 

Voice, My School Survey.

1 The questions ask respondents to indicate how often they discussed 

1

University of Chicago Consortium on School Research (2012).
113

various topics related to preparation for college, including different admissions requirements 

among four-year colleges, how to decide which college to attend, the likelihood of being accepted at different types of colleges, what ACT/SAT scores are needed to get into certain colleges, opportunities to attend out-of-state colleges, and how to pay for college. Responses were 

given on an ordinal scale (“did not discuss,” “discussed briefly,” and “discussed in depth”). Responses of “did not discuss” were coded as 1, “discussed briefly” as 2, and “discussed in depth” 

as 3. Individuals’ responses to the each of the six questions were averaged to create a construct 

outcome measure for each individual. Higher scores indicate more access to college preparatory 

activities.

School Comfort Scale

The school comfort scale is made up of five survey questions that ask respondents to 

indicate how much they agree with a series of statements about the 2012-2013 school year or 

the last year they were in school.2 Statements concern feeling safe at school, enjoying going to 

school, switching schools if the respondent had had the ability to do so, feeling accepted at 

school, and fitting in at school. Responses were given on an ordinal scale (“strongly disagree,” 

“disagree,” “agree,” and “strongly agree”). Responses of “strongly disagree” were coded as 1, 

“disagree” as 2, “agree” as 3, and “strongly agree” as 4. Individuals’ responses to each of the 

five items were averaged to create a construct outcome measure. Since this scale uses reverse 

wording to phrase statements in both positive and negative voices, several scores were reversed 

so that higher scores indicate higher levels of comfort.

Classroom Order Scale

The classroom order scale comprises three survey questions from the UChicago Consortium 2012 My Voice, My School Survey.

3 The questions ask respondents to indicate how 

much they agree with a series of statements about the 2012-2013 school year or the last year 

they were in school. Statements include being distracted by students acting out, classes being 

out of control, and having classmates who did not behave the way teachers wanted them to. Responses were given on an ordinal scale (“strongly disagree,” “disagree,” “agree,” and “strongly 

agree”). Responses of “strongly disagree” were coded as 1, “disagree” as 2, “agree” as 3, 

“strongly agree” as 4. Individuals’ responses to each of the three items were averaged to create a 

construct outcome measure. Since this scale includes statements in only the negative voice

(statements about disorderly classrooms), all scores were reversed so that higher scores indicate 

higher levels of classroom order.

2

These questions are based on questions from the UChicago Consortium 2012 My Voice, My School Survey (University of Chicago Consortium on School Research, 2012). 3

University of Chicago Consortium on School Research (2012).
114

Peers’ Academic Motivation Scale

The peers’ academic motivation scale consists of four survey questions from the UChicago Consortium 2012 My Voice, My School Survey.

4 The questions ask respondents how 

many of their peers in the 2012-2013 school year or the last year of school they attended felt it 

was important to come to school every day, felt it was important to pay attention in class, felt it 

was important to do homework, and tried hard to get good grades. Responses were given on an 

ordinal scale (“none,” “a few,” “about half,” and “most”). Responses of “none” were coded as 

1, “a few” as 2, “about half” as 3, and “most” as 4. Individuals’ responses to each of the four 

items were averaged to create a construct outcome measure. Higher scores indicate more academic motivation among peers.

Caring Adult at School Scale

The caring adult at school scale consists of six survey questions based on the California 

Healthy Kids Survey.

5 The questions ask respondents to indicate how much they agree with a 

series of statements about the school they attended in 2012-2013 or the last year they were in 

school. Statements concern having had teachers or adults at school who really cared about the 

respondent, told the respondent when he or she did a good job, noticed when the respondent 

was not there, always wanted the respondent to do his or her best, listened when the respondent 

had something to say, and believed that the respondent would be a success. Responses were 

given on an ordinal scale (“not at all true,” “a little true,” “pretty much true,” and “very much 

true”). Responses of “not at all true” were coded as 1, “a little true” as 2, “pretty much true” as 

3, and “very much true” as 4. Individuals’ responses to each of the six items were averaged to 

create a construct outcome measure. Higher scores indicate more supportive caregivers.

School-Wide Future Orientation Scale

The school-wide future orientation scale is made up of six survey questions from the 

UChicago Consortium 2012 My Voice, My School Survey.

6 The questions ask respondents to 

indicate how much they agree with a series of statements about the 2012-2013 school year or 

the last year they were in school. Statements included whether students had been at a school 

where teachers made sure that all students were planning for a life after graduation, teachers 

worked hard to make sure that all students were learning, high school was seen as preparation 

for the future, all students were encouraged to go to college, teachers paid attention to all students, and teachers worked hard to make sure that all students stayed in school. Responses were 

given on an ordinal scale (“strongly disagree,” “disagree,” “agree,” and “strongly disagree”). 

4

University of Chicago Consortium on School Research (2012). 5

California Department of Education (2015).

6

University of Chicago Consortium on School Research (2012).
115

Responses of “strongly disagree” were coded as 1, “disagree” as 2, “agree” as 3, “strongly 

agree” as 4. Individuals’ responses to each of the six items were averaged to create a construct 

outcome measure. Higher scores indicate having a school that is more future-oriented.

Order of Home During the Week Scale

The order of home during the week scale comprises 11 survey questions adapted from 

items in the Confusion, Order, and Hubbub Scale.

7 The questions ask respondents to indicate 

how much they agree with a series of statements about where they live from Sunday to Thursday night (the time that SEED students live on campus). Statements concern being able to find 

things when you need them, feeling rushed, being able to stay on top of things, running late, 

being able to talk to others without being interrupted, often having a fuss going on, having plans 

that often do not work out, not being able to hear yourself think, getting drawn into other people’s arguments, having a good place to relax, and having a calm atmosphere. Responses were 

given on an ordinal scale (“very much like where you live during the week,” “somewhat like 

where you live during the week,” “a little bit like where you live during the week,” and “not at 

all like where you live during the week”). Responses of “very much like where you live during 

the week” were coded as 1, “somewhat like where you live during the week” as 2, “a little bit 

like where you live during the week” as 3, and “not at all like where you live during the week” 

as 4. Individuals’ responses to each of the 11 items were averaged to create a construct outcome 

measure. Since this scale uses reverse wording to phrase statements in both positive and negative voices, several scores were reversed so that higher scores indicate having higher levels of 

order.

Caregiver Support During the Week Scale

The caregiver support during the week scale consists of six original survey questions. 

The questions ask respondents to indicate how true they perceive a series of statements to be 

about adults with whom they live from Sunday to Thursday night (the time that SEED students 

live on campus). Statements include whether there is an adult who expects the respondent to 

follow rules, who is interested in the respondent’s schoolwork, who believes that the respondent 

will be a success, who talks with the respondent about his or her problems, who always wants 

the respondent to do his or her best, and with whom the respondent likes to do fun things. Responses were given on an ordinal scale (“not at all true,” “a little true,” “pretty much true,” and 

“very much true”). Responses of “not at all true” were coded as 1, “a little true” as 2, “pretty 

much true” as 3, and “very much true” as 4. Individuals’ responses to each of the six items were 

averaged to create a construct outcome measure. Higher scores indicate a greater level of support from adults during the week.

7

Matheny, Wachs, Ludwig, and Philips (1995). 
116

Study Skills Scale

The study skills scale consists of four survey questions from the UChicago Consortium 

2012 My Voice, My School Survey.

8 The questions ask respondents to indicate how much they 

agree with a series of statements about always studying for tests, setting aside time to do homework and study, trying to do well on schoolwork even when it is not interesting, and forgoing 

time with friends when they need to study. Responses were given on an ordinal scale (“strongly 

disagree,” “disagree,” “agree,” and “strongly agree”). Responses of “strongly disagree” were 

coded as 1, “disagree” as 2, “agree” as 3, “strongly agree” as 4. Individuals’ responses to each 

of the four items were averaged to create a construct outcome measure. Higher scores indicate 

better study skills.

Organizational Skills Scale

The organizational skills scale comprises three survey questions from the UChicago 

Consortium 2012 My Voice, My School Survey.

9 The questions ask respondents to indicate 

how much they agree with a series of statements about keeping track of long-term assignments, 

managing time well enough to get all work done, and keeping their schoolwork and personal 

life organized. Responses were given on an ordinal scale (“strongly disagree,” “disagree,” 

“agree,” and “strongly agree”). Responses of “strongly disagree” were coded as 1, “disagree” as 

2, “agree” as 3, “strongly agree” as 4. Individuals’ responses to each of the three items were 

averaged to create a construct outcome measure. Higher scores indicate better organizational 

skills.

Recent Frequency of Risky Behavior Scale

The recent frequency of risky behavior scale is based on six survey questions from the 

Self-Reported Behavior Index.

10 Respondents are asked to indicate the frequency with which 

they have engaged in specific risky behaviors during the last three months. Behaviors include 

getting into a fight at school or in the neighborhood, having a fight or argument with parents, 

taking something that did not belong to the respondent, skipping school without permission, 

taking something from a store without paying for it, and hitting someone. Responses were given 

on an ordinal scale that was different than the original used in the Self-Reported Behavior Index 

(“you have never done this,” “you have done this, but not in the last 3 months,” “you did this 1-

2 times in the last 3 months,” “you did this 3-4 times in the last 3 months,” and “you did this 5 

or more times in the last 3 months”). Responses of “you have never done this” were coded as 1, 

“you have done this, but not in the last 3 months” as 2, “you did this 1-2 times in the last 3 

8

University of Chicago Consortium on School Research (2012). 9

University of Chicago Consortium on School Research (2012). 10Brown, Clasen, and Eicher (1986).
117

months” as 3, “you did this 3-4 times in the last 3 months” as 4, and “you did this 5 or more 

times in the last 3 months” as 5. Individuals’ responses to each of the six items were averaged to 

create a construct outcome measure. Higher scores indicate more frequent risky behavior in the 

past three months.

Overall Engagement in Risky Behavior Scale

The overall engagement in risky behavior scale comprises four original questions about 

individuals’ involvement in specific risky behaviors since the time of random assignment. The 

questions ask whether the individual has ever been expelled or dismissed from school, has ever 

been in trouble with the police, has ever been arrested or had to go to juvenile court, or has had 

unprotected sex. Responses included “yes” and “no,” “yes” being coded as 1 and “no” as 0. Individuals’ responses to each of the four items were averaged to create a construct outcome 

measure. Higher scores indicate being more likely to have ever engaged in risky behavior since 

random assignment.

Self-Control Scale

The self-control scale uses 13 survey questions that constitute the Brief Self-Control 

Scale.

11 Respondents are asked to indicate how true they perceive a series of statements to be. 

The statements cover topics like breaking bad habits, the ability to refuse things that are not 

good, self-discipline, and thinking through alternatives before acting. Responses were given on 

an ordinal scale (“not at all like you,” “not much like you,” “somewhat like you,” “mostly like 

you,” and “very much like you”). Responses of “not at all like you” were coded as 1, “not much 

like you” as 2, “somewhat like you” as 3, “mostly like you” as 4, and “very much like you” as 

5. Individuals’ responses to each of the 13 items were averaged to create a construct outcome

measure. Since this scale uses reverse wording to phrase statements in both positive and negative voices, several scores were reversed so that higher scores indicate having more self-control.

Grit Scale

The grit scale uses seven survey questions from the Short Grit Scale.

12 Respondents are 

asked to indicate how true they perceive a series of statements to be. The statements cover topics such as whether new ideas distract from previous ones and whether the respondent is a hard 

worker, is diligent, and finishes what he or she begins. Responses were given on an ordinal 

scale (“not at all like you,” “not much like you,” “somewhat like you,” “mostly like you,” and

11Tangney, Baumeister, and Boone (2004). 12Duckworth, Peterson, Matthews, and Kelly (2007).
118

“very much like you”). Responses of “not at all like you” were coded as 1, “not much like you” 

as 2, “somewhat like you” as 3, “mostly like you” as 4, and “very much like you” as 5. Individuals’ responses to each of the seven items were averaged to create a construct outcome measure. 

Since this scale uses reverse wording to phrase statements in both positive and negative voices, 

several scores were reversed so that higher scores indicate having higher levels of grit.
119

References 

Bloom, Howard S., and Rebecca Unterman. 2014. “Can Small High Schools of Choice Improve 

Educational Prospects for Disadvantaged Students?” Journal of Policy Analysis and 

Management 33, 2: 290-319.

Brown, B. Bradford, Donna R. Clasen, and Sue A. Eicher. 1986. “Perceptions of Peer Pressure, Peer 

Conformity Dispositions, and Self-Reported Behavior Among Adolescents.” Developmental 

Psychology 22, 4: 521-530. 

California Department of Education. 2015. “California Healthy Kids Survey Elementary School 

Questionnaire.” Version E19. Website: www.cde.ca.gov.

Curto, Vilsa E., and Roland G. Fryer Jr. 2011. “Estimating the Returns to Urban Boarding Schools: 

Evidence from SEED.” National Bureau of Economic Research Working Paper No. 16746. 

Curto, Vilsa E., Roland G. Fryer Jr., and Meghan L. Howard. 2011. “It May Not Take a Village: 

Increasing Achievement Among the Poor.” Pages 483-505 in Greg J. Duncan and Richard 

J. Murnane (eds.), Whither Opportunity? Rising Inequality, Schools, and Children’s Life

Chances. New York: Russell Sage Foundation.

Duckworth, Angela L., Christopher Peterson, Michael D. Matthews, and Dennis R. Kelly. 2007. 

“Grit: Perseverance and Passion for Long-Term Goals.” Journal of Personality and Social 

Psychology 92, 6: 1087.

Fountas, Irene C., and Gay Su Pinnell. 2012. “The F & P Text Level Gradient: Revision to 

Recommended Grade-Level Goals.” Portsmouth, N.H.: Heinemann. Website: 

www.heinemann.com/fountasandpinnell/pdfs/WhitePaperTextGrad.pdf.

Gennetian, Lisa A., Pamela A. Morris, Johannes M. Bos, and Howard S. Bloom. 2005. 

“Constructing Instrumental Variables from Experimental Data to Explore How Treatments 

Produce Effects.” Pages 75-114 in Howard S. Bloom (ed.), Learning More from Social 

Experiments: Evolving Analytic Approaches. New York: Russell Sage.

Hill, Carolyn J., Howard S. Bloom, Alison Rebeck Black, and Mark W. Lipsey. 2008. 

“Empirical Benchmarks for Interpreting Effect Sizes in Research.” Child Development 

Perspectives 2, 3: 172-177. 

Kemple, James J. 2008. Career Academies: Long-Term Impacts on Work, Education, and 

Transitions to Adulthood. New York: MDRC.

Kemple, James J., Corinne M. Herlihy, and Thomas J. Smith. 2005. Making Progress Toward 

Graduation: Evidence from the Talent Development High School Model. New York: MDRC.

Kena, Grace, Susan Aud, Frank Johnson, Xiaolei Wang, Jijun Zhang, Amy Rathbun, Sidney 

Wilkinson-Flicker, and Paul Kristapovich. 2014. The Condition of Education 2014 (NCES 

2014-083). Washington, DC: National Center for Education Statistics, Institute of 

Education Sciences, U.S. Department of Education. 
120

Matheny, Adam P., Theodore D. Wachs, Jennifer L. Ludwig, and Kay Phillips. 1995. “Bringing 

Order Out of Chaos: Psychometric Characteristics of the Confusion, Hubbub, and Order 

Scale.” Journal of Applied Developmental Psychology 16, 3: 429-444.

Schochet, Peter Z., and Hanley Chiang. 2009. Technical Methods Report: Estimation and 

Identification of the Complier Average Causal Effect Parameter in Education RCTs (NCEE 

2009-4040). Washington, DC: National Center for Education Evaluation and Regional 

Assistance, Institute of Education Sciences, U.S. Department of Education. 

ScholarCentric. 2015. “Resiliency Curriculum.” Website:

www.scholarcentric.com/solutions/resiliency-curriculum. 

Search Institute. 2016. “Search Institute.” Website: www.search-institute.org.

Snipes, Jason C., Glee Ivory Holton, Fred Doolittle, and Laura Sztejnberg. 2006. Striving for Student 

Success: The Effect of Project GRAD on High School Student Outcomes in Three Urban 

School Districts. New York: MDRC.

Tangney, June P., Roy F. Baumeister, and Angie Luzio Boone. 2004. “High Self‐Control Predicts 

Good Adjustment, Less Pathology, Better Grades, and Interpersonal Success.” Journal of 

Personality 72, 2: 271-324.

Tuttle, Christina Clark, Brian Gill, Philip Gleason, Virginia Knechtel, Ira Nichols-Barrer, and 

Alexandra Resch. 2013. KIPP Middle Schools: Impacts on Achievement and Other 

Outcomes, Final Report. Washington, DC: Mathematica Policy Research.

University of Chicago Consortium on School Research. 2012. “My Voice, My School Survey.” 

Website: https://consortium.uchicago.edu. 

What Works Clearinghouse. 2014. What Works Clearinghouse: Procedures and Standards 

Handbook (Version 3.0). Washington, DC: U.S. Department of Education, Institute of 

Education Sciences, What Works Clearinghouse.
About MDRC

MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 

to learning what works to improve the well-being of low-income people. Through its research 

and the active communication of its findings, MDRC seeks to enhance the effectiveness of social and education policies and programs.

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 

for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 

Its projects are a mix of demonstrations (field tests of promising new program approaches) and 

evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 

combination of research and organizational experience to their work, providing expertise on the 

latest in qualitative and quantitative methods and on program design, development, implementation, and management. MDRC seeks to learn not just whether a program is effective but also 

how and why the program’s effects occur. In addition, it tries to place each project’s findings in 

the broader context of related research — in order to build knowledge about what works across 

the social and education policy fields. MDRC’s findings, lessons, and best practices are proactively shared with a broad audience in the policy and practitioner community as well as with the 

general public and the media.

Over the years, MDRC has brought its unique approach to an ever-growing range of policy areas and target populations. Once known primarily for evaluations of state welfare-to-work programs, today MDRC is also studying public school reforms, employment programs for exoffenders and people with disabilities, and programs to help low-income students succeed in 

college. MDRC’s projects are organized into five areas:

•

•

•

•

•

Promoting Family Well-Being and Children’s Development

Improving Public Education

Raising Academic Achievement and Persistence in College

Supporting Low-Wage Workers and Communities

Overcoming Barriers to Employment

Working in almost every state, all of the nation’s largest cities, and Canada and the United 

Kingdom, MDRC conducts its projects in partnership with national, state, and local governments, public school systems, community organizations, and numerous private philanthropies.