NCEE 2010-4019 U . S . D E PA R T M E N T O F E D U C AT I O N

Evaluation of the DC Opportunity 

Scholarship Program

Final Report

Executive Summary
Evaluation of the DC Opportunity 

Scholarship Program 

Final Report

Executive Summary

June 2010 

Patrick Wolf, Principal Investigator, University of Arkansas

Babette Gutmann, Project Director, Westat

Michael Puma, Chesapeake Research Associates

Brian Kisida, University of Arkansas

Lou Rizzo, Westat

Nada Eissa, Georgetown University

Matthew Carr, Westat

Marsha Silverberg, Project Officer, Institute of Education Sciences

NCEE 2010-4019

U.S. Department of Education
U.S. Department of Education 

Arne Duncan 

Secretary 

Institute of Education Sciences

John Q. Easton 

Director

National Center for Education Evaluation and Regional Assistance

John Q. Easton 

Acting Commissioner 

June 2010 

This report was prepared for the Institute of Education Sciences under Contract No. ED-04-CO-0126. The 

project officer was Marsha Silverberg in the National Center for Education Evaluation and Regional 

Assistance. 

IES evaluation reports present objective information on the conditions of implementation and impacts of 

the programs being evaluated. IES evaluation reports do not include conclusions or recommendations or 

views with regard to actions policymakers or practitioners should take in light of the findings in the 

reports.

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While 

permission to reprint this publication is not necessary, the citation should be: Wolf, Patrick, Babette 

Gutmann, Michael Puma, Brian Kisida, Lou Rizzo, Nada Eissa, and Matthew Carr. Evaluation of the DC 

Opportunity Scholarship Program: Final Report Executive Summary (NCEE 2010-4019). Washington, 

DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, 

U.S. Department of Education. 

To order copies of this report, 

• Write to ED Pubs, U.S. Department of Education, P.O. Box 22207, Alexandria, VA 22304. 

• Call in your request toll free to 1-877-4ED-Pubs. If 877 service is not yet available in your area, 

call 800-872-5327 (800-USA-LEARN). Those who use a telecommunications device for the deaf 

(TDD) or a teletypewriter (TTY) should call 1-877-576-7734. 

• Fax your request to 703-605-6794. 

This report also is available on the IES website at http://ies.ed.gov/ncee. 

Upon request, this report is available in alternate formats such as Braille, large print, audiotape, or 

computer diskette. For more information, please contact the Department’s Alternate Format Center at 

202-260-9895 or 202-205-8113. 

• Order online at www.edpubs.gov. 
iii

Acknowledgments

This report is the sixth of a series of annual reports mandated by Congress. We gratefully 

acknowledge the contributions of a significant number of individuals in its preparation and production. 

Staff from the Washington Scholarship Fund provided helpful information and have always 

been available to answer our questions. 

We are also fortunate to have the advice of an Expert Advisory Panel. Members include: 

Julian Betts, University of California, San Diego; Thomas Cook, Northwestern University; Jeffrey Henig, 

Columbia University; William Howell, University of Chicago; Guido Imbens, Harvard University; 

Rebecca Maynard, University of Pennsylvania; and Larry Orr, formerly of Abt Associates and now an 

independent consultant. 

The challenging task of assembling the analysis files was capably undertaken by Yong Lee, 

Quinn Yang, and Yu Cao at Westat. The management and conduct of the data collection was performed 

by Juanita Lucas-McLean and Bonnie Ho of Westat. Expert editorial and production assistance was 

provided by Evarilla Cover and Saunders Freeland of Westat. Jeffery Dean of the University of Arkansas 

ably assisted with the intermediate outcomes analysis and the drafting of chapter 4, and appendices E and 

H. Daniel Bowen helped to confirm the contents of the tables and in the production of appendix F. 
iv

Disclosure of Potential Conflicts of Interests1

The research team for this evaluation consists of a prime contractor, Westat, and two 

subcontractors, Patrick Wolf (formerly at Georgetown University) and his team at the University of 

Arkansas Department of Education Reform and Michael Puma of Chesapeake Research Associates 

(CRA). None of these organizations or their key staff has financial interests that could be affected by 

findings from the evaluation of the DC Opportunity Scholarship Program (OSP). No one on the sevenmember Expert Advisory Panel convened by the research team once a year to provide advice and 

guidance has financial interests that could be affected by findings from the evaluation.

 

1 Contractors carrying out research and evaluation projects for IES frequently need to obtain expert advice and technical assistance from 

individuals and entities whose other professional work may not be entirely independent of or separable from the particular tasks they are 

carrying out for the IES contractor. Contractors endeavor not to put such individuals or entities in positions in which they could bias the 

analysis and reporting of results, and their potential conflicts of interest are disclosed.
v 

Executive Summary 

The District of Columbia School Choice Incentive Act of 2003, passed by Congress in 

January 2004, established the first federally funded, private school voucher program in the United States. 

As part of this legislation, Congress mandated a rigorous evaluation of the impacts of the Program, now 

called the DC Opportunity Scholarship Program (OSP). This final evaluation report presents the longer

term effects of the Program on families who applied and were given the option to move from a public 

school to a participating private school of their choice. 

The evaluation compares the outcomes of 2,300 eligible applicants randomly assigned to 

receive an offer (treatment group) or not receive an offer (control group) of an OSP scholarship through a 

series of lotteries. Although data on most of these outcomes—test scores, high school graduation, 

perceptions of school safety and satisfaction—were collected annually over four or five years, each year’s 

estimated impacts are cumulative in that they represent students’ entire educational experience between 

their application to the Program and the year the data were obtained. Some students offered scholarships 

never used them, while others used their scholarships to attend a participating private school at some 

point during the four- to five-year period. Based on analysis of the final, spring 2009 data we find: 

• There is no conclusive evidence that the OSP affected student achievement. On 

average, after at least four years students who were offered (or used) scholarships had 

reading and math test scores that were statistically similar to those who were not 

offered scholarships (figure ES-2). The same pattern of results holds for students who 

applied from schools in need of improvement (SINI), the group Congress designated as 

the highest priority for the Program. Although some other subgroups of students 

appeared to have higher levels of reading achievement if they were offered or used a 

scholarship, those findings could be due to chance. They should be interpreted with 

caution since the results were no longer significant after applying a statistical test to 

account for multiple comparisons of treatment and control group members across the 

subgroups. 

• The Program significantly improved students’ chances of graduating from high 

school. Although students may not have raised their test scores in reading and math as 

a result of the OSP, they graduated at higher rates. The offer of an OSP scholarship 

raised students’ probability of completing high school by 12 percentage points overall 

(figure ES-3). The graduation rate based on parent-provided information was 82 

percent for the treatment group compared to 70 percent for the control group. The offer 

of a scholarship improved the graduation prospects by 13 percentage points for the 

high priority group of students from schools designated SINI in 2003-05 (79 percent 

graduation rate for the treatment group versus 66 percent for the control group). 
vi

• The OSP raised parents’, but not students’, ratings of school safety and satisfaction

(figures ES-4 and ES-5). Parents were more satisfied and felt school was safer if their 

child was offered or used an OSP scholarship. The Program had no effect on students’ 

reports on school conditions. 

The DC Opportunity Scholarship Program

The purpose of the new scholarship program was to provide low-income residents, 

particularly those whose children attend schools in need of improvement (SINI) or corrective action under 

the Elementary and Secondary Education Act, with “expanded opportunities to attend higher performing 

schools in the District of Columbia” (Sec. 303). The scholarship, worth up to $7,500, could be used to 

cover the costs of tuition, school fees, and transportation to a participating private school. The statute also 

directed that scholarships be awarded by lottery any year in which there are more eligible applicants than 

available scholarships or open slots in private schools and that priority in the lotteries be given first to 

students attending SINI public schools. 

The Program has been operated by the Washington Scholarship Fund (WSF). To date:

• 8,480 students have applied; 

• 5,547 have been deemed eligible; 

• 3,738 have been awarded Opportunity Scholarships; and

• 2,881 students used their scholarships within a year of receiving them.

The Program’s $13-14 million annual appropriation has been sufficient to support about 

1,700 scholarship students each year, if each student uses the full value of his or her scholarship. The 

Program enrolled 1,027 scholarship students in the fall of 2004, its initial year of partial implementation, 

and grew to its peak enrollment of 1,930 students in the fall of 2007 (figure ES-1). Language in a federal 

appropriations statute closed the Program to new applicants in the spring of 2009. In the fall of 2009, the 

OSP supported 1,322 continuing scholarship students attending 1 of the 52 private schools in the District 

participating in the Program that year. 
vii

Figure ES-1. Scholarship Users, Fall 2004-2009 

2,500

2,000 1,930

1,716 1,714

1,805

1,500

1,027

1,322

1,000

500

0

Fall 2004 Fall 2005 Fall 2006 Fall 2007 Fall 2008 Fall 2009

Scholarship users

SOURCE: WSF’s enrollment and payment files.

The Congressionally Mandated Evaluation of the OSP 

Guided by language in the statute, the evaluation of the OSP relied on lotteries of eligible 

applicants—random chance—to create two statistically equivalent groups who were followed over time 

and whose outcomes were compared to estimate Program impacts. A total of 2,308 eligible applicants in 

the first two years of Program implementation were entered into scholarship lotteries (492 in year one, 

called “cohort 1,” and 1,816 in year two, called “cohort 2”). Across the cohorts, 1,387 students were 

randomly assigned to the impact sample’s treatment group (offered a scholarship), while the remaining 

921 were assigned to the control group (not offered a scholarship). 

The OSP law also prescribed what types of impacts or outcomes would be assessed as part 

of the evaluation. These outcomes included student test-score performance in reading and math, 

educational attainment (in our case, parent reports of high school graduation), school safety, the success 

of the Program in expanding options (for which we have used “school satisfaction” as an indicator), and 

the effect of the OSP on District of Columbia Public Schools (DCPS) as well as private schools. To 

provide context for understanding the effects of the Program, the evaluation also provides a description of 

the patterns of school and student participation in the OSP. Data on these outcomes and issues were 
viii

collected primarily through annual surveys of parents, students in grade 4 or higher, and principals of both 

public and private schools in the District. Test scores were derived from evaluation-administered 

assessments using the SAT-9.2

The impacts of the Program were computed by comparing the outcomes of the treatment 

group with those of the control group, controlling for students baseline (pre-Program) reading and math 

scores and other demographic characteristics.3 The impacts of the Program were assessed for the 

complete sample of eligible study participants as well as for several student subgroups, including the high 

priority set of students who applied from SINI public schools.4

Program Impacts

This final report on the OSP examines the effects of the Program on students and their 

parents near the end of the 2008-09 school year. The analysis is both consistent with and different from 

that presented in prior evaluation reports examining shorter-term impacts. It is consistent in that impacts 

are presented in two ways: the impact of the offer of an OSP scholarship, derived straight from comparing 

the average outcomes of the treatment and control groups, and (2) the impact of using an OSP 

scholarship, statistically adjusting for students who declined to use their scholarships. Like the earlier 

reports, the final estimates provide impacts on achievement, safety, and satisfaction. 

Two parts of the analysis are different this year. First, in previous analyses, the two cohorts 

of students in the impact sample had the potential to experience the same number of years in the Program 

(e.g., three years after application). In spring 2009, the last year evaluation data were collected, cohort 1 

students who applied in 2004 (14 percent of the sample) could have used their scholarships for five years 

 

2

 By the 2008-09 school year, a total of 296 students (13 percent of the impact sample) had aged to the point where they would have completed 

12th grade based on their grade upon application to the Program. The primary outcome measure used for the evaluation, the Stanford 

Abbreviated Achievement Test, ninth edition (referred to as the SAT-9 and published by Harcourt Educational Measurement in San Antonio, 

Texas), does not have a version for students beyond 12th grade, so these students effectively “graded-out” of the achievement portion of the 

study for purposes of this final impact report. Among the remaining 2,012 members of the impact sample, 69.5 percent of both the treatment 

and control groups effectively responded to test score data collection efforts in the final year of the study. The data they generated were then 

adjusted to account for nonrespondents before the impact analysis was conducted. For the other measures, response rates were obtained from 

63 to 75 percent of eligible sample members, depending on the survey.

3

 There were no statistically significant differences in baseline measures between the treatment and control group samples overall and for those 

that provided data for this report’s analyses.

4

 In all four years of the impact evaluation, the subgroups included students who had attended SINI 2003-05 and not SINI 2003-05 public 

schools at the time of application, lower baseline test-score performers and higher baseline test-score performers, and males and females. In 

the first three years of the evaluation, impacts were also estimated for two additional subgroup pairs: cohort 1 and cohort 2, and students 

entering grades K-8 and grades 9-12 at baseline. By the final year of the evaluation, all of the students in the baseline grade 9-12 subgroup and 

most of the students in the cohort 1 subgroup had aged to the point that they were no longer eligible for an achievement test, making the cohort 

and grade-level subgroups too small to analyze reliably. 
ix

while cohort 2 students who applied a year later (86 percent of the sample) could only have used their 

scholarship for four years. For this reason, we refer to impacts as “after at least four years” since a small 

portion of the sample—both treatment and control—were in the study a year longer.5 Another important 

difference is that for the first time we are able to estimate the impacts of the Program on educational 

attainment. Most students who applied to the Program were in grades K-5. But by 2009, 22 percent of the 

impact sample (approximately 500 students) had aged to the point that they could have completed 12th 

grade and graduated from high school. This number of students was sufficient to reliably estimate impacts 

on this outcome; this is the first time random assignment has been used to estimate the causal relationship 

between a school voucher program (or private schooling) and educational attainment, thus providing a 

more rigorous estimate than previous studies that have addressed this issue. There are some limitations to 

this analysis, however: it is based on parent reports rather than school administrative records, and it 

represents a relatively small share of the study sample.

In examining the longer term impacts of the Program, we found: 

Student Achievement

• Overall reading and math test scores were not significantly affected by the Program, 

based on our main analysis approach. On average over the 40-plus months of potential 

participation, the treatment group scored 3.90 points higher in reading and .70 points 

higher in math than the control group, but these differences were not statistically 

significant (figure ES-2). 

• No significant impacts on achievement were detected for students who applied from 

SINI 2003-05 schools, the subgroup of students for whom the statute gave top priority, 

or for male students, or those who were lower performing academically when they 

applied. 

• The Program may have improved the reading but not math achievement of the other 

three of six student subgroups. These include students who came from not SINI 2003-

05 schools (by 5.80 scale score points), who were initially higher performing 

academically (by 5.18 points), or who were female (5.27 points). However, the impact 

estimates for these groups may be due to chance after applying a statistical test to 

adjust for multiple comparisons. 

 

5

 Combining the two cohorts in this way was necessary to ensure that the sample size (number of students) for analysis was sufficient to detect 

impacts of a policy-relevant size and to provide results that could be applied to both cohorts. We were unable to collect data from cohort 1 in 

their fourth year after application because the legislative decision to extend the OSP and the evaluation came too late.
x 

Figure ES-2. Achievement (SAT-9 Scale Score Points) After At Least Four 

Years

649.15 645.24 644.06 643.36

470

490

510

530

550

570

590

610

630

650

670

Treatment Control Treatment Control

Reading Math

NOTES: Results are for cohort 1 five years after random assignment and cohort 2 four years after random 

assignment. The possible range of SAT-9 scale scores varies by grade level. The value at which the 

x-axis intersects the y-axis in this figure (470) represents the minimum average reading score 

possible given the grade composition of the control group sample in the final year. The minimum 

average math score possible for the control group sample was 502. The maximum possible reading 

score and math score was 835 and 832, respectively. Valid N for reading = 1,328; math = 1,330. 

Separate reading and math sample weights used.

High School Graduation (Educational Attainment) 

• The offer of an OSP scholarship raised students’ probability of completing high school 

by 12 percentage points overall. The graduation rate based on parent-provided 

information6 was 82 percent for the treatment group compared to 70 percent for the 

control group (figure ES-3). There was a 21 percent difference (impact) for using a 

scholarship to attend a participating private school.

• The offer of a scholarship improved the graduation prospects by 13 percentage points 

for the high-priority group of students from schools designated SINI in 2003-05 (79 

percent for the treatment group versus 66 percent for the control group) (figure ES-3). 

The impact of using a scholarship on this group was 20 percentage points. 

• Two other subgroups had statistically higher graduation rates as a result of the 

Program. Those who entered the Program with relatively higher levels of academic 

performance had a positive impact of 14 percentage points from the offer of a 

scholarship and 25 percentage points from the use of a scholarship. Female students

 

6

 These data were obtained through follow-up telephone surveys with parents of students in the study forecasted to have completed 12th grade 

by the summer of 2009. A total of 63 percent of parents in the target sample responded to this survey.
xi 

had a positive impact of 20 percentage points from the offer of a scholarship and 28 

percentage points from the use of a scholarship. 

• The graduation rates of students from the other subgroups were also higher if they were 

offered a scholarship, but these differences were not statistically significant.

Figure ES-3. High School Graduation Rates for the Overall Sample and the 

SINI 2003-05 Subgroup, 2008-09 

Full Sample

90%

82%**

80%

70%

60%

50%

40%

30%

20%

10%

0%

Treatment

70%

Control

SINI 2003-05 Subgroup

79%*†

66%

Treatment Control

*Statistically significant at the 95 percent confidence level. 

**Statistically significant at the 99 percent confidence level.

† = subgroup impact result remained statistically significant after adjustments for multiple comparisons.

NOTES: Results are for cohort 1 five years after random assignment and cohort 2 four years after random 

assignment. Valid N = 316, including SINI 2003-05 N = 231, not SINI 2003-05 N = 85. High school 

graduation determined via parental self-reports.

School Safety and Satisfaction 

At least four years after random assignment, the OSP had a positive impact overall on parents’ 

ratings of school safety and satisfaction, but not on students’ reports of those same outcomes (figures 

ES-4 and ES-5). For example, parents were 8 percentage points more likely to give their child’s school a 

grade of A or B if offered a scholarship as compared with the control group; however, student reports of 

school satisfaction were comparable whether they were in the treatment or control groups. 
xii 

Figure ES-4. Parent Perceptions and Student Reports of Safety and an 

Orderly School Climate, 2008-09 

10.0

9.00

8.00

7.00

6.00

5.00

4.00

3.00

2.00

1.00

0.00

Group Means: Parent 

Perceptions

0

7.94**

7.47

Treatment Control

10.00

9.00

8.00

7.00

6.00

5.00

4.00

3.00

2.00

1.00

0.00

Group Means: Student 

Reports

6.16 6.01

Treatment Control

**Statistically significant at the 99 percent confidence level.

NOTES: Results are for cohort 1 five years after random assignment and cohort 2 four years after random 

assignment. Parent perceptions are based on a 10-point scale; student reports are based on an 8-point 

scale. For parent perceptions, valid N = 1,224; parent survey weights were used. For student reports, 

valid N = 1,054; student survey weights were used. The survey was given to students in grades 4-12. 

Means are regression adjusted using a consistent set of baseline covariates.
xiii

Figure ES-5. Parent and Student Reports of School Satisfaction, 2008-09 

Group Means: Percentage of 

Parents Who Gave School 

Grade A or B

Group Means: Percentage of 

Students Who Gave School 

Grade A or B

100%

80%

60%

40%

20%

0%

100%

80%

60%

40%

20%

0%

76%**

68%

72%

69%

Treatment Control Treatment Control

**Statistically significant at the 99 percent confidence level.

NOTES: Results are for cohort 1 five years after random assignment and cohort 2 four years after random 

assignment. For parent reports, valid N = 1,227; parent survey weights were used. For student reports, 

valid N = 1,001; student survey weights were used. The survey was given to students in grades 4-12. 

Means are regression adjusted using a consistent set of baseline covariates.

Program Context: Student and School Participation in and Response to the OSP

Understanding how and under what conditions the Program operated is important context for 

interpreting the impacts. For example, the degree to which students used their scholarships provides some 

signal of the attractiveness of the OSP and the ability of the Program and its participating schools to 

accommodate student needs. How the characteristics of the private schools differed from the public 

school options available may have influenced parent choices and students’ educational experiences.

Public and private schools’ exposure to the OSP, through enrollment losses and gains, and any changes 

principals made to retain or attract students could indicate a more complete picture of the OSP and its 

potential for affecting the public and private schools in the area. 
xiv 

Students

As has been true in other school choice programs, not all students offered an OSP 

scholarship actually used it to enroll in a participating private school. And over the years, some students 

lost their eligibility for the Program. For example, by 2008-09, a total of 94 of the 1,387 members of the 

treatment group were no longer eligible to receive scholarships because they had “graded out” of the 

Program, which means that they would have moved beyond 12th grade. Looking across the remaining 

members of the impact sample’s treatment group who had four (cohort 1) or five (cohort 2) years of 

potential Program participation: 

• 282 out of 1,293 (22 percent) never used the OSP scholarships offered to them. 

• 660 treatment students (51 percent) used their scholarships, but not consistently, during 

the school years after the scholarship award. Among these students are an estimated 

147 who may have been forced by circumstances to stop using their scholarship. 

Students could become “forced-decliners” because the school they continued to attend 

converted from a participating Catholic school to a public charter school (confirmed 

for 35 treatment students),7 their family income grew to exceed the Program’s income 

limit (confirmed for 21 treatment students), their family moved out of DC (confirmed 

for 29 students), or they may have faced a lack of space for them in a participating high 

school when they transitioned from 8th to 9th grade (estimated for 62 treatment 

students).8

• The remaining 351 treatment group students (27 percent) used their scholarship during 

all years available to them after the scholarship lottery.

Among the students who partially used their scholarship over at least four 

years after random assignment, 17 percent (9 percent of eligible treatment group 

students overall) used their OSP scholarship in 2008-09. 

Across the years, the most common reasons given by parents for never using an OSP scholarship that was 

awarded to their child was a lack of space at their preferred private school (30.7 percent), the absence of 

special needs services (21.6 percent), and that their child was admitted to a preferred public charter school 

(16.3 percent) (figure ES-6). 

 

7

 Based upon survey data, 35.9 percent of 97 treatment group students who used a scholarship to attend one of these Catholic schools in grades 

K-7 in 2007-08 continued to attend the same school when it converted to a public charter school in 2008-09.

8 The estimate of the number of students forced to decline their scholarships due to the lack of high school slots was calculated by comparing the 

higher rate of scholarship continuation for 7th graders moving to 8th grade with the lower rate of scholarship continuation for 8th graders 

moving to 9th grade. The difference between those two continuation rates, applied to the number of OSP students moving from 8th to 9th 

grade, generates the estimate of forced decliners due to high school slot constraints of 62 (20 in year two plus 30 in year three plus 12 new 

cases in 2008-09). It is impossible to know for certain if all 62 of these students declined to use the scholarship solely or primarily because of 

high school slot constraints, and not for other reasons, or if some treatment students were forced to decline their scholarship at the very start 

due to high school slot constraints. It also is impossible to know if some students declined to even attempt to renew their scholarships because 

they knew their family exceeded the income limit, or how many treatment students moved out of DC and never informed the evaluators that 

they had “moved out” of Program eligibility. Therefore, the total estimate of 147 forced decliners for 2008-09 is simply an estimate based on 

the limited data available.
xv 

Figure ES-6. Reasons Given by Parents of Treatment Students for Never 

Using an OSP Scholarship 

Lack of space 30.7

Lack of special needs services 21.6

Child got into a charter school 16.3

Did not want to leave friends 13.4

Preferred private school not participating 10.8

Moved out of DC 8.8

Transportation issues 7.8

Did not pass admission test 4.2

Child did not want to be held back a grade 3.6

Concerns the work might be too hard 2.9

Lack of sports 2.9

Child did not want religious instruction 2.3

Child did not want to wear a uniform 2.0

Other 0.7

0 5 10 15 20 25 30 35

Percent of parents

NOTES: Responses are unweighted. Respondents were able to select multiple responses each year, and 

some respondents participated in data collection for multiple years. Percentages given represent the 

sum of all responses obtained across years one through four of data collection (i.e., longitudinal 

responses) divided by the sum of all respondents (N = 306) across all of those same years (i.e., 

longitudinal respondents). As a result, this figure includes initial responses from parents of students 

who subsequently graded out of the Program. Categories with responses from fewer than three 

parents in any year are collapsed into the “Other reasons” category for confidentiality reasons.

SOURCE: Impact Evaluation Parent Surveys.

Among students who initially used a scholarship but then left the Program, the most 

common reasons for leaving were that the child was admitted to a preferred public charter school (21.8 

percent), a lack of space at their preferred private school (18.5 percent), and that the family moved out of 

DC (15.2 percent) (figure ES-7). 
xvi

Figure ES-7. Reasons Given by Parents of Treatment Students for Not 

Continuing to Use an OSP Scholarship

1.4

1.4

3.3

3.8

4.3

6.2

7.6

8.5

12.3

13.7

15.2

18.5

21.8

0 5 10 15 20 25

Child did not want to wear a uniform

Did not pass admission test

Child's public school teachers are better

Child did not want to be held back a grade

Did not want to leave friends

Concerns the work might be too hard

Lack of sports

Preferred private school not participating

Lack of special needs services

Transportation issues

Moved out of DC

Lack of space

Child got into a charter school

Percent of parents

NOTES: Responses are unweighted. Respondents were the parents of treatment students who used a 

scholarship in a previous year but not in a subsequent year (N = 211). The reasons for not using 

were drawn from the parent responses the first year after their child stopped using a scholarship. 

Respondents appear in the data only one time (i.e., unique respondents), though they may have 

provided multiple reasons for not continuing to use a scholarship. This figure includes initial 

responses from parents of students who subsequently graded out of the Program.

SOURCE: Impact Evaluation Parent Surveys.

Schools

Fifty-two of 90 private schools in the District of Columbia were participating in the Program 

at the start of the 2008-09 school year, down from a peak of 68 schools in 2005-06.9

 

Among the 22 

schools that participated at some point but left the Program are seven Catholic schools that, in their last 

year in the Program (2007-08), enrolled 112 treatment group students; these schools converted to become 

public charter schools in 2008-09 and therefore no longer could be OSP voucher recipients. Overall in 

2008-09, the last year of the evaluation, 14 percent of treatment group students attended a private school 

9 While, technically, 56 individual campuses were participating in the OSP from the start of the 2008-09 school year, the research team treats 

four of the schools with dual campuses as single entities because they have one financial office that serves both campuses, following the 

classification practice used by the National Center for Education Statistics in its Private School Survey. The 52 schools represent a net loss of 

nine schools since the prior year. Eleven schools stopped participating, while two new schools participated for the first time in 2008-09. The 

total number of private schools operating in DC declined from 109 in 2004-05 to 90 in 2008-09.
xvii

that charged tuition above the statutory cap of $7,500, and 80 percent attended a faith-based school, with 

most of them (53 percent) attending the 15 participating Catholic parochial schools (table ES-1). 

Table ES-1. Features of Participating OSP Private Schools Attended by the 

Treatment Group in 2008-09 

Characteristic Weighted Mean

Charging over $7,500 tuition (percent of treatment students attending) 14.2%

Tuition $7,252

Enrollment 292.1

Faith-based 81.7%

Archdiocesan Catholic 53.3%

NOTES: School N for tuition amounts and religious affiliations = 38; N for enrollment totals = 31. When a tuition 

range was provided, the mid-point of the range was used. The weighted mean was generated by associating 

each student with the characteristics of the school he/she was attending and then computing the average of 

these student-level characteristics. 

SOURCES: OSP School Directory information, 2008-09, WSF; National Center for Education Statistics’ Private School 

Survey, 2007-08.

The schools attended by the evaluation’s treatment group (both those who used their 

scholarship to enroll in a participating private school and those who did not) differed in some ways from 

the schools attended by students who were not offered scholarships (the control group) in 2008-09:10

• Students in the treatment group were less likely than those in the control group to 

attend a school that offered special programs for students who may be academically 

challenged; these include programs or services for non-English speakers (32 vs. 57 

percent) and for students with learning problems (75 vs. 90 percent); 

• Students in the treatment group were less likely to be in schools with special programs 

for advanced learners (38 vs. 49 percent); and 

• Students in the treatment group were less likely than those in the control group to 

attend a school with a cafeteria facility (76 vs. 91 percent), a nurse’s office (50 vs. 82 

percent), counselors (77 vs. 87 percent), and art programs (84 vs. 92 percent).

These features of the public and private schools in DC in 2008-09 could, hypothetically, reflect a 

response by the schools to the OSP. School choice theory suggests that a thriving private school 

scholarship program provides competition to the public schools and could generate improvements to the 

public school system, the private school system, or both (see Chubb and Moe 1990; Henig 1994). Such 

systemic changes could take place if significant percentages of students in the public school system, or in 

 

10 Differences in the characteristics of schools are noted here only if the difference was statistically significant at the .05 level or higher. In 2008-

09, statistically similar proportions of the treatment and control groups were enrolled in schools that offered a computer lab (95 vs. 91 percent),

separate library (77 vs. 79 percent), gyms (68 and 71 percent), individual tutors (58 vs. 63 percent), music programs (93 vs. 91 percent), and 

after-school programs (91 vs. 88 percent).
xviii

specific schools, apply for, receive, and use scholarships to transfer to private schools. Systemic changes 

also could occur in the private sector, if private schools adjust program operations to better attract or 

retain scholarship students. 

As mandated in the statute, we examined how DC public and private schools were affected 

by the OSP by analyzing how these underlying components of the competitive school theory played out 

in DC. A maximum of 1,700 to 2,000 students—about 3 percent of those in DCPS public schools 

(including charter schools)—could be supported by the OSP to attend a private school in any year. We

found that 3.2 percent of students in the DC public schools, cumulatively, used an OSP scholarship to 

transfer out between 2004 and 2009, joining continuing students in the Program. OSP-related student 

transfers ranged from 0 to 21 percent of enrollment across individual schools during that period. Just over 

one-quarter (28 percent) of public school principals reported making any changes to their operations in 

order to retain students who might be interested in the OSP or private schools in general (figure ES-8).11

On average, OSP students made up 16 percent of participating private schools’ student populations, with 

a range of 0 to 65 percent.12 Fifty-two percent of principals at those schools indicated they had made 

changes to encourage OSP students to attend their schools (figure ES-9).13

These findings can be placed in some context. The cumulative exposure of DCPS to the 

Program across five cohorts of students using OSP scholarships represents less than one-fifth of the 

average annual mobility of students in the district (3 percent versus 20 percent).14

 

Given these figures, 

OSP-related transfers to private schools may not have been distinguishable from the larger share of other 

student departures. In addition, school choice theory suggests that if any significant system-wide change 

in public schools is likely, the loss of students to the scholarship program should also entail a loss of 

funding for the public schools and school system affected by such transfers (Hoxby 2003). However, the 

law that established the OSP ensured that DCPS would gain, rather than lose, funds, and district officials 

11 The response rate for the public school principal survey in 2008-09 was 75 percent.

12 Private schools were deemed by the WSF, the program operator, as participating if they agreed to take OSP vouchers even if no OSP students 

were admitted.

13 The response rate for the private school principal survey in 2008-09 was 72 percent.

14 “A student is defined as ‘mobile’ if the student attended a different school or was not enrolled in the snapshot from the prior month.” 

http://www.osse.dc.gov/seo/lib/seo/dc_student_mobility_report2008_06_10.pdf. The DC Office of the State Superintendent of Education 

(OSSE) reported the average monthly student mobility rate in the district was about 2 percent from the fall of 2007 to the spring of 2008.

Taken across a 10-month school year, that monthly rate translates into an annual average mobility rate of 20 percent.
xix 

were not given information to determine how many students left individual public schools as a result of 

the Program.15

Figure ES-8. Public School Responses to the OSP, 2008-09 

8%

14%

15%

17%

20%

20%

21%

22%

24%

28%

0% 5% 10% 15% 20% 25% 30%

Altered class sizes

Offered additional courses (e.g., computer 

technology or art)

Adjusted disciplinary rules

Increased school safety provisions

Added tutoring or other special services to help 

improve academic achievement

Promoted school through the use of flyers, radio ads, 

newspaper ads, or other methods of advertising

Made efforts to improve the physical appearance of 

building

Added parent orientation or meeting to describe 

school offerings and performance

Encouraged greater parental involvement in school 

activities

Reported making any changes to retain students

Percent of principals making a change

NOTES: Responses are unweighted. Respondents were able to select multiple responses. The survey 

question is “In the past five years or since you became principal, have you made any changes 

specifically to encourage students interested in private schools (or the Opportunity Scholarship 

Program) to remain enrolled in your school?” If the principal answered yes, then the principal was 

asked to indicate which (of the following) changes were made. For all percentages, the numerator 

is the number of principals who answered “yes” to making a change and the denominator is the 

total number of survey respondents (N=168). The response rate for the public school principal 

survey was 75 percent.

SOURCE: Impact Evaluation Public School Surveys, 2008-09. 

 

15 The appropriations law that established the OSP and each subsequent appropriations bill that funded the Program provided approximately $13 

million for the OSP, approximately $13 million for DC charter schools, and approximately $13 million for the traditional public schools in 

DCPS. Because of the confidentiality provisions in the law, neither WSF nor IES could reveal information about which or how many students 

left individual DC schools.
xx 

Figure ES-9. Participating Private School Responses to the OSP, 2008-09

Reported making any changes to attract students 52.3

Participated in one or more WSF school fairs 38.6

Promoted school through the use of flyers, radio ads, 

newspaper ads, or other methods of advertising 31.2

Added parent orientation or meeting to describe 

school offerings and performance 27.3

Encouraged greater parent involvement in school 

activities 27.3

Added tutoring or other special services to help 

improve academic achievement 25.0

Made efforts to improve the physical appearance of 

school 25.0

Adjusted disciplinary rules 11.4

Altered class sizes 6.8

Other 11.4

0 20 40 60

Percent of principals making change

NOTES: Responses are unweighted. Respondents (N = 44) were able to select multiple responses. 

Categories with responses from fewer than three principals are collapsed into the “Other reasons” 

category for confidentiality purposes. The response rate for the private school principal survey was 

72 percent.

SOURCE: Impact Evaluation Principal Surveys, 2008-09.

This final report on the impacts of the OSP adds to the growing body of evidence on private 

school voucher programs in the United States. As is the case with previous evaluations of such programs, 

our study had some limitations. We studied early Program applicants, and not all members of our original 

sample participated in data collection each year. By the final year, 13 percent of the students were no 

longer eligible to take the K-12 achievement assessment because they had “graded out,” reducing the size 

of the analysis sample and the precision we had to detect effects. In addition, some of our measures, 

including high school graduation and school safety, are based on respondents’ recall and perceptions and 

not on more conclusive administrative records. Finally, it is important to note that the findings in this 

report are a reflection of the particular Program elements that evolved from the law passed by Congress 

and the characteristics of the students, families, and schools, both public and private, that exist in the 

Nation’s capital. The same program implemented in another city might yield different results, and a 

different scholarship program administered in Washington, DC, might also produce different outcomes. 
xxi 

References 

Chubb, John E., and Terry M. Moe. Politics, Markets, and America’s Schools. Washington, DC: The 

Brookings Institution Press, 1990. 

Henig, Jeffrey. Rethinking School Choice: Limits of the Market Metaphor. Princeton, MA: Princeton 

University Press, 1994. 

Hoxby, Caroline M. “School Choice and School Competition.” Swedish Economic Policy Review 2003, 

10: 11-67.