**U.S. Department of Education**

Institute of Education Sciences

National Center for Education Evaluation and Regional Assistance

**Evaluation of the DC Opportunity Scholarship Program: First Year
Report on Participation**

April 2005

Evaluation of the DC Opportunity Scholarship Program: First Year Report
on Participation

April 2005

Patrick Wolf, Principal Investigator, *Georgetown University*

Babette Gutmann, Project Director, *Westat*

Nada Eissa, *Georgetown University*

Michael Puma, *Chesapeake Research Associates*

Marsha Silverberg, Federal Project Officer

*Institute of Education Sciences*

U.S. Department of Education/Institute of Education Sciences

National Center for Education Evaluation and Regional Assistance (NCEE)

**U.S. Department of Education**

Margaret Spellings

*Secretary*

**Institute of Education Sciences**

Grover J. Whitehurst

*Director*

**National Center for Education Evaluation and Regional Assistance**

Phoebe Cottingham

*Commissioner*

**April 2005**

This report was prepared for the Institute of Education Sciences under
Contract No. ED-01-CO-0082/0016. The project officer was Marsha
Silverberg in the National Center for Education Evaluation and Regional
Assistance. The views expressed herein are those of the contractor.

This publication is in the public domain. Authorization to reproduce it
in whole or in part for educational purposes is granted.

**Suggested Citation**

Wolf, Patrick, Babette Gutmann, Nada Eissa, Michael Puma, and Marsha
Silverberg. *Evaluation of the DC Opportunity Scholarship Program: First
Year Report on Participation.* U.S. Department of Education, National
Center for Education Evaluation and Regional Assistance. Washington, DC:
U.S. Government Printing Office, 2005.

This report is available on the Institute of Education Sciences' Web
site at: [www.ed.gov/ies/ncee](http://ies.ed.gov/ncee/)

ACKNOWLEDGEMENTS

This report is the first of a series of annual reports, as mandated by
Congress. We gratefully acknowledge the contributions of a significant
number of individuals in its preparation and production.

Staff from the U.S. Department of Education and the Mayor's Office
provided ongoing support throughout the process. Special recognition and
thanks go to Marsha Silverberg at the Institute of Education Sciences'
National Center for Education Evaluation, the Contracting Officer's
Representative for this project, for her contributions and her
encouragement. Guidance and comments were also received by Ricky Takai,
Director of IES' National Center for Education Evaluation and Phoebe
Cottingham, Commissioner of IES' National Center for Education
Evaluation and Regional Assistance; Nina Rees, Assistant Deputy
Secretary for the Office of Innovation and Improvement and Michael
Petrilli, Associate Assistant Deputy Secretary for the Office of
Innovation and Improvement; and Michelle Walker, Senior Advisor for
Education, Executive Office of the Mayor of the District of Columbia.

Staff from the Washington Scholarship Fund, the Public School Aid
Service, and the District of Columbia Public Schools provided critical
data and were always there to answer our many questions.

We are also fortunate to have the advice of an Expert Advisory Panel.
Members include: Julian Betts, University of California, San Diego;
Thomas Cook, Northwestern University; Jeffrey Henig, Columbia
University; William Howell, Harvard University; Guido Imbens, University
of California; Rebecca Maynard, University of Pennsylvania; Larry Orr,
Abt Associates.

The challenging task of assembling the many data files was capably
undertaken by Robert Schrack and Yong Lee at Westat. Providing
invaluable assistance in the many analyses were Mike Ingram and Dan
Hoople of Georgetown University. The management and conduct of the data
collection was performed by Juanita Lucas-McLean and Kevin Jay of
Westat. Expert editorial and production assistance was provided by
Evarilla Cover, Saunders Freeland, Joan Murphy, and Beth Sinclair of
Westat. Insightful comments throughout the course of the work were
provided by Tom Stewart of Qwaku Productions. Administrative support for
the Georgetown University project activities was provided ably by
Stephen Cornman.

**Contents**

[Page]{.underline}

Acknowledgements i

Executive Summary ix

1\. Introduction 1

1.1 District of Columbia School Choice Incentive Act of 2003 1

1.2 Congressionally Mandated Evaluation 2

1.3 Organization of This Report 10

2\. Program Implementation in 2004: Recruitment and Applications 11

2.1 Schools 11

2.2 Families and Students 16

3\. Scholarship and Placement Lotteries and Initial Use of Scholarships
Awarded 19

3.1 Statutory Priorities 19

3.2 Scholarship Lottery Design and Implementation 20

3.3 Placement Lotteries 27

3.4 Scholarship Usage 30

4\. Characteristics of Program Applicants 33

4.1 Public School Applicants Compared to Public Nonapplicant Students in
the District 33

4.2 Applicants in the Impact and Non-Impact Samples 41

4.3 Eligible Applicants by Type of Previous School 43

4.4 Scholarship Users and Nonusers 46

4.5 Applicant Response Rates Among Public and Private Schools 50

Appendix A. Analysis of Study Power A-1

Appendix B. Characteristics of Eligible Applicants versus Nonapplicants:
DC Public School Students B-1

Appendix C. Characteristics of Student Eligible Applicants by Type of
Previous School C-1

Appendix D Characteristics of Scholarship Users and Nonusers D-1

**List of Tables**

[Page]{.underline}

Table ES-1 Number of DC Private Schools Participating in the DC
Opportunity Scholarship Program: 2004-05 x

Table ES-2 Number and Percentage of Participants, by Application Status:
Spring 2004 xi

Table ES-3 Characteristics of DC Public School Free or Reduced-Price
Lunch Program Students, Program Applicants Versus Nonapplicants: Spring
2004 xv

Table ES-4 Public School-Level Scholarship Application and Usage Rates:
Spring 2004 and Fall 2004 xviii

Table 1-1 Data Sources 5

Table 2-1 Number of DC Private Schools Participating in the DC
Opportunity Scholarship Program: 2004-05 12

Table 2-2 Enrollment, Race/Ethnicity, and Student-Teacher Ratio of
Students Served by Participating Private Schools Compared with Regular
DCPS Schools: 2003-04 13

Table 2-3 Services, Programs, and Policies Available at Participating
Schools: 2003-04 14

Table 2-4 Average Tuitions Charged by Participating and Nonparticipating
Private Schools: 2004-05 15

Table 2-5 Number and Percentage of Participants, by Application Status:
Spring 2004 18

Table 3-1 Eligible Public School Applicants and Available Private School
Slots: Spring 2004 and Fall 2004 22

Table 3-2 Lottery Probabilities and Assignments, by Applicant Type and
Grade-Level Band: Spring 2004 24

Table 3-3 DC Opportunity Scholarship Program Applicants, by Receipt of
Scholarship and Inclusion in the Impact Analysis Sample: Spring 2004 26

Table 3-4 Results of the Computerized Placement Lottery, by Priority
Group: Spring 2004 30

Table 3-5 Final Results of Lottery and WSF Student Placements, by
Priority Group: Spring 2004 31

Table 4-1 Characteristics of DC Public School Students, Program
Applicants Versus Nonapplicants: Spring 2004 35

Table 4-2 Characteristics of DC Public School Free or Reduced-Price
Lunch Program Students, Program Applicants Versus Nonapplicants: Spring
2004 37

**List of Tables (continued)**

[Page]{.underline}

Table 4-3 Characteristics of DC Public School Free or Reduced-Price
Lunch Students by Grade Band, Program Applicants Versus Nonapplicants,
Spring 2004 39

Table 4-4 Characteristics of DC Public School Free or Reduced-Price
Lunch Students by Grade Band, Program Participants Versus Nonapplicants:
Spring 2004 40

Table 4-5 Characteristics of the Impact Sample and Non-Impact Sample for
Applicants Entering Grades 6 to 12, Spring 2004 42

Table 4-6 Percentage of Applicants in Various Grade Bands, By Type of
School Previously Attended: Spring 2004 44

Table 4-7 Educational Characteristics of Eligible Applicants by School
Type: Spring 2004 45

Table 4-8 Demographic Characteristics of Eligible Applicants by School
Type: Spring 2004 47

Table 4-9 Educational and Demographic Characteristics of the Users and
Nonusers in the Impact Sample and Program Sample, Spring 2004 49

Table 4-10. Public School-Level Scholarship Application and Use Rates:
Spring 2004 and Fall 2004 51

Table 4-11 Private School Reliance on Scholarship Program: Fall 2004 52

Table A1 Required Sample Size: No Baseline Scores, Balanced Treatment
and Control Groups A-3

Table A2 Required Sample Size: No Baseline Scores, Treatment Group
Larger Than Control Group A-3

Table A3 Required Sample Size: Baseline Scores, Balanced Treatment and
Control Groups A-4

Table A4 Required Sample Size: Baseline Scores, Treatment Group Larger
Than Control Group A-4

Table B1 Characteristics of District Public School Students, by Grade
Band, Program Applicants Versus Nonapplicants: Spring 2004 B-2

Table C1 Percent of Students Entering Various Grades, Eligible
Applicants: Spring 2004 C-2

Table C2 Student's Race and Gender, Eligible Applicants: Spring 2004 C-2

Table C3 Adult Applicant's Relationship to Student, Eligible Applicants:
Spring 2004 C-3

Table C4 Parental Involvement with Child's Education, Eligible
Applicants: Spring 2004 C-3

Table C5 School Facilities and Homework, Eligible Applicants: Spring
2004 C-4

**List of Tables (continued)**

[Page]{.underline}

Table C6 Problems at Child's School, Eligible Applicants: Spring 2004
C-5

Table C7 Parental Satisfaction with School, Eligible Applicants: Spring
2004 C-6

Table C8 Valued Features of Schools, Eligible Applicants: Spring 2004
C-7

Table D1 Student's Educational Characteristics, Users and Nonusers:
Spring 2004 D-2

Table D2 Percent of Students by Forecasted Grade Level, Users and
Nonusers: Spring 2004 D-3

Table D3 Family Income, Users and Nonusers: Spring 2004 D-4

Table D4 Mother's Employment and Residence, Users and Nonusers: Spring
2004 D-5

Table D5 Mother's Education, Users and Nonusers: Spring 2004 D-6

Table D6 Mother's Marital Status and Household Composition, Users and
Nonusers: Spring 2004 D-7

Table D7 Mother's Ethnicity, Users and Nonusers: Spring 2004 D-7

Table D8 Student's Ethnicity and Gender, Users and Nonusers: Spring 2004
D-8

Table D9 Parental Involvement with Child's Education, Users and
Nonusers: Spring 2004 D-9

Table D10 School Facilities and Homework, Users and Nonusers: Spring
2004 D-9

Table D11 Problems at Child's School, Users and Nonusers: Spring 2004
D-10

Table D12 Parental Satisfaction with School, Users and Nonusers: Spring
2004 D-11

Table D13 Valued Features of Schools, Users and Nonusers: Spring 2004
D-12

**List of Figures**

[Page]{.underline}

Figure ES-1 Eligible Public School Applicants and Available Private
School Slots, by Grade-Level Band: Spring 2004 and Fall 2004 xii

Figure ES-2 Scholarship Probability, by Type of Applicant and
Grade-Level Band: Spring 2004 xiii

Figure 2-1 DC Private School Participation in the Program: 2004-05 11

Figure 3-1 Eligible Public School Applicants and Available Private
School Slots, by Grade-Level Band: Spring 2004 and Fall 2004 21

Figure 3-2 Scholarship Probability, by Type of Applicant and Grade-Level
Band: Spring 2004 23

Figure 3-3 Participating Private Schools Selected by Scholarship
Recipients, by Ward: Spring 2004 32

EXECUTIVE SUMMARY

The District of Columbia School Choice Incentive Act of 2003 was passed
by Congress in January 2004. The Act provided funds for District of
Columbia Public Schools (DCPS) improvement activities and charter school
facility acquisitions. Most notably, the statute established what is now
called the DC Opportunity Scholarship Program―the first federal
government initiative to provide K-12 education scholarships, or
vouchers, to families to send their children to private schools of
choice.

The DC Opportunity Scholarship Program has the following programmatic
elements:

-   To be eligible, students entering grades K-12 must reside in the
    District and have a family income at or below 185 percent of the
    federal poverty line.

-   Participating students will receive scholarships of up to \$7,500 to
    cover the costs of tuition, school fees, and transportation to a
    participating private school of choice.

-   Scholarships are renewable for up to 5 years (as funds are
    appropriated), as long as students remain eligible for the program
    and remain in good academic standing at the private schools they are
    attending.

-   If there are more eligible applicants than available scholarships or
    open slots in private schools, applicants are to be awarded
    scholarships and admission to private schools random selection, for
    example by lottery.

-   Private schools participating in the program must be located in the
    District, and agree to program requirements regarding
    nondiscrimination in admissions, fiscal accountability, and
    cooperation with the evaluation.

The Act requires that this 5-year scholarship program be rigorously
evaluated by an independent research team, using the "strongest possible
research design for determining the effectiveness" of the program and
addressing a specific set of student comparisons and topics (Section
309). The evaluation thus has several components: (1) an impact
analysis, comparing outcomes of eligible applicants (students and their
parents) from public schools randomly assigned to receive or not receive
a scholarship through a lottery, and (2) a performance reporting
analysis, comparing all students participating in the scholarship
program to students in the same grades in DCPS. All participating
students includes those randomly assigned scholarships and those who
received scholarships automatically, those who were attending public
schools and those attending private schools when they entered the
scholarship program. Because DCPS students who did not apply to the
scholarship program are likely to be quite different from those who
applied and are participating, the impact analysis will be the source of
the reliable, causal evidence on program effectiveness called for in the
legislation.

This document is the first of a series of annual reports from the
evaluation team, as mandated by Congress. Because the initial cohort of
program participants---those who applied in spring 2004 to receive
scholarships for the 2004-05 school year---just recently matriculated at
their new schools, no impact information is available at this time.
Instead, the report describes the purposes and design of the scholarship
program, the first-year implementation activities that generated 1,848
eligible applicants and 58 participating private schools, the process of
awarding scholarships to 1,366 student applicants, and the
characteristics of both applicants and scholarship users. The report
provides an important foundation for the later examination of program
impacts.

# Program Implementation in 2004: Recruitment and Applications (Chapter 2)

The Washington Scholarship Fund (WSF) was awarded a grant by the U.S.
Department of Education (ED) Office of Innovation and Improvement, in
partnership with the DC Mayor's Office, to implement the program,
starting in March 2004. Despite the challenges stemming from the late
start of the program, the implementers recruited 58 schools to
participate in the program in some capacity in 2004-05 and obtained
applications from 1,848 students deemed eligible for the program.

*Participating Schools*

The 58 private schools participating in the program during its inaugural
year represent 53 percent of all private schools in the District (Table
ES-1). All but four of the schools made new slots in their schools
available to scholarship winners. Four schools were willing to enroll
scholarship students only if they had been accepted to the school for
the 2004-05 school year prior to the launch of the scholarship program.

**Table ES-1. Number of DC Private Schools Participating in the DC
Opportunity\
Scholarship Program: 2004-05**

  ------------------------------------------------ ----------- -------------
  **Private Schools**                              **Number of **Percent**
                                                   Schools**   

  In the District of Columbia                      109         100

  Participating in the program in some capacity    58          53

  Set aside slots for DC Opportunity Scholarship   54          50
  Program participants                                         

  Agreed to accept scholarships only for eligible  4           4
  students already admitted                                    
  ------------------------------------------------ ----------- -------------

NOTE: Detail may not sum to totals because of rounding.

SOURCE: "School Directory, D.C. K-12 Scholarship Program, 2004-05 School
Year," Washington\
Scholarship Fund, June 2004.

The characteristics of the private schools that chose to participate in
the program the first year include the following:

-   Mostly religiously affiliated: about 51 percent are affiliated with
    the Roman Catholic Church, another 21 percent are affiliated with
    various non-Catholic religions, and approximately 28 percent are
    independent private schools.

-   Long established in the area: more than three-quarters of the
    schools have been in existence since 1983 and the most recently
    established participating private school opened in 2002.

-   Already serve a high proportion of students of color: on average, 82
    percent of students in participating private schools are from
    minority racial/ethnic groups, compared to 95 percent for DCPS
    schools.

-   On average, have lower school size and student-teacher ratios than
    do DCPS schools.

-   Mostly (about 70 percent) charge tuitions that are under the \$7,500
    maximum provided through the federal scholarship program.

-   More likely to be religiously affiliated and serve higher
    proportions of students of color than are private schools that chose
    not to participate in the program the first year. Nonparticipating
    schools also tend to charge higher tuitions and have smaller class
    sizes than do private schools that are currently involved in the
    program.

### Participating Families and Students

The program implementer -- WSF -- conducted most of the outreach to and
recruitment of families between March and May 2004. Perhaps as many as
40,000 DC children were eligible for the scholarship program, based on
data from the U.S. Census (Table ES-2). Inquiries about the program were
made on behalf of almost 6,000 students, and nearly 2,700 applications
were submitted during the recruitment period. A total of 1,848
applicants (69 percent of those who applied) provided all of the
required documents and were deemed eligible for the program. Seventy-two
percent of those eligible applicants were attending public school during
2003-04, whereas 28 percent were already attending private schools but
met the eligibility requirements in the statute.

**Table ES-2. Number and Percentage of Participants, by Application
Status: Spring 2004**

  -------------- ------------ --------------- ---------------- -------------- -------------- --------------
  **Measure**    **Eligible   **Inquiries**   **Applicants**   **Eligible     **Public       **Private
                 Base**                                        Applicants**   Eligible       Eligible
                                                                              Applicants**   Applicants**

  Number of      40,507       5,863           2,692            1,848          1,330          518
  students                                                                                   

  \% of base     100          14              7                5              NA             NA

  \% of                       100             46               32             NA             NA
  inquiries                                                                                  

  \% of                                       100              69             NA             NA
  applicants                                                                                 

  \% of                                                        100            72             28
  eligibles                                                                                  
  -------------- ------------ --------------- ---------------- -------------- -------------- --------------

NOTE: Because the eligible base, inquiries, and applicants included an
unknown combination of public and private school students, it would not
be appropriate to express the number of public or private eligibles as a
percentage of those bases.

SOURCE: Figure for the "Eligible Base" is based on data from the U.S.
Census, population of the District of Columbia age 5 to 17 under 185
percent of the federal poverty line in 2000. The exact number for 2004
is likely to differ somewhat from this 2000 figure. Figure for
"Inquiries" provided by Fight For Children. Figures for "Applicants" and
"Eligible Applicants" were drawn from the applicant database, with
eligibility determined by Private School Aid Service (PSAS). Numbers of
applicants from public or private schools were determined by
cross-matching the name of the school each child was attending against a
list of public and private schools in DC compiled from the DCPS web site
and the NCES Common Core of Data.

# Scholarship and Placement Lotteries and Initial Use of Scholarships Awarded (Chapter 3)

The program statute requires that scholarship recipients be randomly
selected (e.g., by lottery), if the program or specific schools are
"oversubscribed"―that is, have more demand for them than can be
accommodated. The law also details congressional priorities to guide the
award of scholarships and scarce seats to eligible applicants: (1)
students attending a public school designated as in need of improvement
(SINI) under the No Child Left Behind Act at the time of application to
the program, and (2) families that lack the resources to take advantage
of the educational choices available to them.

A total of 79 eligible applicants (4 percent) were from one of the 15
SINI-designated schools in spring 2004 and were, therefore, given the
highest priority in the lotteries.[^1] An additional 1,251 eligible
applicants (68 percent) were attending non-SINI public schools and were
assigned the second-highest priority in the lotteries. The 518 eligible
applicants (28 percent) from private schools were given the lowest
priority, because they were considered to meet neither of the
congressional priorities. These priority groups were used both to award
scholarships and, later, to place scholarship recipients in the
participating private schools of their choice.

## *The Scholarship Lottery*

The first lottery was to distribute scholarships to eligible applicants.
For public school applicants, each student's probability of obtaining a
scholarship was dependent not only on his or her membership in a
priority group but also on the availability of new private school seats
at various grade levels. The new seats in participating private schools
were highly concentrated in the K-5 elementary grades, meaning there
were more available seats in those grades than there were eligible
applicants (Figure ES-1).

![](media/image2.wmf)

Thus, in the lottery, K-5 students would receive scholarships
automatically. The middle school grades (6-8) were modestly
oversubscribed, and the high school grades (9-12) were severely
oversubscribed. Only in those grades was there random assignment as part
of the lottery. Since eligible private school applicants already held
slots in their private schools, they were not constrained by slot
availability in the same way as public school applicants.

Accounting for both the statutory priorities and the slot constraints
within the grade-level bands, scholarship award probabilities were
assigned to the various groups of eligible applicants and a
custom-designed computer program awarded scholarships to students within
each group (Figure ES-2).

![](media/image2.wmf)

A total of 1,366 scholarships were awarded in June 2004, including the
following:

-   All 79 applicants from SINI-designated schools.[^2]

```{=html}
<!-- -->
```
-   All 772 non-SINI public school applicants in K to 5.

-   255 (76 percent) of the non-SINI public school applicants in grades
    6 to 8 and 44 (28 percent) of them in grades 9 to 12.

-   A total of 216 (43 percent) of applicants already attending private
    schools, with probabilities considerably lower (55 percent for those
    in grades K to 5, 42 percent for grades 6 to 8, and 17 percent for
    grades 9 to 12) than public school applicants in all three grade
    bands.

    In sum, the scholarship lottery produced two groups of students for
    purposes of meeting statutory requirements for the DC Choice
    Opportunity Scholarship program evaluation. The 1,366 scholarship
    recipients will be the subject of annual performance reporting and
    comparison to DCPS nonapplicants. The 492 public school applicants
    who were entering grades subject to random assignment (grades 6-12)
    will contribute to the annual impact analysis: these include 299
    students assigned to the treatment group (and also included in the
    performance reporting sample) and 193 students assigned to the
    control group. The 289 private school applicants who were not
    awarded scholarships belong to neither group. Since they previously
    attended and presumably will continue to attend private school using
    resources outside of the scholarship program, following these
    students after baseline would not contribute meaningfully to the
    evaluation. To conserve resources, this group of initial applicants
    will not be part of the evaluation going forward.

    However, the impact sample in the first year of the program is not,
    on its own, large enough for the evaluation to reliably draw
    conclusions about any differences in achievement outcomes that might
    be expected from an intervention of this kind.[^3] Instead, the
    treatment and control groups from the first year lottery will be
    combined with those from the lottery for second year applicants,
    expected in April 2005, to provide a sufficient sample for the
    rigorous evaluation of program impacts.

*Placement Lottery and Follow Up*

After being notified of their scholarship offer, families were required
to meet with officials at participating private schools and obtain
conditional acceptance to the schools that they wanted their children to
attend. Parents then submitted school preference forms to indicate and
rank the top four private schools of their choice. These forms were used
to place students, through a combination of a custom-designed computer
placement lottery and followup case-by-case placements by WSF. There
were a total of 1,366 scholarship winners:

-   A total of 1,040 were successfully placed, with the overwhelming
    majority of students placed in their most-preferred school.

-   The remaining 326 scholarship recipients did not complete the school
    search process and, therefore, could not be assigned a placement in
    a participating private school.

Of the 1,040 who were placed, 1,027 had matriculated at their preferred
private school by September 10, 2004. This represents an overall initial
scholarship usage rate of 75 percent. The usage rate for the impact
sample's treatment group is lower---62 percent---because that group
excludes students in grades K-5 and those who were already attending
private schools when they applied to the program, subgroups that have
significantly higher rates of scholarship use than do public school
students in the middle and high school grades.

# Characteristics of Program Applicants (Chapter 4)

In determining and interpreting program effectiveness later on, it is
important to know how well the program is targeted to the disadvantaged
families who are the focus of the program and, beyond that, what types
of families and students apply, win scholarships in the lottery, and
choose to use them to enroll in a private school. It is also useful to
identify the extent to which public and private schools in the District
are experiencing a significant loss or gain of students due to the first
year of program implementation, because that information provides the
foundation for our later examination of the impact of the program on DC
schools.

### 

### Public School Applicants Compared to Similar Public School Students in DC

There are several reasons to examine the eligible public school
applicants to the DC Opportunity Scholarship Program in relation to
other DCPS students. Most clearly, the comparison provides a context for
considering the kinds of students who might be attracted to the program
in future years in the District or to a similar program in other
locations. The income eligibility criteria for the program---family
income within 185 percent of the federal poverty line---matches up quite
closely with eligibility for the federal free or reduced-price lunch
(FRL) program. Using this income indicator to compare public school
program applicants to similarly low-income DCPS nonapplicants, we find
some differences and similarities (Table ES--3):

**Table ES-3. Characteristics of DC Public School Free or Reduced-Price
Lunch\
Program Students, Program Applicants Versus Nonapplicants: Spring 2004**

+-----------------------------------+-----------+----------+----------+
| **Characteristic**                | **Applic  | **DCPS   | **Diff   |
|                                   | ants**^1^ | Sample** | erence** |
+-----------------------------------+-----------+----------+----------+
| **Baseline Test Scores**^2^       |           |          |          |
+-----------------------------------+-----------+----------+----------+
| > Average Reading Percentile      | 40.2      | 36.4     | 3.9\*\*  |
+-----------------------------------+-----------+----------+----------+
| > *Percent missing*               | *25*      | *26*     |          |
+-----------------------------------+-----------+----------+----------+
| >                                 |           |          |          |
+-----------------------------------+-----------+----------+----------+
| > Average Mathematics Percentile  | 46.7      | 43.0     | 3.7\*\*  |
+-----------------------------------+-----------+----------+----------+
| > *Percent missing*               | *25*      | *24*     |          |
+-----------------------------------+-----------+----------+----------+
| ** **                             |           |          |          |
+-----------------------------------+-----------+----------+----------+
| **Percent in Special Education**  | 17        | 15       | 1        |
+-----------------------------------+-----------+----------+----------+
| > *Percent missing*               | *24*      | *22*     |          |
+-----------------------------------+-----------+----------+----------+
| ** **                             |           |          |          |
+-----------------------------------+-----------+----------+----------+
| **Percent, by Race**              |           |          |          |
+-----------------------------------+-----------+----------+----------+
| > African American                | 93        | 88       | 5\*\*    |
+-----------------------------------+-----------+----------+----------+
| > Hispanic                        | 6         | 10       | -5\*\*   |
+-----------------------------------+-----------+----------+----------+
| > Other race^3^                   | 2         | 2        | 0        |
+-----------------------------------+-----------+----------+----------+
| > *Percent missing*               | *0*       | *0*      |          |
+-----------------------------------+-----------+----------+----------+
|                                   |           |          |          |
+-----------------------------------+-----------+----------+----------+
| ** Percent, by Gender**           |           |          |          |
+-----------------------------------+-----------+----------+----------+
| > Female                          | 51        | 51       | 0        |
+-----------------------------------+-----------+----------+----------+
| > *Percent missing*               | *0*       | *0*      |          |
+-----------------------------------+-----------+----------+----------+
| ** **                             |           |          |          |
+-----------------------------------+-----------+----------+----------+
| **Percent Participating in        | 100       | 100      | 0        |
| Free/Reduced-price Lunch          |           |          |          |
| Program**                         |           |          |          |
+-----------------------------------+-----------+----------+----------+
| > *Percent missing*               | *0*       | *0*      |          |
+-----------------------------------+-----------+----------+----------+
|                                   |           |          |          |
+-----------------------------------+-----------+----------+----------+
| **Sample size**                   | 894       | 44,740   |          |
+-----------------------------------+-----------+----------+----------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ The sample size of 894 for the applicant group compared here differs
from the total sample of 1343 public school applicants for two reasons.
First, only 1,077 (80 percent) of the public school applicants could be
identified conclusively in the DCPS accountability testing database, the
only source of comparable data for both applicants and nonapplicants
from DCPS. Most missing observations were in pre-K, first, or second
grade, where accountability testing is optional. An additional 183
public school applicants in the accountability database were not
enrolled in the free or reduced-price lunch (FRL) program. Because it is
most analytically sound to compare similarly disadvantaged groups of
applicants and DCPS nonapplicants, we included in this comparison only
those students who had test scores in the database and who were
confirmed eligible for FRL. To examine whether there was any substantial
bias in our analysis, we compared the demographic characteristics of
students who were tested by DCPS with those who were not tested, both
within and across the applicant and nonapplicants samples, using a *t*
test for difference of means to identify statistically significant
differences. The testers did not differ significantly from the
nontesters on any characteristic, except for grade, since testing is
mandatory only in grades 3 and higher. Therefore, we are confident that
tester/nontester bias does not affect the comparisons that we make here.

^2^ Test-score results are in terms of National Percentile Ranks, with
50 as the median score.

^3^ "Other race" includes students who were identified as white, Asian,
American Indian, or Alaska Native.

NOTE: Detail may not sum to totals because of rounding. Applicant sample
includes all eligible applicants identified in the DCPS data base that
were participating in the free- and reduced-price lunch program.

SOURCE: Accountability testing database for DC public and charter
schools, DCPS Office of Communications and Public Information.

-   Program applicants scored somewhat higher on reading and mathematics
    accountability tests than the nonapplicants.

-   Applicants were more likely than nonapplicants to be African
    American and less likely to be Hispanic.

-   The two groups were similar regarding special education enrollment,
    gender, and enrollment in the FRL program (the latter by design).

Similar patterns are evident when we use a subset of the
applicants---just the program participants---to compare to economically
disadvantaged DCPS students in the same grades, as the program statute
requires for performance reporting.[^4]

#### *Applicants in the Impact and Non-Impact Samples*

While it is important to examine who applies to the DC Opportunity
Scholarship Program, for the evaluation it is equally critical to assure
that the applicants who will be the focus of the impact analysis---the
applicants who were randomly assigned to treatment and control groups
randomly assigned to treatment by the lottery---are similar prior to the
beginning of the program. It is on this similarity between the groups,
not only in characteristics easily measured but also in those not
observed, that the scientific benefits of the randomized control trial
(RCT) approach rests. We find no statistically significant differences
between the two groups on any major educational or family background
measures, confirming that the assignment lottery was conducted
appropriately.

It is also useful to review how the characteristics of the impact sample
differ from the characteristics of students who applied but will not be
included in the analysis of program effectiveness (the "non-impact
sample" of applicants). The existence of many significant differences
between the impact and non-impact samples limits extrapolation of the
results of the impact analysis to characterize overall program impact.
There are a few differences between the two groups. Compared to their
non-impact sample counterparts, members of the impact sample: (1) scored
higher in reading in grades 9 through 12, (2) are more likely to have a
learning disability, and (3) are less likely to be of Hispanic
ethnicity.

## *Applicants by Type of Previous School*

The schools students previously attended may be associated with
students' educational or background characteristics, their parents'
attitudes, and ultimately the extent to which the program is effective
for students seeking scholarships. Eligible applicants to the program
came from four different types of schools. Four percent came from
SINI-designated public schools, 54 percent from non-SINI regular public
schools, 14 percent from public charter schools, and 28 percent from
various private schools. The most important differences among the
applicants include the following:

-   Among high school applicants, SINI public applicants scored lower in
    reading, whereas non-SINI public applicants scored higher.[^5]

-   The average family income of all applicants was \$18,742, with SINI
    and non-SINI public applicants reporting somewhat lower incomes and
    private school applicants reporting somewhat higher incomes.

-   The mothers of applicants reported an average of almost 13 years of
    formal education. The mothers of non-SINI public applicants reported
    slightly lower levels of education, and the mothers of private
    school applicants slightly higher levels.

### Applicant Response Rates Among Public and Private Schools

A central question in the debate surrounding school choice---and one of
the topics the statute requires the evaluation to address---is whether a
scholarship program has an impact on the larger public and private
school systems. Such "systemic effects" could take place if significant
percentages of students in the public school system or in specific
schools apply for, receive, and use scholarships to transfer to private
schools. With regard to the public schools, the DC Opportunity
Scholarship Program could have either positive or negative effects. One
theoretical argument suggests that scholarship programs will divert
funding and the most motivated students from public schools to private
schools, leaving the public school system with fewer resources with
which to educate the remaining student population. Another theory is
that schools behave in a manner similar to firms and will respond to
competition by becoming more efficient. In the case of schools, it is
the risk of losing students and subsequently funding that may provide an
impetus for public schools to provide better services and produce better
student outcomes. Private schools in jurisdictions with greater school
choice may face similar incentives to improve or expand in order to
retain as many of their current students and attract more.

To provide the basis for a later analysis of the effects of the DC
Opportunity Scholarship Program on schools, we first need to describe
the extent to which public schools have so far been affected by program
applications and scholarship users---a possible predictor of the level
of competitive pressure the public schools may experience. The
school-level scholarship application and use rates this first year
suggest that relatively few public schools have experienced a
significant loss of students as a result of the scholarship program
(Table ES-4):

-   Over one-quarter of the public schools in the District experienced
    no student losses due to the program.

-   Another 56 percent of DCPS schools had program-related transfers out
    that totaled less than 2 percent of their student populations.

-   Seventeen percent of District schools lost about 2 to 4 percent of
    their students.

-   Finally, 2 percent of the public schools in the District experienced
    more significant student transfers of over 4 percent under the
    program.

**Table ES-4. Public School-Level Scholarship Application and Usage
Rates:\
Spring 2004 and Fall 2004**

  ------------------ ------------- ------------- --------------- -------------
                     **Applied for               **Used a        
                     the Program**               Scholarship**   

  **Percent of       **Number of   **Percent of  **Number of     **Percent of
  Student Body**     Schools**     Schools**     Schools**       Schools**

                                                                 

  0                  25            13            51              26

  0.1 -- 1.0         47            24            63              32

  1.1 -- 2.0         56            28            47              24

  2.1 -- 3.0         38            19            21              11

  3.1 -- 4.0         12            6             12              6

  4.1 --             19            11            3               2

                                                                 

  Total              197           100           197             100
  ------------------ ------------- ------------- --------------- -------------

NOTE: Detail may not sum to totals because of rounding.

SOURCES: Application and usage numbers by school generated from the
Applicant Database and Washington Scholarship Fund (WSF) Placement
Database. Enrollment figures for DCPS are from the 2002-03 school year
and were obtained from Membership in the District of Columbia Public
Schools by School and Grade, October 7, 2003, available on the DCPS web
site,
www.k12.dc.us/dcps/data/enrollment/membership-Oct.703_official\_.pdf.
Enrollment figures for DC public charter schools chartered by the
District Board of Education are from the 2002-03 school year and were
obtained from the Common Core of Data, National Center for Education
Statistics. Enrollment figures for the DC public charter schools
chartered by the DC Public Charter School Board are from the 2003-04
school year and were obtained from the Board's web site at
www.dcpubliccharter.com.

In contrast, a similar analysis of the participating private schools
suggests that students using DC Opportunity Scholarships make up a
significant share of their enrollments. In more than one-quarter of
those private schools, nearly 20 percent of their students are using
Opportunity Scholarships; in another 37 percent, scholarship students
make up between 5 and 20 percent of their student populations in
2004-05.

1\. INTRODUCTION

The District of Columbia School Choice Incentive Act of 2003 was passed
by Congress in January 2004. The Act provided funds for District of
Columbia Public School (DCPS) improvement activities and charter school
facility acquisitions. Most notably, the statute established what is now
called the DC Opportunity Scholarship Program─the first federal
government initiative to provide K-12 education scholarships, or
vouchers, to families to send their children to private schools of
choice.

The statute requires that this 5-year scholarship program be rigorously
evaluated by an independent research team. This document is the first of
a series of annual reports from that team, as mandated by Congress. The
report describes the purposes and design of the scholarship program, the
first-year implementation activities that generated 1,848 eligible
applicants and 58 participating private schools, the process of randomly
awarding scholarships to 1,366 student applicants, and the
characteristics of both applicants and scholarship users. The report
provides an important foundation for the later examination of program
impacts.

**1.1 District of Columbia School Choice Incentive Act of 2003**

In January 2004, the U.S. Congress passed the DC School Choice Incentive
Act of 2003, Title III of Division C of the Consolidated Appropriations
Act, 2004, P.L. 108-199. The statute established a new, 5-year school
choice program for low-income residents of Washington, DC. The key
elements of the program include the following:

-   To be eligible, students entering grades K-12 must reside in the
    District and have a family income at or below 185 percent of the
    federal poverty line.

-   Participating students will receive scholarships of up to \$7,500 to
    cover the costs of tuition, school fees, and transportation to a
    participating private school of choice.

-   Scholarships are renewable for up to 5 years (as funds are
    appropriated), as long as students remain eligible for the program
    and remain in good academic standing at the private schools they are
    attending.

-   If there are more eligible applicants than available scholarships or
    open slots in private schools, applicants are to be awarded
    scholarships and admission to private schools by random selection
    (e.g., by a lottery).

-   Private schools participating in the program must be located in
    Washington, DC, and agree to program requirements regarding
    nondiscrimination in admissions, fiscal accountability, and
    cooperation with the evaluation.

Certain groups of students have priority in obtaining access to the
program: (1) those coming from public schools identified as in need of
improvement (SINI) under the federal Elementary and Secondary Education
Act and (2) those whose families lack the financial resources to take
advantage of available educational options.

> The law also charged the U.S. Secretary of Education and the Mayor of
> the District of Columbia with selecting both a program implementer and
> an independent evaluator of the program. The Secretary designated the
> Office of Innovation and Improvement (OII) within the U.S. Department
> of Education (ED) as the lead agency for funding and monitoring the
> implementation of the program, and the Institute of Education Sciences
> (IES) to take the lead in funding and monitoring the independent
> evaluation. Under current appropriation levels---about \$13 million
> annually---the program is likely to support 1,800 to 2,000
> scholarships, depending on the tuition levels of the selected private
> schools.

**1.2 Congressionally Mandated Evaluation**

Section 309 of the Act describes the requirements for an independent
evaluation of the DC Opportunity Scholarship Program. The Secretary of
Education is to ensure the following:

-   "The evaluation is conducted using the strongest possible research
    design for determining the effectiveness" of the school choice
    program;

-   The results of the evaluation regarding the impact of the program on
    the participating students and nonparticipating students and schools
    in the District are disseminated widely.

Early on, IES determined that the foundation of the DC Opportunity
Scholarship Program evaluation would be a randomized controlled trial
(RCT), comparing outcomes of eligible applicants (students and their
parents) randomly assigned to receive or not receive a scholarship.[^6]
This decision was based on the mandate to use rigorous evaluation
methods, the expectation that there would be more applicants than funds
and private school spaces available, and the requirement to use random
selection to determine who receives a scholarship. In addition, the law
clearly specified that such a comparison in outcomes be made.[^7] This
component represents the **impact analysis** and will provide evidence
on the effectiveness of the program.[^8]

The law also called for the evaluation to track program progress in
other ways. For example, the evaluation must compare students
participating in the scholarship program to students in the same grades
in the DC Public Schools. However, DCPS students who did not apply to
the scholarship program are likely to be quite different from those who
applied and are participating---in ways we can observe and ways we
cannot. Comparing outcomes between participants and nonapplicants is,
therefore, not a reliable measure of program effects. Instead, this type
of **performance reporting** will be combined with other data collection
and analysis that examines the context in which the program is
operating.

In spring 2004, IES initiated a competitive bidding process to select an
initial technical advising team as well as an entity to design and
implement the 5-year impact evaluation of the program. The technical
advising contract was competed quickly, so that a group of experts would
be in place to advise ED and the program operator regarding the baseline
data collection and the lotteries that are essential to both the
effective launch of the program and the subsequent impact evaluation. In
March 2004, the technical advising contract was awarded to a research
consortium led by Westat and including Georgetown University and
Chesapeake Research Associates. Later, in July 2004, the competition for
the 5-year impact evaluation contract concluded with an award to the
Westat-Georgetown-Chesapeake team.

##### Research Questions

Based on guidance in the statute, the research team plans to conduct a
comprehensive and rigorous RCT evaluation of the impact of the
scholarship program on participating students and families.
Specifically, the impact analysis will address the following research
questions:

1.  ***What is the impact of the program on student academic
    achievement?*** The law places high priority on examining whether
    the program---the availability and offer of scholarships---improves
    the academic achievement of eligible students. This question can be
    addressed most rigorously by comparing the academic achievement of
    student applicants randomly assigned by a lottery to receive and not
    receive scholarships.

2.  ***What is the impact of attending private versus public schools?***
    Because some students offered scholarships will choose not to use
    them, the research team will use accepted statistical methods to
    examine the effects for students who take the scholarship offer and
    successfully enroll in a private school.

3.  ***What is the impact of the program on other student measures?***
    The law calls for examining other indicators of student and school
    success, including persistence, retention, graduation, and, if
    possible, college enrollment. In addition, the legislation requires
    the evaluation to assess the school safety of students who receive
    the scholarships relative to those who did not receive scholarships.

4.  ***What effect does the program have on student and parent
    satisfaction with the educational options available in the District
    and with children's actual school experiences?*** A key desired
    outcome of scholarship programs is an increase in both the school
    choices possible and parents' and students' satisfaction with the
    choices they have made. These issues will be examined by comparing
    the satisfaction and reasons for applying to the DC Opportunity
    Scholarship Program among applicants assigned by lottery to receive
    scholarships and those assigned to not receive scholarships.

> The evaluation will also address other issues posed in the law through
> program performance analysis:

5.  ***How well are scholarship recipients performing relative to
    students in DCPS?*** As noted earlier, the law asks for a comparison
    of the academic achievement of students who participate in the
    program with the achievement of their grade level counterparts in
    the DCPS.[^9]

6.  ***To what extent is the program influencing public schools and
    expanding choice options for parents in Washington, DC?***
    Scholarship programs have been hypothesized to affect not only the
    students who receive the scholarships but also the broader
    population of public schools and students. Theory suggests that
    these broader outcomes could occur if a significant number of
    students move from public to private schools. The public schools
    might experience a reduction in per student funding that affects
    their offerings, a change in average student performance, or they
    may respond to the competition for students by changing curricula,
    adopting new themes or missions, and modifying existing policies and
    practices to make the public schools more attractive to students
    with schooling options. Choice programs might also affect the larger
    population of private schools, beyond those in which the programs'
    participants are currently enrolled; if choice programs are
    successful, additional private schools may choose to participate,
    new schools may be established to meet enrollment demand, or
    existing schools might expand capacity.

*Data Collection*

To answer these questions, the evaluation will draw on different types
of data---some available from DCPS, some collected for the purposes of
this study. These data will include preprogram (baseline) measures of
family background and student achievement. The baseline measures allow
us to verify that students randomly assigned to the scholarship and
nonscholarship groups were, in fact, similar before the program; the
measures also enable us to create subgroups of students whose impacts we
might want to examine separately, such as students with low prior
achievement. Additional data collected will include annual "in program"
measures (e.g., parents' satisfaction with their children's school,
students' academic achievement), which will serve as outcomes for the
rigorous evaluation of program impacts (see\
Table 1-1)

*Impact Analysis*

It is well known that the independent effects of school choice on
student outcomes are difficult to estimate. Perhaps the most significant
difficulty faced by researchers is selection bias─the self-selection of
families to even seek out a new school choice for their child, and the
mutual student/school decision process that selects students into
different types of schools. Because this bias is generally a result of
unmeasurable factors, most researchers have preferred the use of an RCT
to a dependence on non-experimental (nonrandomized) statistical methods.
Since the DC Opportunity Scholarship Program provides for the random
distribution of scholarships through a lottery, we will, therefore, use
RCT methods to estimate most program impacts.[^10]

**Table 1-1. Data Sources**

+--------------+-------------------------------------------------------+
| **Data       | **Description**                                       |
| Source**     |                                                       |
+--------------+-------------------------------------------------------+
| ***Student   | -   Baseline, or preprogram measures of student       |
| as           |     achievement for public school applicants will     |
| sessments*** |     come primarily from the SAT-9 standardized        |
|              |     assessment administered by DCPS as part of their  |
|              |     spring testing program.^1^                        |
|              |                                                       |
|              | ```{=html}                                            |
|              | <!-- -->                                              |
|              | ```                                                   |
|              | -   Each spring after the baseline year, the study    |
|              |     will administer the SAT-9 to all students who     |
|              |     were offered scholarships as well as all members  |
|              |     of the control group who did not receive a        |
|              |     scholarship                                       |
|              |                                                       |
|              | -   DCPS test score data will be obtained for all     |
|              |     public school students in those years, to compare |
|              |     with the students participating in the program.   |
+--------------+-------------------------------------------------------+
| ***Parent    | -   Surveys of parents (of all applicants) will be    |
| surveys***   |     conducted in all 4 years of impact evaluation     |
|              |     data collection.                                  |
|              |                                                       |
|              | -   Topics will include reasons for applying,         |
|              |     satisfaction with school choices, and perceptions |
|              |     of school safety, educational climate, and        |
|              |     offerings.                                        |
+--------------+-------------------------------------------------------+
| ***Student   | -   Each year after baseline, surveys will be         |
| surveys***   |     conducted with all applicants who are in grades   |
|              |     four and higher.                                  |
|              |                                                       |
|              | -   Topics will include students' satisfaction with   |
|              |     their schools, perceptions of safety, and other   |
|              |     characteristics of their school program and       |
|              |     environment.                                      |
+--------------+-------------------------------------------------------+
| ***Principal | -   Surveys will be conducted each year of principals |
| surveys***   |     of all 109 private schools in DC and principals   |
|              |     of all of the 160 regular public and charter      |
|              |     schools in DCPS.                                  |
|              |                                                       |
|              | -   Topics will include school organization, safety,  |
|              |     climate, principals' awareness of and response to |
|              |     the DC Opportunity Scholarship Program, and, for  |
|              |     private school principals, why they are/are not   |
|              |     participating.                                    |
+--------------+-------------------------------------------------------+

^1^ Baseline achievement will be collected only for applicants from
public schools because, as described in the Impact Analysis Sample
section, applicants who were already attending private schools will not
be included in the impact analysis. For public school applicants who did
not participate in regular DCPS testing in the year they applied to the
program (e.g., particularly children below grade 3), the study will
administer the equivalent DCPS assessment to these students in the fall
after application. All other data will be collected for all applicants,
both from public and private schools.

[Impact Analysis Sample]{.underline}

The RCT approach rests on random assignment or, in the case of the DC
Opportunity Scholarship Program evaluation, a lottery to create two
statistically equivalent groups of students from among program
applicants: (1) a "treatment" group that receives a scholarship, and (2)
a "control" group that does not receive a scholarship. Because the two
groups are generated from the same pool of applicants, they are equally
likely to be motivated to participate in the program and to reap any
benefits from it. And as long as the pool of applicants is sufficiently
large, the random assignment of students into treatment and control
groups should produce groups that are similar in other characteristics,
both those we can observe and measure (e.g., family income, prior
academic achievement) and those we cannot (e.g., motivation to succeed).
The random assignment assures that all observed and unobserved
characteristics are equally represented in both groups.

However, according to the statute, the random assignment that is the
means to create the treatment and control groups can only be used to
help allocate scholarships under particular circumstances. Finally, it
may not be appropriate to include some student applicants in the impact
evaluation, even if they participate in the lottery. As a result of all
of these conditions, the impact analysis sample will perform the
following:

-   Exclude applicants who are already attending private schools. The
    statute contained no provision to exclude from the program students
    who were currently enrolled in private schools but otherwise
    eligible to participate. [^11] A substantial number of private
    school students did apply to the program, as described in Chapters 2
    and 3. However, because those students intended to use the DC
    Opportunity Scholarship to *continue* to attend private schools,
    measuring the difference in outcomes between private school
    applicants who did and did not receive a scholarship through the
    lottery would likely only answer the question of whether a different
    type or amount of scholarship funds affect student outcomes. While
    that question is of some policy interest, it is not the main focus
    of the evaluation as specified in the legislation. Therefore,
    applicants currently enrolled in private schools will not be part of
    the impact analysis sample.

-   Include only public school applicants in grades where there are more
    applications than there are available private school slots. A
    lottery is a fair and efficient way of distributing scholarships
    when there are too many applicants, but the law specifies that
    random selection be used only when the program or particular grades
    are oversubscribed.

-   Exclude public school students who automatically receive a
    scholarship. It is possible that even if many students apply to
    participate in the program, there may not be oversubscription (i.e.,
    more demand than slots) in some grade levels. In those grade levels,
    all applicants will receive scholarships and there will be no
    control group.

Thus, the impact evaluation of the DC Opportunity Scholarship Program
depends on the extent to which large numbers of eligible DC families
with public school students apply to the program. The treatment and
control groups must be of a sufficient size to allow us to detect and
measure any difference in outcomes between the two groups (the "impact")
with statistical certainty. A procedure called "power analysis" is used
to determine the sample sizes necessary to enable the study to answer
the central research questions and to measure program effects that are
large enough to be both meaningful in students' lives and relevant to
policy debates about the efficacy of school choice interventions. The
power calculations conducted for the evaluation suggest that a total of
at least 1,240 public school applicants must be randomly assigned, with
over 800 assigned to the treatment (scholarship groups) and over 400
assigned to the control (non-scholarship) group in order for the impact
analysis to be able to detect moderately large test score effects (see
Appendix A for more detail).

[General Statistical Approach: Estimating the Impact of the Offer of a
Scholarship]{.underline}

Given appropriately sized treatment and control groups, the strategy for
analyzing impacts is well established. To motivate the discussion of how
we identify the effect of the scholarship program on test scores, it is
useful to begin with a simple representation of the selection problem as
a missing data problem, using the potential outcomes approach. This
approach defines causal effects in terms of *potential outcomes* or
*counterfactuals*. Conceptually, the causal effect of treatment---the
scholarship---is defined as the difference between the "outcome for
individuals assigned to the *treatment* group" and the "outcome for the
*treatment* group if it had not received the treatment," or:

(E.1) "E(Y~i~\| X~i~, T~i~ =1)" - "E(Y~i~ \|X~i~, T~i~ =0)" Equation 1:
expectation open bracket Y sub i conditional on X sub i comma T sub i
equal 1 close bracket -- expectation open bracket Y sub i conditional on
X sub i comma T sub i equal 0 close bracket.

In the case of scholarships, the treatment effect─the effect of the
scholarships on academic achievement─would be defined as the difference
between "test scores for program students" and "test scores for program
students if they had not received a scholarship." The fundamental
problem is that a student is never observed simultaneously in both
states of the world. What is observed is a student in the treatment
group (T~i~ =1) or in the control group (T~i~ =0). The outcome in the
absence of treatment, E(Y~i~ \|X~i~, T~i~ =0), is then the
counterfactual─what would have occurred to those students receiving the
scholarships if they had not received them.

If students receiving scholarships were identical to other students in
both observable and unobservable characteristics, the counterfactual
could be generated directly from an appropriately selected comparison
group. Valid comparison groups are rarely found in practice, however.
The random assignment of students into the program generates the
counterfactual from the control group─eligible applicants who did not
receive a scholarship.[^12] If correctly implemented, random assignment
yields statistically equivalent groups and allows estimation of the
program impact through differences in mean outcomes between the two
groups.

Consistent with this approach is the following basic analytic model of
the effects of school choice scholarships on outcomes. Consider first
the outcome equation for the test score of student *i* in year *t*. It
is reasonable to assume that test scores (Y~it~ ) are determined as
follows:

(E.2) Y~it~ =α+ τ T~it~ + X~i~ γ+ ε~it~ if t\>k (period after program
takes effect) Equation 2: Y sub i-t equals alpha plus tau T sub i-t plus
X sub i-t gamma plus epsilon sub i-t uf t greater than k open bracket
period after program takes effect close bracket.

Equation (E.2) estimates the effect of the ***offer*** of a scholarship
on student outcomes. Under this model, commonly referred to as the
"Intent to Treat" (ITT) estimation, all students who were randomly
assigned by virtue of the lottery are included in the analysis,
regardless of whether a member of the treatment group uses the
scholarship to attend a private school. In E.2, T~it~ is equal to one if
the student *has the opportunity to participate* in the scholarship
program (i.e., the award rather than the actual use of the scholarship)
and equal to zero otherwise. X~i~ is a vector of student characteristics
(measured at baseline) known to influence future academic achievement,
such as prior test scores, mother's level of education, family income,
etc. In this model, τ represents the effect of scholarships on test
scores for students in the program, conditional on X~i~. With a properly
designed RCT, using a concise and judiciously chosen set of statistical
controls for characteristics that predict future achievement should
improve the precision of the estimated impact.[^13] That treatment
effect, τ, should be identical to the difference in mean outcomes
between the treatment and the control groups.

Since the initial applicants were randomized within certain relevant
subgroups, as described in Chapter 3, we will analyze program impacts
using a randomized block design. We are interested in how academic
achievement (Y) is affected by the assignment into the scholarship
program within each block (B) or group of size n. The impacts are then
estimated as:

(E.3) Y~ikt~ = μ+ τ T~ikt~ +∑^b^~j=2~ ρ~j~ B~ik~+ X~ik~ γ+ ε~ik,t~
Equation 3: Y sub i-k-t equals mu plus tau T sub i-k-t plus summation j
equals 2 to b rho sub j B sun i-k + X sub i-k gamma plus epsilon sub i-k
comma t.

where

i = 1,.....,n observations and k=1,....,b blocks(defined by grade and
priority status);

Y~ji~ is the outcome for student i in block j, at time t;

μ is the overall mean outcome (e.g., test score);

τ is the treatment (scholarship program) effect;

ρ~j~ is the j^th^ block effect;

T~it~ is assignment into the scholarship program;

B~ji~ is the block assignment;

X~ji~ represents observable characteristics, measured at baseline; and

ε~ij~ is the random error; independent, Ν(0,σ~ε~^2^ ).

This analytical framework follows naturally from the group randomization
and is easily implemented and interpreted. Y can be measured in several
different dimensions, including test scores, school satisfaction,
parental satisfaction, grade completion, including where appropriate,
high school graduation, etc. μ is average outcome for all program
members, ρ~j~ is the average block effect, and τ is the effect of
scholarships on academic achievement.[^14]

[Estimating the Impact of the Use of Scholarships]{.underline}

Even with a properly implemented RCT, we may expect that not all
applicants placed by random assignment into the treatment (scholarship
offer) group will actually use the scholarship at a private school. That
is, some scholarship recipients may choose not to use their scholarship
and instead attend a public school. This type of non-participation or
underutilization of treatment services has been observed across all RCT
settings, including medical trials, job training and health insurance
experiments, as well as in previous school scholarship RCTs such as the
one of the Milwaukee Parental Choice Program.

Policymakers are typically interested in the effect of scholarship
***use*** on student achievement, in addition to the offer of the
scholarship. To estimate the impact, we will use a model commonly
referred to as the "Impact of the Treated" (IOT), which statistically
estimates the impact of actual scholarship use. Instrumental variable
analysis provides us with a well-established method to generate an
unbiased estimate of the scholarship impact on the treated from the ITT
estimator.[^15]

*Performance Reporting Analysis*

To fulfill the requirements specified in the law, we will compare the
outcomes of *all* participating students with the outcomes of similar
DCPS students who did not apply to the program.

[Performance Reporting Sample]{.underline}

The group of student applicants that will be examined as part of
performance reporting differs from the impact analysis sample in several
important ways. The impact analysis sample includes all public school
applicants who were randomly assigned to receive or not receive a
scholarship as part of the lottery (i.e., students in grade bands for
which there were more applicants than there were private school slots
available). In contrast, the performance reporting sample performs the
following:

-   Excludes students who did not receive a scholarship as part of the
    lottery (the control group).

-   Includes students who both received a scholarship automatically and
    those who received one through the lottery (treatment group)

```{=html}
<!-- -->
```
-   Includes scholarship recipients who were already attending private
    schools at the time of application.

-   Will focus on students who chose to use their scholarship; while the
    law does not define what it means to "participate," performance
    reporting in the evaluation's later reports will examine the
    differences between students who were given the *option* of
    participating (i.e., received a scholarship) and those who exercised
    that option (i.e., who used the scholarship to attend a private
    school). Both groups will be compared to DCPS students in the same
    grades who did not apply to the DC Opportunity Scholarship Program.

[Statistical Approach]{.underline}

The performance reporting comparisons will focus on student achievement,
both as specified in the statute and because that is the only measure
that will be similar for DCPS students and those participating in
private schools through the DC Opportunity Scholarship program.[^16] In
order to ensure comparability in student assessment, the evaluation will
make every effort to administer the same test to program participants
that is used by DCPS. DCPS will provide the evaluation team with test
score and background data on public school students.

The analysis will be conducted by comparing the mean test scores of
program participants and DCPS nonapplicants, testing for the statistical
significance of the difference. To create the most relevant group of
DCPS students for comparison, we will draw from the DCPS database the
group of non-applicant students who qualify for the program (i.e.,
eligible for free/reduced-price lunch), stratified by grade level to
match our scholarship performance reporting sample. We will present
these comparative results as descriptive findings, since the absence of
random-assignment to the scholarship or public school conditions would
render any causal claims highly speculative.

*Reports*

The law requires the Secretary to submit to the Congress annual reports
resulting from the independent evaluation by December 1 each year and a
final report not more than a year after the 5-year program ends. These
reports should provide the Congress, other policymakers, the research
community, and the public at large with important new information about
what happens to students, families, schools, and communities when
educational options are expanded for urban low-income families through
public policy.

**1.3 Organization of This Report**

This report is the first in the series of required evaluation reports to
Congress. While the focus of the evaluation is on examining the
effectiveness of the program, no impact information is available at this
point because the initial cohort of program participants---those who
applied in spring 2004 to receive scholarships for the 2004-05 school
year---just recently matriculated at their new schools. Instead, this
report examines the extent of student and school interest in the program
and the characteristics of those participating.

-   Chapter 2 describes the recruitment and application activities that
    resulted in the submission of 2,692 initial applications for the
    program, of which 1,848 were deemed eligible. It also describes the
    characteristics of the group of 58 private schools in the District
    that agreed to participate in the first year of the program.

-   Chapter 3 presents information about the lotteries that determined
    which eligible applicants would receive scholarships and which
    scholarship students would be assigned seats when individual schools
    were oversubscribed.

-   Chapter 4 describes the characteristics of program applicants,
    including their average test scores, family background, parental
    involvement in their education, type of school they attended
    previously, and parental assessments of their previous school. It
    also presents initial data regarding the extent to which DC schools
    are exposed to greater competition for students as a result of
    program implementation.

2\. Program Implementation in 2004: Recruitment and Applications

Any evaluation, particularly a randomized control trial (RCT), must take
into consideration the number and flow of program applicants and
participating institutions. In the case of the DC Opportunity
Scholarship Program evaluation, the design was predicated on attracting
a sufficient number of applicants to be able to create two sizable and
randomly assigned groups of students whose outcomes could be compared:
(1) eligible applicants who receive scholarships to attend participating
private schools as part of the lottery and (2) eligible applicants who
do not receive scholarships by virtue of the lottery.

The recruitment and application period for the program began at the end
of March 2004, immediately after a partnership led by the Washington
Scholarship Fund (WSF) was selected to implement the DC Opportunity
Scholarship Program, and continued throughout the spring and early
summer of 2004. The process produced agreements from 58 private schools
in the District to participate in the federal program during the first
year and applications from 2,692 students seeking scholarships, of whom
1,848 were eligible. This chapter describes in more detail the
activities and results of these efforts to recruit schools and students
to participate in the program.

**2.1 Schools**

An initial task of the WSF-led implementation team was to recruit DC
private schools to participate in the program during the first year. The
team directly contacted each of the 109 private schools in the District
and also worked closely with umbrella organizations that represent
groups of schools, such as the Catholic Archdiocese of Washington, the
Council for American Private Education, and the Association of Christian
Schools International. The team also hosted an information session for
private school officials in April. Several representatives of
independent private schools at that meeting said that their admissions
process for 2004-05 had already concluded and their schools were fully
enrolled. However, some of them expressed an interest in participating
in the program anyway, and others suggested that they would participate
in the future if the application and admissions schedule could be moved
to earlier in the school year.

![](media/image2.wmf)

Despite the challenges stemming from the late start of the program, the
implementers recruited 58 DC private schools to participate in the
program in 2004-05, comprising just over one-half of the private schools
in the District (Figure 2-1 and Table 2-1).[^17]

**Table 2-1. Number of DC Private Schools Participating in the DC
Opportunity Scholarship Program: 2004-05**

  ------------------------------------------------ ----------- -------------
  **Private Schools**                              **Number of **Percent**
                                                   Schools**   

  In the District of Columbia                      109         100

  Participating in the program in some capacity    58          53

  Set aside slots for DC Opportunity Scholarship   54          50
  Program participants                                         

  Agreed to accept scholarships only for eligible  4           4
  students already admitted                                    
  ------------------------------------------------ ----------- -------------

NOTE: Detail may not sum to totals because of rounding.

SOURCE: "School Directory, D.C. K-12 Scholarship Program, 2004-05 School
Year," Washington Scholarship Fund, June 2004.

Fifty-four of the 58 schools made new slots available to scholarship
students; the remaining four schools had already closed their admissions
and were only willing to accept new students awarded Opportunity
Scholarships who already had been accepted to those schools during their
regular admissions period. This small subgroup of scholarship winners
that we describe as private school "pre-admits" were attending public
schools in 2003-04 but obtained acceptance to a private school
independent of and prior to the launch of the scholarship program. Some
data were obtained regarding 53 of the 58 participating schools.[^18]

### Religious Affiliation, Location, and Students Served by Participating Schools

The private schools that chose to participate in the DC Opportunity
Scholarship Program in the first year are a diverse group but have some
similar characteristics. Most are religiously affiliated and have been
long established in the area. About half (27 of the 52 schools that
reported their affiliation, or 51 percent) are affiliated with the
Catholic Church; 11 (21 percent) are affiliated with a non-Catholic
religion; and 15 (28 percent) are independent nonsectarian schools. More
than three-quarters of the schools were established before 1983, with a
large share of them founded before 1955. The most recently established
private school to participate in the program opened in 2002.

The private schools that are participating in the DC Opportunity
Scholarship Program are located in every ward of the District. The
highest concentration of participating schools, nearly one-quarter of
them, are in Ward 4. The remaining participating schools are almost
evenly distributed throughout the other seven wards of the city. [^19]

Most of the participating private schools already are serving a high
proportion of students of color (Table 2-2). On average among the
schools, 82 percent of their students in the current year are African
American, Latino, Asian, or Native American, compared to 95 percent
average minority populations in regular DCPS schools. Sixty percent of
the participating private schools have student bodies that are entirely
from minority racial/ethnic groups, but a few schools serve fewer than
10 percent of students from those groups.

**Table 2-2. Enrollment, Race/Ethnicity, and Student-Teacher Ratio of
Students Served by Participating Private Schools Compared with Regular
DCPS Schools: 2003-04**

  -------------------------------- ------------- ------------- ------------ ---------
  **Characteristic**               **Average**   **Highest**   **Lowest**   **Valid
                                                                            N^1^**

  Enrollment (number of students)                                           

  Private schools                  206\*\*       1,056         12           52

  DCPS schools^2^                  414           1,442         127          139

  Percentage of students from                                               
  racial/\                                                                  
  ethnic minority groups                                                    

  Private schools                  82\*\*        100           8            52

  DCPS schools                     95            100           29           139

  Average student-teacher ratio^3^                                          

  Private schools                  11:1\*\*      20:1          5:1          47

  DCPS schools                     14:1          21:1          7:1          133
  -------------------------------- ------------- ------------- ------------ ---------

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ "Valid N" refers to the number of schools for which information on a
particular characteristic was available.

^2^ The comparison group of regular DCPS schools excludes public charter
schools, alternative schools, and learning centers.

^3^ Whenever ranges were given for student-teacher ratio, the midpoint
of the range was selected.

SOURCES: Data on participating private schools drawn from "School
Directory, D.C. K-12 Scholarship Program, 2004-05 School Year,"
Washington Scholarship Fund, June 2004, supplemented by the National
Center for Education Statistics, Common Core of Data. Data on DCPS drawn
from the web sites of the District of Columbia government and DCPS.

When it comes to the size of overall enrollments or of individual
classes, there are substantial differences across schools, although most
of these stem from the different grade levels served. Nearly half of the
participating private schools enroll students in either kindergarten
through eighth grade or pre-K through eighth grade, with the other half
enrolling various combinations of elementary-, middle-, and
high-school-aged students. It is, therefore, no surprise that
enrollments range from 1,000 in a relatively large high school to only
12 students in a very small school. The average enrollment of the
participating private schools is about half the average enrollment of
regular DCPS schools. The average ratio of students to teachers in the
participating private schools is somewhat smaller than the average for
DCPS schools. Five of the participating private schools (less than 10
percent) serve students of only one gender.

Although there is no data source that allows a systematic examination of
the extent to which the participating private schools are representative
of the entire population of private schools in the District, analysis of
web sites suggests that there may be some differences. Schools not
currently participating in the program appear to have slightly higher
enrollments than the participating schools, probably because more
nonparticipating schools serve students in the high school grades.
Compared to participating schools, nonparticipating schools seem to
enroll a substantially smaller share of minorities, have smaller
student-teacher ratios, and are much less likely to be religiously
affiliated. However, these comparisons should be interpreted with
caution, because data were available from only 39 of the 51
nonparticipating schools and were obtained from difference sources and
school years.

### Programs and Services Available at Participating Schools

The participating private schools offer many but not all of the services
and activities that are commonly available in public schools (Table
2-3). An overwhelming majority of participating schools offer computers,
art, a library, tutoring, after-school care, music, and before-school
care as part of their educational programs. Nearly three-quarters of the
participating schools include religious instruction and/or worship as
part of their school program, and a similar proportion require that
applicants complete an exam to determine grade placement for enrollment.
Nearly 12 percent require applicants to pass an entrance exam in order
to be deemed admissible to their school. Only 60 percent have a gym and
less than half have a school cafeteria. Gifted and talented programs are
available in about 16 percent of the participating private schools.

**Table 2-3. Services, Programs, and Policies Available at
Participating\
Schools: 2003-04**

  --------------------------------- ------------ ------------- ------------
  **Characteristic or Service**     **Number of  **Percent**   **Valid
                                    Schools**                  N^1^**

  Computers available to students   49           96            51

  Art                               46           92            50

  Tutoring                          45           88            51

  After care                        45           87            52

  Music                             40           85            47

  Before care                       40           77            52

  Placement exam                    37           76            49

  Library                           39           75            52

  Religious instruction/worship     37           73            51

  After-school sports               34           67            51

  Gym                               31           60            52

  Cafeteria                         24           47            51

  Gifted program                    8            16            51

  Entrance exam                     6            12            52
  --------------------------------- ------------ ------------- ------------

^1^ "Valid N" refers to the number of schools that provided information
on a particular characteristic.

SOURCE: "School Directory, D.C. K-12 Scholarship Program, 2004-05 School
Year," Washington Scholarship Fund, June 2004.

Participating schools were asked whether they accept and how they
accommodate students with diagnosed learning disabilities. Private
schools are not required to admit students with learning disabilities if
this would present an undue burden on the schools (according to the
standards in the Americans with Disabilities Act), unless the private
schools receive federal funds (other than those under the DC Opportunity
Scholarship Program). However, many of the participating schools do
admit such students. Seventeen of the schools (32 percent) responded
that they do not accept students with learning disabilities. Twenty-two
schools (42 percent) reported that they accept such students and fully
include them in the educational environment of the school. An additional
eight schools (15 percent) said that they accept and provide special
services for students with limited or mild learning disabilities, and
five participating private schools (9 percent) expressed a willingness
to accept students with moderate to severe learning disabilities.

It should be noted that the District has a sizable "private placement"
program for special education students with needs that cannot be served
effectively in a DCPS school. The program currently pays the full
tuition to send 2,595 DC students to specialized private schools or
public schools outside of the District for educational services
specified in student individualized educational plans (IEPs), at an
average tuition cost of \$27,575 per student.[^20] At least eight
private schools in the District receive private placements of special
education students from DCPS.[^21] None of the eight schools decided to
participate in the DC Opportunity Scholarship Program this year, perhaps
because the scholarship ceiling is below the amount that they typically
spend to educate students with significant disabilities.

**Table 2-4. Average Tuitions Charged by Participating and
Nonparticipating\
Private Schools: 2004-05**

  ------------------------ ----------------- ------------- -------------------- -------------
                           **Participating                 **Nonparticipating   
                           Schools**                       Schools**            

  **Average Tuition^1^**   **Number of       **Percent**   **Number of          **Percent**
                           Schools**                       Schools**            

  Above \$7,500            15                30            21                   84

  7,500 or less            35                70            4^2^                 16

  **Total**                **50**            **100**       **25**               **100**
  ------------------------ ----------------- ------------- -------------------- -------------

^1^ For schools that charge a range of tuitions, the midpoint of the
range was selected. Tuition rates were unavailable for 8 of the
participating private schools and 26 of the nonparticipating private
schools.

^2^ Three schools charged no tuition either because of foundation
support or because the school serves groups such as DC-placed special
education students funded by the government.

NOTE: Detail may not sum to totals because of rounding.

SOURCES: Data on participating private schools drawn from "School
Directory, D.C. K-12 Scholarship Program, 2004-05 School Year,"
Washington Scholarship Fund, June 2004. Data on nonparticipating private
schools were obtained from school web sites.

Finally, a majority (70 percent) of the private schools that chose to
participate in the DC Opportunity Scholarship Program the first year
charge tuitions that are under the \$7,500 maximum amount provided
through the federal scholarship (Table 2-4). Most ask families to pay
annual per student tuitions at or below \$5,500. Although 30 percent of
the participating schools typically charge tuitions above the
scholarship ceiling, with a couple of exceptions, they agreed to accept
a \$7,500 scholarship as full compensation for enrolling scholarship
students, at least for the 2004-05 school year.[^22] Nonparticipating
private schools appear to charge higher tuitions than the participating
schools, however, this conclusion is based on data for a subset of
nonparticipating private schools.

**2.2 Families and Students**

At the same time it was recruiting schools to participate in the DC
Opportunity Scholarship Program, WSF also was recruiting families and
students. Given appropriations of approximately \$13 million the first
year, WSF estimated that approximately 1,800 students could be supported
with scholarships. The technical advising team hoped for 1,250-1,600
eligible public school applicants in oversubscribed grades to generate
the conditions for a statistically well-powered experimental impact
analysis in future years (see Appendix A).

Although ***outreach*** to families in the District began even before
WSF was officially designated as the program implementer, major efforts
were initiated at the end of March when the organization and its
partners were awarded a 5-year program management grant. These
activities included the following:[^23]

-   Mailing personalized letters to all families that inquired about the
    program;

-   Mailing notices to officials at Head Start and after-school centers,
    the leaders of DC faith-based organizations, and officials of
    various other organizations that serve low-income DC residents such
    as the Greater Washington Urban League;

-   Making personalized phone calls to all families that inquired about
    the new program or that had previously applied to WSF's or other
    privately funded scholarship program; and

-   Distributing flyers at community organizations, after-school
    centers, DC churches, DC Metro stations, and several DC
    neighborhoods populated with low-income families.

The implementation team then scheduled and advertised a series of
***application*** events at the DC Convention Center for April 28 to
May 1. The application events included a mandatory orientation session
for families and the distribution of the application form, which
collected both baseline characteristics of the families and students and
information about the families' eligibility for the program.[^24]

The application form required parents to confirm that student applicants
met all eligibility criteria and to provide documentation for
verification purposes. For example, income was verified with reference
to a family's 2003 income tax return and supporting documents such as
official benefits statements pertaining to income assistance programs. A
student's entering grade level was verified using a dated report card
and student birth date (for rising kindergartners). DC residency was
established using a family's tax return or a recent utility bill.
Families were encouraged to bring full documentation to application
events, where copy machines were available to them. If they submitted an
application that lacked complete documentation they were told exactly
what additional documentation they needed to provide and where they
needed to send it. Program staff repeatedly contacted families with
incomplete applications in order to encourage completion.

At the conclusion of the series of application events, the program
operators decided to extend the application period in order to enable
more families to apply. The WSF initiated and advertised extended
evening and weekend office hours to assist applicants on a walk-in basis
from May 3 through May 17. The implementation team and support
organizations staffed a series of 10 application meetings in community
centers located within the residential boundaries of DC schools that had
been declared in need of improvement under No Child Left Behind
(NCLB).[^25] Throughout this period, volunteers assisting the
implementation team visited homebound parents with disabilities to help
them in completing applications.

On May 26, the implementation team closed new applications to the
program. Applicants were given until June 1 to provide all of the
documentation necessary to complete their applications and verify their
eligibility for the program. Members of the technical advising team
based at Westat consolidated the information that had been collected to
that point about participating schools and applicants into a master
database.

The results of the outreach, application, and eligibility verification
process provide some sense of the demand for this scholarship program,
given its particular requirements and the compressed time period within
which it was launched in 2004 (Table 2-5).

-   Just over 40,000 students were estimated to be potentially eligible
    for the program, based on data from the U.S. Census regarding the
    total number of children in grades K-12 in the District with family
    incomes at or below 185 percent of poverty in 2000.

-   Distinct families representing 5,863 school-age children inquired
    about the scholarship program and expressed an interest in applying,
    according to figures obtained from the nonprofit organization that
    advertised the program before WSF was awarded the program management
    grant. If all of these children were eligible for the program (an
    unlikely assumption), then they would represent approximately 14
    percent of the eligible population of students.

-   Applications were submitted on behalf of 2,692 students,
    representing 7 percent of the eligible population and roughly half
    of the number of student inquiries.

-   A total of 1,848 students were confirmed to be eligible for the
    program; an unknown number of additional applicants might have been
    deemed eligible had they completed their application.[^26] The
    number of confirmed eligible applicants represents 5 percent of the
    eligible population, about one-third of the inquiries, and 69
    percent of all applicants.

-   Of these eligible applicants, 1,330 (72 percent) had been enrolled
    in DCPS in the 2003-04 school year, and 6 percent of those were
    coming from schools designated as in need of improvement under NCLB
    as of spring 2004.[^27] Another 518 (28 percent of all eligible
    applicants) were already attending private schools but satisfied all
    of the program requirements and, therefore, were deemed eligible for
    scholarships.

**Table 2-5. Number and Percentage of Participants, by Application
Status: Spring 2004**

  -------------- ------------ --------------- ---------------- -------------- -------------- --------------
  **Measure**    **Eligible   **Inquiries**   **Applicants**   **Eligible     **Public       **Private
                 Base**                                        Applicants**   Eligible       Eligible
                                                                              Applicants**   Applicants**

  Number of      40,507       5,863           2,692            1,848          1,330          518
  students                                                                                   

  \% of base     100          14              7                5              NA             NA

  \% of                       100             46               32             NA             NA
  inquiries                                                                                  

  \% of                                       100              69             NA             NA
  applicants                                                                                 

  \% of                                                        100            72             28
  eligibles                                                                                  
  -------------- ------------ --------------- ---------------- -------------- -------------- --------------

NOTE: Because the eligible base, inquiries, and applicants included an
unknown combination of public and private school students, it would not
be appropriate to express the number of public or private eligibles as a
percentage of those bases.

SOURCE: Figure for the "Eligible Base" is based on data from the U.S.
Census, population of the District of Columbia age 5 to 17 under 185
percent of the federal poverty line in 2000. The exact number for 2004
is likely to differ somewhat from this 2000 figure. Figure for
"Inquiries" provided by Fight For Children. Figures for "Applicants" and
"Eligible Applicants" were drawn from the applicant database, with
eligibility determined by PSAS. Numbers of applicants from public or
private schools were determined by cross-matching the name of the school
each child was attending against a list of public and private schools in
the District compiled from the DCPS web site and the NCES Common Core of
Data.

3\. Scholarship and Placement Lotteries AND INITIAL\
USE OF SCHOLARSHIPS AWARDED

As discussed in Chapter 1, the DC School Choice Incentive Act requires
that a random selection process (e.g., a lottery) be used to select
scholarship recipients, if the program or specific schools are
"oversubscribed"---that is, have more demand for them than there are
slots available. These lotteries were to serve not only as the fairest
way to allocate scholarships and placements, but also as the basis for
creating the randomly determined treatment and control groups for the
impact evaluation. In the initial year, there were more eligible
applicants than new slots only in the upper grades of participating
private schools. Later, among scholarship recipients, there was more
demand than slots at some grade levels in particular private schools.

As a result, the technical advising team assisted the program in
designing and executing two separate lotteries. The scholarship lottery
determined which of the eligible applicants in oversubscribed grades
would receive scholarships. The placement lottery drew upon the school
preferences expressed by the parents of scholarship winners to determine
which students would receive scarce seats in specific private schools.
Because of the priority categories specified in the law, SINI
scholarship recipients were placed first, followed by recipients from
non-SINI public schools, then recipients from private schools. This
chapter describes the design, operation, and results of the lotteries
and other placement efforts.

**3.1 Statutory Priorities**

Section 306 of the Act includes congressional priorities to guide the
award of scholarships and scarce seats to eligible applicants. Program
operators are instructed to "provide students and families with the
widest range of educational options"[^28] and to perform the following:

1.  Give priority to students attending a public school in need of
    improvement under NCLB at the time of application to the program;
    and

2.  "Target resources to students and families that lack the financial
    resources to take advantage of available educational options."

Regarding the first priority, the program implementer, the technical
advising team, and ED agreed that policymakers will want to know if the
program is having an effect on all types of students, including those
who transferred out of NCLB-eligible schools. Thus, it would be
desirable to include applicants from SINI schools in the rigorous impact
evaluation and, therefore, in the random assignment of students as part
of the scholarship lottery. Awarding scholarships to all SINI students
would prevent the evaluation from generalizing findings of program
impacts to those students from designated low-performing schools that
are a high priority of the program.

However, because the 79 SINI students represented such a small share (4
percent) of the overall applicant pool in the first year, there was
little possibility of separately analyzing the impacts for this
subgroup.[^29] Therefore, for *the first year only*, every eligible
applicant who was currently attending a SINI school was automatically
awarded a scholarship.[^30] The SINI scholarship winners also were the
first group of students assigned scarce seats in preferred private
schools through the subsequent placement lottery.

In response to the second statutory priority, eligible applicants from
non-SINI public schools were assigned the second-highest probabilities
for scholarship award and were the second group in line for the
placement lottery, after the SINI applicants. The fact that public
school applicants were not yet attending private schools but wanted to
was interpreted as evidence that their families lacked the resources to
enroll in a school of choice.

The eligible applicants already attending private schools at baseline
were given the lowest priority in the scholarship and placement
lotteries, based on the first two statutory priorities. Thus lotteries
had to take into account three priority strata: (1) SINI applicants, (2)
non-SINI public school applicants, and (3) private school applicants.

**3.2 Scholarship Lottery Design and Implementation**

The scholarship lottery was designed to accomplish several
objectives.[^31] First, the lottery needed to take into account the
legislated priorities, which essentially give some groups of eligible
students a greater chance of obtaining a conditional scholarship than
other students. Second, the lottery had to lay the foundation for the
rigorous evaluation of program impacts and effectiveness called for in
the law. Finally, the lottery had to balance the program's desire to
maximize the use of the scholarships with a need to limit the number of
disappointed families not able to find school placements; that is, to
ensure that there would not be too many students seeking the fixed
number of private school slots in each grade. Therefore, an important
part of the lottery design was the creation of scholarship award
probabilities that reflected the number of applicants in relation to the
number of slots available in private schools by grade level groups or
"bands."

*Applicants and Available Private School Slots, by Grade Band*

The awarding of scholarships to public school applicants for the first
year was constrained by the limited availability of new private school
seats at certain grade levels, not necessarily by the amount of funds
available for scholarships. The distribution of eligible public school
applicants by grade did not closely match the distribution of new slots
by grade in participating private schools (Figure 3‑1):

-   In grades K to 5, more private school seats were available than
    eligible public school applicants interested in filling them (i.e.,
    there was more than one slot available to each such applicant in the
    elementary grades).

> ![](media/image2.wmf)

-   Grades 6 to 8 were modestly oversubscribed given seat availability.

-   Grades 9 to 12 were severely oversubscribed given seat availability.

Since eligible private school applicants already held slots in their
private schools, they were not constrained by slot availability in the
same way as public school applicants.

## *Creating Probabilities of Scholarship Award*

The first step in generating the probabilities for the scholarship
lottery was to create base probabilities that matched the ratio of slots
to applicants. The base probability for K to 5 public school applicants
was above 100 percent, meaning that all would receive scholarships. The
base probabilities for eligible public school applicants entering grades
6 to 8 and 9 to 12 were 66 percent and 31 percent, respectively
(Table 3-1).

In finalizing the probabilities, the technical advising team also took
into account that some students awarded scholarships are likely to
choose not to use them. Low-income inner-city families are highly
mobile, and some of them would likely move out of the District before
the start of the school year, making them ineligible for the program.
Other scholarship winners might choose to remain in their current public
school or enroll in another public school in the District such as a
charter school. Therefore, a decision was made to assign more
scholarships than the number of slots available in the middle and high
school grades.[^32] A previous evaluation of the partial-tuition
scholarship program for elementary

**Table 3-1. Eligible Public School Applicants and Available Private
School\
Slots: Spring 2004 and Fall 2004**

  -------------------- ---------------- ---------------- --------------------
  **Grade-Level        **Number of      **Number of      **Ratio of
  Band^1^**            Eligible         Available        Slots/Applicants**
                       Applicants**     Slots**          

  K-5                  796              968              1.22

  6-8                  356              236              0.66

  9-12                 191              60               0.31

  **Totals**           **1,343**        **1,264**        **0.94**
  -------------------- ---------------- ---------------- --------------------

^1^ Forecasted grade level of each applicant based on parental reports
of "Grade entering for the 2004-05 school year" on the application form.
All applications for students forecasted to be entering kindergarten in
2004-05 were examined to determine if the child's age was above the
cutoff for kindergarten enrollment.

SOURCES: Figure for the "Number of Eligible Applicants" drawn from the
applicant database. "Number of Available Slots" based on official
reports from all participating private schools as of June 1, the slot
cutoff for purposes of designing the scholarship lottery.

school students in New York City reported that 18 percent of students
awarded scholarships there did not use them in the first year.[^33] We
might expect that the rate of non-usage would be somewhat lower for the
middle school students who receive Opportunity Scholarships, since they
cover the full tuition, but somewhat higher for the high school
students, who were able to select from relatively few participating
schools in the first year. [^34]

Based on these expectations, we set the scholarship "overage" as 16
percent for middle school students and 28 percent for high school
students. Thus, the overall scholarship probability for middle school
students applying from public schools was 82 percent (66 percent + 16
percent) and the probability for public high school applicants was 59
percent (31 percent + 28 percent). Since the SINI public school
applicants within these grade bands were automatically awarded
scholarships this first year, the award probabilities for the non-SINI
public school applicants were somewhat lower than the grade-band
average, depending on the proportion of slots that automatically were
awarded to SINI public applicants.

## *Results of the Scholarship Lottery*

Once these decisions had been made, the scholarship lottery operated as
a series of mini-lotteries for each of the priority strata and grade
bands within them. The computer program applied separate selection
probabilities to the groups of eligible applicants who met the various
priority and grade-band criteria (Figure 3-2 and Table 3-2).

-   Because the award of scholarships to the public school applicants in
    grades K to 5 was not constrained by the number of slots available,
    all those applicants were automatically awarded scholarships.

> ![](media/image2.wmf)

-   The SINI public school applicants, representing the highest
    statutory priority, all received scholarships even if they were
    entering the slot-constrained middle and high school grades.

-   A modest number of non-SINI public school applicants entering middle
    school and a moderate number of non-SINI public school applicants
    entering high school were randomly assigned to the control group (81
    and 112, respectively).

-   A total of 216 of the applicants already attending private schools
    were awarded scholarships. This last number was determined by ED and
    the DC Mayor, based on setting award probabilities at about half of
    the probabilities for non-SINI public school applicants in the
    relevant grade bands.

**Table 3-2. Lottery Probabilities and Assignments, by Applicant Type
and Grade-Level\
Band: Spring 2004**

+----------------------+-----------+-----------+-----------+----------+
|                      | Gr        |           |           |          |
|                      | ade-Level |           |           |          |
|                      | Band      |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
| ##                   | **K - 5** | **6 - 8** | **9 -     | **To     |
| ## Type of Applicant |           |           | 12**      | tal^1^** |
+----------------------+-----------+-----------+-----------+----------+
| **Public SINI**      | ** **     | ** **     | ** **     | ** **    |
+----------------------+-----------+-----------+-----------+----------+
| > Eligibles          | 24        | 20        | 35        | 79       |
+----------------------+-----------+-----------+-----------+----------+
| > Probability        | 100.0%    | 100.0%    | 100.0%    | 100.0%   |
+----------------------+-----------+-----------+-----------+----------+
| > Scholarships       | 24        | 20        | 35        | 79       |
+----------------------+-----------+-----------+-----------+----------+
| > Nonrecipients      | 0         | 0         | 0         | 0        |
+----------------------+-----------+-----------+-----------+----------+
|                      |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
| **Non-SINI Public**  |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
| > Eligibles          | 772       | 336       | 156       | 1,264    |
|                      |           | shaded    | shaded    |          |
+----------------------+-----------+-----------+-----------+----------+
| > Probability        | 100.0%    | 75.9%     | 28.2%     | 84.7%    |
|                      |           | shaded    | shaded    |          |
+----------------------+-----------+-----------+-----------+----------+
| > Scholarships       | 772       | 255       | 44 shaded | 1,071    |
|                      |           | shaded    |           |          |
+----------------------+-----------+-----------+-----------+----------+
| > Nonrecipients      | 0         | 81 shaded | 112       | 193      |
|                      |           |           | shaded    |          |
+----------------------+-----------+-----------+-----------+----------+
|                      |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
| **Private**          |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
| > Eligibles          | 239       | 156       | 110       | 505      |
+----------------------+-----------+-----------+-----------+----------+
| > Probability        | 55.2%     | 41.7%     | 17.3%     | 42.8%    |
+----------------------+-----------+-----------+-----------+----------+
| > Scholarships       | 132       | 65        | 19        | 216      |
+----------------------+-----------+-----------+-----------+----------+
| > Nonrecipients      | 107       | 91        | 91        | 289      |
+----------------------+-----------+-----------+-----------+----------+
| >                    |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
| **Totals**           |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
| > Eligibles          | 1,035     | 512       | 301       | 1,848    |
+----------------------+-----------+-----------+-----------+----------+
| > Probability        | 89.7%     | 66.4%     | 32.6%     | 73.9%    |
+----------------------+-----------+-----------+-----------+----------+
| > Public             | 796       | 275       | 79        | 1,150    |
| > Scholarships       |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
| > Private            | 132       | 65        | 19        | 216      |
| > Scholarships       |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
|                      |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
| **Total              | **928**   | **340**   | **98**    | *        |
| Scholarships**       |           |           |           | *1,366** |
+----------------------+-----------+-----------+-----------+----------+
| > Public             | 0         | 81        | 112       | 193      |
| > Nonrecipients      |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
| > Private            | 107       | 91        | 91        | 289      |
| > Nonrecipients      |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
|                      |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+
| **Total              | **107**   | **172**   | **203**   | **482**  |
| Nonrecipients**      |           |           |           |          |
+----------------------+-----------+-----------+-----------+----------+

^1^ The totals here for combined public school applicants (79+1264 =
1343) and private school applicants (505) differ from the figures of
1330 and 518 presented elsewhere (e.g., in Table 2-5). The reason is
that 13 applicants were originally classified as attending non-SINI
public schools based on parental responses to the school type and name
of school questions in the baseline survey. In all 13 cases, the parent
indicated that the child was attending a public school and provided
either no school name (4 cases) or a school name that did not obviously
signify the type of school (9 cases―4 of which were preschools). Further
investigations by the technical advising team indicated that these 13
students were attending private schools or private preschools, and their
school type classification was corrected accordingly.

NOTE: The shaded boxes represent the randomization of the impact
analysis sample.

SOURCE: Evaluation database (based on information from multiple sources:
(1) the DC Opportunity Scholarship Program applications, (2) the program
operator's files, and (3) data files from DCPS).

As a result of the automatic awarding of scholarships to some students
and the randomization process in oversubscribed grades:[^35]

-   A total of 1,366 eligible applicants were awarded program
    scholarships in the initial year.[^36] They comprise the program
    sample that will be the subject of performance reporting in future
    evaluation reports. Of these, 1,150 (84 percent) were applicants
    from public schools, and 216\
    (16 percent) were applicants from private schools.

-   A total of 299 (22 percent) of the students in the program sample
    comprise the treatment group for the impact sample (Table 3-3). They
    were all public school applicants entering grades 6 to 12, where the
    number of eligible applicants exceeded the number of available
    private school slots.

-   A total of 482 students were randomly assigned to nonrecipient
    status. Of the students who did not receive scholarships through the
    lottery, 193 (40 percent) were public school applicants who will
    comprise the control group for the impact analysis. We will track
    their academic achievement and experiences over time and compare
    them with those of the 299 scholarship recipients in the treatment
    group to assess program impact.

-   The remaining 289 (60 percent) nonrecipients who were private school
    applicants are members of neither the program sample (because they
    did not receive a scholarship) nor the impact sample (because they
    are nonrecipients likely to continue to attend private school).
    Thus, this group of eligible applicants from the 2003-04 cohort will
    not be followed in the future as part of the evaluation.

    In sum, the scholarship lottery produced two groups of students for
    purposes of meeting statutory requirements for the DC Choice
    Opportunity Scholarship program evaluation. The 1,366 scholarship
    recipients will be the subject of annual performance reporting. The
    492 public school applicants who were entering grades subject to
    random assignment will contribute to the annual impact analysis (see
    shaded cells of Table 3-3): these include 299 students assigned to
    the treatment group (and also included in the performance reporting
    sample) and 193 students assigned to the control group. The 289
    private school applicants who were not awarded scholarships belong
    to neither group. Since they previously attended and presumably will
    continue to attend private school using resources outside of the
    scholarship program, following these students after baseline would
    not contribute meaningfully to the evaluation. To conserve
    resources, this group of initial applicants will not be part of the
    evaluation going forward.

**Table 3-3. DC Opportunity Scholarship Program Applicants, by Receipt
of Scholarship and Inclusion in the Impact Analysis Sample: Spring
2004**

  -------------- ------------ -------------- ------------ ------------- --------------- --------------- ---------------
                              Applicants who                            Applicants who                  
                              Received                                  Did Not Receive                 
                              Scholarships                              Scholarships                    
                              (Performance                                                              
                              Reporting                                                                 
                              Sample)                                                                   

  School Type at Eligible     Non-Impact     Impact       Total         Impact Sample   Non-Impact      Total
  Application    Applicants   Sample         Sample       Scholarship   Nonrecipients   Sample          Nonrecipients
  and Grade Band              Recipients     Recipients   Recipients                    Nonrecipients   

  SINI                                                                                                  

  Grades K-5     24           24             0            24            0               0               0

  Grades 6-8     20           20             0            20            0               0               0

  Grades 9-12    35           35             0            35            0               0               0

  **Subtotal**   **79**       **79**         **0**        **79**        **0**           **0**           **0**

                                                                                                        

  Non-SINI                                                                                              
  Public                                                                                                

  Grades K-5     772          772            0            772           0               0               0

  Grades 6-8     336          0              255 shaded   255           81 shaded       0               81

  Grades 9-12    156          0              44 shaded    44            112 shaded      0               112

  **Subtotal**   **1,264**    **772**        **299**      **1,071**     **193** shaded  **0**           **193**
                                             shaded                                                     

                                                                                                        

  Private                                                                                               

  Grades K-5     239          132            0            132           0               107             107

  Grades 6-8     156          65             0            65            0               91              91

  Grades 9-12    110          19             0            19            0               91              91

  **Subtotal**   **505**      **216**        **0**        **216**       **0**           **289**         **289**

                                                                                                        

  **Totals**     **1,848**    **1,067**      **299**      **1,366**     **193** shaded  **289**         **482**
                                             shaded                                                     
  -------------- ------------ -------------- ------------ ------------- --------------- --------------- ---------------

NOTE: The shaded boxes represent the randomization of the impact
analysis sample.

SOURCE: Evaluation database (based on information from multiple sources:
(1) the DC Opportunity Scholarship Program applications, (2) the program
operator's files, and (3) data files from DCPS).

However, the impact sample in the first year of the program is not, on
its own, sufficiently sizable to identify program impacts unless they
are very large. The statistical power analysis conducted for the
evaluation (see Chapter 1 and Appendix A) suggests this initial
treatment (scholarship) and control (no scholarship) group in the impact
sample is too small for the evaluation to reliably draw conclusions
about any differences in achievement outcomes that might be expected
from an intervention of this kind.[^37] Instead, the treatment and
control groups from the first year lottery will be combined with those
from the lottery for second year applicants, expected in April 2005, to
bring more evidence to bear on the rigorous evaluation of program
impacts.

**3.3 Placement Lotteries**

To receive a seat under the DC Opportunity Scholarship Program,
scholarship winners were instructed to identify and visit each private
school that they were interested in attending. To facilitate this
process, the WSF and its partners conducted school fairs where parents
and children were given the opportunity to meet with representatives
from participating schools to learn about the programs and admission
requirements to enter the schools.[^38] Scholarship winners were
strongly encouraged to meet with representatives from more than one
school to maximize their chance of placement in a school of their
choice.

It was a requirement of the scholarship program that to receive a seat
in a private school, scholarship winners must (1) visit with private
school representatives and secure a provisional seat in that school and
(2) complete and submit a school preference form to the WSF listing
schools by first, second, third, and fourth choice.

Although the program operators initially planned to conduct a single
placement lottery, this was not possible because many participating
schools were late to finalize the number of seats available to
scholarship winners, and many families failed to complete their school
visits and other admissions requirements by the lottery deadline of July
12.[^39] A decision was made to go forward with the lottery so that the
substantial number of families that *had* completed the school search
process in time could be quickly informed about where their children
would be attending school in the fall. The program implementer
determined that they would assist the remaining scholarship recipients
in completing their search and in finding appropriate placements. As a
result, placements were made using a combination of a lottery for
children meeting all of the requirements by the specified deadline
followed by WSF placing children in schools on a flow basis as schools
finalized their seat openings and scholarship winners completed all of
the requirements of their school search. For the purposes of the
placement process, no distinction was made between scholarship winners
in the impact sample and those in the non-impact sample.

*Lottery Methodology Used To Distribute Seats to Scholarship Recipients*

To ensure that scholarship winners were assigned seats in a school
without bias or favoritism, Westat programming staff wrote and
thoroughly tested a computer application program that distributed seats
in accordance with the following rules:

-   Scholarship winners currently attending a public school but already
    admitted to a specific private school before the DC Opportunity
    Scholarship Program began (i.e., "pre-admits") were automatically
    given a seat in the school in which they were pre-admitted. The
    automatic award of these seats had no effect on the seats available
    to other scholarship winners.

-   Scholarship winners already enrolled in a participating private
    school who submitted a form to WSF indicating that they wished to
    remain in that school were also automatically assigned a seat. As
    noted earlier, there was no effect on seats available to the
    remaining scholarship winners.

After eliminating these students from the seat competition, the
remaining students were assigned seats under the following rules:

-   Students were organized into three categories corresponding to the
    priority groups used for the scholarship lottery (i.e., all students
    attending SINI schools were grouped together, all students in
    non-SINI public schools were grouped together, and all students in
    private schools who were seeking to change schools were grouped
    together).

-   The students in each separate category were randomly ordered.

-   Once the random order was established within each group, the three
    groups were placed in a single file for school assignment with the
    SINI public students first, followed by non-SINI public students,
    and finally private school students seeking to switch schools.

-   Placement was based on the school preferences─up to four schools
    could be named and ranked in order of priority─indicated by each
    family for each child. Only schools that had agreed to accept the
    child could be assigned as part of the placement lottery. Since a
    seat may not have been available in a student's first school
    preference, an attempt was made to place the student in their second
    preference, then third, and so on until all preferences were
    exhausted.

-   The seat assignment described here was used for each child, with the
    exception of families with more than one scholarship student who
    wanted to place their children in the same school.[^40] When this
    occurred, the children were processed together. The first school
    preference was examined to determine whether seats were available at
    that school for all scholarship children in the family. If not, the
    second preference was examined followed by the third, and so on
    until all school preferences were checked. If a family placement was
    not possible, then children were placed individually, as described
    earlier.

*Results of the First Placement Lottery*

The total number of scholarship winners at the time of the first
placement lottery was 1,277.[^41] However, 513 of them were not prepared
to participate in the first placement lottery because they failed to
visit with school representatives to secure a seat, failed to return a
school preference form by the deadline, or both. Some of the reasons
given include that they could not attend school fairs because of
scheduling conflicts, transportation problems, insufficient length of
time to visit with school representatives, could not afford to miss work
to visit schools, and some simply lost interest in the program.

Next, 196 scholarship winners automatically received a seat in a
particular private school. These included (1) students who were
currently attending a public school but who had already been admitted to
a participating private school for the fall term prior to the late-April
program application period, and (2) applicants from participating
private schools who completed a school preference form indicating that
they intended to remain in their current school. In neither of these
cases did a new slot need to be taken at a participating private school.
That left a total of 568 students who were entered into the placement
lottery to compete for available seats in private schools.[^42]

The computerized placement lottery assigned seats to 549 of the 568
students seeking seats in participating private schools for the school
year beginning in fall 2004 (Table 3-4). As predicted by the
legislation, there were more scholarship recipients applying to some
private schools than could be accommodated. However, of the 568 students
who were competing for new slots, 531 (93 percent) were assigned their
first choice school. Another 3 percent wound up in their second or third
preferred school.[^43] Because the school preferences clustered around
certain geographically or programmatically desirable schools, 19
students (3 percent) could not be placed in a preferred school by the
computerized placement lottery.

*Followup to the Placement Lottery*

Since the start of the school year was rapidly approaching and some
participating schools were making additional slots available to
scholarship students, the remaining 295 scholarship students who
completed their school search after the operation of the placement
lottery were assigned to available slots in private schools by WSF on a
case-by-case basis. The added slots also permitted WSF eventually to
place almost all students who could not be placed by the initial
placement lottery, including every student who remained interested in
participating in the program. Parents and children were contacted either
by telephone or letter notifying them of their successful placement.

**Table 3-4. Results of the Computerized Placement Lottery, by Priority
Group:\
Spring 2004**

  ------------------ ------------- ------------- ------------- -------------
  **Priority Group** **Students    **Students    **Students    **Total
                     Competing for Assigned      Assigned      Students
                     Available     First         Second or     Placed**
                     Seats**       Choice^1^**   Third         
                                                 Choice**      

  SINI               47            44            0             44

  Non-SINI Public    503           471           18            489

  Private            18^2^         16            0             16

  **Total**          **568**       **531**       **18**        **549**
  ------------------ ------------- ------------- ------------- -------------

^1^ If a child submitted an invalid first school preference, and the
second school preference was valid and they were able to obtain a seat
in that school, the child is counted as being placed in their first
preference on this table.

^2^ These students were private school scholarship winners who wanted to
switch to a different participating private school and, therefore, were
entered into the competition for new private school seats.

SOURCE: Applicant database and school preference forms submitted by
parents.

At the end of the placement process, 1,040 scholarship students were
assigned available seats in participating private schools selected by
their parents (Table 3-5). Every scholarship winner, whether in the
impact analysis treatment group or not, who completed a school search
and remained interested in the program was eventually placed in a
private school chosen by the family.

**3.4 Scholarship Usage**

Of the 1,366 students awarded scholarships through the computerized
scholarship lottery and appeals/\
extended completion period, 1,040 obtained a placement, and 1,027 had
matriculated at a preferred private school as of September 10, 2004.
This represents an initial scholarship usage rate of 75 percent for the
program sample that will be used for performance reporting. This
program-wide scholarship usage rate is at the high end of the range for
initial usage among the previous large-scale school voucher or private
scholarship programs.[^44]

The initial scholarship usage rate for the 299 members of the treatment
group in the impact sample was somewhat lower, at 62 percent. The lower
rate for this group, in comparison to both the overall program sample
and that of other scholarship programs reflects two factors:

**Table 3-5. Final Results of Lottery and WSF Student Placements, by
Priority Group:\
Spring 2004**

+---------+-------+----------+--------+----------+--------+----------+
| **P     | **Sc  | **Failed | **     | **Not    | *      | **C      |
| riority | holar | To       | Placed | Ch       | *Total | ompleted |
| Group** | -ship | Complete | T      | anging** | St     | Search   |
|         | Winn  | School   | hrough |          | udents | but      |
|         | ers** | Search** | Lot    | **Scho   | with   | Un       |
|         |       |          | tery** | ols**^1^ | Place  | placed** |
|         |       |          |        | **or     | ment** |          |
|         |       |          |        | Placed   |        |          |
|         |       |          |        | by WSF** |        |          |
+---------+-------+----------+--------+----------+--------+----------+
| SINI    | 79    | 28       | 33     | 18       | 51     | 0        |
| public  |       |          |        |          |        |          |
+---------+-------+----------+--------+----------+--------+----------+
| N       | 1,071 | 276      | 497    | 298      | 795    | 0        |
| on-SINI |       |          |        |          |        |          |
| public  |       |          |        |          |        |          |
+---------+-------+----------+--------+----------+--------+----------+
| Private | 216   | 22       | 19     | 175      | 194    | 0        |
+---------+-------+----------+--------+----------+--------+----------+
| **      | **1,  | **326**  | *      | **491**  | **1    | **0**    |
| Total** | 366** |          | *549** |          | ,040** |          |
+---------+-------+----------+--------+----------+--------+----------+

^1^ Not changing schools includes public school applicants who obtained
admission to a particular private school prior to applying to the
program and private school applicants continuing in their previous
school. There were 196 students in this category.

SOURCE: Computerized placement lottery database (Westat) and WSF
placement records.

-   Scholarship usage rates vary significantly by grade level, with the
    highest usage among K-5 students (Appendix D, Table D2). Since the
    treatment group in the impact sample does not include any K-5
    students, the initial usage rate for the treatment group is lower
    than the rate for the program sample as a whole and lower than that
    of other scholarship programs, almost all of which focused on
    elementary grades.

-   In addition, about 16 percent of the DC Opportunity Scholarship
    recipients are students who were already attending private schools
    when they applied and thus were virtually certain to use their
    scholarships. These students are not represented among either the
    299 members of the treatment group or recipients of other
    scholarship programs.

Students who chose to take advantage of their scholarships were
distributed throughout the participating schools, although there was
some clustering by religious affiliation and location. For example,
although about half of the schools are affiliated with the Catholic
Church, 61 percent of scholarship users enrolled in a Catholic school.
Seventeen percent of the students using scholarships chose a
non-Catholic religious school. The proportion of scholarship students
attending nonsectarian private schools, 22 percent, is high compared to
previous government-sponsored school choice programs. [^45]

This pattern of scholarship use in the District is a function of the
types of private schools established and willing to participate in the
program and differences in the numbers of new seats that schools of
different types made available.

As with the distribution of participating schools described in Chapter
2, the schools that scholarship students have selected are slightly
concentrated in a few wards but are also found throughout the
District[^46] (Figure 3-3). Almost one-quarter of scholarship users are
attending private schools in Ward 4, and Ward 6 is hosting nearly 20
percent. The remaining scholarship students are scattered throughout the
city, with Ward 2 private schools serving the smallest proportion of
participating students.

![](media/image2.wmf)

# 4. Characteristics of Program Applicants

The implementation and impacts of most programs are shaped by the
characteristics of those who apply to participate. The DC Opportunity
Scholarship Program is no exception; the law's provisions make it clear,
for example, that the program is intended to serve economically
disadvantaged families. In determining and interpreting program effects
later on, it is important to know how well the program is targeted to
disadvantaged families and, beyond that, what types of families and
students apply, win scholarships in the lottery, and choose to use them
to enroll in a private school. This chapter describes the 1,848 eligible
applicants to the DC Opportunity Scholarship Program in part by
comparing them, and subsets of them, to several policy-relevant groups.
The chapter concludes with some initial evidence regarding the extent
that public and private schools in the District are experiencing a
significant loss or gain of students due to the first year of program
implementation.

**4.1 Public School Applicants Compared to Public Nonapplicant Students
in the District**

There are several reasons to examine the eligible public school
applicants to the DC Opportunity Scholarship Program in relation to
other DCPS students. Most importantly, the comparison provides a context
for considering the kinds of students who might be attracted to the
program in future years in the District or to a similar program in other
locations.

Of course, program applicants differ from public nonapplicant students
in important ways, some by design of the program. Applicants must have
family incomes at or below 185 percent of the federal poverty line. In
addition, the majority of applicants were young students, in the early
elementary grades, while DCPS students are somewhat more evenly spread
out across grade levels. These differences are important because they
can affect average test scores, racial/ethnic group membership, or other
characteristics of interest. As a result, we compare applicants to
several different subgroups of DCPS students, in addition to DCPS
students overall, and make comparisons within certain grade bands
whenever it is practical to do so.

Because the evaluation's primary outcome measure is academic
achievement, it is useful to understand whether applicants to the DC
Opportunity Scholarship Program *started out* with higher, lower, or
similar levels of achievement than nonapplicants. This focus necessarily
limits the analysis to the 784 eligible applicants who participated in
the accountability testing in DC public and charter schools in the
spring of 2004 and could be identified conclusively in the database
provided by DCPS.[^47] The analysis cannot include private school
applicants, for whom baseline (pre-application) test scores are not
available or are not comparable to testing in DCPS.[^48]

The accountability database contains the results of DCPS administration
of the Stanford Achievement Test, Version 9 (SAT-9) to all students in
grades 3 through 11, and some students in grades 1 and 2, in the spring
of 2004. These tests covered the areas of reading and mathematics, with
results expressed in terms of National Percentile Ranks (NPRs). The NPR
for a particular student test score describes the percentage of students
who scored below that level on a nationally normed test for the specific
grade level, subject, and testing period that applies to the student
score.[^49] Expressing test-score norms in this way makes them
comparable across grade levels. The DCPS database also includes a modest
number of student demographic characteristics with which to organize
comparisons.

### All Public School Applicants Compared to All Nonapplicant DCPS Students

To answer the question---what kinds of students apply to the DC
Opportunity Scholarship Program---program applicants as a whole were
considered in relation to all DCPS students who did not apply to the
program. This analysis yields the following conclusions (Table 4-1):

-   Applicants performed similarly to DCPS nonapplicants in both reading
    and mathematics.[^50]

-   Applicants are somewhat more likely (16 percent) than public
    nonapplicant students (14 percent) to have been enrolled in special
    education.[^51]

-   Public school students attracted to the DC Opportunity Scholarship
    Program were more likely to be African American than were
    nonapplicant public school students (92 percent vs. 85 percent) but
    less likely to be Hispanic (6 percent vs. 9 percent) or other races
    (2 percent vs. 6 percent).

-   Applicants are substantially more economically disadvantaged than
    are DCPS students overall; applicants were more likely to be
    enrolled in the federal Free or Reduced-price Lunch Program (FRL)
    than were nonapplicants (85 percent vs. 68 percent), a result that
    is not surprising given the income ceiling for program
    eligibility.[^52]

**Table 4-1. Characteristics of DC Public School Students, Program
Applicants\
Versus Nonapplicants: Spring 2004**

+-------------------------------------+----------+----------+----------+
| > **Characteristic**                | **Appl   | **DCPS   | **Diff   |
|                                     | icants** | Sample** | erence** |
+-------------------------------------+----------+----------+----------+
| > **Baseline Test Scores^1^**       |          |          |          |
+-------------------------------------+----------+----------+----------+
|                                     |          |          |          |
+-------------------------------------+----------+----------+----------+
| > Average Reading Percentile        | 41.5     | 40.1     | 1.4      |
+-------------------------------------+----------+----------+----------+
| > *Percent missing*                 | *26*     | *26*     |          |
+-------------------------------------+----------+----------+----------+
|                                     |          |          |          |
+-------------------------------------+----------+----------+----------+
| > Average Mathematics Percentile    | 47.2     | 46.3     | 1.0      |
+-------------------------------------+----------+----------+----------+
| > *Percent missing*                 | *25*     | *25*     |          |
+-------------------------------------+----------+----------+----------+
|                                     |          |          |          |
+-------------------------------------+----------+----------+----------+
| > **Percent in Special Education**  | 16       | 14       | 2\*      |
+-------------------------------------+----------+----------+----------+
| > *Percent missing*                 | *24*     | *23*     |          |
+-------------------------------------+----------+----------+----------+
|                                     |          |          |          |
+-------------------------------------+----------+----------+----------+
| > **Percent, by Race**              |          |          |          |
+-------------------------------------+----------+----------+----------+
| > African American                  | 92       | 85       | 8\*\*    |
+-------------------------------------+----------+----------+----------+
| > Hispanic                          | 6        | 9        | -4\*\*   |
+-------------------------------------+----------+----------+----------+
| > Other race^2^                     | 2        | 6        | -4\*\*   |
+-------------------------------------+----------+----------+----------+
| > *Percent missing*                 | *1*      | *1*      |          |
+-------------------------------------+----------+----------+----------+
|                                     |          |          |          |
+-------------------------------------+----------+----------+----------+
| > **Percent, by Gender**            |          |          |          |
+-------------------------------------+----------+----------+----------+
| > Female                            | 51       | 50       | 0        |
+-------------------------------------+----------+----------+----------+
| > *Percent missing*                 | *0*      | *1*      |          |
+-------------------------------------+----------+----------+----------+
|                                     |          |          |          |
+-------------------------------------+----------+----------+----------+
| > **Percent Participating in        | 85       | 68       | 17\*\*   |
| > Free/Reduced-price Lunch          |          |          |          |
| > Program**                         |          |          |          |
+-------------------------------------+----------+----------+----------+
| > *Percent missing*                 | *2*      | *2*      |          |
+-------------------------------------+----------+----------+----------+
|                                     |          |          |          |
+-------------------------------------+----------+----------+----------+
| > **Sample size**                   | 1,077    | 66,868   |          |
+-------------------------------------+----------+----------+----------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ Test-score results are in terms of National Percentile Ranks, with
50 as the median score.

^2^ "Other race" includes students who were identified as white, Asian,
American Indian, or Alaska Native.

NOTE: Detail may not sum to totals because of rounding. Applicant sample
includes all applicants identified in the DCPS database, regardless of
whether or not they participated in the accountability testing.

SOURCE: Accountability testing database for District of Columbia public
and charter schools, DCPS Office of Communications and Public
Information.

*All Public School Applicants Compared to Economically Disadvantaged
Nonapplicant DCPS Students*

Since, by virtue of the eligibility criteria, program applicants are
more economically disadvantaged than DCPS students overall, it is
appropriate to consider the applicants' characteristics relative to
those of public school students who are similarly disadvantaged. The
income criterion for the DC Opportunity Scholarship Program---at or
below 185 percent of the federal poverty line---is similar to that
defining eligibility for enrollment in the federal FRL program or for
students that are Title 1 eligible (Table 4-2). [^53]

Using those indicators of economic disadvantage for both applicants and
nonapplicants we conclude the following:

-   Program applicants have higher test score performance than
    economically disadvantaged nonapplicants. Applicants scored nearly
    four NPR points higher in both reading and mathematics than similar
    nonapplicants.[^54]

-   Among economically disadvantaged students, there is no statistically
    significant difference between applicants and nonapplicants in the
    proportion enrolled in special education.

-   Lower income applicants remain somewhat more likely to be African
    American (93 percent) and less likely to be Hispanic (6 percent)
    than nonapplicant public school students with similar income levels
    (88 percent and 10 percent, respectively).

-   By design, the two groups are similar regarding their eligibility
    for the FRL program.

### All Public School Applicants Compared to Economically Disadvantaged Nonapplicant DCPS Students, by Grade Bands

In addition to their economic circumstances, program applicants differ
from DCPS students overall in terms of their grade levels: applicants,
including public school applicants, are more likely to be in the
elementary grades and less likely to be in the high school grades than
are DCPS nonapplicants. To make the groups as comparable as possible,
we, therefore, present data on eligible public school applicants in each
of the three grade bands used for the lottery and nonapplicant DCPS
students in the identical grade bands who were income-eligible for the
program. Students are assigned to a subgroup based on the grade they
were forecasted to be entering in the fall of 2004, as is the case with
all of the grade-level comparisons in this report. We include here the
sample that is limited to economically disadvantaged students, since
controlling for family income, which is associated with test scores,
generates a more reliable comparison. A comparison by grade band using
the full sample of students, including students above the FRL
eligibility ceiling, is presented in Appendix B (Table B1).

**Table 4-2. Characteristics of DC Public School Free or Reduced-Price
Lunch\
Program Students, Program Applicants Versus Nonapplicants: Spring 2004**

+------------------------------------+----------+----------+----------+
| **Characteristic**                 | **Appl   | **DCPS   | **Diff   |
|                                    | icants** | Sample** | erence** |
+------------------------------------+----------+----------+----------+
| **Baseline Test Scores**^1 ^       |          |          |          |
+------------------------------------+----------+----------+----------+
|                                    |          |          |          |
+------------------------------------+----------+----------+----------+
| > Average Reading Percentile       | 40.2     | 36.4     | 3.9\*\*  |
+------------------------------------+----------+----------+----------+
| > *Percent missing*                | *25*     | *26*     |          |
+------------------------------------+----------+----------+----------+
| >                                  |          |          |          |
+------------------------------------+----------+----------+----------+
| > Average Mathematics Percentile   | 46.7     | 43.0     | 3.7\*\*  |
+------------------------------------+----------+----------+----------+
| > *Percent missing*                | *25*     | *24*     |          |
+------------------------------------+----------+----------+----------+
| > ** **                            |          |          |          |
+------------------------------------+----------+----------+----------+
| **Percent in Special Education**   | 17       | 15       | 1        |
+------------------------------------+----------+----------+----------+
| > *Percent missing*                | *24*     | *22*     |          |
+------------------------------------+----------+----------+----------+
| > ** **                            |          |          |          |
+------------------------------------+----------+----------+----------+
| **Percent, by Race**               |          |          |          |
+------------------------------------+----------+----------+----------+
| > African American                 | 93       | 88       | 5\*\*    |
+------------------------------------+----------+----------+----------+
| > Hispanic                         | 6        | 10       | -5\*\*   |
+------------------------------------+----------+----------+----------+
| > Other race^2^                    | 2        | 2        | 0        |
+------------------------------------+----------+----------+----------+
| > *Percent missing*                | *0*      | *0*      |          |
+------------------------------------+----------+----------+----------+
| > ** **                            |          |          |          |
+------------------------------------+----------+----------+----------+
| **Percent, by Gender**             |          |          |          |
+------------------------------------+----------+----------+----------+
| > Female                           | 51       | 51       | 0        |
+------------------------------------+----------+----------+----------+
| > *Percent missing*                | *0*      | *0*      |          |
+------------------------------------+----------+----------+----------+
| > ** **                            |          |          |          |
+------------------------------------+----------+----------+----------+
| **Percent Participating in         | 100      | 100      | 0        |
| Free/Reduced-price Lunch Program** |          |          |          |
+------------------------------------+----------+----------+----------+
| > *Percent missing*                | *0*      | *0*      |          |
+------------------------------------+----------+----------+----------+
| >                                  |          |          |          |
+------------------------------------+----------+----------+----------+
| **Sample size**                    | 894      | 44,740   |          |
+------------------------------------+----------+----------+----------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ Test-score results are in terms of National Percentile Ranks, with
50 as the median score.

^2^ "Other race" includes students who were identified as white, Asian,
American Indian, or Alaska Native.

NOTE: Detail may not sum to totals because of rounding. Applicant sample
includes all eligible applicants identified in the DCPS database that
were participating in the free and reduced price lunch program.

SOURCE: Accountability testing database for District of Columbia public
and charter schools, DCPS Office of Communications and Public
Information.

This most detailed comparison of applicants and nonapplicants is nearly
identical to the other analyses but provides some additional information
(Table 4-3):

-   Although program applicants have higher reading and mathematics
    achievement than non-SINI public school students who are similar in
    terms of income and grade level, these differences are only
    statistically significant among elementary students.[^55]

-   There are no statistically significant differences in the rates of
    special education enrollment between applicants and similar
    nonapplicants in any grade band.

-   The higher rates of African American applicants and lower rates of
    applicants of other races are evident at both the elementary and
    high school levels but not at the middle school level.

-   Again, by design, the applicant group is similar to the nonapplicant
    comparison group in all three grade bands regarding the proportions
    that are enrolled in the FRL program.

*Public School Participants Compared to Economically Disadvantaged
Nonapplicant DCPS Students, by Grade Bands*

The legislation requires that the evaluation include a comparison
between participating eligible students and DCPS students in the same
grades, an analysis we refer to as performance reporting. Participants
differ from the group of public school applicants above, in that
participants: (1) include students who were already attending private
school when they applied to the program, and (2) exclude students who
did not receive scholarships as part of the lottery for those entering
grades 6 through 12 (the impact sample's control group). Unfortunately,
at baseline (before the program begins), it is not possible to compare
the prior academic achievement of scholarship recipients who were
already enrolled in private schools because no test data comparable to
that of the DCPS accountability assessment is available from private
schools.[^56]

Because public school *participants* are such a large subset of public
*applicants*, comparing either group to similarly economically
disadvantaged DCPS students in the same grade levels yields consistent
similarities and differences (Table 4-4):

-   Program participants in grades K to 5 score somewhat higher in
    reading and mathematics than comparable nonapplicants. They also are
    more likely to be African American and less likely to be Hispanic
    than are nonapplicants. These results are identical to the K to 5
    comparisons of eligible public school applicants to DCPS
    nonapplicants in Table 4-3 because all eligible public school
    applicants in grades K to 5 automatically became program
    participants.[^57]

**Table 4-3. Characteristics of DC Public School Free or Reduced-Price
Lunch Students by Grade Band, Program Applicants Versus Nonapplicants:
Spring 2004**

<table style="width:100%;">
<colgroup>
<col style="width: 26%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 9%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
</colgroup>
<tbody>
<tr class="odd">
<td></td>
<td colspan="3"><strong>Grades K-5</strong></td>
<td colspan="3"><strong>Grades 6-8</strong></td>
<td colspan="3"><strong>Grades 9-12</strong></td>
</tr>
<tr class="even">
<td><strong>Characteristic</strong> </td>
<td><strong>Applicants</strong></td>
<td><p><strong>DCPS</strong></p>
<p><strong>Sample</strong></p></td>
<td><strong>Difference</strong></td>
<td><strong>Applicants</strong></td>
<td><p><strong>DCPS</strong></p>
<p><strong>Sample</strong></p></td>
<td><strong>Difference</strong></td>
<td><strong>Applicants</strong></td>
<td><p><strong>DCPS</strong></p>
<p><strong>Sample</strong></p></td>
<td><strong>Difference</strong></td>
</tr>
<tr class="odd">
<td><strong>Baseline Test Scores</strong><sup>1</sup></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Average Reading Percentile</p>
</blockquote></td>
<td><blockquote>
<p>46.5</p>
</blockquote></td>
<td>42.6</td>
<td>4.0*</td>
<td>38.0</td>
<td>36.4</td>
<td>1.5</td>
<td>29.7</td>
<td>28.8</td>
<td>0.8</td>
</tr>
<tr class="even">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>42</em></p>
</blockquote></td>
<td><em>44</em></td>
<td></td>
<td><em>3</em></td>
<td><em>5</em></td>
<td></td>
<td><em>3</em></td>
<td><em>14</em></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p> </p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="even">
<td><blockquote>
<p>Average Mathematics Percentile</p>
</blockquote></td>
<td><blockquote>
<p>51.2</p>
</blockquote></td>
<td>47.8</td>
<td>3.5*</td>
<td>43.8</td>
<td>42.3</td>
<td>1.5</td>
<td>41.7</td>
<td>37.9</td>
<td>3.8</td>
</tr>
<tr class="odd">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>41</em></p>
</blockquote></td>
<td><em>42</em></td>
<td></td>
<td><em>2</em></td>
<td><em>5</em></td>
<td></td>
<td><em>3</em></td>
<td><em>13</em></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p><strong> </strong></p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="odd">
<td><strong>Percent in Special Education</strong></td>
<td><blockquote>
<p>17</p>
</blockquote></td>
<td>13</td>
<td>3</td>
<td>18</td>
<td>17</td>
<td>1</td>
<td>14</td>
<td>16</td>
<td>-2</td>
</tr>
<tr class="even">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>41</em></p>
</blockquote></td>
<td><em>41</em></td>
<td></td>
<td><em>2</em></td>
<td><em>3</em></td>
<td></td>
<td><em>2</em></td>
<td><em>9</em></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p><strong> </strong></p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="even">
<td><strong>Percent, by Race</strong></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="odd">
<td><blockquote>
<p>African American</p>
</blockquote></td>
<td><blockquote>
<p>94</p>
</blockquote></td>
<td>87</td>
<td>7**</td>
<td>88</td>
<td>90</td>
<td>-1</td>
<td>95</td>
<td>89</td>
<td>6*</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Other race<sup>2</sup></p>
</blockquote></td>
<td><blockquote>
<p>6</p>
</blockquote></td>
<td>13</td>
<td>-7**</td>
<td>12</td>
<td>10</td>
<td>1</td>
<td>5</td>
<td>11</td>
<td>-6*</td>
</tr>
<tr class="odd">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>0</em></p>
</blockquote></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>1</em></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p><strong> </strong></p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="odd">
<td><strong>Percent, by Gender</strong></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p>Female</p>
</blockquote></td>
<td><blockquote>
<p>52</p>
</blockquote></td>
<td>51</td>
<td>1</td>
<td>51</td>
<td>51</td>
<td>0</td>
<td>46</td>
<td>52</td>
<td>-6</td>
</tr>
<tr class="odd">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>0</em></p>
</blockquote></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>1</em></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p><strong> </strong></p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="odd">
<td><strong>Percent Participating in Free/Reduced-price Lunch
Program</strong></td>
<td><blockquote>
<p>100</p>
</blockquote></td>
<td>100</td>
<td>0</td>
<td>100</td>
<td>100</td>
<td>0</td>
<td>100</td>
<td>100</td>
<td>0</td>
</tr>
<tr class="even">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>0</em></p>
</blockquote></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p> </p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="even">
<td><strong>Sample size</strong></td>
<td><blockquote>
<p>509</p>
</blockquote></td>
<td>20,893</td>
<td> </td>
<td>259</td>
<td>12,364</td>
<td> </td>
<td>126</td>
<td>11,483</td>
<td> </td>
</tr>
</tbody>
</table>

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ Test-score results are in terms of National Percentile Ranks, with
50 as the median score.

^2^ "Other race" includes students who were identified as Hispanic,
white, Asian, American Indian, or Alaska Native. Because of small cell
sizes, Hispanic was combined with other race in this table.

NOTE: Detail may not sum to totals because of rounding. Applicant
samples include all eligible applicants identified in the DCPS database
that were participating in the free or reduced-price lunch program.

SOURCE: Accountability testing database for District of Columbia public
and charter schools, DCPS Office of Communications and Public
Information.

**Table 4-4. Characteristics of DC Public School Free or Reduced-Price
Lunch Students by Grade Band, Program Participants Versus Nonapplicants:
Spring 2004**

<table>
<colgroup>
<col style="width: 24%" />
<col style="width: 8%" />
<col style="width: 7%" />
<col style="width: 8%" />
<col style="width: 9%" />
<col style="width: 7%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 7%" />
<col style="width: 8%" />
</colgroup>
<tbody>
<tr class="odd">
<td></td>
<td colspan="3"><strong>Grades K-5</strong></td>
<td colspan="3"><strong>Grades 6-8</strong></td>
<td colspan="3"><strong>Grades 9-12</strong></td>
</tr>
<tr class="even">
<td><strong>Characteristic</strong> </td>
<td><strong>Participants</strong></td>
<td><p><strong>DCPS</strong></p>
<p><strong>Sample</strong></p></td>
<td><strong>Difference</strong></td>
<td><strong>Participants</strong></td>
<td><p><strong>DCPS</strong></p>
<p><strong>Sample</strong></p></td>
<td><strong>Difference</strong></td>
<td><strong>Participants</strong></td>
<td><p><strong>DCPS</strong></p>
<p><strong>Sample</strong></p></td>
<td><strong>Difference</strong></td>
</tr>
<tr class="odd">
<td><strong>Baseline Test Scores</strong><sup>1</sup></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Average Reading Percentile</p>
</blockquote></td>
<td><blockquote>
<p>46.5</p>
</blockquote></td>
<td>42.6</td>
<td>4.0*</td>
<td>37.7</td>
<td>36.4</td>
<td>1.3</td>
<td>24.0</td>
<td>28.8</td>
<td>-4.8</td>
</tr>
<tr class="even">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>42</em></p>
</blockquote></td>
<td><em>44</em></td>
<td></td>
<td><em>3</em></td>
<td><em>5</em></td>
<td></td>
<td><em>4</em></td>
<td><em>14</em></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p> </p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td></td>
<td> </td>
<td></td>
<td></td>
<td> </td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p>Average Mathematics Percentile</p>
</blockquote></td>
<td><blockquote>
<p>51.2</p>
</blockquote></td>
<td>47.8</td>
<td>3.5*</td>
<td>43.0</td>
<td>42.3</td>
<td>0.7</td>
<td>41.0</td>
<td>37.9</td>
<td>3.1</td>
</tr>
<tr class="odd">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>41</em></p>
</blockquote></td>
<td><em>42</em></td>
<td></td>
<td><em>1</em></td>
<td><em>5</em></td>
<td></td>
<td><em>4</em></td>
<td><em>13</em></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p><strong> </strong></p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td></td>
<td> </td>
<td></td>
<td></td>
<td> </td>
<td></td>
</tr>
<tr class="odd">
<td><strong>Percent in Special Education</strong></td>
<td><blockquote>
<p>17</p>
</blockquote></td>
<td>13</td>
<td>3</td>
<td>17</td>
<td>17</td>
<td>0</td>
<td>19</td>
<td>16</td>
<td>2</td>
</tr>
<tr class="even">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>41</em></p>
</blockquote></td>
<td><em>41</em></td>
<td></td>
<td><em>1</em></td>
<td><em>3</em></td>
<td></td>
<td><em>2</em></td>
<td><em>9</em></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p><strong> </strong></p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td></td>
<td> </td>
<td></td>
<td></td>
<td> </td>
<td></td>
</tr>
<tr class="even">
<td><strong>Percent, by Race</strong></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td></td>
<td> </td>
<td></td>
<td></td>
<td> </td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p>African American</p>
</blockquote></td>
<td><blockquote>
<p>94</p>
</blockquote></td>
<td>87</td>
<td>7**</td>
<td>88</td>
<td>90</td>
<td>-2</td>
<td>95</td>
<td>89</td>
<td>6</td>
</tr>
<tr class="even">
<td><blockquote>
<p>Hispanic</p>
</blockquote></td>
<td><blockquote>
<p>5</p>
</blockquote></td>
<td>12</td>
<td>-7**</td>
<td>9</td>
<td>9</td>
<td>1</td>
<td>0</td>
<td>9</td>
<td>-9*</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Other race<sup>2</sup></p>
</blockquote></td>
<td><blockquote>
<p>1</p>
</blockquote></td>
<td>2</td>
<td>-1</td>
<td>2</td>
<td>1</td>
<td>1</td>
<td>5</td>
<td>2</td>
<td>4*</td>
</tr>
<tr class="even">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>0</em></p>
</blockquote></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>1</em></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p><strong> </strong></p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td></td>
<td> </td>
<td></td>
<td></td>
<td> </td>
<td></td>
</tr>
<tr class="even">
<td><strong>Percent, by Gender</strong></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Female</p>
</blockquote></td>
<td><blockquote>
<p>52</p>
</blockquote></td>
<td>51</td>
<td>1</td>
<td>50</td>
<td>51</td>
<td>-2</td>
<td>45</td>
<td>52</td>
<td>-6</td>
</tr>
<tr class="even">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>0</em></p>
</blockquote></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>1</em></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p><strong> </strong></p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td></td>
<td> </td>
<td></td>
<td></td>
<td> </td>
<td></td>
</tr>
<tr class="even">
<td><strong>Percent Participating in Free/Reduced-price Lunch
Program</strong></td>
<td><blockquote>
<p>100</p>
</blockquote></td>
<td>100</td>
<td>0</td>
<td>100</td>
<td>100</td>
<td>0</td>
<td>100</td>
<td>100</td>
<td>0</td>
</tr>
<tr class="odd">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><blockquote>
<p><em>0</em></p>
</blockquote></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p> </p>
</blockquote></td>
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td></td>
<td> </td>
<td> </td>
</tr>
<tr class="odd">
<td><strong>Sample size</strong></td>
<td><blockquote>
<p>509</p>
</blockquote></td>
<td>20,893</td>
<td> </td>
<td>201</td>
<td>12,364</td>
<td> </td>
<td>55</td>
<td>11,483</td>
<td> </td>
</tr>
</tbody>
</table>

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ Test-score results are in terms of National Percentile Ranks, with
50 as the median score.

^2^ "Other race" includes students who were identified as white, Asian,
American Indian, or Alaska Native.

NOTE: Detail may not sum to totals because of rounding. Participants
samples differ from previous samples because they are limited to
scholarship recipients identified in the DCPS database that were
participating in the free or reduced-price lunch program.

SOURCE: Accountability testing database for District of Columbia public
and charter schools, DCPS Office of Communications and Public
Information.

-   The only statistically significant differences between public school
    program participants and nonapplicants in grades 6 to 12 involve
    student race and ethnicity. High school participants are less likely
    to be Hispanic and more likely to be a race other than African
    American or Hispanic than are nonapplicants.

-   Within all three grade bands, program participants are statistically
    similar to DCPS nonapplicants in their special education status,
    gender, and (by design) eligibility for free or reduced-price lunch.

**4.2 Applicants in the Impact and Non-Impact Samples**

While it is important to examine who applies to the DC Opportunity
Scholarship Program, for the evaluation it is equally critical to ensure
that the applicants who will be the focus of the impact analysis---the
treatment and control groups randomly assigned as part of the
lottery---are similar prior to the beginning of the program. It is on
this similarity between the groups, not only in characteristics easily
measured but also in those not observed, that the scientific benefits of
the RCT approach rests. As described in Chapters 1 and 2, the impact
sample consists of public school applicants who were not from schools in
need of improvement (non-SINI) and who were entering the oversubscribed
grades of 6 through 12.

However, because the impact sample, at least in the first year of the
program, is only a subset of all program applicants, it is also useful
to review how the characteristics of the impact sample differ from the
characteristics of students who will not be included in the analysis of
program effectiveness (or what we will call the "non-impact sample" of
applicants).[^58] The existence of many significant differences between
the impact and non-impact samples might limit one's ability to
extrapolate the results of the impact analysis to measuring the program
impacts as a whole.[^59]

Because grade level can account for significant differences in
characteristics and the impact sample includes only students entering
grades 6 through 12, the comparisons are limited to those grades and
suggest that (Table 4-5):

-   The treatment and control groups are statistically similar on
    important educational and family background measures that could
    affect outcomes, such as baseline test scores, racial/ethnic
    composition, mother's average years of school, and family income.
    Although the two groups are not identical, the differences that
    exist are within the range of what we might expect due to mere
    chance. None of the differences are statistically significant.

**Table 4-5. Characteristics of the Impact Sample and Non-Impact Sample
for Applicants Entering Grades 6 to 12, Spring 2004**

+-----------------------+------+------+------+------+------+-------+
| Characteristics       | T    | Con  | Di   | Im   | No   | Diffe |
|                       | reat | trol | ffer | pact | n-Im | rence |
|                       | ment |      | ence | Sa   | pact |       |
|                       |      |      |      | mple | Sa   |       |
|                       |      |      |      |      | mple |       |
+-----------------------+------+------+------+------+------+-------+
| **Baseline Test       |      |      |      |      | *    | ** ** |
| Scores**^1^           |      |      |      |      | * ** |       |
+-----------------------+------+------+------+------+------+-------+
| Average Reading       |      |      |      |      |      |       |
| Percentile            |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| Grades 6-8            | 39.6 | 42.7 | -3.1 | 40.3 | 35.1 | 5.2   |
+-----------------------+------+------+------+------+------+-------+
| Grades 9-12           | 32.9 | 35.6 | -2.7 | 34.9 | 19.5 | 15.   |
|                       |      |      |      |      |      | 3\*\* |
+-----------------------+------+------+------+------+------+-------+
| *Percent missing*     | *17* | *22* |      | *19* | *84* |       |
+-----------------------+------+------+------+------+------+-------+
|                       |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| Average Mathematics   |      |      |      |      |      |       |
| Percentile            |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| Grades 6-8            | 44.2 | 48.1 | -4.0 | 45.1 | 42.0 | 3.1   |
+-----------------------+------+------+------+------+------+-------+
| Grades 9-12           | 47.6 | 43.8 | 3.8  | 44.9 | 36.3 | 8.6   |
+-----------------------+------+------+------+------+------+-------+
| *Percent missing*     | *16* | *22* |      | *18* | *85* |       |
+-----------------------+------+------+------+------+------+-------+
|                       |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| **Percent of Students | 18   | 23   | -5   | 20   | 12   | 7\*\* |
| with a Learning or    |      |      |      |      |      |       |
| Physical Disability** |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| > *Percent            | *6*  | *7*  |      | *6*  | *3*  |       |
| > missing(average)*   |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
|                       |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| **Percent, by Race**  |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| African American      | 93   | 94   | -1   | 93   | 90   | 3     |
+-----------------------+------+------+------+------+------+-------+
| Other race^2^         | 7    | 7    | 0    | 7    | 10   | -2    |
+-----------------------+------+------+------+------+------+-------+
| *Percent missing*     | *5*  | *6*  |      | *5*  | *10* |       |
+-----------------------+------+------+------+------+------+-------+
|                       |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| **Percent Hispanic    | 9    | 7    | 2    | 8    | 16   | -     |
| (any race)**          |      |      |      |      |      | 8\*\* |
+-----------------------+------+------+------+------+------+-------+
| *Percent missing*     | *4*  | *7*  |      | *5*  | *4*  |       |
+-----------------------+------+------+------+------+------+-------+
|                       |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| **Mother's Average    | 12.6 | 12.8 | -.2  | 12.7 | 12.8 | -.2   |
| Years of Schooling**  |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| *Percent missing*     | *17* | *16* |      | *17* | *15* |       |
+-----------------------+------+------+------+------+------+-------+
|                       |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| **Average Family      | \$18 | \$19 | -\   | \$18 | \$19 | -     |
| Income**              | ,767 | ,029 | $262 | ,870 | ,334 | \$464 |
+-----------------------+------+------+------+------+------+-------+
| *Percent missing*     | *0*  | *0*  |      | *0*  | *0*  |       |
+-----------------------+------+------+------+------+------+-------+
|                       |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| **Percent of Students | 49   | 54   | -5   | 51   | 48   | 4     |
| whose Mothers are     |      |      |      |      |      |       |
| Employed Full Time**  |      |      |      |      |      |       |
+-----------------------+------+------+------+------+------+-------+
| *Percent missing*     | *19* | *19* |      | *19* | *17* |       |
+-----------------------+------+------+------+------+------+-------+
| **Sample size**       | 299  | 193  |      | 492  | 321  |       |
+-----------------------+------+------+------+------+------+-------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ Test-score results are in terms of National Percentile Ranks, with
50 as the median score.

^2^ "Other race" includes respondents who were identified as white,
Asian, American Indian, Alaska Native, Pacific Islander, or multiracial.
Respondents classified as "multiracial" if more than one race category
was selected.

NOTE: Detail may not sum to totals because of rounding.

SOURCE: Eligible Applicant Database. Test-scores obtained from the
accountability testing database for District of Columbia public and
charter schools, DCPS Office of Communications and Public Information.

-   There are a few significant differences between the impact sample as
    a whole and the non-impact sample, at least for those entering
    grades 6 through 12. While the extent of missing baseline
    achievement data for the non-impact sample precludes a reliable
    comparison of test scores, members of the impact sample are more
    likely than members of the non-impact sample to report having a
    learning or physical disability (20 percent vs. 12 percent) but are
    less likely to be of Hispanic ethnicity (8 percent vs. 16 percent).

These results give us confidence that the random assignment process
worked as designed to generate two comparable subgroups for the impact
analysis. However, the impact analysis sample does not appear to be
comparable to the sample of applicants who will not be included in the
rigorous part of the evaluation on at least a few dimensions.

**4.3 Eligible Applicants by Type of Previous School**

The differences between evaluation subsamples underscore that the
program applicants are a diverse group, and these differences may play a
role in the extent to which the program is effective and for which
students. In particular, applicants are likely to vary somewhat in their
educational and background characteristics depending on the type of
school that they attended previously---SINI, non-SINI public schools,
charter schools, and private schools.[^60]

As described in Chapter 3, the eligible applicants were drawn primarily
from public schools that were not designated in need of improvement
under NCLB at the time of application.[^61] A total of 15 schools,
representing 9 percent of all DCPS, had been designated as SINI as of
the program application period. Seventy-nine students, or 4 percent of
all eligible applicants, were drawn from the schools designated as SINI
during the 2003-04 school year. A total of 993 students, or 54 percent
of all eligible applicants, were drawn from non-SINI public schools.
Public charter schools generated 258 applicants, or 14 percent of the
total, with private schools accounting for the remaining 518 eligible
applicants (28 percent). Detailed information on all applicants was
collected as part of the application process and analyzed for
differences and similarities by type of school previously attended. As a
way of summarizing, in these analyses an average for a school type
subgroup was designated as significant if it differs statistically from
the average for all applicants with that particular subgroup
excluded.[^62]

*Educational Characteristics of Eligible Applicants*

We examined the grade-level distributions, average test scores,
disability status, and English proficiency for all applicants by
previous type of school. Pre-application test score data were only
available for public and charter school applicants who completed the
DCPS accountability tests in spring 2004. Comparable test data were not
available for private school applicants. But other significant findings
include the following:

-   More than half of all applicants (55 percent) were at the elementary
    level (Table 4-6). SINI public schools were less likely to have
    elementary school program applicants and more likely to have high
    school applicants than other types of schools, although this largely
    reflects the proportion of SINI-designated schools that were high
    schools (7 of 15). [^63] Non-SINI public schools, which had the
    majority of program applicants, were substantially more likely than
    other schools to send elementary-level applicants and less likely to
    generate high school applicants.

-   Overall, applicants tended to score somewhat higher in mathematics
    than in reading and higher in the younger grades than in the older
    grades (Table 4-7). At the high school level, applicants from SINI
    public schools scored substantially lower in reading―and applicants
    from non-SINI public schools significantly higher―than applicants
    from non-SINI public schools. The test scores for the remaining
    grade-band and school-typesubgroups in reading, and all of the
    subgroup averages in mathematics, are statistically similar to the
    applicant norms.[^64]

**Table 4-6. Percentage of Applicants in Various Grade Bands, By Type of
School\
Previously Attended: Spring 2004**

+---------------------+--------+--------+--------+--------+--------+
| **Forecasted Grade  | **T    | **SINI | **Non- | **Cha  | **Pri  |
| Level Fall 2004**   | otal** | Pu     | SINI** | rter** | vate** |
|                     |        | blic** |        |        |        |
|                     |        |        | **Pu   |        |        |
|                     |        |        | blic** |        |        |
+---------------------+--------+--------+--------+--------+--------+
|                     |        |        |        |        |        |
+---------------------+--------+--------+--------+--------+--------+
| Kindergarten-5      | 55     | 32\*\* | 66\*\* | 45\*   | 47\*\* |
+---------------------+--------+--------+--------+--------+--------+
| 6-8                 | 27     | 26     | 25     | 33     | 31     |
+---------------------+--------+--------+--------+--------+--------+
| 9-12                | 16     | 44\*\* | 10\*\* | 20     | 22     |
+---------------------+--------+--------+--------+--------+--------+
| Total               | 100    | 100    | 100    | 100    | 100    |
+---------------------+--------+--------+--------+--------+--------+
|                     |        |        |        |        |        |
+---------------------+--------+--------+--------+--------+--------+
| **Sample size**     | 1,848  | 79     | 993    | 258    | 518    |
+---------------------+--------+--------+--------+--------+--------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

SOURCE: Eligible Applicant Database.

-   For the total population of eligible applicants, 14 percent have a
    learning or physical disability, according to parental reports
    (Table 4-7). Applicants from non-SINI public schools are slightly
    more likely to report a learning or physical disability, whereas
    private school applicants are somewhat less likely to report such a
    disability.[^65]

**Table 4-7. Educational Characteristics of Eligible Applicants by
School Type: Spring 2004**

+-----------------------+-------+---------+---------+-------+--------+
| **Characteristics**   | **To  | *       | **N     | *     | **Pri  |
|                       | tal** | *Public | on-SINI | *Char | vate** |
|                       |       | SINI**  | P       | ter** |        |
|                       |       |         | ublic** |       |        |
+-----------------------+-------+---------+---------+-------+--------+
| **Baseline Test       | ** ** |         | ** **   | ** ** | ** **  |
| Scores**^1^           |       |         |         |       |        |
+-----------------------+-------+---------+---------+-------+--------+
| > Average Reading     |       |         |         |       |        |
| > Percentile          |       |         |         |       |        |
+-----------------------+-------+---------+---------+-------+--------+
| > Grades K-5          | 47.0  | 49.9    | 47.5    | 43.8  | NA     |
+-----------------------+-------+---------+---------+-------+--------+
| > Grades 6-8          | 39.9  | 35.1    | 39.9    | 41.3  | NA     |
+-----------------------+-------+---------+---------+-------+--------+
| > Grades 9-12         | 31.7  | 1       | 3       | 27.9  | NA     |
|                       |       | 9.5\*\* | 8.0\*\* |       |        |
+-----------------------+-------+---------+---------+-------+--------+
| > *Percent missing*   | *57*  | *22*    | *42*    | *38*  | *100*  |
+-----------------------+-------+---------+---------+-------+--------+
| >                     |       |         |         |       |        |
+-----------------------+-------+---------+---------+-------+--------+
| > Average Mathematics |       |         |         |       |        |
| > Percentile          |       |         |         |       |        |
+-----------------------+-------+---------+---------+-------+--------+
| > Grades K-5          | 51.1  | 67.1    | 51.0    | 48.5  | NA     |
+-----------------------+-------+---------+---------+-------+--------+
| > Grades 6-8          | 44.8  | 42.0    | 44.5    | 46.7  | NA^2^  |
+-----------------------+-------+---------+---------+-------+--------+
| > Grades 9-12         | 43.1  | 36.3    | 46.0    | 42.4  | NA     |
+-----------------------+-------+---------+---------+-------+--------+
| > *Percent missing*   | *56*  | *23*    | *41*    | *38*  | *100*  |
+-----------------------+-------+---------+---------+-------+--------+
| >                     |       |         |         |       |        |
+-----------------------+-------+---------+---------+-------+--------+
| > **Percent with a    |       | 16      | 16\*    | 16    | 10\*\* |
| > Learning or         |       |         |         |       |        |
| > Physical            | 14    |         |         |       |        |
| > Disability**        |       |         |         |       |        |
+-----------------------+-------+---------+---------+-------+--------+
| > *Percent missing    | *7*   | *11*    | *9*     | *5*   | *6*    |
| > (average)*          |       |         |         |       |        |
+-----------------------+-------+---------+---------+-------+--------+
|                       |       |         |         |       |        |
+-----------------------+-------+---------+---------+-------+--------+
| > **Sample size**     | 1,848 | 79      | 993     | 258   | 518    |
+-----------------------+-------+---------+---------+-------+--------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ Test-score results are in terms of National Percentile Ranks, with
50 as the median score.

SOURCE: Eligible Applicant Database. Test scores obtained from the
accountability testing database for District of Columbia public and
charter schools, DCPS Office of Communications and Public Information.

*Background Characteristics of Eligible Applicants*

We also collected information about the demographics and socioeconomic
status of applicants:

-   The overwhelming majority of eligible applicants to the program, 95
    percent, are African American (Table 4-8). Private school applicants
    were somewhat less likely to identify their race as African American
    (92 percent), whereas applicants from charter schools were somewhat
    more likely to do so (98 percent).[^66]

```{=html}
<!-- -->
```
-   The average family income of applicants in 2003 was \$18,742. The
    family incomes of SINI public and non-SINI public applicants
    averaged slightly less than the other groups, whereas the incomes of
    private school applicants were somewhat higher than the norm.

-   Mothers' education is also considered an important indicator of
    socioeconomic status. The mothers of applicants averaged almost 13
    years of postkindergarten education.[^67] Mothers of applicants from
    non-SINI public schools averaged slightly less formal education than
    those in the other subgroups, and the mothers of private school
    applicants averaged somewhat more formal education (13.2 years).

-   Nearly half of the mothers of applicants (47 percent) were employed
    full time at the time of application. Applicants from non-SINI
    public schools were somewhat less likely to have mothers who worked
    full time (45 percent), and charter school applicants were
    significantly more likely (54 percent). About one-fifth of the
    program applications indicated that the student's mother is married,
    a percentage that did not vary significantly among the various
    applicant groups.

Additional information about such baseline survey items as parental
views of and satisfaction with their child's school are presented in
Appendix C.

**4.4 Scholarship Users and Nonusers**

Some people who are offered the benefit of a new government program
inevitably decide not to avail themselves of it. This has been the case
for both government-financed and private-funded school choice programs
in the United States. As discussed in Chapter 3, the initial scholarship
usage rate for the DC Opportunity Scholarship Program was 75 percent,
though the usage rate for members of the impact subsample was somewhat
lower, at 62 percent.

Those earlier studies of school choice programs found that students who
actually use scholarships tend to differ modestly from students who
receive scholarships but do not use them.[^68] Most, but not all,
differences suggested that scholarship users are somewhat more
advantaged academically and in terms of family background than students
who decline to use their scholarships.

**Table 4-8. Demographic Characteristics of Eligible Applicants by
School Type: Spring 2004**

+-------------------------+-------+--------+--------+-------+--------+
| **Characteristics**     | **To  | **     | **No   | *     | **Pri  |
|                         | tal** | Public | n-SINI | *Char | vate** |
|                         |       | SINI** | Pu     | ter** |        |
|                         |       |        | blic** |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| **Percent, by Race**    |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| > African American      | 95    | 96     | 96     | 98\*  | 92\*\* |
+-------------------------+-------+--------+--------+-------+--------+
| > Other race^1^         | 5     | 4      | 4      | 2\*   | 8\*\*  |
+-------------------------+-------+--------+--------+-------+--------+
| > *Percent missing*     | *9*   | *1*    | *8*    | *5*   | *13*   |
+-------------------------+-------+--------+--------+-------+--------+
|                         |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| **Percent of Students   |       |        |        |       |        |
| whose Mothers have the  |       |        |        |       |        |
| Following Level of      |       |        |        |       |        |
| Education**             |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| > No high school        | 13    | 20     | 15\*\* | 9\*   | 10\*   |
| > diploma               |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| > High school           | 29    | 27     | 32     | 30    | 24     |
| > diploma/GED           |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| > Some college, no      | 32    | 32     | 31     | 37    | 32     |
| > degree                |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| > Associates degree or  | 25    | 22     | 21\*   | 23    | 33\*   |
| > higher                |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| > *Percent missing*     | *13*  | *16*   | *13*   | *18*  | *12*   |
+-------------------------+-------+--------+--------+-------+--------+
|                         |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| **Mother's Average      | 12.77 | 12.62  | 12.    | 12.85 | 13.    |
| Years of Schooling**    |       |        | 56\*\* |       | 15\*\* |
+-------------------------+-------+--------+--------+-------+--------+
| > *Percent missing*     | *13*  | *16*   | *13*   | *17*  | *12*   |
+-------------------------+-------+--------+--------+-------+--------+
|                         |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| **Average Family        | \$1   | \$16   | \$18   | \$1   | \$20,2 |
| Income**                | 8,742 | ,033\* | ,021\* | 9,328 | 46\*\* |
+-------------------------+-------+--------+--------+-------+--------+
| > *Percent missing*     | *0*   | *0*    | *0*    | *0*   | *0*    |
+-------------------------+-------+--------+--------+-------+--------+
|                         |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| **Percent of Students   | 47    | 43     | 45\*   | 54\*  | 49     |
| whose Mothers are       |       |        |        |       |        |
| Employed Full Time**    |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| > *Percent missing*     | *16*  | *20*   | *16*   | *18*  | *14*   |
+-------------------------+-------+--------+--------+-------+--------+
|                         |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| **Percent of Students   | 19    | 21     | 19     | 16    | 22     |
| whose Mothers are       |       |        |        |       |        |
| Married**               |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| > *Percent missing*     | *12*  | *14*   | *11*   | *13*  | *13*   |
+-------------------------+-------+--------+--------+-------+--------+
|                         |       |        |        |       |        |
+-------------------------+-------+--------+--------+-------+--------+
| > **Sample size**       | 1,848 | 79     | 993    | 258   | 518    |
+-------------------------+-------+--------+--------+-------+--------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ "Other race" includes respondents who were identified as white,
Asian, American Indian, Alaska Native, Pacific Islander, or multiracial.
Respondents classified as "multiracial" if more than one race category
was selected.

NOTE: Detail may not sum to totals because of rounding.

SOURCE: Eligible Applicant Database.

Differences between scholarship users and nonusers in the impact sample
are a particular concern, since it could affect the ability of the
evaluation to generate reliable assessments of program impact.
Fortunately, the scholarship user and nonuser subgroups within the
impact are similar in almost all key respects (Table 4-9, Impact Sample
columns). Although users are less likely than nonusers to report having
a learning or physical disability (12 percent versus 27 percent), users
are statistically comparable to nonusers regarding a number of important
academic and family background factors including baseline test scores,
race and ethnicity, income, and mother's years of schooling and
employment status.

A comparison of the user and nonuser subgroups in the larger Program
Sample reveals a few more statistically significant differences than in
the Impact Sample. Among all the scholarship recipients in the DC
Opportunity Scholarship Program, the 1,027 who actually used the
scholarship to enroll in a preferred private school differed from the
339 nonusers in several important respects (Table 4-9, Program Sample
columns). The notable differences between scholarship users and nonusers
in the Program Sample included the following (see Appendix D for the
full set of comparisons):

-   Users entering grades K to 5 scored somewhat higher in reading (8.8
    NPR) and mathematics (9.7 NRP) than nonusers. However, this result
    should be interpreted with extreme caution, as baseline test scores
    were available for less than half of the students.

-   Nine percent of users reported having a learning or physical
    disability compared to 29 percent of nonusers.

-   Mothers of scholarship users reported .2 more years of education
    than mothers of nonusers.

There were no statistically significant differences between scholarship
users and nonusers in the Program Sample regarding the following:

-   Test-score performance for students in grades 6 to 8 or 9 to 12,

-   Student's race or ethnicity,

-   Family income, or

-   Mother's employment.

It is impossible to know at this point if the sizeable and consistent
difference between scholarship users and nonusers regarding learning or
physical disabilities is the result of parental decisions regarding what
is best for their child or limitations in the ability of participating
private schools to accommodate the special educational challenges that
some scholarship students face. At the same time, there is no evidence
in the data that scholarship recipients of differing races, ethnicities,
or family incomes are differentially willing or able to use an
Opportunity Scholarship to switch to a private school of their parent's
choosing.

**Table 4-9. Educational and Demographic Characteristics of the Users
and Nonusers in the Impact Sample and Program Sample, Spring 2004**

+-----------------------+------+------+-------+------+------+-------+
|                       | **Im |      |       | *    |      |       |
|                       | pact |      |       | *Pro |      |       |
|                       | Samp |      |       | gram |      |       |
|                       | le** |      |       | Samp |      |       |
|                       |      |      |       | le** |      |       |
+-----------------------+------+------+-------+------+------+-------+
| **Characteristics**   | *    | **No | **Di  | *    | **No | **Di  |
|                       | *Use | nuse | ffere | *Use | nuse | ffere |
|                       | rs** | rs** | nce** | rs** | rs** | nce** |
+-----------------------+------+------+-------+------+------+-------+
| **Baseline Test       |      |      |       |      | *    | ** ** |
| Scores**^1^           |      |      |       |      | * ** |       |
+-----------------------+------+------+-------+------+------+-------+
| Average Reading       |      |      |       |      |      |       |
| Percentile            |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| Grades K-5            | NA   | NA   | NA    | 49.1 | 40.3 | 8.    |
|                       |      |      |       |      |      | 8\*\* |
+-----------------------+------+------+-------+------+------+-------+
| Grades 6-8            | 39.9 | 38.9 | 1.1   | 39.8 | 38.1 | 1.8   |
+-----------------------+------+------+-------+------+------+-------+
| Grades 9-12           | 39.6 | 27.7 | 11.9  | 30.0 | 23.6 | 6.4   |
+-----------------------+------+------+-------+------+------+-------+
| *Percent missing      | *16* | *20* |       | *56* | *41* |       |
| (average)*            |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
|                       |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| Average Mathematics   |      |      |       |      |      |       |
| Percentile            |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| Grades K-5            | NA   | NA   | NA    | 53.4 | 43.8 | 9.    |
|                       |      |      |       |      |      | 7\*\* |
+-----------------------+------+------+-------+------+------+-------+
| Grades 6-8            | 44.8 | 43.0 | 1.8   | 44.9 | 42.3 | 2.7   |
+-----------------------+------+------+-------+------+------+-------+
| Grades 9-12           | 49.4 | 46.1 | 3.4   | 40.3 | 43.8 | -3.5  |
+-----------------------+------+------+-------+------+------+-------+
| *Percent missing      | *14* | *19* |       | *56* | *40* |       |
| (average)*            |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
|                       |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| **Percent of Students | 12   | 27   | -1    | 9    | 29   | -2    |
| with a Learning or    |      |      | 5\*\* |      |      | 0\*\* |
| Physical Disability** |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| > *Percent missing    | *6*  | *5*  |       | *6*  | *8*  |       |
| > (average)*          |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
|                       |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| **Percent, by Race**  |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| African American      | 92   | 96   | -4    | 94   | 96   | -2    |
+-----------------------+------+------+-------+------+------+-------+
| Other race^2^         | 8    | 4    | 4     | 6    | 4    | 2     |
+-----------------------+------+------+-------+------+------+-------+
| *Percent missing*     | *7*  | *5*  |       | *6*  | *6*  |       |
+-----------------------+------+------+-------+------+------+-------+
|                       |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| **Percent Hispanic    | 10   | 7    | 2     | 7    | 7    | 1     |
| (any race)**          |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| *Percent missing*     | *3*  | *5*  |       | *4*  | *7*  |       |
+-----------------------+------+------+-------+------+------+-------+
|                       |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| **Average Family      | \$18 | \$19 | -\$   | \$18 | \$17 | \$812 |
| Income**              | ,342 | ,457 | 1,115 | ,652 | ,841 |       |
+-----------------------+------+------+-------+------+------+-------+
| *Percent missing*     | *0*  | *0*  |       | *0*  | *0*  |       |
+-----------------------+------+------+-------+------+------+-------+
|                       |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| **Mother's Average    | 12.5 | 12.7 | -0.3  | 12.7 | 12.5 | .2\*  |
| Years of Schooling**  |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| *Percent missing*     | *14* | *23* |       | *11* | *19* |       |
+-----------------------+------+------+-------+------+------+-------+
|                       |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| **Percent of Students | 49   | 50   | -1    | 46   | 44   | 2     |
| whose Mothers are     |      |      |       |      |      |       |
| Employed Full Time**  |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| *Percent missing*     | *16* | *25* |       | *13* | *23* |       |
+-----------------------+------+------+-------+------+------+-------+
|                       |      |      |       |      |      |       |
+-----------------------+------+------+-------+------+------+-------+
| **Sample size**       | 185  | 114  |       | 1    | 339  |       |
|                       |      |      |       | ,027 |      |       |
+-----------------------+------+------+-------+------+------+-------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ Test-score results are in terms of National Percentile Ranks, with
50 as the median score.

^2^ "Other race" includes respondents who were identified as white,
Asian, American Indian, Alaska Native, Pacific Islander, or multiracial.
Respondents classified as "multiracial" if more than one race category
was selected.

NOTE: Detail may not sum to totals because of rounding.

SOURCE: Eligible Applicant Database. Test-scores obtained from the
accountability testing database for District of Columbia public and
charter schools, DCPS Office of Communications and Public Information.

**4.5 Applicant Response Rates Among Public and Private Schools**

In addition to examining the impact on students, the law also calls for
the evaluation to examine the impact of the program on schools in the
District. Such "systemic effects" could take place if significant
percentages of students in the public school system or in specific
schools apply for, receive, and use scholarships to transfer to private
schools. With regard to the public schools, the DC Opportunity
Scholarship Program could have either positive or negative effects. One
theoretical argument suggests that scholarship programs will divert
funding and the most motivated students from public schools to private
schools, leaving the public school system with fewer resources with
which to educate the remaining student population.[^69] Another theory
is that schools behave in a manner similar to firms and will respond to
competition by becoming more efficient.[^70] In the case of schools, it
is the risk of losing students and subsequently funding that may provide
an impetus for public schools to provide additional services and produce
better student outcomes. Private schools in jurisdictions with greater
school choice may face similar incentives to improve or expand in order
to retain as many of their current students and attract more.

In practice, two features of choice programs are required if effects on
public schools are to be observed. First, the threat of losing students
to the choice program must be significant.[^71] It is important to note
that the threat need not be that students actually take the scholarship
(at least initially) but that the program is available to a large share
of the student population. The second feature is that the loss of
students to the choice program also entails a loss of funding. However,
according to a Memorandum of Understanding between the Office of the DC
Mayor and DCPS, public schools that lose students as a result of the
Opportunity Scholarship program will be reimbursed for any lost
resources, thereby eliminating any fiscal impact of the program on
specific public schools, at least in the short run. Still, an analysis
of the response of public schools to the scholarship program is both
mandated and of policy interest.

*Public School Application and Use Rates*

To provide a foundation for a later analysis of the effects of the DC
Opportunity Scholarship Program on DC schools, we first need to describe
the extent to which public schools have so far been affected by program
applications and scholarship users---an indicator of the seriousness of
the pressure that the public schools may face. There are 197 public
schools in Washington, DC, including all DCPS schools and all public
charter schools.[^72] We ranked those schools based on the proportion of
students in the school who applied for the program or used a scholarship
to transfer out of the school, from 0 percent to at least 5 percent
(Table 4-10):

**Table 4-10. Public School-Level Scholarship Application and Use
Rates:\
Spring 2004 and Fall 2004**

  ------------------ -------------- ------------ --------------- -------------
                     **Applied for               **Used a        
                     the Program**               Scholarship**   

  **Percent of       **Number of    **Percent of **Number of     **Percent of
  Student Body**     Schools**      Schools**    Schools**       Schools**

                                                                 

  0                  25             13           51              26

  0.1 -- 1.0         47             24           63              32

  1.1 -- 2.0         56             28           47              24

  2.1 -- 3.0         38             19           21              11

  3.1 -- 4.0         12             6            12              6

  4.1 --             19             11           3               2

                                                                 

  Total              197            100          197             100
  ------------------ -------------- ------------ --------------- -------------

NOTE: Detail may not sum to totals because of rounding.

SOURCES: Application and usage numbers by school generated from the
Applicant Database and WSF Placement Database. Enrollment figures for
DCPS are from the 2002-23 school year and were obtained from Membership
in the District of Columbia Public Schools by School and Grade, October
7, 2003, available on the DCPS web site,
www.k12.dc.us/dcps/data/enrollment/membership-Oct.703_official\_.pdf.
Enrollment figures for DC public charter schools chartered by the
District Board of Education are from the 2002-03 school year and were
obtained from the Common Core of Data, National Center for Education
Statistics. Enrollment figures for the DC public charter schools
chartered by the DC Public Charter School Board are from the 2003-04
school year and were obtained from the Board's web site at
www.dcpubliccharter.com.

-   Over one-quarter of the public schools in the District experienced
    no student losses due to the program.

-   Another 56 percent of DCPS schools had program-related transfers out
    that totaled less than 2 percent of their student populations.

-   Seventeen percent of District schools lost about 2 to 4 percent of
    their students.

-   Finally, 2 percent of the public schools in the District experienced
    more significant student transfers of over 4 percent under the
    program.

For now, it appears that very few public schools in the District have
experienced a significant loss of students to the DC Opportunity
Scholarship Program.

*Private School Reliance on DC Choice Program*

Much of the discussion surrounding K to 12 scholarship programs revolves
around the composition of students entering private schools. Schools
participating in choice programs make a second, perhaps equally
important, set of decisions about the number of students to admit. The
larger the share of entering students that come through these programs,
the greater is the investment of the private school in the program and
the potential exposure to the challenges of assimilating a sizable
population of new private school students who are using scholarships.
Moreover, private schools with a large proportion of their student body
comprised of Opportunity Scholarship students may become relatively
dependent on the program for the necessary resources to operate. Or they
may choose to continue to expand their facilities in order to serve
additional scholarship students.

Fifty-eight DC private schools participated in the DC Opportunity
Scholarship Program in the first year. Of those, reliable enrollment
data were available for 54 schools. Our analysis indicates that, for
most of the participating private schools, students using DC Choice
Opportunity Scholarships make up a significant share of their
enrollments (Table 4-11)[^73]:

-   In 28 percent of the schools over 20 percent of their students are
    using Opportunity Scholarships this year, including 9 percent of the
    schools with over half of their students using program scholarships.

-   In an additional 37 percent of the private schools, scholarship
    students make up between 5 and 20 percent of their student
    population this school year.

-   Slightly more than one-third will have student bodies comprised of
    less than 5 percent program scholarship students in 2004-05.

The extent of student losses and gains due to the program suggest that
private schools in the District may be more affected by the initial
implementation of the DC Opportunity Scholarship Program than are DC
public schools at this point in time.

**Table 4-11. Private School Reliance on Scholarship Program:\
Fall 2004**

  ---------------------------- --------------------- ---------------------
                               **Schools with        
                               Scholarship Users**   

  **Percent of Student Body**  **Number of Schools** **Percent of\
                                                     Schools**

                                                     

  0                            4                     7

  0.1 -- 5.0                   15                    28

  5.1 -- 10.0                  6                     11

  10.1 -- 15.0                 7                     13

  15.1 -- 20.0                 7                     13

  20.1 -- 33.3                 7                     13

  33.4 -- 50.0                 3                     6

  50.1 --                      5                     9

                                                     

  Total                        54                    100
  ---------------------------- --------------------- ---------------------

NOTE: Detail may not sum to totals because of rounding.

SOURCE: Scholarship users drawn from the WSF Placement Database. Private
school enrollments are for the 2003-04 school year and are drawn from
the WSF DC Opportunity Scholarship Directory of Participating Schools.

**Appendix A**

# Appendix A: Analysis of Study Power

To ensure that the experimental evaluation of program impact will
produce reliable findings, the sample size must be large enough to
enable the analysis to answer the study's central questions and to
measure program effects that are large enough to be both meaningful in
students' lives and relevant to policy debates about the efficacy of
educational interventions. The ability of a study to do so is a function
of the study's precision or "power."

Minimum detectable effects are a simple way to express the statistical
precision of an impact study design. Intuitively, a minimum detectable
effect is the smallest program impact that could be measured with
confidence given random sampling and statistical estimation error.[^74]
For example, from a benefit-cost perspective one might ask whether a
proposed sample could reliably detect the smallest impact needed for a
program to "break even" (that is, produce benefits equal to costs). One
would want a sample that was large enough to ensure that an estimated
impact around this "break-even" point was a reliable indicator of the
program's true impact and not just due to chance variation. A smaller
sample might enable the study to detect only impacts that are well above
this break-even point and thus very difficult or costly to attain. A
study that used a sample of this size would therefore miss an
opportunity to produce reliable estimates of policy-relevant impacts.
Hence, it would be "under-powered" statistically.

The goals of statistical power analysis and sample size estimation are
to determine how large a sample is needed to make accurate and reliable
statistical judgments; and how likely a statistical test will detect
effects of a given magnitude.

To determine the sample size required for the scholarship program, we
use the following information. Suppose the statistical model is given
by:

T= γ~1~ + γ~2~ P + γ~3~ X+ ε T equals gamma sub one plus gamma sub 2 P
plus gamma sub three X plus epsilon

where T= test score,

P= treatment status, and

X demographics, baseline score and time controls.

γ~2~ is the treatment effect Gamma sub two is the treatment effect

The null hypothesis, i.e. the initial assumption that the treatment has
no effect, is γ~2~=0. The power is the probability of rejecting this
null if the treatment actually has an effect.

Let:

α Alpha be the statistical significance level, set equal to 0.05 (i.e.
95 percent confidence).

(1-β) (1-beta) be the power of the test (acceptable range of 80-90
percent).

***n~T\ ~*** n sub T be the sample size for the treatment sample.

***n~C~*** n sub C be the sample size for the control sample.

***σ*** sigma be the standard deviation for an outcome of interest, in
this case, test scores.

*ρ* rho be the correlation between a given student's test scores at
baseline and outcome year 1.

*Estimated Sample Size Required For Desired Power Analyzing Test Scores
in Outcome Years 1 and 3*

Although the evaluation will analyze outcomes such as test scores in
every year post baseline, for simplicity, we present the power analysis
numbers for two representative years -- the first and third outcome
years. Central to analytic power is the sample size of study
participants *who actually provide outcome information in a given year*.
Thus, this power analysis factors in expected study attrition over time.
It also takes account of the correlation between baseline test scores
and outcome test scores. By including baseline test scores in the
statistical estimation of outcome test scores, analysts make the
estimation of the impact of the treatment on the outcome more precise,
thus increasing power. Since baseline test scores were obtained for most
but not all members of the impact sample, the power analysis is
conducted under conditions of the absence of baseline test scores and
the presence of baseline test scores. Finally, all else equal, power is
greatest when the treatment and control groups are the same size. It is
unlikely that this condition will be met in practice. Therefore, the
sample size estimations are made under the condition of balanced
treatment and control groups and the more likely scenario of twice as
many treatment as control group members.

Assume: H~o~: γ~2~ = 0 H naught colon gamma two equals zero

> H~1~: γ~2~ $$`<!-- -->`{=html}0 H one colon gamma two does not equal
> zero
>
> α = 0.05 Alpha equals zero point zero five
>
> ***σ*** = 20 Sigma equals twenty
>
> *ρ* = 0.57 (correlation between baseline and post test scores) Rho
> equals zero point five seven
>
> Annual study attrition of 15 percent from treatment and control group

We would like our statistical tests of program impacts to have at least
**80 percent power** for detecting a difference in the means of the
treatment and control groups of 0.15 to 0.20 standard deviations.[^75]
That is, power = (1-β) = 0.80, and ∆ = \|γ~2~\| = 0.15σ in the first
case; and ∆ = \|γ~2~\| = 0.20σ in the second case. The assumptions above
regarding test score standard deviations and correlations are drawn from
the actual data obtained from the previous experimental evaluation of
the privately-funded Washington Scholarship Fund program,
1998-2001.[^76] Though characterized as assumptions, they are likely to
be more accurate than mere educated guesses because they are based on
actual data from a similar analysis.

Without the aid of baseline test scores, over 1,600 students would need
to be subject to random assignment for the impact analysis to have 80
percent power in detecting a program effect of 0.15 standard deviations
in outcome year 1(Table A1). Accounting for study attrition, the
required sample size would need to be almost 2,300 students in order to
have sufficient power to identify such a moderately-size effect in
outcome year 3. Should the test score impact of the program be somewhat
larger, 0.20 standard deviations, the analysis would be adequately
powered for year 1 if only a little over 900 students were randomly
assigned and for year 3 if slightly less than 1300 students were
randomized.

**Table A1. Required Sample Size: No Baseline Scores, Balanced Treatment
and Control\
Groups**

+--------------------+------------+------------+-----------+-----------+
|                    | **No       |            | **No      |           |
|                    | Baseline   |            | Baseline  |           |
|                    | Test       |            | Test      |           |
|                    | Score**    |            | Scores**  |           |
|                    |            |            |           |           |
|                    | **0.15σ**  |            | **0.20σ** |           |
+--------------------+------------+------------+-----------+-----------+
| **Treatment Group  | **Year of  |            | **Year of |           |
| Equal to Control   | Ev         |            | Eva       |           |
| Group**            | aluation** |            | luation** |           |
+--------------------+------------+------------+-----------+-----------+
|                    | **One**    | **Three**  | **One**   | **Three** |
+--------------------+------------+------------+-----------+-----------+
| ***N~T~***         | 821        | 1,137      | 462       | 640       |
+--------------------+------------+------------+-----------+-----------+
| ***N~c~***         | 821        | 1,137      | 462       | 640       |
+--------------------+------------+------------+-----------+-----------+
| Total *N*          | 1,642      | 2,274      | 924       | 1,280     |
+--------------------+------------+------------+-----------+-----------+

SOURCE: Unnecessary, as numbers are from statistical formulas and not
actual data.

Under the more likely condition of twice as many students in the
treatment than control groups, the sample sizes required to detect a
0.15 or 0.20 standard deviation program effect after one and three years
are slightly larger (Table A2). Still, should the test score impact be
as large as 0.20 standard deviations, the analysis would be able to
detect it in year 1 if somewhat more than 1,000 students were
randomized, and in year 3 if somewhat more than 1,400 students were
randomly assigned.

**Table A2. Required Sample Size: No Baseline Scores, Treatment Group
Larger Than\
Control Group**

+--------------------+------------+------------+-----------+-----------+
|                    | **No       |            | **No      |           |
|                    | Baseline   |            | Baseline  |           |
|                    | Test       |            | Test      |           |
|                    | Score**    |            | Scores**  |           |
|                    |            |            |           |           |
|                    | **0.15σ**  |            | **0.20σ** |           |
+--------------------+------------+------------+-----------+-----------+
| **Treatment Group  | **Year of  |            | **Year of |           |
| Larger Than        | Ev         |            | Eva       |           |
| Control**          | aluation** |            | luation** |           |
+--------------------+------------+------------+-----------+-----------+
|                    | **One**    | **Three**  | **One**   | **Three** |
+--------------------+------------+------------+-----------+-----------+
| ***N~T~***         | 1,232      | 1,705      | 693       | 959       |
+--------------------+------------+------------+-----------+-----------+
| ***N~c~***         | 616        | 853        | 347       | 480       |
+--------------------+------------+------------+-----------+-----------+
| Total *N*          | 1,848      | 2,558      | 1,040     | 1,439     |
+--------------------+------------+------------+-----------+-----------+

SOURCE: Unnecessary, as numbers are from statistical formulas and not
actual data.

Since including baseline test scores in the estimation of experimental
test score impacts improves the precision and subsequent power of the
analysis, and over 75 percent of the members of the initial impact
sample were tested at baseline, the actual sample size needed for a
sufficiently powered analysis may be closer to the figures presented
below (Table A3). With the aid of baseline test scores, about 2,000
would need to be randomly assigned to detect a 0.15 standard deviation
effect in year 3, and only 1,100 would need to be randomized in order to
detect a 0.20 standard deviation effect in year 3.

**Table A3. Required Sample Size: Baseline Scores, Balanced Treatment
and Control\
Groups**

+--------------------+------------+------------+-----------+-----------+
|                    | **Baseline |            | *         |           |
|                    | Test       |            | *Baseline |           |
|                    | Scores**   |            | Test      |           |
|                    |            |            | Scores**  |           |
|                    | **0.15σ**  |            |           |           |
|                    |            |            | **0.20σ** |           |
+--------------------+------------+------------+-----------+-----------+
| **Treatment Group  | **Year of  |            | **Year of |           |
| Equal to Control   | Ev         |            | Eva       |           |
| Group**            | aluation** |            | luation** |           |
+--------------------+------------+------------+-----------+-----------+
|                    | **One**    | **Three**  | **One**   | **Three** |
+--------------------+------------+------------+-----------+-----------+
| ***N~T~***         | 707        | 979        | 398       | 550       |
+--------------------+------------+------------+-----------+-----------+
| ***N~c~***         | 707        | 979        | 398       | 550       |
+--------------------+------------+------------+-----------+-----------+
| Total *N*          | 1,414      | 1,960      | 796       | 1,100     |
+--------------------+------------+------------+-----------+-----------+

SOURCE: Unnecessary, as numbers are from statistical formulas and not
actual data.

Under the more likely condition of twice as many students in the control
than treatment, the sample sizes required to detect a 0.15 or 0.20
standard deviation program effect after one and three years are slightly
larger (Table A4). Still, should the test score impact be as large as
0.20 standard deviations, the analysis would be able to detect it in
year 1 if just under 900 students were randomized and in year 3 if
somewhat more than 1,200 students were randomly assigned.

**Table A4. Required Sample Size: Baseline Scores, Treatment Group
Larger Than\
Control Group**

+--------------------+------------+------------+-----------+-----------+
|                    | **Baseline |            | *         |           |
|                    | Test       |            | *Baseline |           |
|                    | Score**    |            | Test      |           |
|                    |            |            | Scores**  |           |
|                    | **0.15σ**  |            |           |           |
|                    |            |            | **0.20σ** |           |
+--------------------+------------+------------+-----------+-----------+
| **Treatment Group  | **Year of  |            | **Year of |           |
| Larger Than        | Ev         |            | Eva       |           |
| Control**          | aluation** |            | luation** |           |
+--------------------+------------+------------+-----------+-----------+
|                    | **One**    | **Three**  | **One**   | **Three** |
+--------------------+------------+------------+-----------+-----------+
| ***N~T~***         | 1,060      | 1,467      | 596       | 826       |
+--------------------+------------+------------+-----------+-----------+
| ***N~c~***         | 531        | 734        | 300       | 414       |
+--------------------+------------+------------+-----------+-----------+
| Total *N*          | 1,591      | 2,201      | 896       | 1,240     |
+--------------------+------------+------------+-----------+-----------+

SOURCE: Unnecessary, as numbers are from statistical formulas and not
actual data.

**Appendix B**

**Appendix B**

**Characteristics of Eligible Applicants versus Nonapplicants:\
DC Public School Students**

The statute requires that the annual evaluation reports on the program
include a comparison between program participants and nonapplicants from
District public schools. The DCPS Office of Communications and Public
Information provided the technical evaluation team with data that permit
comparisons such as between public school applicants and nonapplicants
as well as participants and nonapplicants, using information regarding a
modest number of educational and background characteristics that DCPS
collects on all of its students. Because the only individual signifiers
that were available in the DCPS database were student names, birthdates,
and grades, we were only able to confirm the identity in the database of
1077 of 1343 (i.e., 80 percent) public school eligible applicants to the
program.[^77] Thus, the comparisons presented below should be
interpreted with caution.

Chapter 4 of the report includes four tables that compare various
samples of program applicants and participants to DCPS nonapplicants.
Appendix Table B1 provides a fifth comparison, of all eligible
applicants in the database to all nonapplicant DCPS students, regardless
of free or reduced-price lunch status and broken out by grade band.
Because this applicant sample includes students who did not complete the
accountability testing and students who were not participating in the
free or reduced-price lunch program, it is a more expansive applicant
sample than some of the more restrictive samples used in the comparison
tables in Chapter 4. Although this comparison is informative,
conclusions regarding the comparisons should be made with caution, as
the sample of eligible applicants is limited to low-income students
whereas the sample of DCPS nonapplicants includes at least some middle
to high income students.

**Table B1. Characteristics of District Public School Students, by Grade
Band, Program Applicants Versus Nonapplicants: Spring 2004**

<table style="width:100%;">
<colgroup>
<col style="width: 27%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
</colgroup>
<tbody>
<tr class="odd">
<td> </td>
<td colspan="3"><strong>Grades K-5</strong></td>
<td colspan="3"><strong>Grades 6-8</strong></td>
<td colspan="3"><strong>Grades 9-12</strong></td>
</tr>
<tr class="even">
<td><strong>Characteristic</strong></td>
<td><strong>Applicants</strong></td>
<td><p><strong>DCPS</strong></p>
<p><strong>Sample</strong></p></td>
<td><strong>Difference</strong></td>
<td><strong>Applicants</strong></td>
<td><p><strong>DCPS</strong></p>
<p><strong>Sample</strong></p></td>
<td><strong>Difference</strong></td>
<td><strong>Applicants</strong></td>
<td><p><strong>DCPS</strong></p>
<p><strong>Sample</strong></p></td>
<td><strong>Difference</strong></td>
</tr>
<tr class="odd">
<td><strong>Baseline Test Scores</strong><sup>1</sup></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p>Average Reading Percentile</p>
</blockquote></td>
<td>46.86</td>
<td>46.37</td>
<td>0.49</td>
<td>39.98</td>
<td>40.65</td>
<td>-0.68</td>
<td>31.66</td>
<td>32.91</td>
<td>-1.24</td>
</tr>
<tr class="odd">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><em>43</em></td>
<td><em>44</em></td>
<td></td>
<td><em>3</em></td>
<td><em>6</em></td>
<td></td>
<td><em>4</em></td>
<td><em>14</em></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Average Mathematics Percentile</p>
</blockquote></td>
<td>50.95</td>
<td>51.18</td>
<td>-0.23</td>
<td>44.91</td>
<td>46.11</td>
<td>-1.20</td>
<td>43.09</td>
<td>41.26</td>
<td>1.83</td>
</tr>
<tr class="even">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><em>42</em></td>
<td><em>43</em></td>
<td></td>
<td><em>2</em></td>
<td><em>5</em></td>
<td></td>
<td><em>4</em></td>
<td><em>13</em></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p><strong> </strong></p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="even">
<td><strong>Percent in Special Education</strong></td>
<td>16</td>
<td>12</td>
<td>4*</td>
<td>18</td>
<td>16</td>
<td>2</td>
<td>13</td>
<td>15</td>
<td>-1</td>
</tr>
<tr class="odd">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><em>41</em></td>
<td><em>42</em></td>
<td></td>
<td><em>1</em></td>
<td><em>3</em></td>
<td></td>
<td><em>3</em></td>
<td><em>10</em></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p><strong> </strong></p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="odd">
<td><strong>Percent, by Race</strong></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="even">
<td><blockquote>
<p>African American</p>
</blockquote></td>
<td>94</td>
<td>82</td>
<td>11**</td>
<td>89</td>
<td>87</td>
<td>2</td>
<td>94</td>
<td>86</td>
<td>8**</td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Other race<sup>2</sup></p>
</blockquote></td>
<td>6</td>
<td>18</td>
<td>-11**</td>
<td>11</td>
<td>13</td>
<td>-2</td>
<td>6</td>
<td>14</td>
<td>-8**</td>
</tr>
<tr class="even">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><em>2</em></td>
<td><em>2</em></td>
<td></td>
<td><em>0</em></td>
<td><em>1</em></td>
<td></td>
<td><em>0</em></td>
<td><em>1</em></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p><strong> </strong></p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="even">
<td><strong>Percent, by Gender</strong></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td><blockquote>
<p>Female</p>
</blockquote></td>
<td>52</td>
<td>50</td>
<td>2</td>
<td>49</td>
<td>51</td>
<td>-1</td>
<td>48</td>
<td>51</td>
<td>-3</td>
</tr>
<tr class="even">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>1</em></td>
<td></td>
</tr>
<tr class="odd">
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td><strong>Percent Participating in Free/Reduced-price Lunch
Program</strong></td>
<td>86</td>
<td>71</td>
<td>15**</td>
<td>84</td>
<td>72</td>
<td>12**</td>
<td>81</td>
<td>60</td>
<td>21**</td>
</tr>
<tr class="odd">
<td><blockquote>
<p><em>Percent missing</em></p>
</blockquote></td>
<td><em>4</em></td>
<td><em>4</em></td>
<td></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
<td><em>0</em></td>
<td><em>0</em></td>
<td></td>
</tr>
<tr class="even">
<td><blockquote>
<p> </p>
</blockquote></td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
<td> </td>
</tr>
<tr class="odd">
<td><strong>Sample size</strong></td>
<td>614</td>
<td>30,468</td>
<td> </td>
<td>308</td>
<td>17,279</td>
<td> </td>
<td>155</td>
<td>19,121</td>
<td> </td>
</tr>
</tbody>
</table>

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ Test-score results are in terms of National Percentile Ranks, with
50 as the median score.

^2^ "Other race" includes students who were identified as Hispanic,
white, Asian, American Indian, or Alaska Native. Because of small cell
sizes, Hispanic was combined with other race in this table.

SOURCE: Accountability testing database for DC public and charter
schools, DCPS Office of Communications and Public Information.

**Appendix C**

**Appendix C**

**Characteristics of Student Eligible Applicants by Type of Previous
School**

This appendix presents additional detail about the eligible applicants
to the program in the initial year of implementation. Table C1 describes
the forecasted entering grades of applicants, by school type. Table C2
presents information about student race and gender. Tables C3 and C4
describe the relationship of the adult applicant to the child applicant
as well as parental involvement in the child's education. Finally,
Tables C5-C8 describe the responses of applicants to various questions
about school facilities, problems at school, parental satisfaction with
school, and the features that parents are looking for in a school.

These comparisons are merely descriptive in nature. Since students with
different characteristics and motivations self-select themselves into
different types of schools prior to applying for the program, none of
the comparisons presented here should be interpreted as meaning that the
types of schools that the students attended actually were responsible
for the differences. For example, most applicants from private schools
seek a scholarship in order to remain in their school, while all
applicants from public schools are trying to switch to a private school.
Therefore, it is not surprising that the parents of private school
applicants were much more satisfied with their child's school than were
the parents of public school applicants.

**Table C1. Percent of Students Entering Various Grades, Eligible
Applicants: Spring 2004**

+-------------------+---------+---------+---------+---------+---------+
| **Grade           | **      | **P     | **Non   | **Ch    | **Pr    |
| Level**^1^        | Total** | ublic** | -SINI** | arter** | ivate** |
|                   |         |         |         |         |         |
|                   |         | *       | **P     |         |         |
|                   |         | *SINI** | ublic** |         |         |
+-------------------+---------+---------+---------+---------+---------+
| Kindergarten -1   | 20      | 10\*    | 25\*\*  | 13\*\*  | 16\*\*  |
+-------------------+---------+---------+---------+---------+---------+
| 2-3               | 18      | 11      | 19      | 16      | > 18    |
+-------------------+---------+---------+---------+---------+---------+
| 4-5               | 18      | 9\*     | 21\*\*  | 16      | 15\*    |
+-------------------+---------+---------+---------+---------+---------+
| 6-7               | 19      | 11      | 19      | 21      | 20      |
+-------------------+---------+---------+---------+---------+---------+
| 8-9               | 15      | 23      | 11\*\*  | 22\*\*  | 19\*\*  |
+-------------------+---------+---------+---------+---------+---------+
| 10-12             | 10      | 35\*\*  | 5\*\*   | 12      | 13\*\*  |
+-------------------+---------+---------+---------+---------+---------+
| > *Percent        | *0*     | *0*     | *0*     | *0*     | *0*     |
| > missing *       |         |         |         |         |         |
+-------------------+---------+---------+---------+---------+---------+
|                   |         |         |         |         |         |
+-------------------+---------+---------+---------+---------+---------+
| **Sample size**   | 1,848   | 79      | 993     | 258     | 518     |
+-------------------+---------+---------+---------+---------+---------+

\* Significantly different from the average of the other subgroups at
the 95 percent confidence level.

\*\* Significantly different from the average of the other subgroups at
the 99 percent confidence level.

^1^ Grade levels were combined because cell sizes were too small to
report separately by grade.

SOURCE: Eligible Applicant Database.

**Table C2. Student's Race and Gender, Eligible Applicants: Spring
2004**

+----------------------------+-------+-------+-------+-------+-------+
| **Characteristic**         | **To  | **Pub | **    | *     | *     |
|                            | tal** | lic** | Non-S | *Char | *Priv |
|                            |       |       | INI** | ter** | ate** |
|                            |       | **S   |       |       |       |
|                            |       | INI** | **Pub |       |       |
|                            |       |       | lic** |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| **Percent, by Race**       |       |       |       |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| African American           | 95    | 96    | 96    | 98\*  | 9     |
|                            |       |       |       |       | 2\*\* |
+----------------------------+-------+-------+-------+-------+-------+
| Other race^1^              | 5     | 4     | 4     | 2\*   | 8\*\* |
+----------------------------+-------+-------+-------+-------+-------+
| > *Percent missing*        | *9*   | *1*   | *8*   | *5*   | *13*  |
+----------------------------+-------+-------+-------+-------+-------+
| **Percent, by Gender**     |       |       |       |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| Female                     | 51    | 51    | 50    | 49    | 54    |
+----------------------------+-------+-------+-------+-------+-------+
| > *Percent missing*        | *1*   | *1*   | *1*   | *0*   | *1*   |
+----------------------------+-------+-------+-------+-------+-------+
| ** **                      |       |       |       |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| **Sample size**            | 1,848 | 79    | 993   | 258   | 518   |
+----------------------------+-------+-------+-------+-------+-------+

\* Significantly different from the average of the other subgroups at
the 95 percent confidence level.

\*\* Significantly different from the average of the other subgroups at
the 99 percent confidence level.

[]{.mark}

^1^ "Other race" includes respondents who were identified as white,
Asian, American Indian, Alaska Native, Pacific Islander, or multiracial.
Respondents classified as "multiracial" if more than one race category
was selected.

NOTE: Detail may not round to totals because of rounding.

SOURCE: Eligible Applicant Database.

**Table C3. Adult Applicant's Relationship to Student, Eligible
Applicants: Spring 2004**

+----------------------------+-------+-------+-------+-------+-------+
| **Characteristic**         | **To  | **Pub | **    | *     | *     |
|                            | tal** | lic** | Non-S | *Char | *Priv |
|                            |       |       | INI** | ter** | ate** |
|                            |       | **S   |       |       |       |
|                            |       | INI** | **Pub |       |       |
|                            |       |       | lic** |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| **Percent of Applicants    |       |       |       |       |       |
| who are the Child's**      |       |       |       |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| Parent                     | 90    | 92    | 90    | 91    | 89    |
+----------------------------+-------+-------+-------+-------+-------+
| Other^1^                   | 10    | 8     | 10    | 9     | 11    |
+----------------------------+-------+-------+-------+-------+-------+
| > *Percent missing*        | *3*   | *5*   | *3*   | *2*   | *2*   |
+----------------------------+-------+-------+-------+-------+-------+
|                            |       |       |       |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| **Percent of Applicants    | 82    | 84    | 83    | 86    | 79\*  |
| who are the Child's        |       |       |       |       |       |
| Mother**                   |       |       |       |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| *Percent missing*          | *18*  | *16*  | *17*  | *14*  | *21*  |
+----------------------------+-------+-------+-------+-------+-------+
|                            |       |       |       |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| **Sample size**            | 1,848 | 79    | 993   | 258   | 518   |
+----------------------------+-------+-------+-------+-------+-------+

\* Significantly different from the average of the other subgroups at
the 95 percent confidence level.

\*\* Significantly different from the average of the other subgroups at
the 99 percent confidence level.

^1^ "Other" includes grandparent, step parent, other relative, and other
adult.

NOTE: Detail may not round to totals because of rounding.

SOURCE: Eligible Applicant Database.

**Table C4. Parental Involvement with Child's Education, Eligible
Applicants: Spring 2004**

+------------------------------+-------+-------+-------+-------+-------+
| **Characteristic**           | **To  | **P   | **    | *     | *     |
|                              | tal** | ublic | Non-S | *Char | *Priv |
|                              |       | S     | INI** | ter** | ate** |
|                              |       | INI** |       |       |       |
|                              |       |       | **Pub |       |       |
|                              |       |       | lic** |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| **Percent of Parents who     |       |       |       |       |       |
| Participated in the          |       |       |       |       |       |
| Following Activities with    |       |       |       |       |       |
| Their Child in the Past      |       |       |       |       |       |
| Month**                      |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| Discussed experiences at     | 99    | 97    | 99    | 100   | 99    |
| school                       |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
|                              |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| > Helped with mathematics or | 91    | 7     | 9     | 92    | 88\*  |
| > reading not                |       | 8\*\* | 3\*\* |       |       |
| >                            |       |       |       |       |       |
| > related to homework        |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
|                              |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| Worked on homework           | 95    | 8     | 9     | 94    | 94    |
|                              |       | 6\*\* | 6\*\* |       |       |
+------------------------------+-------+-------+-------+-------+-------+
|                              |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| Worked on a school project   | 87    | 7     | 8     | 87    | 9     |
|                              |       | 6\*\* | 5\*\* |       | 3\*\* |
+------------------------------+-------+-------+-------+-------+-------+
|                              |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| Attended school activities   | 89    | 6     | 88    | 87    | 9     |
| with child                   |       | 5\*\* |       |       | 5\*\* |
+------------------------------+-------+-------+-------+-------+-------+
|                              |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| > *Average percent missing*  | *7*   | *9*   | *8*   | *5*   | *6*   |
+------------------------------+-------+-------+-------+-------+-------+
| ** **                        |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| **Sample size**              | 1,848 | 79    | 993   | 258   | 518   |
+------------------------------+-------+-------+-------+-------+-------+

\* Significantly different from the average of the other subgroups at
the 95 percent confidence level.

\*\* Significantly different from the average of the other subgroups at
the 99 percent confidence level.

SOURCE: Eligible Applicant Database.

**Table C5. School Facilities and Homework, Eligible Applicants: Spring
2004**

+------------------------------+-------+-------+-------+-------+-------+
| **Characteristic**           | **To  | **Pub | **    | *     | *     |
|                              | tal** | lic** | Non-S | *Char | *Priv |
|                              |       |       | INI** | ter** | ate** |
|                              |       | **S   |       |       |       |
|                              |       | INI** | **Pub |       |       |
|                              |       |       | lic** |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| **Percent of Parents         |       |       |       |       |       |
| Reporting that Students Have |       |       |       |       |       |
| the Following Resources at   |       |       |       |       |       |
| Their School**               |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| Special programs for         | 41    | 43    | 5     | 2     | 3     |
| non-English                  |       |       | 0\*\* | 7\*\* | 3\*\* |
|                              |       |       |       |       |       |
| speakers                     |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| Nurse's office               | 77    | 9     | 9     | 73    | 4     |
|                              |       | 2\*\* | 2\*\* |       | 5\*\* |
+------------------------------+-------+-------+-------+-------+-------+
| Prepared Lunches             | 67    | 8     | 8     | 64    | 4     |
|                              |       | 4\*\* | 1\*\* |       | 0\*\* |
+------------------------------+-------+-------+-------+-------+-------+
| Cafeteria                    | 80    | 9     | 9     | 76\*  | 5     |
|                              |       | 7\*\* | 2\*\* |       | 9\*\* |
+------------------------------+-------+-------+-------+-------+-------+
| Special programs for         | 41    | 30    | 3     | 35    | 5     |
| advanced                     |       |       | 7\*\* |       | 3\*\* |
|                              |       |       |       |       |       |
| learners                     |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| Arts program                 | 70    | 5     | 6     | 66    | 7     |
|                              |       | 3\*\* | 6\*\* |       | 9\*\* |
+------------------------------+-------+-------+-------+-------+-------+
| Special education programs   | 60    | 57    | 6     | 58    | 4     |
|                              |       |       | 7\*\* |       | 7\*\* |
+------------------------------+-------+-------+-------+-------+-------+
| Computer lab                 | 79    | 79    | 7     | 80    | 8     |
|                              |       |       | 4\*\* |       | 9\*\* |
+------------------------------+-------+-------+-------+-------+-------+
| Gym                          | 63    | 8     | 63    | 4     | 67\*  |
|                              |       | 9\*\* |       | 6\*\* |       |
+------------------------------+-------+-------+-------+-------+-------+
| Child counselors             | 79    | 84    | 8     | 75    | 75\*  |
|                              |       |       | 2\*\* |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| Library                      | 82    | 85    | 8     | 6     | 85    |
|                              |       |       | 5\*\* | 2\*\* |       |
+------------------------------+-------+-------+-------+-------+-------+
| After-school program         | 80    | 70\*  | 80    | 75\*  | 8     |
|                              |       |       |       |       | 5\*\* |
+------------------------------+-------+-------+-------+-------+-------+
| Music program                | 71    | 61    | 6     | 6     | 8     |
|                              |       |       | 8\*\* | 0\*\* | 4\*\* |
+------------------------------+-------+-------+-------+-------+-------+
| Individual tutors            | 44    | 27\*  | 3     | 36\*  | 6     |
|                              |       |       | 7\*\* |       | 2\*\* |
+------------------------------+-------+-------+-------+-------+-------+
|                              |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| **Average Hours of Daily     | 1.04  | .6    | .8    | 1.08  | 1.4   |
| Homework**                   |       | 8\*\* | 5\*\* |       | 1\*\* |
+------------------------------+-------+-------+-------+-------+-------+
| > *Percent missing*          | *8*   | *13*  | *9*   | *8*   | *5*   |
+------------------------------+-------+-------+-------+-------+-------+
| ** **                        |       |       |       |       |       |
+------------------------------+-------+-------+-------+-------+-------+
| **Sample size**              | 1,848 | 79    | 993   | 258   | 518   |
+------------------------------+-------+-------+-------+-------+-------+

\* Significantly different from the average of the other subgroups at
the 95 percent confidence level.

\*\* Significantly different from the average of the other subgroups at
the 99 percent confidence level.

SOURCE: Eligible Applicant Database.

**Table C6. Problems at Child's School, Eligible Applicants: Spring
2004**

+----------------------------+-------+-------+-------+-------+-------+
| **Characteristic**         | **To  | **Pub | **    | *     | *     |
|                            | tal** | lic** | Non-S | *Char | *Priv |
|                            |       |       | INI** | ter** | ate** |
|                            |       | **S   |       |       |       |
|                            |       | INI** | **Pub |       |       |
|                            |       |       | lic** |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| **Percent of Parents who   |       |       |       |       |       |
| Believe the Following      |       |       |       |       |       |
| Problems at School are     |       |       |       |       |       |
| Serious**                  |       |       |       |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| Fighting                   | 44    | 8     | 5     | 45    | 1     |
|                            |       | 5\*\* | 8\*\* |       | 2\*\* |
+----------------------------+-------+-------+-------+-------+-------+
| Tardiness                  | 44    | 8     | 5     | 44    | 2     |
|                            |       | 6\*\* | 2\*\* |       | 3\*\* |
+----------------------------+-------+-------+-------+-------+-------+
| Destruction of property    | 32    | 7     | 4     | 2     | 1     |
|                            |       | 9\*\* | 2\*\* | 2\*\* | 0\*\* |
+----------------------------+-------+-------+-------+-------+-------+
| Truancy                    | 32    | 7     | 4     | 31    | 1     |
|                            |       | 6\*\* | 2\*\* |       | 0\*\* |
+----------------------------+-------+-------+-------+-------+-------+
| Cheating                   | 23    | 6     | 2     | 23    | 9\*\* |
|                            |       | 4\*\* | 8\*\* |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| Guns or other weapons      | 18    | 5     | 2     | 13    | 6\*\* |
|                            |       | 4\*\* | 2\*\* |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| Drug distribution          | 14    | 3     | 1     | 10    | 6\*\* |
|                            |       | 8\*\* | 8\*\* |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| Drug and alcohol use       | 14    | 3     | 1     | 11    | 8\*\* |
|                            |       | 5\*\* | 6\*\* |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| *Average percent missing*  | 9     | 11    | 11    | 5     | 7     |
+----------------------------+-------+-------+-------+-------+-------+
|                            |       |       |       |       |       |
+----------------------------+-------+-------+-------+-------+-------+
| **Sample size**            | 1,848 | 79    | 993   | 258   | 518   |
+----------------------------+-------+-------+-------+-------+-------+

\* Significantly different from the average of the other subgroups at
the 95 percent confidence level.

\*\* Significantly different from the average of the other subgroups at
the 99 percent confidence level.

SOURCE: Eligible Applicant Database.

**Table C7. Parental Satisfaction with School, Eligible Applicants:
Spring 2004**

+-------------------------------+------+------+------+------+------+
| **Characteristic**            | *    | **   | **No | **C  | **P  |
|                               | *Tot | Publ | n-SI | hart | riva |
|                               | al** | ic** | NI** | er** | te** |
|                               |      |      |      |      |      |
|                               |      | **S  | **   |      |      |
|                               |      | INI^ | Publ |      |      |
|                               |      | 1^** | ic** |      |      |
+-------------------------------+------+------+------+------+------+
| **Percent of Parents 'Very    |      |      |      |      |      |
| Satisfied' with the Following |      |      |      |      |      |
| Aspects of Their Child's      |      |      |      |      |      |
| School**                      |      |      |      |      |      |
+-------------------------------+------+------+------+------+------+
| Amount of information from    | 29   |      | 17   | 25   | 58   |
| teachers                      |      |      | \*\* |      | \*\* |
+-------------------------------+------+------+------+------+------+
| Freedom to observe religious  | 24   |      | 7    | 13   | 63   |
| traditions                    |      |      | \*\* | \*\* | \*\* |
+-------------------------------+------+------+------+------+------+
| Class size                    | 23   |      | 8    | 15   | 57   |
|                               |      |      | \*\* | \*\* | \*\* |
+-------------------------------+------+------+------+------+------+
| Safety                        | 27   |      | 12   | 22\* | 60   |
|                               |      |      | \*\* |      | \*\* |
+-------------------------------+------+------+------+------+------+
| Respect between teachers and  | 28   |      | 13   | 23\* | 63   |
| students                      |      |      | \*\* |      | \*\* |
+-------------------------------+------+------+------+------+------+
| Location                      | 33   |      | 25   | 23   | 58   |
|                               |      |      | \*\* | \*\* | \*\* |
+-------------------------------+------+------+------+------+------+
| Parental support for school   | 28   |      | 16   | 22   | 58   |
|                               |      |      | \*\* | \*\* | \*\* |
+-------------------------------+------+------+------+------+------+
| Academic quality              | 26   |      | 11   | 21\* | 61   |
|                               |      |      | \*\* |      | \*\* |
+-------------------------------+------+------+------+------+------+
| Discipline                    | 27   |      | 12   | 24   | 61   |
|                               |      |      | \*\* |      | \*\* |
+-------------------------------+------+------+------+------+------+
| Racial mix of students        | 21   |      | 12   | 17   | 42   |
|                               |      |      | \*\* |      | \*\* |
+-------------------------------+------+------+------+------+------+
| > *Average percent missing*   | *7*  |      | *9*  | *5*  | *5*  |
+-------------------------------+------+------+------+------+------+
|                               |      |      |      |      |      |
+-------------------------------+------+------+------+------+------+
| **Percent of Parents who Gave |      |      |      |      |      |
| the School a Grade of**       |      |      |      |      |      |
+-------------------------------+------+------+------+------+------+
| A or B                        | 53   | 9    | 40   | 46   | 88   |
|                               |      | \*\* | \*\* | \*\* | \*\* |
+-------------------------------+------+------+------+------+------+
| C                             | 29   | 26   | 37   | 37   | 10   |
|                               |      |      | \*\* | \*\* | \*\* |
+-------------------------------+------+------+------+------+------+
| D                             | 11   | 32   | 14   | 11   | 1    |
|                               |      | \*\* | \*\* |      | \*\* |
+-------------------------------+------+------+------+------+------+
| F                             | 7    | 32   | 8\*  | 7    | 1    |
|                               |      | \*\* |      |      | \*\* |
+-------------------------------+------+------+------+------+------+
| > *Percent missing*           | *5*  | *6*  | *7*  | *2*  | *4*  |
+-------------------------------+------+------+------+------+------+
|                               |      |      |      |      |      |
+-------------------------------+------+------+------+------+------+
| **Average Grade Parent Gave   | 2.50 | 1.15 | 2.20 | 2.   | 3.33 |
| School (4.0 Scale)**          |      | \*\* | \*\* | 36\* | \*\* |
+-------------------------------+------+------+------+------+------+
| *Percent missing*             | *5*  | *6*  | *7*  | *2*  | *4*  |
+-------------------------------+------+------+------+------+------+
|                               |      |      |      |      |      |
+-------------------------------+------+------+------+------+------+
| **Sample size**               | 1    | 79   | 993  | 258  | 518  |
|                               | ,848 |      |      |      |      |
+-------------------------------+------+------+------+------+------+

\* Significantly different from the average of the other subgroups at
the 95 percent confidence level.

\*\* Significantly different from the average of the other subgroups at
the 99 percent confidence level.

^1^ Parent satisfaction for students in public SINI schools could not be
reported because the cell sizes were too small for some response
categories

NOTE: Detail may not round to totals because of rounding.

SOURCE: Eligible Applicant Database.

**Table C8. Valued Features of Schools, Eligible Applicants: Spring
2004**

+---------------------------------+------+------+------+------+------+
| **Characteristic**              | *    | **   | **No | **C  | **P  |
|                                 | *Tot | Publ | n-SI | hart | riva |
|                                 | al** | ic** | NI** | er** | te** |
|                                 |      |      |      |      |      |
|                                 |      | **SI | **   |      |      |
|                                 |      | NI** | Publ |      |      |
|                                 |      |      | ic** |      |      |
+---------------------------------+------+------+------+------+------+
| **Percent of Parents who Cited  |      |      |      |      |      |
| the Following as the Most       |      |      |      |      |      |
| Important Reason in Choosing a  |      |      |      |      |      |
| School (Top Four Choices)^1^**  |      |      |      |      |      |
+---------------------------------+------+------+------+------+------+
| Academic quality                | 48   | 54   | 52\* | 43   | 44   |
+---------------------------------+------+------+------+------+------+
| Services for students with      | 5    | 5    | 6\*  | 3    | 4    |
| special needs                   |      |      |      |      |      |
+---------------------------------+------+------+------+------+------+
| Convenient location             | 13   | 12   | 11   | 15   | 15   |
+---------------------------------+------+------+------+------+------+
| School safety                   | 17   | 17   | 16   | 21   | 15   |
+---------------------------------+------+------+------+------+------+
| > *Average percent missing*     | *30* | *25* | *32* | *27* | *29* |
+---------------------------------+------+------+------+------+------+
|                                 |      |      |      |      |      |
+---------------------------------+------+------+------+------+------+
| **Sample size**                 | 1    | 79   | 993  | 258  | 518  |
|                                 | ,848 |      |      |      |      |
+---------------------------------+------+------+------+------+------+

\* Significantly different from the average of the other subgroups at
the 95 percent confidence level.

\*\* Significantly different from the average of the other subgroups at
the 99 percent confidence level.

^1^ Parents were given a wide variety of reasons to choose from, but
only the top four are reported in this table, as cell sizes were too
small to report for all response categories. The average percent missing
is across all response categories.

SOURCE: Eligible Applicant Database.

**Appendix D**

**Appendix D**

**Characteristics of Scholarship Users and Nonusers**

In addition to considering the characteristics of eligible applicants to
a scholarship program, it is also important to examine the
characteristics of the students who actually use a scholarship compared
to those who do not use one. A total of 1,027 scholarship recipients had
used their scholarship to matriculate at a private school as of
September 10, 2004. A total of 339 recipients had not used their
scholarship by that date. The tables in this appendix present
comparisons between the scholarship users and nonusers on a variety of
educational, background, and experiential characteristics. Selected
items from these comparisons are summarized in section 4.4 of the
report. The most noteworthy additional findings from these data are
that:

-   Users were more likely than nonusers to be entering kindergarten or
    first grade, less likely than nonusers to be entering the more
    slot-constrained grades 7 through 12.

-   Users had spent an average of 6.1 years in their current residence
    compared to 7.7 years for nonusers.

-   The mothers of scholarship users were somewhat less likely to lack a
    high school diploma and somewhat less likely to have a graduate
    degree than were the mothers of nonusers.

-   Fifty-three percent of users were female compared to just 45 percent
    of nonusers.

-   Students were less likely to use a scholarship if their parents
    reported that their previous school had special programs for
    non-English speakers, advanced learners, or students in special
    education.

-   Students were more likely to use a scholarship if their parents
    described drugs or alcohol use as a problem at their previous
    school.

-   Students were more likely to use a scholarship if their parents
    listed "school safety" as the most important reason for choosing a
    school, less likely to use a scholarship if they listed "services
    for students with special needs" as the most important reason.

There were no statistically significant differences between scholarship
users and nonusers regarding a number of key educational and background
factors discussed in section 4.4 as well as mother's marital status and
the overall grade that parents gave to their child's previous school.

**Table D1. Student's Educational Characteristics, Users and Nonusers:
Spring 2004**

+----------------------------------------+---------+---------+---------+
| **Characteristic**                     | **      | **Non   | **Diffe |
|                                        | Users** | users** | rence** |
+----------------------------------------+---------+---------+---------+
| **Baseline Test Scores**^1^            |         | ** **   | ** **   |
+----------------------------------------+---------+---------+---------+
| > Average Reading Percentile           |         |         |         |
+----------------------------------------+---------+---------+---------+
| > Grades K-5                           | 49.12   | 40.30   | 8       |
|                                        |         |         | .82\*\* |
+----------------------------------------+---------+---------+---------+
| > Grades 6-8                           |  39.82  | 38.06   | 1.76    |
+----------------------------------------+---------+---------+---------+
| > Grades 9-12                          | 30.00   | 23.57   | 6.43    |
+----------------------------------------+---------+---------+---------+
| > *Percent missing*                    | *56*    | *41*    |         |
+----------------------------------------+---------+---------+---------+
| >                                      |         |         |         |
+----------------------------------------+---------+---------+---------+
| > Average Mathematics Percentile       |         |         |         |
+----------------------------------------+---------+---------+---------+
| > Grades K-5                           | 53.47   | 43.75   | 9       |
|                                        |         |         | .72\*\* |
+----------------------------------------+---------+---------+---------+
| > Grades 6-8                           | 44.91   | 42.26   | 2.65    |
+----------------------------------------+---------+---------+---------+
| > Grades 9-12                          | 40.30   | 43.83   | -3.53   |
+----------------------------------------+---------+---------+---------+
| > *Percent missing*                    | *56*    | *40*    |         |
+----------------------------------------+---------+---------+---------+
| > ** **                                |         |         |         |
+----------------------------------------+---------+---------+---------+
| **Percent with a Learning or Physical  |  9      | 29      | -20\*\* |
| Disability**                           |         |         |         |
+----------------------------------------+---------+---------+---------+
| > *Percent missing*                    | *6*     | *8*     |         |
+----------------------------------------+---------+---------+---------+
| >                                      |         |         |         |
+----------------------------------------+---------+---------+---------+
| **Sample size**                        | 1,027   | 339     |         |
+----------------------------------------+---------+---------+---------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ Test-score results are in terms of National Percentile Ranks, with
50 as the median score.

SOURCE: Eligible Applicant Database. Test-scores obtained from the
accountability testing database for District of Columbia public and
charter schools, DCPS Office of Communications and Public Information.

**Table D2. Percent of Students by Forecasted Grade Level, Users and
Nonusers:\
Spring 2004**

+----------------------+---------------+---------------+---------------+
| **Grade Level**      | **Users**     | **Nonusers**  | *             |
|                      |               |               | *Difference** |
+----------------------+---------------+---------------+---------------+
| Kindergarten         | 14            | 9             | 5\*           |
+----------------------+---------------+---------------+---------------+
| 1                    | 13            | 6             | 8\*\*         |
+----------------------+---------------+---------------+---------------+
| 2                    | 11            | 9             | 3             |
+----------------------+---------------+---------------+---------------+
| 3                    | 12            | 9             | 3             |
+----------------------+---------------+---------------+---------------+
| 4                    | 11            | 9             | 2             |
+----------------------+---------------+---------------+---------------+
| 5                    | 11            | 13            | -2            |
+----------------------+---------------+---------------+---------------+
| 6                    | 8             | 9             | -1            |
+----------------------+---------------+---------------+---------------+
| 7                    | 7             | 13            | -6\*\*        |
+----------------------+---------------+---------------+---------------+
| 8                    | 7             | 11            | -4\*          |
+----------------------+---------------+---------------+---------------+
| 9                    | 2             | 4             | -1            |
+----------------------+---------------+---------------+---------------+
| 10                   | 2             | 6             | -4\*\*        |
+----------------------+---------------+---------------+---------------+
| 11-12^1^             | 1             | 4             | -3\*\*        |
+----------------------+---------------+---------------+---------------+
| > *Percent missing*  | *0*           | *0*           |               |
+----------------------+---------------+---------------+---------------+
|                      |               |               |               |
+----------------------+---------------+---------------+---------------+
| **Sample size**      | 1,027         | 339           |               |
+----------------------+---------------+---------------+---------------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ Cell size too small to report separately by grade.

SOURCE: Eligible Applicant Database.

**Table D3. Family Income, Users and Nonusers: Spring 2004**

  --------------------------------------- ------------- -------------- ----------------
  **Characteristic **                     **Users**     **Nonusers**   **Difference**

  **Percent of Students whose Family                    ** **          ** **
  Income is**                                                          

  Less than \$5,000                       11            11             0

  \$5,000-\$10,999                        18            24             -6\*

  \$11,000-\$24,999                       43            38             5

  \$25,000-\$39,999                       25            24             1

  \$40,000 or more                        3             4              0

                                                                        

  **Average Family Income**               \$18,652.49   \$17,840.70    \$811.79

  *Percent missing*                       *0*           *0*            

                                                                        

  **Percent of Students whose Families                                 
  Received the Following Forms of                                      
  Government Assistance**                                              

  Total wages, salaries, tips             53            44             9\*\*

  Housing Assistance                      7             6              1

  TANF (Welfare)                          22            28             -7\*

  Supplemental Security Income            3             6              4\*\*

  Social Security                         10            14             -3

  Child Support                           8             8              0

  Gifts from family/friends               1             1              0

  Interest and dividend income            1             2              0

  Other income                            7             8              -2

                                                                        

  **Sample size**                         1,027         339             
  --------------------------------------- ------------- -------------- ----------------

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

SOURCE: Eligible Applicant Database.

**Table D4. Mother's Employment and Residence, Users and Nonusers:
Spring 2004**

+---------------------------------------+---------+---------+---------+
| **Characteristic**                    | **      | **Non   | **Diffe |
|                                       | Users** | users** | rence** |
+---------------------------------------+---------+---------+---------+
| **Percent of Students whose Mothers   |         | ** **   | ** **   |
| are Employed**                        |         |         |         |
+---------------------------------------+---------+---------+---------+
| Full time                             | 46      | 44      | 2       |
+---------------------------------------+---------+---------+---------+
| Part time                             | 15      | 15      | 0       |
+---------------------------------------+---------+---------+---------+
| Looking for work                      | 30      | 30      | -1      |
+---------------------------------------+---------+---------+---------+
| Not looking                           | 10      | 11      | -1      |
+---------------------------------------+---------+---------+---------+
| > *Percent missing*                   | *13*    | *23*    | * *     |
+---------------------------------------+---------+---------+---------+
|                                       |         |         |         |
+---------------------------------------+---------+---------+---------+
| **Mother's Average Years Work         | 7.81    | 7.98    | -17     |
| Experience**                          |         |         |         |
+---------------------------------------+---------+---------+---------+
| > *Percent missing*                   | *51*    | *57*    |         |
+---------------------------------------+---------+---------+---------+
|                                       |         |         |         |
+---------------------------------------+---------+---------+---------+
| **Average Years at Current            | 6.11    | 7.69    | -1      |
| Residence**                           |         |         | .58\*\* |
+---------------------------------------+---------+---------+---------+
| > *Percent missing*                   | *2*     | *3*     |         |
+---------------------------------------+---------+---------+---------+
| ** **                                 |         |         |         |
+---------------------------------------+---------+---------+---------+
| **Percent who Rent and do not Own**   | 86      | 82      | 3       |
+---------------------------------------+---------+---------+---------+
| > *Percent missing*                   | *14*    | *17*    |         |
+---------------------------------------+---------+---------+---------+
| ** **                                 |         |         |         |
+---------------------------------------+---------+---------+---------+
| **Monthly Rent/Mortgage Payment**     | \       | \       | \$32.02 |
|                                       | $618.38 | $586.36 |         |
+---------------------------------------+---------+---------+---------+
| > *Percent missing*                   | *5*     | *5*     |         |
+---------------------------------------+---------+---------+---------+
| ** **                                 |         |         |         |
+---------------------------------------+---------+---------+---------+
| **Sample size**                       | 1,027   | 339     |         |
+---------------------------------------+---------+---------+---------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

NOTES: Detail may not sum to total because of rounding. Mother response
is for biological parent only.

SOURCE: Eligible Applicant Database.

**Table D5. Mother's Education, Users and Nonusers: Spring 2004**

+----------------------------------------+---------+---------+---------+
| **Characteristic**                     | **      | **Non   | **Diffe |
|                                        | Users** | users** | rence** |
+----------------------------------------+---------+---------+---------+
| **Percent of Students whose Mothers    |         | ** **   | ** **   |
| Attained the Following Level of        |         |         |         |
| Education**                            |         |         |         |
+----------------------------------------+---------+---------+---------+
| No high school diploma                 | 11      | 19      | -7\*\*  |
+----------------------------------------+---------+---------+---------+
| GED                                    | 6       | 6       | 0       |
+----------------------------------------+---------+---------+---------+
| High school diploma                    | 25      | 25      | 0       |
+----------------------------------------+---------+---------+---------+
| Some college                           | 35      | 30      | 5       |
+----------------------------------------+---------+---------+---------+
| Vocational (2 yr) degree or            | 15      | 14      | 1       |
| certificate                            |         |         |         |
+----------------------------------------+---------+---------+---------+
| College graduate                       | 7       | 4       | 3       |
+----------------------------------------+---------+---------+---------+
| Graduate degree                        | 1       | 2       | -2\*    |
+----------------------------------------+---------+---------+---------+
| > *Percent missing*                    | *11*    | *19*    |         |
+----------------------------------------+---------+---------+---------+
|                                        |         |         |         |
+----------------------------------------+---------+---------+---------+
| **Average Years of Mother's            | 12.73   | 12.50   | 23\*    |
| Education**                            |         |         |         |
+----------------------------------------+---------+---------+---------+
| > *Percent missing*                    | *11*    | *19*    |         |
+----------------------------------------+---------+---------+---------+
| ** **                                  |         |         |         |
+----------------------------------------+---------+---------+---------+
| **Sample size**                        | 1,027   | 339     |         |
+----------------------------------------+---------+---------+---------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

NOTE: Mother response is for biological parent only.

SOURCE: Eligible Applicant Database.

**Table D6. Mother's Marital Status and Household Composition, Users and
Nonusers:\
Spring 2004**

+------------------------------------+----------+----------+----------+
| **Characteristic**                 | *        | **No     | **Diff   |
|                                    | *Users** | nusers** | erence** |
+------------------------------------+----------+----------+----------+
| **Percent of Students whose        |          | ** **    | ** **    |
| Mothers are**                      |          |          |          |
+------------------------------------+----------+----------+----------+
| Single, never married              | 61       | 58       | 2        |
+------------------------------------+----------+----------+----------+
| Married                            | 19       | 16       | 2        |
+------------------------------------+----------+----------+----------+
| Divorced                           | 10       | 13       | -3       |
+------------------------------------+----------+----------+----------+
| Other^1^                           | 10       | 12       | -1       |
+------------------------------------+----------+----------+----------+
| > *Percent missing*                | *10*     | *18*     |          |
+------------------------------------+----------+----------+----------+
|                                    |          |          |          |
+------------------------------------+----------+----------+----------+
| **Household Composition**          |          |          |          |
+------------------------------------+----------+----------+----------+
| Adults in household                | 1.33     | 1.31     | 2        |
+------------------------------------+----------+----------+----------+
| > *Percent missing*                | *0*      | *3*      |          |
+------------------------------------+----------+----------+----------+
|                                    |          |          |          |
+------------------------------------+----------+----------+----------+
| Children in household              | 2.83     | 2.70     | 12       |
+------------------------------------+----------+----------+----------+
| > *Percent missing*                | *0*      | *3*      |          |
+------------------------------------+----------+----------+----------+
|                                    |          |          |          |
+------------------------------------+----------+----------+----------+
| Household size                     | 4.10     | 3.94     | 15       |
+------------------------------------+----------+----------+----------+
| > *Percent missing*                | *0*      | *0*      |          |
+------------------------------------+----------+----------+----------+
|                                    |          |          |          |
+------------------------------------+----------+----------+----------+
| **Sample size**                    | 1,027    | 339      |          |
+------------------------------------+----------+----------+----------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ "Other" includes widowed, separated, and divorced, remarried.

NOTE: Mother responses is for biological parent only.

SOURCE: Eligible Applicant Database.

**Table D7. Mother's Ethnicity, Users and Nonusers: Spring 2004**

+------------------------------------+----------+----------+----------+
| **Characteristic**                 | *        | **No     | **Diff   |
|                                    | *Users** | nusers** | erence** |
+------------------------------------+----------+----------+----------+
| **Percent of Students whose        |          | ** **    |          |
| Mothers are**                      |          |          |          |
+------------------------------------+----------+----------+----------+
| African American                   | 93       | 93       | 0        |
+------------------------------------+----------+----------+----------+
| Other race^1^                      | 7        | 7        | 0        |
+------------------------------------+----------+----------+----------+
| > *Percent missing*                | *15*     | *22*     |          |
+------------------------------------+----------+----------+----------+
| **Percent of Students whose        | 8        | 7        |  1       |
| Mothers are Hispanic (any race)**  |          |          |          |
|                                    | *10*     | *18*     |          |
| *Percent missing*                  |          |          |          |
+------------------------------------+----------+----------+----------+
| **Sample size**                    | 1,027    | 339      |          |
+------------------------------------+----------+----------+----------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

[]{.mark}

^1^ "Other race" includes respondents who were identified as white,
Asian, American Indian, Alaska Native, Pacific Islander, or multiracial.
Respondent classified as "multiracial" if more than one race category
selected.

NOTES: Detail may not sum to total because of rounding. Mother response
is for biological parent only.

SOURCE: Eligible Applicant Database.

**Table D8. Student's Ethnicity and Gender, Users and Nonusers: Spring
2004**

+-------------------------------------+----------+----------+----------+
| **Characteristic**                  | *        | **No     | **Diff   |
|                                     | *Users** | nusers** | erence** |
+-------------------------------------+----------+----------+----------+
| **Percent, by Race**                |          | ** **    |          |
+-------------------------------------+----------+----------+----------+
| African American                    | 94       | 96       | -2       |
+-------------------------------------+----------+----------+----------+
| Other race^1^                       | 6        | 4        | 2        |
+-------------------------------------+----------+----------+----------+
| > *Percent missing*                 | *6*      | *6*      |          |
+-------------------------------------+----------+----------+----------+
|                                     | 7        | 7        | 1        |
|                                     |          |          |          |
| **Percent Hispanic (any race)**     | *4*      | 7        |          |
|                                     |          |          |          |
| > *Percent missing*                 |          |          |          |
+-------------------------------------+----------+----------+----------+
| **Percent, by Language Most Spoken  |          |          |          |
| in Home**                           |          |          |          |
+-------------------------------------+----------+----------+----------+
| English                             | 93       | 96       | -3       |
+-------------------------------------+----------+----------+----------+
| Other language^2^                   | 7        | 4        | 3        |
+-------------------------------------+----------+----------+----------+
| > *Percent missing*                 | *4*      | *4*      |          |
+-------------------------------------+----------+----------+----------+
|                                     |          |          |          |
+-------------------------------------+----------+----------+----------+
| **Percent, by Gender**              |          |          |          |
+-------------------------------------+----------+----------+----------+
| Female                              | 53       | 45       | 9\*\*    |
+-------------------------------------+----------+----------+----------+
| > *Percent missing*                 | *1*      | *2*      |          |
+-------------------------------------+----------+----------+----------+
| ** **                               |          |          |          |
+-------------------------------------+----------+----------+----------+
| **Sample size**                     | 1,027    | 339      |          |
+-------------------------------------+----------+----------+----------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ "Other race" includes respondents who were identified as white,
Asian, American Indian, Alaska Native, Pacific Islander, or multiracial.
Respondent classified as "multiracial" if more than one race category
selected.

^2^ "Other language" includes all other languages including Spanish and
Amharic.

NOTE: Detail may not sum to total because of rounding.

SOURCE: Eligible Applicant Database.

**Table D9. Parental Involvement with Child's Education, Users and
Nonusers: Spring 2004**

+------------------------------------+----------+----------+----------+
| **Characteristic**                 | *        | **No     | **Diff   |
|                                    | *Users** | nusers** | erence** |
+------------------------------------+----------+----------+----------+
| **Percent of Parents who           |          |          |          |
| Participated in the Following      |          |          |          |
| Activities with Their Child in the |          |          |          |
| Past Month**                       |          |          |          |
+------------------------------------+----------+----------+----------+
| > Discussed experiences at school  | 99       | 98       | 0        |
+------------------------------------+----------+----------+----------+
| > Helped with mathematics or       | 94       | 90       | 4\*\*    |
| > reading not related to homework  |          |          |          |
+------------------------------------+----------+----------+----------+
| > Worked on homework               | 97       | 92       | 5\*\*    |
+------------------------------------+----------+----------+----------+
| > Worked on a school project       | 86       | 85       | 1        |
+------------------------------------+----------+----------+----------+
| > Attended school activities with  | 90       | 85       | 5\*      |
| > child                            |          |          |          |
+------------------------------------+----------+----------+----------+
| > *Average percent missing*        | *8*      | *6*      |          |
+------------------------------------+----------+----------+----------+
| ** **                              |          |          |          |
+------------------------------------+----------+----------+----------+
| **Sample size**                    | 1,027    | 339      |          |
+------------------------------------+----------+----------+----------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

SOURCE: Eligible Applicant Database.

**Table D10. School Facilities and Homework, Users and Nonusers: Spring
2004**

+-------------------------------------+----------+----------+----------+
| **Characteristic **                 | *        | **No     | **Diff   |
|                                     | *Users** | nusers** | erence** |
+-------------------------------------+----------+----------+----------+
| **Percent of Parents Reporting that |          | ** **    | ** **    |
| Students have the Following         |          |          |          |
| Resources at Their School**         |          |          |          |
+-------------------------------------+----------+----------+----------+
| Special programs for non-English    | 37       | 54       | -17\*\*  |
| speakers                            |          |          |          |
+-------------------------------------+----------+----------+----------+
| Nurse's office                      | 80       | 89       | -9\*\*   |
+-------------------------------------+----------+----------+----------+
| Prepared Lunches                    | 70       | 77       | -7\*     |
+-------------------------------------+----------+----------+----------+
| Cafeteria                           | 81       | 90       | -9\*\*   |
+-------------------------------------+----------+----------+----------+
| Special programs for advanced       | 33       | 46       | -12\*\*  |
| learners                            |          |          |          |
+-------------------------------------+----------+----------+----------+
| Arts program                        | 67       | 72       | -4       |
+-------------------------------------+----------+----------+----------+
| Special education programs          | 57       | 73       | -15\*\*  |
+-------------------------------------+----------+----------+----------+
| Computer lab                        | 75       | 79       | -5       |
+-------------------------------------+----------+----------+----------+
| Gym                                 | 58       | 68       | -10\*\*  |
+-------------------------------------+----------+----------+----------+
| Child counselors                    | 78       | 84       | -7\*     |
+-------------------------------------+----------+----------+----------+
| Library                             | 80       | 86       | -6\*     |
+-------------------------------------+----------+----------+----------+
| After-school program                | 82       | 78       | 4        |
+-------------------------------------+----------+----------+----------+
| Music program                       | 69       | 72       | -4       |
+-------------------------------------+----------+----------+----------+
| Individual tutors                   | 37       | 45       | -8\*     |
+-------------------------------------+----------+----------+----------+
|                                     |          |          |          |
+-------------------------------------+----------+----------+----------+
| **Average Hours of Daily Homework** | .93      | .96      | -.04     |
+-------------------------------------+----------+----------+----------+
| > *Percent missing*                 | *9*      | *7*      |          |
+-------------------------------------+----------+----------+----------+
| ** **                               |          |          |          |
+-------------------------------------+----------+----------+----------+
| **Sample size**                     | 1,027    | 339      |          |
+-------------------------------------+----------+----------+----------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

SOURCE: Eligible Applicant Database.

**Table D11. Problems at Child's School, Users and Nonusers: Spring
2004**

+---------------------------------+-----------+-----------+-----------+
| **Characteristic**              | **Users** | **N       | **Dif     |
|                                 |           | onusers** | ference** |
+---------------------------------+-----------+-----------+-----------+
| **Percent of Parents who        |           | ** **     | ** **     |
| Believe the Following Problems  |           |           |           |
| at School are Serious**         |           |           |           |
+---------------------------------+-----------+-----------+-----------+
| Fighting                        | 50        | 46        | 4         |
+---------------------------------+-----------+-----------+-----------+
| Tardiness                       | 48        | 48        | 1         |
+---------------------------------+-----------+-----------+-----------+
| Destruction of property         | 35        | 35        | 1         |
+---------------------------------+-----------+-----------+-----------+
| Truancy                         | 34        | 39        | -4        |
+---------------------------------+-----------+-----------+-----------+
| Cheating                        | 26        | 22        | 4         |
+---------------------------------+-----------+-----------+-----------+
| Guns or other weapons           | 20        | 15        | 4         |
+---------------------------------+-----------+-----------+-----------+
| Drug distribution               | 16        | 10        | 6\*       |
+---------------------------------+-----------+-----------+-----------+
| Drug and alcohol use            | 14        | 10        | 5\*       |
+---------------------------------+-----------+-----------+-----------+
| > *Average percent missing*     | *10*      | *9*       |           |
+---------------------------------+-----------+-----------+-----------+
| ** **                           |           |           |           |
+---------------------------------+-----------+-----------+-----------+
| **Sample size**                 | 1,027     | 339       |           |
+---------------------------------+-----------+-----------+-----------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

SOURCE: Eligible Applicant Database.

**Table D12. Parental Satisfaction with School, Users and Nonusers,
Spring 2004**

+---------------------------------+-----------+-----------+-----------+
| **Characteristic**              | **Users** | **N       | **Dif     |
|                                 |           | onusers** | ference** |
+---------------------------------+-----------+-----------+-----------+
| **Percent of Parents 'Very      |           | ** **     | ** **     |
| Satisfied' with the Following   |           |           |           |
| Aspects of Their Child's        |           |           |           |
| School**                        |           |           |           |
+---------------------------------+-----------+-----------+-----------+
| Amount of information from      | 26        | 23        | 3         |
| teachers                        |           |           |           |
+---------------------------------+-----------+-----------+-----------+
| Freedom to observe religious    | 19        | 13        | 6\*       |
| traditions                      |           |           |           |
+---------------------------------+-----------+-----------+-----------+
| Class size                      | 18        | 17        | 1         |
+---------------------------------+-----------+-----------+-----------+
| Safety                          | 22        | 20        | 2         |
+---------------------------------+-----------+-----------+-----------+
| Respect between teachers and    | 23        | 21        | 2         |
| students                        |           |           |           |
+---------------------------------+-----------+-----------+-----------+
| Location                        | 31        | 25        | 6\*       |
+---------------------------------+-----------+-----------+-----------+
| Parental support for school     | 23        | 24        | 0         |
+---------------------------------+-----------+-----------+-----------+
| Academic quality                | 21        | 18        | 3         |
+---------------------------------+-----------+-----------+-----------+
| Discipline                      | 22        | 20        | 2         |
+---------------------------------+-----------+-----------+-----------+
| Racial mix of students          | 17        | 15        | 2         |
+---------------------------------+-----------+-----------+-----------+
| > *Average percent missing*     | *8*       | *9*       |           |
+---------------------------------+-----------+-----------+-----------+
|                                 |           |           |           |
+---------------------------------+-----------+-----------+-----------+
| **Percent of Parents who Gave   |           |           |           |
| the School a Grade of**         |           |           |           |
+---------------------------------+-----------+-----------+-----------+
| A                               | 19        | 15        | 4         |
+---------------------------------+-----------+-----------+-----------+
| B                               | 29        | 31        | -3        |
+---------------------------------+-----------+-----------+-----------+
| C                               | 32        | 32        | 0         |
+---------------------------------+-----------+-----------+-----------+
| D                               | 13        | 13        | 0         |
+---------------------------------+-----------+-----------+-----------+
| F                               | 8         | 9         | -1        |
+---------------------------------+-----------+-----------+-----------+
|                                 |           |           |           |
+---------------------------------+-----------+-----------+-----------+
| **Average Grade Parent Gave     | 2.37      | 2.30      | 7         |
| School (4.0 Scale)**            |           |           |           |
+---------------------------------+-----------+-----------+-----------+
| > *Percent missing*             | *6*       | *6*       |           |
+---------------------------------+-----------+-----------+-----------+
|                                 |           |           |           |
+---------------------------------+-----------+-----------+-----------+
| **Sample size**                 | 1,027     | 339       |           |
+---------------------------------+-----------+-----------+-----------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

SOURCE: Eligible Applicant Database.

**Table D13. Valued Features of Schools, Users and Nonusers: Spring
2004**

+------------------------------------+----------+----------+----------+
| **Characteristic**                 | *        | **No     | **Diff   |
|                                    | *Users** | nusers** | erence** |
+------------------------------------+----------+----------+----------+
| **Percent of Parents who Cited the |          | ** **    | ** **    |
| Following as the Most Important    |          |          |          |
| Reason in Choosing a School (Top   |          |          |          |
| Four Choices)^1^**                 |          |          |          |
+------------------------------------+----------+----------+----------+
| Academic quality                   | 49       | 47       | 2        |
+------------------------------------+----------+----------+----------+
| Services for students with special | 3        | 13       | -10\*\*  |
| needs                              |          |          |          |
+------------------------------------+----------+----------+----------+
| Convenient location                | 13       | 11       | 2        |
+------------------------------------+----------+----------+----------+
| School safety                      | 19       | 11       | 8\*\*    |
+------------------------------------+----------+----------+----------+
| > *Average percent missing*        | *30*     | *31*     |          |
+------------------------------------+----------+----------+----------+
|                                    |          |          |          |
+------------------------------------+----------+----------+----------+
| **Sample size**                    | 1,027    | 339      |          |
+------------------------------------+----------+----------+----------+

\* Statistically significant at the 95 percent confidence level.

\*\* Statistically significant at the 99 percent confidence level.

^1^ Parents were given a wide variety of reasons to choose from, but
only the top four are reported in this table, as cell sizes were too
small to report for all response categories. The average percent missing
is across all response categories.

SOURCE: Eligible Applicant Database.

[^1]: While there were 79 applicants from the 15 schools designated as
    SINI in 2003, in advance of the first year lottery in May 2004, DCPS
    designated an additional 73 schools as in need of improvement in
    August 2004, two months after the scholarship lottery (See
    <http://silicon.k12.dc.us/NCLB/index.asp>, accessed February 15,
    2004). A total of 535 program applicants came from 2003-designated
    or 2004-designated SINI schools, representing 29 percent of all
    program applicants this first year. Of these students, 433 (81
    percent) were awarded scholarships. Twice as many Opportunity
    Scholarships were awarded to students applying from 2003- or
    2004-designated SINI schools than were awarded to students applying
    from private schools.

[^2]: Because they represented such a small share of the overall
    applicant pool the first year and there was little possibility of
    separately analyzing impacts for this subgroup, all SINI applicants
    were awarded a scholarship. However, with more SINI schools
    identified for the next year, it is likely that a much higher number
    of students in this priority group will apply to the program in the
    future. Therefore, in succeeding years the group will be given
    highest priority but be part of the random assignment process that
    will enable that important category of students to be included in
    the rigorous impact evaluation.

[^3]: Evaluation sample size needs are calculated using statistical
    power analysis. It is an important part of evaluation design and
    demonstrates how well the design will be able to distinguish real
    impacts from chance differences between groups. The analysis takes
    into consideration such factors as the size of the impact expected,
    the proportion of students anticipated to be assigned to the
    treatment and control groups, and the likely attrition from the
    study's data collection. The power analysis conducted for the DC
    Opportunity Scholarship Program suggests a target of 1,240 randomly
    assigned students, with slightly more than 800 students assigned to
    the treatment group and slightly more than 400 assigned to the
    control group, so long as the test score impacts in the outcomes in
    future years are at least moderately large (Appendix A).

[^4]: The sample size of 894 for the applicant group compared here
    differs from the total sample of 1343 public school applicants for
    two reasons. First, only 1,077 (80 percent) of the public school
    applicants could be identified conclusively in the DCPS
    accountability testing database. Most missing observations were in
    pre-K, first, or second grade, where accountability testing is
    optional. An additional 183 public school applicants in the
    accountability database were not enrolled in the free or
    reduced-price lunch program. To keep the comparison balanced, they
    were excluded from the applicant group for purposes of this
    analysis.

[^5]: Test scores were not available for the private school applicants.

[^6]: RCTs are commonly referred to as the "gold standard" for
    evaluating educational interventions; when mere chance determines
    which eligible applicants receive access to school choice, the
    students who apply but are not admitted comprise an ideal "control
    group" for comparison with the school choice "treatment group." Both
    groups of participants are equally motivated to obtain new
    educational options, and nothing except a random draw distinguishes
    those who receive the opportunity from those who do not. Therefore,
    any differences in the two groups in subsequent years can be
    attributed to the impact of the program. In contrast, the results of
    school choice studies that are not based on RCTs must be interpreted
    and used more cautiously, because comparisons between the applicants
    and a group of students who chose not to apply will likely reflect
    not only the impact of the program but also differences between the
    groups in motivation and other unmeasured characteristics. See below
    for more detail on the RCT analysis approach.

[^7]: See 309 (a)(4)(A)(ii).

[^8]: The RCT approach was also used by researchers conducting impact
    evaluations of the New York City, Dayton, Ohio, and Washington, DC,
    private scholarship programs.

[^9]: Because answering this question requires analysts to deviate from
    the random assignment design, and the testing conditions for
    scholarship students and DCPS students will differ, inferences from
    the results of this specific comparison will be drawn with caution.

[^10]: For this report, which is descriptive, as well as for the later
    impact analysis reports, we will use several tests for calculating
    statistical significance, or the level of confidence that evaluators
    have that a difference between groups did not occur merely by
    chance. For most of the comparisons that we make, we use the
    "Student's *t* test." The *t* test is commonly used when the factor
    being considered, such as test scores, tends to be distributed on a
    normal, bell-shaped curve. Unlike some significance tests, the *t*
    test incorporates information about the distribution of values in
    both comparison groups, and not just the overall population, and
    thus is a more precise measure of statistical significance than the
    *Z* test, for example (see Russell A. Langley, *Practical Statistics
    Simply Explained* (New York: Dover, 1970), 160-165. When we compare
    the characteristics of one specific group to another specific group,
    the application of the *t* test is conventional and straightforward.
    When we compare the characteristics of one specific group with
    several other specific groups, as in the comparisons of applicants
    from various types of schools in Chapter 4, we apply the *t* test to
    calculate the significance of the difference between the average
    score for the group in question and the average score for all of the
    other groups combined. When the characteristic in question is not
    normally distributed―such as gender, which is an either/or and not a
    more-or-less―we use the "chi-squared" test of statistical
    significance.

[^11]: Some of these applicants from private schools were already
    relying on scholarship funds in order to attend those schools.
    However, the scholarships they were receiving may have been less
    generous than those available under the federal DC Opportunity
    Scholarship Program.

[^12]: See the following studies, which all use the same data from an
    evaluation of a New York City privately funded scholarship program:
    William G. Howell, Patrick J. Wolf, David E. Campbell, and Paul E.
    Peterson, "School Vouchers and Academic Performance: Results from
    Three Randomized Field Trials," *Journal of Policy Analysis and
    Management* 21 (2000): 2; John Barnard, Constantine E. Frangakis,
    Jennifer L. Hill, and Donald B. Rubin, "Principal Stratification
    Approach to Broken Randomized Experiments: A Case Study of School
    Choice Vouchers in New York City," *Journal of the American
    Statistical Association* 98 (2003): 462; Alan B. Krueger and Pei
    Zhu, *Another Look at the New York City School Voucher Experiment,*
    Working Paper Series, Education Research, (Princeton, NJ: Princeton
    University, March 2003).

[^13]: For a spirited debate about the use of this technique in the
    context of school choice research, see William G. Howell and Paul E.
    Peterson, "Uses of Theory in Randomized Field Trials: Lessons from
    School Voucher Research on Disaggregation, Missing Data, and the
    Generalization of Findings," *American Behavioral Scientist* 47
    (Jan. 2004): 634-657; Krueger and Zhu, *"Another Look,"* 658-698;
    Paul E. Peterson and William G. Howell, "Efficiency, Bias, and
    Classification Schemes: A Response to Krueger, A.B. and Zhu, P.,
    'Another Look at the New York City School Voucher Experiment,'"
    Working Paper Series, Education Research (Princeton, NJ: Princeton
    University, March 2003), 699-717; Alan B. Kruger and Pei Zhu,
    "Inefficiency, Subsample Selection Bias, and Nonrobustness: A
    Response to Peterson, P.E. and Howell, W.G., 'Another Look at the
    New York City School Voucher Experiment,'" Working Paper Series,
    Education Research (Princeton, NJ: Princeton University, March
    2003): 718-728; Paul E. Peterson and William G. Howell, "Voucher
    Research Controversy: New Looks at the New York City Evaluation,"
    *Education Next* 4 (Spring 2004): 73-78.

[^14]: Depending on the extent to which the randomly assigned applicants
    are clustered in their schools, some adjustments to the standard
    error estimates may be necessary.

[^15]: For an extended discussion of the use of this technique under
    such circumstances, see William G. Howell and Paul E. Peterson, with
    Patrick J. Wolf and David E. Campbell, *The Education Gap: Vouchers
    and Urban Schools* (Washington, DC: Brookings Institution Press,
    2002): 49-51.

[^16]: That is, there are no readily available measures of student
    safety or parent satisfaction for students in DCPS that are
    comparable to those being collected from program participants.

[^17]: To participate in the program, schools are required to sign a
    formal letter of agreement that requires them to (1) submit an
    independent financial and management audit to WSF annually to
    document financial stability, (2) provide a Certificate of
    Occupancy, (3) document that the school is operating lawfully in the
    District and is in compliance with DC health, safety, and fire
    codes, and (4) agree to abide by all applicable nondiscrimination
    laws.

[^18]: WSF surveyed the participating schools about their
    characteristics and offerings in order to provide applicants and
    scholarship winners with information they could use in seeking a
    school placement. Five schools did not respond to the survey.

[^19]: The ward location of one participating school that provided data
    was not available.

[^20]: "Proposed FY2005 Budget, Office of the Mayor, Washington, DC,"
    www.dc.gov/mayor/budget/proposed/index2.shtm/ (accessed November 11,
    2004); "Establishing a Baseline: A Report on the State of Education
    in the District of Columbia," draft report of the DC State Education
    Office, June 2004,
    http://seo.dc.gov/seo/frames.asp?doc=/seo/lib/seo/SER/
    State_of_Education_rpt04.pdfandgroup=1507/ (accessed November 11,
    2004).

[^21]: DC School Search web site,
    www.dcschoolsearch.dc.gov/schools/report_results.asp?report_id=14/
    (accessed November 11, 2004).

[^22]: Participating schools that increase their tuition by more than 20
    percent in a given year or increase their annual enrollment by more
    than 50 percent are required to submit to an inspection by a task
    force of education experts before being permitted to initiate or
    continue their participation in the scholarship program.

[^23]: Shortly after the legislation was enacted in January 2004, Fight
    For Children (FFC), a 501(c)3 organization, was designated to serve
    as the interim entity to promote awareness of the program and lay
    the foundation for its implementation. Among other activities, FFC
    initiated an advertising campaign that featured radio and bus ads,
    established a web page and call center to inform families about the
    program, and built a database of families that expressed an initial
    interest in subsequently applying for the program and another
    database of private schools interested in participating in the
    program.

[^24]: As part of the orientation sessions, officials of the WSF
    explained the requirements of the program and the procedure for
    applying, being selected for, and using a scholarship; members of
    the technical advising team and Institute of Education Sciences
    (IES) also were present to discuss the evaluation component of the
    program and explain the rights and responsibilities of participants.
    Also present were officials from the Public School Aid Service
    (PSAS), a company that verifies eligibility for government and
    school-sponsored scholarship programs. PSAS was under contract to
    WSF to provide those services and, as part of the application
    events, to train volunteers and assist families in documenting their
    income and residency eligibility.

[^25]: One or more meetings took place from May 4 through May 13 at the
    Langston Terrace housing project, Tyler House housing project,
    Fletcher Johnson Education Center, Barry Farm Community Center,
    Barry Farm Recreation Center, and Woodland Terrace Community Center.

[^26]: A total of 604 student applications were never completed.
    Applicants provided a number of reasons for failing to complete
    their applications, including that they received a spot in a
    preferred public or charter school, that they were not confident
    that they would find a desirable private school near their home, or
    that they realized that they would not be eligible. An additional
    240 student applications were completed, but the applicants were
    confirmed to be ineligible for the program.

[^27]: While there were 79 applicants from the 15 schools designated as
    SINI in 2003, in advance of the first year lottery in May 2004, DCPS
    designated an additional 73 schools as in need of improvement in
    August 2004, two months after the scholarship lottery (See
    <http://silicon.k12.dc.us/NCLB/index.asp>, accessed February 15,
    2004). A total of 535 program applicants came from 2003-designated
    or 2004-designated SINI schools, representing 29 percent of all
    program applicants this first year. Of these students, 433 (81
    percent) were awarded scholarships. Twice as many Opportunity
    Scholarships were awarded to students applying from 2003- or
    2004-designated SINI schools than were awarded to students applying
    from private schools.

[^28]: To meet the legislation's requirement that the program provide a
    wide range of education options, parents of scholarship recipients
    were encouraged to list up to four preferred private schools for
    each scholarship student. This was expected to improve their
    likelihood of being able to take advantage of the school choices
    available to them.

[^29]: While there were 79 applicants from the 15 schools designated as
    SINI in 2003, in advance of the first year lottery in May 2004, DCPS
    designated an additional 73 schools as in need of improvement in
    August 2004, two months after the scholarship lottery (See
    <http://silicon.k12.dc.us/NCLB/index.asp>, accessed February 15,
    2004). A total of 535 program applicants came from 2003-designated
    or 2004-designated SINI schools, representing 29 percent of all
    program applicants this first year. Of these students, 433 (81
    percent) were awarded scholarships. Twice as many Opportunity
    Scholarships were awarded to students applying from 2003- or
    2004-designated SINI schools than were awarded to students applying
    from private schools.

[^30]: With more schools identified as SINI and the program
    implementer's plan for aggressive recruitment of students from these
    schools in 2005, it is likely that a much higher number of students
    in this priority group will apply to the DC Opportunity Scholarship
    Program next year. Under these conditions, the group will be given
    highest but not absolute priority and will be part of the random
    assignment process that will enable that important category of
    students to be included in the rigorous impact evaluation.

[^31]: This first stage lottery results in a "conditional scholarship"
    for lottery winners, because to use the scholarship, families must
    ultimately go through a placement process at any participating
    school in which they are interested.

[^32]: No overassignment of scholarships was necessary at the elementary
    grades, since more slots were available than there were eligible
    applicants.

[^33]: Howell et al., *The Education Gap,* 44.

[^34]: Only nine participating private schools served students in any of
    the high school grades.

[^35]: The customized computer program designed by the technical
    advising team to implement the lottery was executed at the WSF
    offices on June 16, 2004. Members of the public and officials from
    ED were present to witness the lottery. A diagnostic program
    confirmed that the lottery operated exactly as designed, including
    the creation of comparable treatment and control groups for
    applicants in the oversubscribed grades 6 to 12.

[^36]: Of the 1,366 scholarship winners, 1,249 were selected via the
    initial execution of the lottery program. As described in Chapter 2,
    after the lottery, additional applicants became eligible for the
    program either by providing necessary documentation that completed
    their original application or through an appeals process conducted
    by the WSF. These additional eligible applicants were automatically
    awarded scholarships if they were attending SINI public or public
    school grades that were not oversubscribed (i.e., entering K-5),
    otherwise they were randomly assigned to the scholarship or control
    group using the same probabilities that had been applied to the
    initial group of eligible applicants. This subsequent lottery
    process resulted in an additional award of 117 scholarships and
    eight additional control group members, since most of the newly
    eligible applicants qualified for the automatic scholarship awards.

[^37]: As discussed in Chapter 1 and Appendix A, statistical power
    analysis is an important part of evaluation design and demonstrates
    how well the design will be able to distinguish real impacts from
    chance differences. The analysis takes into consideration such
    factors as the size of the impact expected, the proportion of
    students anticipated to be assigned to the treatment and control
    groups, and the likely attrition from the study's data collection.

[^38]: The law allows participating private schools to use the same
    admissions criteria for federal scholarship recipients as for other
    applicants to the school.

[^39]: Because of delays in obtaining a final list of available seats at
    participating schools, the actual computerized placement lottery
    could not be run until August 6.

[^40]: There were 348 families that won scholarships for multiple
    children. Of these multi-award families, 129 submitted school
    preference forms in time for the placement lottery. A total of 119
    of the 129 preference forms (92 percent) included a request that all
    the children in the family be placed in the same school.

[^41]: Another 89 students were awarded scholarships after the first
    placement lottery because they were deemed eligible upon finally
    completing their application or as a result of the appeals process.
    All of these students were public school applicants entering grades
    K-5 and thus were automatically awarded scholarships upon being
    certified eligible for the program.

[^42]: The number of scholarship winners listed for SINI public,
    Non-SINI public, and Private schools differ from the numbers shown
    in the scholarship lottery. For the placement lottery, if a parent
    indicated on the school preference form that he/she wanted all of
    his/her children placed in the same school [and]{.underline} the
    children were currently attending schools that fell under two or
    more of the classifications below, all children in the family were
    listed in the highest priority and most advantageous classification.
    For example, if there were two children in a family and one attended
    a SINI school while the other did not, [both]{.underline} children
    were classified as SINI. This was done so that SINI students would
    not be penalized for applying along with one or more non-SINI
    siblings.

[^43]: All the students who listed a fourth school choice were placed in
    their first, second, or third choice school.

[^44]: The initial scholarship usage rate was 82 percent in New York
    City, 78 percent in Dayton, Ohio, and 68 percent in Washington, DC,
    for the earlier experimental evaluation of private scholarship
    programs in those three cities. Voucher usage rates were 61 percent
    in the first year of the Milwaukee Parental Choice Program and
    estimated to be about 50 percent in the first year of the Cleveland
    Scholarship and Tutoring Program. See Howell et al., *The Education
    Gap,* 44; John F. Witte, *First Year Report: Milwaukee Parental
    Choice Program* (University of Wisconsin-Madison, November 1991), 3;
    John F. Witte, Andrea B. Bailey, and Christopher A. Thorn, *Second
    Year Report: Milwaukee Parental Choice Program* (University of
    Wisconsin-Madison, December 1992), 8; Paul E. Peterson and Bryan C.
    Hassel, eds., *Learning From School Choice* (Washington, DC:
    Brookings, 1998), 360.

[^45]: For example, only 4 percent of the students participating in the
    Cleveland Scholarship and Tutoring Program in 1999-2000 attended
    secular private schools, a fact that figured prominently in the
    deliberations surrounding the constitutionality of that school
    choice program (See *Zelman, Superintendent of Public Instruction of
    Ohio, et al. v. Simmons-Harris et al*. 536 U.S. 647 (2002), p. 1).

[^46]: The ward locations of schools were determined using the NCES
    Common Core of Data and the Citizen Atlas at the District of
    Columbia Government web site, available at
    citizenatlas.dc.gov/atlasapps/reporthometab.aspx.

[^47]: The DCPS Office of Communications and Public Information provided
    the data file to the technical advising team consistent with the
    requirement in the legislation that all schools and school systems
    that are covered by the DC School Choice Incentive Act of 2003
    cooperate with the program evaluator in providing information
    necessary to the evaluation. The DCPS accountability database
    included demographic information on 1077 eligible applicants,
    representing 80 percent of all public school eligible applicants.
    The data base also included a reading or math test score for 784 of
    these program applicants, or 58 percent of all eligible public
    school applicants to the program. Of the 42 percent of applicants
    without such data: (1) 65 percent are students entering grades K-1,
    for whom there are no DCPS tests offered; (2) another 28 percent are
    students entering grades 2 and 3 who are not required to take the
    DCPS accountability tests, although some did and are included in the
    comparisons; (3) the remaining 17 percent are students entering
    grades 4 through 12. To examine whether there was any substantial
    bias in our analysis, we compared the demographic characteristics of
    students who were tested by DCPS with those who were not tested,
    both within and across the applicant and nonapplicants samples,
    using a *t* test for difference of means to identify statistically
    significant differences. The testers did not differ significantly
    from the nontesters on any characteristic, except for grade, since
    testing is mandatory only in grades 3 and higher. Therefore, we are
    confident that tester/nontester bias does not affect the comparisons
    that we make here.

[^48]: There were not sufficient evaluation resources or time during the
    first year of the program to consider having the evaluation team
    test all program applicants, including those from private schools.

[^49]: For example, the score average of nearly 47 NPRs for all K to 5
    applicants in reading is about 3 percentile points below the
    national median for those grades in reading, as approximately 47
    percent of students across the country who took the reading test
    last spring in those grades scored lower than the average for the
    program applicants.

[^50]: Test scores were missing from about one-quarter of the sample, so
    these results should be interpreted with caution. However, as
    discussed in footnote 47, the background characteristics of testers
    and non-testers do not indicate any clear biases that would
    necessarily undermine the comparisons.

[^51]: Students generally are classified as enrolled in special
    education if they have an individualized education plan consistent
    with the federal Individuals with Disabilities Education Act. As
    with the test score comparison, the significant amount of missing
    data regarding disability status requires that results be
    interpreted with caution.

[^52]: Although all applicants who were deemed eligible for the program
    were verified as having family incomes that made them eligible for
    the federal lunch program, some of them were not enrolled in that
    program. It is well established that not all students who are
    eligible for the lunch program enroll in it, and the percentage of
    eligible students who decline to enroll tends to increase in the
    upper grades. For example, our data indicate that 14 percent of K-5
    eligible applicants, 16 percent of grades 6 to 8 eligible
    applicants, and 19 percent of grades 9 to 12 eligible applicants
    were not enrolled in the lunch program (Appendix B, Table B1). Any
    bias that this underenrollment introduces into the sample is likely
    to be roughly equal for both the program applicant and nonapplicant
    groups, so it probably does not bias the comparison between the two.

[^53]: The data from the accountability test score file for charter
    students did not include information on FRL status. The charter
    student data did include an indicator for whether the student was
    eligible for Title 1. We confirmed with the DC Office of
    Communications and Public Information that the income cutoffs for
    Title 1 and FRL in the District are identical. Thus, we included
    charter students in the comparison limited to FRL if they were
    designated as Title 1 eligible.

[^54]: Test scores were missing from about one-quarter of the sample, so
    these results should be interpreted with caution. However, as
    discussed in footnote 47, the background characteristics of testers
    and non-testers do not indicate any clear biases that would
    necessarily undermine the comparisons.

[^55]: Test scores were missing from about one-quarter of the sample, so
    these results should be interpreted with caution. However, as
    discussed in footnote 47, the background characteristics of testers
    and non-testers do not indicate any clear biases that would
    necessarily undermine the comparisons.

[^56]: In addition, DCPS measures other characteristics, such as
    race/ethnicity and special education status, differently than does
    the baseline survey the evaluation team administered

[^57]: Test scores were missing from about one-quarter of the sample, so
    these results should be interpreted with caution. However, as
    discussed in footnote 47, the background characteristics of testers
    and non-testers do not indicate any clear biases that would
    necessarily undermine the comparisons.

[^58]: As described in Table 3-3, the non-impact sample consists of
    1,356 eligible applicants. A total of 1,067 are scholarship
    recipients -- 851 of whom were public school applicants who were
    automatically awarded scholarships and 216 of whom were private
    school applicants. The non-impact sample also includes 289
    scholarship nonrecipients who were private school applicants.

[^59]: It is not appropriate, statistically, to assume that the impact
    analysis results would apply to the student groups not included in
    the impact sample. In particular, applicants already enrolled in
    private schools, excluded from the impact sample, are likely to be
    quite different than public school applicants.

[^60]: The data presented in this chapter were drawn almost exclusively
    from the application form and baseline survey, with the exception of
    test-score data taken from the DCPS database.

[^61]: The numbers of eligible applicants by subgroup differ somewhat in
    this chapter from the totals used for the priority-group
    classifications described in Chapters 2 and 3. First, the 258
    applicants from charter schools are a significant component of the
    non-SINI public group, and, thus, were separated out from that group
    for the purposes of this analysis. Second, 13 applicants were
    originally classified as attending non-SINI public schools based on
    parental responses to the school type and name of school questions
    in the baseline survey. In all 13 cases, the parent indicated that
    the child was attending a public school and provided either no
    school name (4 cases) or a school name that did not obviously
    signify the type of school (9 cases―4 of which were preschools).
    Further investigations by the technical advising team indicated that
    these 13 students were attending private schools or private
    preschools, and their school type classification was corrected
    accordingly.

[^62]: Specifically, for each characteristic, each subgroup (e.g., SINI
    Public, Non-SINI Public, Charter, Private) average was compared with
    the population average excluding that subgroup, using a *t* test to
    determine if the subgroup score was statistically different from the
    collective members of the other subgroups. For example, the average
    reading score of 19.5 NPRs for SINI public applicants entering
    grades 9 to 12 was significantly different from the average for all
    other applicants excluding those from SINI public schools. This
    approach was selected instead of conducting multiple pair-wise
    comparisons because it directly and efficiently addresses the
    question "Is this subgroup average different from the rest?" The *t*
    test was used instead of a *Z* test because the *t* test is a more
    powerful statistical test, which is an important concern when
    subgroups are small (e.g., the SINI public subgroup).

[^63]: Seven of the 15 SINI schools were high schools, and high school
    students were significantly less likely to apply for the program
    than were students in the elementary and middle school grades.

[^64]: Test scores were missing from all of the private school
    applicants and one-quarter to one-half of the members of the other
    subgroups, so these results should be interpreted with great
    caution.

[^65]: Both the size of the difference and the size of the samples being
    compared factor into the determination of statistical significance.
    As a result, even though the subgroup of SINI public applicants
    reported the highest rate of learning or physical disabilities (16
    percent), that level does not differ statistically from the rates of
    students from the other subgroups because the sample of SINI public
    applicants is so small―just 79 students.

[^66]: Again, the proportion of applicants who are African American was
    highest among the SINI public subgroup, but the small size of that
    subgroup means that we cannot know with confidence that it is
    significantly different from the norm in that regard.

[^67]: Comparing the applicant data to census data for the DC for all
    adults ages 25 to 64 years old in households that are at or below
    185 percent of poverty in the District indicates that the mothers of
    program applicants have higher levels of educational attainment.
    This may or may not suggest that applicant families are relatively
    advantaged, because the census numbers include men, whose
    educational attainment tends to be lower than that of women.
    Although the baseline application and survey requested information
    about the applicant's birth father, only 10 to 15 percent of
    applications included responses to those questions.

[^68]: William G. Howell, "Dynamic Selection Effects in Means-Tested,
    Urban School Voucher Programs," *Journal of Policy Analysis and
    Management 23*, No. 2, Spring 2004, pp. 225-250; William G. Howell
    and Paul E. Peterson, with Patrick J. Wolf and David E. Campbell,
    *The Education Gap: Vouchers and Urban Schools* (Washington, DC:
    Brookings, 2002), pp. 65-80; John F. Witte, *The Market Approach to
    Education* (Princeton, 2000), pp. 60-61.

[^69]: Henry M. Levin, *Privatizing Education: Can the Marketplace
    Deliver Choice, Efficiency, Equity, and Social Cohesion?* (Boulder,
    CO: Westview, 2001), 6.

[^70]: See especially Caroline M. Hoxby, "School Choice and School
    Competition: Evidence from the United States," *Swedish Economic
    Policy Review* 10 (2003): 11-67.

[^71]: For example, see David L. Armour and Brett M. Peiser,
    "Interdistrict Choice in Massachusetts," in Paul E. Peterson and
    Bryan C. Hassel, eds., *Learning From School Choice* (Washington,
    DC: Brookings, 1998), 157-186.

[^72]: The list of schools was obtained from the web sites of DCPS and
    the DC Public Charter School Board.

[^73]: Students using DC Opportunity Scholarships at private schools
    include both students who transferred to those schools from public
    schools and students who were already attending the private school
    before they applied for and received a scholarship through the
    lottery.

[^74]: We define a minimum detectable effect as the smallest true
    program impact that would have an 80 percent chance of being
    detected (have 80 percent power) using a two-tail hypothesis test at
    the 0.05 level of statistical significance. We use a two-tail test
    because it is conceivable that the scholarship program could have
    either a negative or positive effect on test scores, even though the
    policy question is about improved test scores. Clustering of
    students may also affect the sample size calculations, however the
    extent of that clustering is unknown at this time. When the data is
    analyzed, it may be necessary to increase the sample needed or the
    size of the effect that can be detected.

[^75]: A number of prominent evaluations of educational interventions
    have reported statistically significant impacts near or within the
    range of 0.15 to 0.20 standard deviations over one to three years.
    Such impacts are generally characterized as moderate in size and
    policy relevant. For examples, see Howell et al., *The Education
    Gap,* 151; Alan B. Krueger, "Experimental Estimates of Education
    Production Functions," *Quarterly Journal of Economics* 114, no. 2,
    (May 1999): 525; Ann Flanagan, Jennifer Kawata, and Stephanie
    Williamson, *Improving Student Achievement: What NAEP Test Scores
    Tell Us* (Santa Monica, CA: RAND Corporation, 2000), 59.

[^76]: See Patrick J. Wolf, Paul E. Peterson, and Martin R. West,
    *Results of a School Voucher Experiment: The Case of Washington,
    D.C. After Two Years,* Paper delivered at the National Center for
    Education Statistics 2001 Data Conference, Mayflower Hotel,
    Washington, D.C., July 25-27.

[^77]: Many of the names of eligible applicants are subject to
    alternative spellings and the birth date variable was missing some
    data and included some entries that did not appear to be valid.