![](media/image1.png){width="8.604166666666666in"
height="11.134722222222223in"}

## Study of the  {#study-of-the .unnumbered}

## Voluntary Public School Choice Program {#voluntary-public-school-choice-program .unnumbered}

## Final Report {#final-report .unnumbered}

##  {#section .unnumbered}

For

U.S. Department of Education

Office of Planning, Evaluation and Policy Development

Prepared by

Robert K. Yin

Pirkko S. Ahonen

COSMOS Corporation

Bethesda, Md.

2008

This report was prepared for the U.S. Department of Education under
Contract Number ED04CO0047 (Task Order No. 001) with COSMOS Corporation.
Reeba Daniel served as the contracting officer's representative. The
views expressed herein do not necessarily represent the positions or
policies of the U.S. Department of Education. No official endorsement by
the U.S. Department of Education is intended or should be inferred.

**U.S. Department of Education**

Margaret Spellings

*Secretary*

**Office of Planning, Evaluation and Policy Development**

Bill Evers

*Assistant Secretary*

**Policy and Program Studies Service**

Alan Ginsburg

*Director*

**Program and Analytic Studies Division**

David Goodwin

*Director*

November 2008

This report is in the public domain, except for the photograph on the
front cover, which is used with permission and copyright, 2007, Getty
Images. Authorization to reproduce the report in whole or in part is
granted. While permission to reprint this publication is not necessary,
the suggested citation is: U.S. Department of Education, Office of
Planning, Evaluation and Policy Development, Policy and Program Studies
Service, *Study of the Voluntary Public School Choice Program: Final
Report*, Washington, D.C., 2008.

This report is available on the Department's Web site at
http://www.ed.gov/about/offices/list/opepd/ppss/reports.html.

On request, this publication is available in alternate formats, such as
Braille, large print, or computer diskette. For more information, please
contact the Department's Alternate Format Center at 202-260-0852 or
202-260-0818.

**Contents**

**Page**

**LIST OF EXHIBITS** iv

**ACKNOWLEDGMENTS** vii

**EXECUTIVE SUMMARY** ix

**1.** The Voluntary Public School Choice (VPSC) Program and Its
Evaluation 1

**2.** Summary of Evaluation Methodology 5

> 2.1 Evaluation Design for a Program Evaluation 5
>
> 2.2 Data Collection 6
>
> 2.3 Analytic Priorities 8

**3.** Characteristics of the VPSC Program's Sites 11

> 3.1 Overview of VPSC Sites 11
>
> 3.2 Four Types of Choice Arrangements at the VPSC Sites 14
>
> 3.3 Core Activities in the VPSC Choice Initiatives 17
>
> 3.4 Initial Implications for Federal Education Policy and
>
> Local Education Practice 23

**4.** Promoting Educational Equity and Excellence 27

> 4.1 Student Participation at the VPSC Sites, 2002--07:
>
> Eligibles, Applicants, and Enrollees 27
>
> 4.2 Progress on Program Priorities 31
>
> 4.3 Implications for Federal Education Policy and Local Education
> Practice:
>
> Further Issues of Educational Equity and Excellence 37

**5.** Student Academic Achievement Trends Concurrent with the VPSC
Initiatives 41

> 5.1 Collection of Individual-Level Student Data 42
>
> 5.2 Methodology for Analyzing Student Achievement Trends 46
>
> 5.3 Findings on Student Achievement Trends 49

**6.** Useful Choice Practices and Overall Lessons Learned about\
Voluntary Public School Choice 55

> 6.1 Useful Choice Practices for Future Choice Initiatives 55
>
> 6.2 Overall Lessons Learned from the National Evaluation of the VPSC
> Program 58
>
> 6.3 Two Challenges for Improving Voluntary Public School Choice 61

**REFERENCES** 63

**APPENDIXES**

**Page**

A. Detailed Evaluation Methodology A-1

B. Instruments Used in the National Evaluation B-1

C. Supplemental Data about Individual VPSC Sites C-1

**EXHIBITS**

1-1 Location of the VPSC Grantees (2002 Awards) 2

2-1 Other Public School Choice Options at the VPSC Sites 7

3-1 Community Characteristics 12

3-2 Brief Descriptions of the VPSC Sites' Choice Initiatives 13

3-3 Grade Span Distribution of Participating Schools 15

3-4 Four Types of Choice Arrangements 15

3-5 Parent Involvement Activities Reported by School Officials in School
Survey 18

3-6 Schools' Reported Efforts to Notify Parents of Their Choice Options,
2006--07 19

3-7 Parents' and Families' Understanding of Choice

Reported by School Officials, 2006--07 19

3-8 Capacity-Enhancing Activities 21

3-9 Illustrative Curriculum Topics at the VPSC Sites 22

3-10 Variations in Sites' Uses of VPSC Funds to Support Choice Options
26

4-1 Student Participation in the VPSC's Initiatives, 2005--06 28

4-2 Participation Rates, 2003--07 30

4-3 Demographic Characteristics of Schools, 2006--07 33

4-4 Reported AYP Status of Schools, 2004--05 through 2006--07 34

4-5 Student Transfers from Low- to Higher-Performing Schools, 2005--06
35

4-6 Eligibility to Participate in VPSC's Choice Arrangements

An Equity Perspective 39

5-1 Desired Student Record 42

5-2 Individual-level Data Submitted by VPSC Sites 45

5-3 Baseline Characteristics of VPSC Enrollees and Matched Comparison
Students 50

5-4 Performance of VPSC and Matched Comparison Students,

Analyzed Separately 51

5-5 Aggregate Comparison of VPSC and Matched Comparison Groups,

Across Cohorts 52

6-1 Potentially Useful Choice Practices from the VPSC Program 56

A-1 Schedule of VPSC Field Visits A-6

A-2 VPSC School Survey: Number of Schools Surveyed and\
Response Rate A-8

C-1 State Education Agencies Used VPSC Funds to

> Support Widely Different Choice Priorities C-3

C-2 VPSC Initiatives Coincided With Existing, Districtwide Choice
Options C-3

C-3 Sites Split Their VPSC Funds to Support a Variety of Activities,

Not Limited to a Single Choice Initiative C-4

C-4 VPSC Initiatives Could Be Separate From or Coincide With Title I
Choice C-4

C-5 Five Years of VPSC Participants C-5

C-6 Choice Initiatives Limited to Zones Within a District

Nevertheless Involved a Broad Array of Sending Schools C-6

C-7 Eligible Sending Schools Were Restricted,\
but Not Necessarily to Low-Performing Schools C-6

C-8 Capacity-building Occurred at Receiving Schools

That Already Were (Presumably) Higher-performing Schools C-6

C-9 Local Sites May Need to Focus on a "Tapestry" of Choice\
Options, Rather Than on Any Single Initiative C-7

C-10 "Attractor" Programs at Receiving Schools May Not\
Have Been Needed to Attract Incoming Students C-7

C-11 Explicit Efforts May Be Needed to Maintain the

> Performance of Receiving Schools C-7

**ACKNOWLEDGMENTS**

This report has benefited from the advice and comments of many expert
reviewers, who have given generously of their time. A Technical Working
Group (TWG) met four times during the course of the project (March 2003,
July 2005, April 2006, and December 2007), each time providing helpful
feedback on the evaluation's progress. At its final meeting, the TWG
discussed an earlier version of this final report. The following is a
list of the TWG members:

Frank Brown, professor of education, University of North Carolina

Peter Cookson, dean of graduate school, Lewis and Clark College

> David Heistad, executive director, Testing, Evaluation, and Student
> Information, Minneapolis Public Schools

Valerie Lee, professor of education, University of Michigan

Janelle T. Scott, assistant professor of education, New York University

> Paul Teske, dean, Graduate School of Public Affairs, University of
> Colorado at Denver
>
> Patrick Wolf, professor and endowed chair in school choice, University
> of Arkansas

Todd Ziebarth, senior policy analyst, National Alliance for Public
Charter Schools

Patrick McKnight of George Mason University's Department of Psychology
conducted the analyses described in section 3 of appendix A. These
analyses support the findings in chapter 5. In addition, Clive Belfield
of the Teachers College at Columbia University served as an independent
consultant throughout the entire project, reviewing drafts of reports,
including this final report. The final report also benefited from
reviews of earlier versions by David Goodwin, Daphne Kaplan, and Reeba
Daniel at the U.S. Department of Education (the Department), along with
comments by two anonymous reviewers.

The bulk of the data came from the 13 sites, with Iris Lane and John
Fiegel of the Department's VPSC Program Office graciously facilitating
the process. Without the cooperation of all these parties, the
evaluation could not have been conducted.

All this assistance notwithstanding, the authors are responsible for
this final report and any errors it may contain.

**EXECUTIVE SUMMARY**

**The Voluntary Public School Choice (VPSC) Program**

**And Its Evaluation**

The *No Child Left Behind Act of 2001 (NCLB)*[^1] expanded public school
choice opportunities for students. First, the new accountability
requirements in Title I of the *Elementary and Secondary Education Act
(ESEA)* (Title I, Section 1116(b)) required districts to offer public
school choice to students in Title I schools that are identified for
improvement, corrective action, or restructuring as a result of not
meeting state definitions for adequate yearly progress (U.S. Department
of Education, 2004).[^2]

Second, the law created the VPSC Program (Title V, Subpart 3, Section
5241 of the *ESEA*), which is the subject of the current report. The
program supports the emergence and growth of choice initiatives across
the country, by assisting states and local school districts in
developing innovative strategies to expand choice options for students.

The VPSC Program functions independently from the choice provisions in
Title I and provides funds to a relatively small number of sites across
the country. In October 2002, the U.S. Department of Education (the
Department) awarded five-year competitive grants to 13 applicants.
Awards ranged in size from \$2.8 million to \$17.8 million for an
average award of \$9.2 million for five years, or approximately \$1.8
million per year. The VPSC-funded locations included: the state of
Arkansas; Albany, N.Y.; Chicago, Ill.; the state of Florida; Hartsdale,
N.Y.; Hillsborough County, Fla.; La Quinta, Calif.; Miami, Fla.; the
state of Minnesota; Swanzey, N.H.; New Haven, Conn.; Portland, Oreg.;
and Rockford, Ill.

***This report contains the final assessment of the first five years of
the VPSC Program (2002***--***07).*** The evaluation was charged with
assessing the VPSC Program's progress in meeting the goals and
fulfilling the intent of the original legislation. The evaluation
addressed three central questions:

1\) What are the characteristics of the VPSC Program's sites?

2\) How and to what extent does the VPSC Program promote educational

> equity and excellence?

3\) What academic achievement is associated with the VPSC Program?

To address the third question, only a few of the VPSC sites provided the
needed data. As a result, the evaluation's findings for this third
question may come from too small a sample of sites or of VPSC choice
enrollees to cover the VPSC Program's experience adequately.

In addition to the three central questions, the analysis also focused on
four priorities in the original VPSC legislation: a) providing the
widest possible choice to students in participating schools; b)
promoting transfers of students from low- to higher-performing schools;
c) forming interdistrict partnerships to allow students to transfer to a
school in another district from that of their original school; and d)
requiring sites to use funds to support transportation services for
students (on the assumption that this would allow students to attend
more attractive but also more distant schools).

***The evaluation covers choice initiatives at 13 sites that received
VPSC grant awards from the U.S. Department of Education in the fall of
2002.*** The 2002 grantees were nine districts, three state education
agencies, and one nonprofit organization.[^3] However, the findings of
this report draw conclusions about the VPSC Program as a whole, not the
activities at any given VPSC site. At the same time, the evaluation does
not cover a second round of 14 five-year grants made by the VPSC Program
in the summer of 2007.

The relevant evaluation data came from several sources: multiple site
visits to the VPSC sites; three rounds of surveys covering an average of
50 participating schools at each VPSC site; reviews of grantees' annual
reports; and the collection of archival data about student performance.
The archival data tracked individual student achievement trends,
covering one or more academic years prior to the start of a VPSC
initiative and as many academic years as possible during the
implementation of the initiative. The trends were then compared to those
for a group of students not enrolled in a VPSC choice initiative.

**Brief Summary of Key Findings**

Briefly summarized, at the end of five years, the VPSC Program, though
limited to grant awards at only 13 sites, showed a variety of different
public school choice arrangements working in a diverse array of
communities across the country. Among the four legislative priorities,

-   The VPSC Program made progress on the first priority in providing
    the widest variety of choice.

-   Transfers from low- to higher-performing schools comprised only a
    portion of the students enrolled in the VPSC initiatives, with only
    three of the 13 sites limiting their enrollment to transfers from
    low- to higher-performing schools.

-   Most of the VPSC sites limited their choice initiatives to
    within-district options, rather than developing interdistrict
    options.

-   Relative to enrollment, transportation costs did not increase
    proportionately as might have been expected, because the VPSC
    initiatives permitted many students who were already attending
    distant schools to select schools closer to home.

Regarding student achievement trends, data from a highly limited set of
VPSC sites and students enrolling in their choice initiatives showed
improved trends in math and reading, compared to matched groups of
non-enrolling students.

The gains were statistically significant, but the findings need to be
tempered by several cautions:

-   the voluntary nature of choosing to enroll in a VPSC initiative and
    the possibility that the VPSC enrollees were more highly motivated
    students than non-enrollees, thereby accounting for some or all of
    the differences in student achievement;

-   the procedure for selecting the matched groups of non-enrolling
    students;

-   the length of time covered by the trends (more annual data points
    came from the years before rather than after enrollment); and

-   the fact that the data came from only four of the 13 VPSC sites and
    only six of 38 cohorts of annual enrollees across all of the VPSC
    sites.

Given all the cautions, the findings in this analysis of six cohorts
from four VPSC sites offer early promise regarding the potential
benefits of the VPSC Program.

**Overview of the VPSC's Sites**

The 13 VPSC sites were located across the country. Ten were located in
predominantly urban communities, two in areas that cover both urban and
rural regions, and one in an entirely rural area.

Participating schools covered all grade levels: 64 percent spanned the
elementary grades, 15 percent the middle school grades, 11 percent the
high school grades, and 10 percent were classified as "other" or had
missing data. As a comparison, approximately 58 percent of schools
nationwide are elementary schools, 17 percent are middle schools, 19
percent are high schools, and 6 percent are "other."

***The VPSC\'s choice initiatives varied in the timing of their
implementation.*** Some of the sites started enrolling choice students
as early as January 2003, while other sites started enrollment the
following year or even later. Because all sites began some type of
planning if not enrollment activity in 2002--03, the school year
2001--02 is considered the base year prior to the implementation of VPSC
initiatives. The evaluation traced the sites' activities for the
duration of their five-year awards through 2006--07, although the
majority of sites received no-cost extensions to support their
operations for another year.

Once they started enrollment, the VPSC sites offered choice options to a
new set of students every year. Therefore, for each cohort of
"first-time enrollees," the "intervention period" varied. At the student
level, this variation was taken into account by defining different base
years for each cohort of students. A consequence of this staggered
pattern is that more years of annual data were available for the older
cohorts than for the younger ones.

***As with many other school districts in the country, nearly all of the
13 sites already had a broad variety of public school options prior to
the VPSC Program.*** The program permitted sites to enhance or expand
existing options, and sites did not need to start entirely new
initiatives. Sites tried to make their entire array of choice options
work well, but they did not necessarily track students' participation on
an option-by-option basis. Acknowledging this complication also is
necessary for interpreting the findings in this report. For some of the
sites, the VPSC funds were only part of the support for the identified
initiative. As a result, the contribution of the VPSC Program may have
been overestimated. At other sites, the VPSC funds not only supported
the identified initiative but also partially supported other choice
options at the same site. As a result, the contribution of the VPSC
funds may have been underestimated. Unfortunately, the analysis was not
able to distinguish the extent of these over- or underestimations.

The VPSC Program also started in the same year that federal legislation
expanded support for Title I choice options. The legislation allowed
spending up to 20 percent of Title I funds to support transportation
costs for students transferring away from schools designated under Title
I as "identified for improvement." Three of the 13 sites defined their
VPSC initiatives to coincide or overlap closely with their Title I
choice options. However, most VPSC sites defined schools participating
in their VPSC initiatives as a broader set of schools than simply those
designated under Title I as identified for improvement.

**Four Types of Choice Arrangements at the VPSC Sites**

The sites varied greatly in the design of their choice initiatives,
differing in the number of students served, the number of participating
public schools, and the capacity to accommodate transfers. In addition,
the sites differed by how they defined choice zones and managed the flow
of students among participating schools. Despite this variation, sites
also pursued some common paths.

First, although unique, the VPSC's choice initiatives tended to fall
under four major categories, based on how sites defined choice
arrangements and directed the flow of transferring students:

-   Five of the sites designated specific schools to be either sending
    schools or receiving schools but not both.

-   Five sites defined initiatives whereby the same schools could be
    both sending and receiving.

-   One site established a within-school initiative, in which students
    chose from education programs within the same school and did not
    transfer between schools.

-   Two sites had initiatives involving a mixture of the first three
    types.

Second, all sites focused on two core activities throughout the
implementation process: 1) engaging parents and community members, and
2) building capacity at schools to attract and accommodate choice
transfers.

Parent and community activities included a rich array of outreach,
marketing, and communication efforts. Sites also engaged parents and
community representatives in developing and implementing choice
initiatives.

Capacity-enhancing activities included starting new academic programs or
subjects, purchasing supplies and equipment for schools, and providing
professional development to teaching staff. However, the sites'
capacity-enhancing activities were not necessarily accompanied by an
expansion of seats or classrooms at many of the receiving schools. For
instance, none of the sites reported hiring more teaching staff or
taking other steps simply to expand the number of seats to accommodate
higher enrollments at the existing schools.

**Student Participation at the VPSC Sites**

***In 2005--06, across 12 of the 13 VPSC sites, 24,921 students enrolled
in choice initiatives, reflecting an overall participation rate of 2.8
percent of the students eligible to enroll.*** This number of students
represented those who had enrolled for the first time that year
("first-time enrollees"). The cumulative first-time enrollees, starting
from the inception of the VPSC Program in 2002--03 until 2005--06,
included 49,616 students. Thus, the most accurate estimate of
participation falls somewhere between 24,921 and 49,616. (However, the
cumulative enrollment overestimates the total enrollment because of an
attrition factor that was not tracked by the sites.)

***Overall, the number of enrolling students in the VPSC Program
increased during the earlier years of the program but declined in the
program's fifth year.*** Of the VPSC sites' enrolling students, ten
provided eligibility and enrollment data for four consecutive years
(2003--04 to 2006--07). These data permitted the estimation of trends
through the fifth VPSC year (2006--07). The yearly data captured the
number of first-time (i.e., new) enrollees each year, not the total
number across all years. For these "first-time enrollees," the trends
showed that the VPSC Program at first averaged 696 enrollees per site in
2003--04, then reached a peak of 2,459 per site in 2005--06, and then
declined to 2,167 per site in 2006--07.

The decline may have reflected the actual saturation of the VPSC
initiatives because a good (but unmeasured) portion of the earlier
years' enrollees still remained enrolled in the later years, possibly
limiting the seats available for first-time enrollees in the final year.

***Participation rates also showed the same pattern, increasing from
2003--04 to 2005--06, and then declining in 2006--07.*** For the same
ten sites reporting data from 2003--04 to 2006--07, the participation
rates by first-time enrollees also increased from 1.5 percent to 4.1
percent, from 2003--04 to 2005--06, and then dropped to 3.0 percent in
2006--07.

**Progress on Program Priorities**

The VPSC legislation had four program priorities. Three priorities were:
to provide the widest variety of choices; to encourage transfers from
low- to higher-performing schools; and to provide opportunities for
students to transfer to schools outside of their home districts. The
fourth priority directed sites to use some of their VPSC funds to
support transportation services and costs. Findings on these four
priorities are discussed next.

***The VPSC Program made progress on the first priority of providing the
widest variety of choice.*** Sites expanded the assortment of choice
options in participating schools and offered a diverse number of
academic programs to transferring students. The VPSC initiatives also
made efforts, through media campaigns and related activities, to
increase parents' awareness of the variety of education options
available to them.

***Transfers from low- to higher-performing schools comprised only a
portion of the students enrolled in VPSC initiatives*.** Only three VPSC
sites limited their enrollment to transfers from low- to
higher-performing schools. Another five sites permitted transfers within
an entire district or zone, but only two of the sites tracked the
portion of the transfers from low- to higher-performing schools. Aside
from the five sites that either limited or tracked their transfers from
low- to higher-performing schools, none of the other eight sites could
provide such information. At the five sites, the confirmed transfers
from low- to higher-performing schools represented 1,295 of 5,927, or
21.8 percent, of their total transfers from 2005--06.

The other eight VPSC sites either permitted a wider variety of transfers
or had VPSC enrollments involving no transfers. Among the sites not
tracking the low- to higher-performing transfers, one received a waiver
from the Department to omit such tracking because all of the site's
enrollees were low-performing students (scoring "below proficient" on
the state assessment), but they were not necessarily transferring from
low- to higher-performing schools. Thus, across the entire VPSC Program,
the actual portion of low- to higher-performing transfers could be
larger or smaller, depending on the number of transfers at the eight
sites that did not track or document the pattern of their transfers.

***Most of the VPSC sites limited their choice initiatives to
within-district options, rather than developing interdistrict
partnerships.*** Although the implementation of formal interdistrict
choice options might have expanded the variety of choices available to
students even further, only five of the 13 VPSC sites created formal
interdistrict options. At the eight remaining sites, many had existing
transfer agreements with neighboring districts. However, these transfer
options were separate from VPSC initiatives and were generally reviewed
on a case-by-case basis. The majority of the VPSC initiatives did not
directly promote student transfers to other school districts, and VPSC
funds were not used to support interdistrict transfers at these sites.

***Relative to enrollment, transportation costs did not increase
proportionately as might have been expected.*** This is partly because
the VPSC initiatives permitted many students who were already attending
distant schools to select schools closer to home. As an example, two of
the VPSC sites had recently emerged from court-ordered school busing,
from which students had been assigned to more distant schools as part of
the original desegregation order. The VPSC-funded initiative gave
affected students the choice of enrolling at neighborhood schools. Under
these circumstances, sites experienced minimal or reduced transportation
costs.

**Findings on Student Achievement Trends**

The final student achievement trends came from six cohorts of VPSC
enrollees and six corresponding groups of non-VPSC enrollees, across
four of the 13 VPSC sites. The cross-site analysis first estimated the
student achievement trends for these two groups separately and then
compared the trends between the two groups.

These two analyses served two different purposes. The first, estimating
trends separately, was needed to establish whether either the enrollee
or matched comparison group was alone moving in a positive or negative
direction. The goal was to determine whether the enrollee group might
have been performing worse, regardless of any relative difference
between it and the comparison students. The second, estimating trends
relative to each other, then captured the comparison between the two
groups. (The two analyses involved two different units of analysis,
students and cohorts, and the values should not be compared across these
two analyses.)

***When the VPSC and non-VPSC trends were examined separately, the VPSC
enrollees' trends were neutral in math and positive in reading, but not
statistically significant.*** More important, the enrollee group's
scores were not found to be declining in any way. In contrast, the
non-enrollee group showed a declining trend in math proficiency that was
statistically significant at the 99 percent confidence level.[^4] The
non-enrollees' trends for reading were positive but not significant.

***When the VPSC and non-VPSC trends were compared, students enrolling
in the VPSC initiatives had better student achievement trends than those
not enrolling.*** The enrollees surpassed the non-enrollees in the
trends for both mathematics and reading, with the differences in
mathematics being statistically significant at the 99.9 percent
confidence level and in reading at the 99 percent level. The comparison
was based on a meta-analysis of effect sizes across all six of the
cohorts, with the individual effect sizes for each cohort having already
accounted for the demographic and baseline differences between the VPSC
and non-VPSC groups. Because the units of measure were standardized
scores, the trends cannot easily be translated into everyday educational
units, but the effect sizes (.020 and .028 for mathematics and reading
respectively) appear to be modest.

These findings need to be accompanied by several cautions about
interpreting the differences in achievement trends between the enrolling
and non-enrolling groups. The main cautions are as follows.

First, in all of the choice initiatives, students can choose to enroll
or not. As a result, an enrollee group may represent more highly
motivated students or differ in other unobserved ways from a
non-enrollee group, accounting for some or all of any subsequent
differences in student achievement.[^5]

Second, the trends in the present analysis leaned in the direction of
having more of the data points precede rather than follow enrollment in
the VPSC initiative. Confidence about the trends concurrent with a VPSC
initiative, much less inferences about any effects, would be increased
if analyses were based on a larger number of data points *following*
enrollment.

Third, the data only came from a small sample of sites (four of 13) and
from only a small sample of the enrolling students (six of 38 cohorts of
annual enrollees across all of the VPSC sites by 2005-06). Data from
more sites and cohorts would produce a firmer set of findings about the
program as a whole. The ongoing VPSC Program can put renewed emphasis on
obtaining such data, given the new round of five-year awards made in
2007. The possibilities of such additional data are especially strong,
given that seven of the newly awarded sites were continuations from the
first round of awards.

Given the cautions, the findings in this analysis of six cohorts from
four VPSC sites offer early promise regarding the potential benefits of
the VPSC Program.

Recommendations for improving future evaluations as the VPSC Program
progresses with its 2007 cohort of grantees are detailed at the end of
the full report. Aside from taking steps to overcome the earlier
cautions about interpreting student achievement findings, two specific
procedures could improve the robustness of future analyses:

-   *Careful recordkeeping and tracking of all participants.* Sites
    should keep accurate and aggregate counts of all three types of
    choice participants (eligibles, applicants, and enrollees) and also
    should compile individual student-level data for choice enrollees'
    demographic characteristics and student achievement scores.

-   *A modified procedure for defining comparison students.* Sites
    should not try to match any particular group of non-enrolling
    students, as the sites did in the current evaluation. Rather, the
    more desirable procedure would be for sites to provide data from a
    larger but nonselected set of non-enrolling students, such as those
    students remaining in the sending schools or even a districtwide set
    of students. Analytic procedures such as propensity score matching
    or some similar procedure conducted by the evaluation team could
    then provide a fairer selection of matched comparison group than the
    sites' procedures in the current evaluation.

# 1. THE VOLUNTARY PUBLIC SCHOOL CHOICE (VPSC) PROGRAM

# AND ITS EVALUATION

The *No Child Left Behind Act of 2001* (*NCLB*)[^6] expanded public
school choice opportunities for students. First, the new accountability
requirements in Title I of the *Elementary and Secondary Education Act*
(*ESEA*) (Title I, Section 1116(b)) required districts to offer public
school choice to students in Title I schools that are identified for
improvement, corrective action, or restructuring as a result of not
meeting state definitions for adequate yearly progress (U.S. Department
of Education, 2004).[^7]

Second, and the subject of the present report, Congress created the VPSC
Program (Title V, Subpart 3, Section 5241 of the *ESEA*) to support the
emergence and growth of choice initiatives across the country. The
purpose of the program is to assist state and local school districts in
the development of innovative strategies to expand options for students,
and to encourage transfers of students from low- to higher-performing
schools.

The VPSC Program functions independently from the choice provisions in
Title I and provides funds to a relatively small number of sites across
the country. The program has made two rounds of awards, only the first
of which is the subject of the present evaluation. The first round
occurred in the fall of 2002, when the U.S. Department of Education (the
Department) awarded five-year grants to 13 applicants (see exhibit 1-1).
The second round occurred in the summer of 2007, when the Department
made another set of five-year grants to 14 applicants, seven of which
had received awards in the first round.

The VPSC Program was designed and initiated against a backdrop of
increasing interest in public school choice. Such initiatives give
students the option of enrolling at a public school other than the one
to which they are assigned as a result of their residential location.
Their choices can include: all the other public schools in a system;
some of the schools (e.g., schools in a geographic region within the
system); or specific schools (e.g., magnet schools, charter schools, or
schools identified under Title I choice). Despite the variation in
options, all choice is limited to public schools only.

The evaluation of the VPSC Program, focusing on the first round of
awards only, addressed three questions to assess the VPSC Program's
progress in meeting the goals and fulfilling the intent of the original
legislation. The three central questions Congress asked were:

+-----------------------------------------------------------------------+
| **Exhibit 1-1**                                                       |
|                                                                       |
| **Location of the VPSC Grantees**                                     |
|                                                                       |
| **(2002 Awards)**                                                     |
|                                                                       |
| ![](media/image2.pct){width="5.996527777777778in" height="3.98125in"} |
|                                                                       |
| Exhibit Reads: One of the 13 VPSC grantees, Portland Public Schools   |
| in Oregon, received a five-year VPSC award of \$6,467,122.            |
+-----------------------------------------------------------------------+

> 1\) What are the characteristics of the VPSC Program's sites?
>
> 2\) How and to what extent does the VPSC Program promote educational
> equity and excellence?
>
> 3\) What academic achievement is associated with the VPSC Program?

Question one relates to basic descriptive information about the program
sites and their implementation strategies, including activities related
to community outreach and capacity-building within participating
schools.

Question two relates to the extent and nature of student participation
in the choice initiatives funded by the program, as well as sites'
pursuit of the stated goals of the VPSC legislation. The legislation
stipulated four priorities in the selection of sites:

> a\) Provide the widest possible choice to students in participating
> schools;
>
> b\) Promote transfers of students from low-performing to
> higher-performing
>
> schools;
>
> c\) Include interdistrict partnerships to allow students to transfer
> to a school
>
> in another district from that of their original school; and
>
> d\) Require sites to use funds to support transportation services for
> students (on the assumption that this would allow students to attend
> more attractive but also more distant schools).

Question three pertains to Congress's interest in having the evaluation
investigate the achievement outcomes associated with the VPSC Program.
It asks whether students who participated in the VPSC Program had
greater academic outcomes in reading and math compared to similar
students that had not participated in the program.

This report covers the three evaluation questions and related topics.
After a brief description of the methodology in chapter 2, the report is
organized as follows:

Chapter 3 addresses the first evaluation question by providing a
detailed analysis of the characteristics of the VPSC Program's grantees.
This chapter also categorizes the sites by type of choice arrangement,
documents important program practices, and describes other choice
initiatives at the VPSC sites.

Chapter 4 addresses the second evaluation question and discusses how and
to what extent the program has promoted educational equity and
excellence. The chapter presents trends in eligibility, applicants, and
enrollees at the sites. This chapter also examines the degree to which
the program has provided students with: a wide variety of choice;
opportunities to transfer from low- to higher-performing schools;
interdistrict choice options; and transportation support.

Chapter 5 addresses the third evaluation question by presenting and
discussing student achievement trends concurrent with the VPSC Program.

Finally, Chapter 6 of the report identifies useful choice practices
emerging from the VPSC Program and discusses implications for the
program and future research.

**2. SUMMARY OF EVALUATION METHODOLOGY**

The purpose of the national evaluation was to assess the experience of
the VPSC Program. The evaluation followed a mixed quantitative and
qualitative methods research design, with data coming from a variety of
original and archival sources (for a full description of the evaluation
methodology, see appendix A: *Detailed Evaluation Methodology*).

**2.1 Evaluation Design for a Program Evaluation**

The evaluation covered the 13 sites that received VPSC awards from the
Department in the fall of 2002. However, the analysis and findings in
this report aimed at drawing conclusions about the VPSC Program as a
whole, not the activities at any given VPSC site.[^8]

Because choice initiatives can involve schools and students in a variety
of roles, the evaluation defined the schools' roles in the following
manner:

> \(a\) *Sending* schools: schools from which students transfer;
>
> \(b\) *Receiving* schools: schools to which students transfer;
>
> \(c\) *Both sending and receiving*: schools that have students both
> transferring in and out as part of the same choice initiative; and
>
> \(d\) *Within-school* initiatives: initiatives with no inter-school
> transfers, because students choose among different education programs
> within the same school.

Similarly, the evaluation defined the students' roles in the following
manner:

> \(a\) *Eligible students*: all students who could potentially
> participate in a VPSC initiative;
>
> \(b\) *Applicants*: the set of eligible students who applied either to
> attend another public school or to participate in an academic program
> within their original school; and
>
> \(c\) *Enrollees*: those students who successfully applied and
> enrolled in a school or program as a result of a VPSC initiative.

The evaluation accounted for the varied timing of the VPSC's choice
initiatives in the following manner. After receiving their five-year
awards in the fall of 2002, some of the sites started enrolling choice
students as early as January 2003. Other sites carried out planning
activities first and then started enrollments the following year or
later. Because all sites began some type of activity in 2002--03, the
school year 2001--02 is considered the base year prior to the
implementation of the VPSC initiatives. The evaluation traced the sites'
activities for the duration of their five-year awards, through 2006--07.

Sites also offered choice options to a new set of students every year.
Therefore, for each cohort of students, the "intervention period"
varied. At the student level, this variation was taken into account by
defining different base years for each cohort of students. A consequence
of this staggered pattern is that more years of annual data were
available for the older cohorts than for the younger ones.

Finally, the evaluation design called for the collection of data from
comparison groups for every VPSC site. The groups mainly consisted of
students who were not enrolled in the VPSC initiatives but who had
similar demographic and academic characteristics as the enrolling
students. The VPSC sites provided data about these comparison groups.
(See chapter 5 for additional details.)

From a design standpoint, two complications diminished the evaluation's
ability to assess the VPSC Program. First, all of the sites already had
one or more choice options ongoing at the onset of the VPSC initiative.
These other options included magnet and charter schools, Title I public
school choice, and a variety of other choice options offered by the
district or state. Overall, even when sites used VPSC funds exclusively
to support a new VPSC Program initiative, the sites had other unrelated
choice options operating at the same time (see exhibit 2-1).

Second, the VPSC Program permitted sites to expand existing arrangements
rather than to establish new ones. The majority of the sites chose to
expand existing initiatives. Separating the VPSC-funded portion of a
choice initiative from other aspects of the initiative has not been
easily delineated.

**2.2 Data Collection**

The data used in this evaluation came from a variety of sources. These
included: multiple site visits to the VPSC sites; surveys covering an
average of 50 participating schools at each VPSC site; and the
collection of archival data about student achievement as well as reviews
of the grantees' reports to the Department.

The team conducted three rounds of site visits to every grantee site.
The site visits covered the VPSC project site as well as one sending
school and one receiving school (or two participating schools at those
sites not designating specific sending and receiving

+-----------------------------------------------------------------------+
| **Exhibit 2-1**                                                       |
|                                                                       |
| **Other Public School Choice Options at the VPSC Sites**              |
|                                                                       |
| +----------------------------+-------------------+----------------+   |
| | **Other choice options**   | **Number of sites |                |   |
| |                            | with other public |                |   |
| |                            | school choice     |                |   |
| |                            | options in        |                |   |
| |                            | addition\         |                |   |
| |                            | to the            |                |   |
| |                            | VPSC-funded       |                |   |
| |                            | initiative**\*    |                |   |
| +----------------------------+-------------------+----------------+   |
| | ##### **Magnet schools**   | 13                |                |   |
| +----------------------------+-------------------+----------------+   |
| | **Charter schools**        | 11                |                |   |
| +----------------------------+-------------------+----------------+   |
| | **Title I choice**         | 11                |                |   |
| +----------------------------+-------------------+----------------+   |
| | **Interdistrict options**  | 9                 |                |   |
| +----------------------------+-------------------+----------------+   |
| | **Other district options** | 9                 |                |   |
| +----------------------------+-------------------+----------------+   |
| | **Other state options**    | 5                 |                |   |
| +----------------------------+-------------------+----------------+   |
| | **Total for all 13 sites** | 58                |                |   |
| +----------------------------+-------------------+----------------+   |
| | **Average for each site**  | 4 or more\*\*     |                |   |
| +----------------------------+-------------------+----------------+   |
|                                                                       |
| Exhibit Reads: All 13 sites had magnet schools in addition to the     |
| VPSC-funded initiative.                                               |
|                                                                       |
| \*Individual VPSC sites can appear under more than one type of choice |
| option.                                                               |
|                                                                       |
| \*\*Each of the 13 sites had an average of four or more public school |
| choice options in addition to the VPSC-funded initiative.             |
|                                                                       |
| Sources: Analysis of site visit data and Grant Performance Reports,   |
| by COSMOS Corporation, 2007.                                          |
+-----------------------------------------------------------------------+

schools). The sites chose two schools that had a lot of choice activity
(i.e., many transferring students), so that the site visit team could
observe the VPSC initiative in action at the school level.

A two-person team conducted all three rounds of field visits. The team's
data collection activities were guided by field protocols (see appendix
B). These protocols included interviews with the VPSC site's project
director, staff, and other key participants in the choice initiative.
The protocols also called for the collection of documents and archival
data related to the interviews.

The team also conducted three rounds of surveys with participating
schools at VPSC sites. The surveys were brief, mainly gathering data to
corroborate the schools' participation in initiatives but not to
investigate the schools' other conditions in any depth. [^9]

The surveys were based on a closed-ended questionnaire (see appendix B)
sent to school principals. The questionnaire requested information
about: student demographics; school performance; the choice options
available to students; the percentages of students taking part in these
options; the methods by which choice information was being shared with
parents; and whether staff members were receiving professional
development related to school choice.

The survey included all of the schools that were eligible to participate
in the choice initiative. One statewide initiative was an exception: no
schools were surveyed in this initiative; instead, the districts with
five or more students participating in the choice initiative were asked
to complete the survey at the district level.[^10]

In the survey's third and final round, the evaluation team distributed
questionnaires to 689 schools. Respondents at 630 schools completed and
returned the questionnaire, resulting in an overall response rate of 91
percent. The response rates for the earlier two rounds were 75 and 91
percent, respectively (see appendix A, exhibit A-2).

The data collection also included archival data. The data covered
individual student achievement scores for one or more academic years
prior to the start of a VPSC initiative through as many academic years
as possible during the implementation of the initiative. The data
covered both the students enrolled in the VPSC choice initiatives and
matched comparison groups of students.

####### 

The archival data also came from reviews of annual performance reports
(2003--06) submitted by the VPSC grantees to the VPSC Program Office at
the Department. Each year, the office issues reporting requirements,
based in part on the data collection suggestions by the national
evaluation team. However, the review did not include the grantees' final
performance reports, which were to be submitted after the conclusion of
the national evaluation.

**2.3 Analytic Priorities**

**2.3.1 Cross-site Analysis**

The general analytic strategy in dealing with evaluation data was to
examine the extent of any relationship between a VPSC initiative and
concurrent educational outcomes, including equity, excellence, and
student achievement.

The analysis began by focusing on within-site conditions. A separate
database, consisting of narrative and numeric material, brought together
all of the data about a single VPSC initiative and addressed such issues
as: (1) distinguishing ongoing choice options from the VPSC initiative
at each site; and (2) calling attention to contextual conditions
potentially related to the VPSC initiative. The creation of this
database was itself a qualitative task and aimed at strengthening the
possible association between VPSC-supported actions at a site and later
outcomes. An important part of this procedure was to search for and
understand the role of possible rival explanations (Yin, 2000). For
instance, ongoing district or school policies apart from the VPSC
initiative may be strongly associated with the observed outcomes.

The primary focus of the national evaluation was then to conduct a
cross-site analysis to arrive at findings for the VPSC Program as a
whole, rather than to assess the accomplishments at any given site. The
cross-site analysis identified patterns across the VPSC sites. For
instance, findings and lessons from the VPSC Program revealed different
types of choice initiatives, with the 13 sites arrayed into subgroups
based on the design of their VPSC-funded choice arrangement (see chapter
3). The subgroups helped provide greater insight into the associated
outcomes from choice initiatives and also served as practical examples
to be considered for implementation by other districts in the future.

The practical examples were part of an effort that came from an
expressed need, by the Department, for the national evaluation to report
new information about useful practices for conducting choice
initiatives. From the national evaluation, such information on choice
initiatives would serve at least two audiences: 1) districts
implementing the choice provisions of Title I, and 2) districts wanting
to start or strengthen their own public school choice initiatives
independent of the Title I requirements.

**2.3.2 Student-level Analysis**

The evaluation collected and analyzed student achievement trends at the
VPSC sites. The evaluation invested extensive efforts in getting sites
to collect the needed student achievement data. Data cleaning and
clarification of the data files submitted by the sites also consumed
time and resources. Chapter 5 describes the methodological techniques
and the findings (also see section 3, "Methods for Analyzing Student
Achievement Data" in appendix A for the detailed analysis plan).

**3. CHARACTERISTICS OF THE VPSC PROGRAM'S SITES**

The Department awarded grants to three types of organizations in the
fall of 2002: 1) nine local or regional school districts; 2) three state
education agencies; and 3) one nonprofit, charter school
organization.[^11] Each type of organization engaged in a unique choice
initiative. Despite this variation, sites pursued some common paths.
First, the school choice initiatives tended to fall under four major
categories based on how sites defined choice arrangements and directed
the flow of enrolling students. Second, all sites focused on two core
activities throughout the implementation process: 1) engaging parents
and community members; and 2) building capacity at schools to attract
and accommodate choice enrollment.

**3.1 Overview of VPSC Sites**

**3.1.1 Sites' Characteristics**

***The 13 VPSC sites were located across the country.*** Ten were
located in predominantly urban communities, two in areas that cover both
urban and rural regions, and one in an entirely rural area (see exhibit
3-1). Ten of the locales represented a total population of over 100,000
people each. The public school student populations were mostly diverse
and of low-income backgrounds. Nonwhite students comprised over 60
percent of the student population at seven school systems represented by
the sites. Similarly, over 60 percent of the students were eligible for
the Free and Reduced-Price Lunch Program[^12] at seven sites.

**3.1.2 Brief Descriptions of Sites' Choice Initiatives**

***The VPSC sites varied greatly in the design of their choice
initiatives.*** The sites differed widely in the number of students
served, the number of participating public schools, and the capacity to
accommodate transfers (see exhibit 3-2 for brief descriptions). They
differed by how they defined choice zones and managed the flow of
students among participating schools. Furthermore, four sites used VPSC
funds to support a preexisting

+-----------------------------------------------------------------------+
| **Exhibit 3-1**                                                       |
|                                                                       |
| **Community Characteristics**                                         |
|                                                                       |
| +------------------------------------------------+----------------+   |
| | # Characteristic                               | **No. of       |   |
| |                                                | sites**        |   |
| +------------------------------------------------+----------------+   |
| | **Type of community:**                         |                |   |
| +------------------------------------------------+----------------+   |
| | **Urban**                                      | 10             |   |
| +------------------------------------------------+----------------+   |
| | **Rural**                                      | 1              |   |
| +------------------------------------------------+----------------+   |
| | **Mixed**                                      | 2              |   |
| +------------------------------------------------+----------------+   |
| | **Population:**                                |                |   |
| +------------------------------------------------+----------------+   |
| | > **Under 100,000**                            | 3              |   |
| +------------------------------------------------+----------------+   |
| | **100,000 to 1 million**                       | 6              |   |
| +------------------------------------------------+----------------+   |
| | **Over 1 million**                             | 4              |   |
| +------------------------------------------------+----------------+   |
| | **Public school enrollment:**                  |                |   |
| +------------------------------------------------+----------------+   |
| | **No. of students enrolled**                   |                |   |
| +------------------------------------------------+----------------+   |
| | **Under 25,000**                               | 4              |   |
| +------------------------------------------------+----------------+   |
| | **25,000**--**100,000**                        | 4              |   |
| +------------------------------------------------+----------------+   |
| | **Over 100,000**                               | 5              |   |
| +------------------------------------------------+----------------+   |
| | **Percent nonwhite**                           |                |   |
| +------------------------------------------------+----------------+   |
| | **Under 30**                                   | 1              |   |
| +------------------------------------------------+----------------+   |
| | **30**--**60**                                 | 5              |   |
| +------------------------------------------------+----------------+   |
| | **Over 60**                                    | 7              |   |
| +------------------------------------------------+----------------+   |
| | **Percent eligible for the Free and            |                |   |
| | Reduced-Price Lunch Program**                  |                |   |
| +------------------------------------------------+----------------+   |
| | **Under 30**                                   | 1              |   |
| +------------------------------------------------+----------------+   |
| | **30**--**60**                                 | 5              |   |
| +------------------------------------------------+----------------+   |
| | **Over 60**                                    | 7              |   |
| +------------------------------------------------+----------------+   |
|                                                                       |
| Exhibit Reads: Ten of the VPSC sites are located in urban areas.      |
|                                                                       |
| Source: NCES, Common Core of Data, 2004--05.                          |
+-----------------------------------------------------------------------+

or preplanned expansion of districtwide choice. Nearly all of the others
used the funds to support a specific function within an environment that
already had other existing choice

options. Only two sites had minimal choice options prior to VPSC because
they had no Title I schools and only a preexisting magnet program.

+----------------------------------------------------------------------+
| ###### Exhibit 3-2                                                   |
|                                                                      |
| **Brief Descriptions of the VPSC Sites' Choice Initiatives**         |
|                                                                      |
| +---------+-----+------------------------------------------------+   |
| | **VPSC  | *   | **Description of initiative, from 2002--03 to  |   |
| | gr      | *Aw | 2005--06**                                     |   |
| | antee** | ard |                                                |   |
| |         | Amt |                                                |   |
| |         | .** |                                                |   |
| |         |     |                                                |   |
| |         | **( |                                                |   |
| |         | \$m |                                                |   |
| |         | )** |                                                |   |
| +=========+=====+================================================+   |
| | **A.    |     |                                                |   |
| | PREE    |     |                                                |   |
| | XISTING |     |                                                |   |
| | OR      |     |                                                |   |
| | PRE     |     |                                                |   |
| | PLANNED |     |                                                |   |
| | DISTR   |     |                                                |   |
| | ICTWIDE |     |                                                |   |
| | INITIA  |     |                                                |   |
| | TIVES** |     |                                                |   |
| +---------+-----+------------------------------------------------+   |
| | **      | \$1 | Has augmented preexisting districtwide choice  |   |
| | Chicago | 0.2 | by using VPSC funds to support schools in      |   |
| | Public  |     | neighborhood learning clusters (NLCs) of       |   |
| | S       |     | four-to-six K**--**8 schools each. Four        |   |
| | chools, |     | clusters started in 2003**--**04, three        |   |
| | Ill.**  |     | started in 2004**--**05, and three started in  |   |
| |         |     | 2005**--**06. There is a new school in each    |   |
| |         |     | cluster, and existing schools develop magnet   |   |
| |         |     | themes; clusters have coordinators; and        |   |
| |         |     | schools receive VPSC funds.                    |   |
| +---------+-----+------------------------------------------------+   |
| | *       | \$1 | Has supported two cohorts of students          |   |
| | *School | 0.2 | (K**--**12) starting in 2004**--**05,          |   |
| | D       |     | enrolling in a districtwide controlled choice  |   |
| | istrict |     | initiative, involving seven regions (and an    |   |
| | of      |     | urban "zone" within each region) and the       |   |
| | Hills   |     | creation or expansion of "attractor" programs  |   |
| | borough |     | at existing schools to maintain or increase    |   |
| | County, |     | student diversity. Plans for the initiative    |   |
| | Fla.**  |     | were in place well before VPSC, which only     |   |
| |         |     | partially supports the initiative.             |   |
| +---------+-----+------------------------------------------------+   |
| | **Mi    | \$1 | Has supported three cohorts of transfer        |   |
| | nnesota | 1.8 | students in a preexisting program allowing MPS |   |
| | Dep     |     | students qualifying for the Free and           |   |
| | artment |     | Reduced-Price Lunch Program to transfer to     |   |
| | of      |     | nearly 60 schools in eight surrounding         |   |
| | Educ    |     | suburban districts, and suburban or MPS        |   |
| | ation** |     | students to attend 12 schools in MPS           |   |
| |         |     | (K**--**12). VPSC funds partially support      |   |
| |         |     | Parent Information Centers, some               |   |
| |         |     | transportation, and related support services.  |   |
| +---------+-----+------------------------------------------------+   |
| | **P     | \$  | Has supported three cohorts of transfer        |   |
| | ortland | 6.5 | students through the augmentation of a         |   |
| | Public  |     | districtwide, previously available choice      |   |
| | S       |     | program (K**--**12). VPSC funds help support   |   |
| | chools, |     | new enrollment and transfer policies;          |   |
| | Oreg.** |     | selection, lottery, and transition services;   |   |
| |         |     | and collaborative curricula planning.          |   |
| +---------+-----+------------------------------------------------+   |
| | **B.    |     |                                                |   |
| | INIT    |     |                                                |   |
| | IATIVES |     |                                                |   |
| | EN      |     |                                                |   |
| | HANCING |     |                                                |   |
| | OTHER   |     |                                                |   |
| | E       |     |                                                |   |
| | XISTING |     |                                                |   |
| | OP      |     |                                                |   |
| | TIONS** |     |                                                |   |
| +---------+-----+------------------------------------------------+   |
| | **B     | \$  | Has supported the first cohort of students by  |   |
| | righter | 3.4 | opening three new charter middle schools in    |   |
| | Choice  |     | Albany in 2005**--**06. The site has continued |   |
| | Charter |     | to support charter school development and to   |   |
| | S       |     | coordinate supplemental educational services.  |   |
| | chools, |     |                                                |   |
| | N.Y.**  |     |                                                |   |
| +---------+-----+------------------------------------------------+   |
| | *       | \$  | Has supported four cohorts of students         |   |
| | *Desert | 7.9 | transferring from low-performing sending       |   |
| | Sands   |     | schools to higher-performing receiving schools |   |
| | Unified |     | (K**--**12). Funds help augment curricula at   |   |
| | School  |     | receiving schools with environmental studies   |   |
| | Di      |     | theme to make them more attractive to          |   |
| | strict, |     | transferring students.                         |   |
| | C       |     |                                                |   |
| | alif.** |     |                                                |   |
| +---------+-----+------------------------------------------------+   |
| | **      | \$1 | Has provided technical assistance to school    |   |
| | Florida | 7.8 | districts as they develop and implement their  |   |
| | Dep     |     | choice plans. Funds assist a subset of mentor  |   |
| | artment |     | districts (already successful at choice        |   |
| | of      |     | options) and mentee districts (needing to      |   |
| | Educ    |     | expand options), and postsecondary             |   |
| | ation** |     | institutions (to support school choice         |   |
| |         |     | information and technical assistance centers). |   |
| +---------+-----+------------------------------------------------+   |
| | **Mia   | \$1 | Has supported three cohorts of students        |   |
| | mi-Dade | 1.7 | (K**--**12) in two of eight transportation     |   |
| | County  |     | zones in the district. Provides funds to       |   |
| | Public  |     | create choice programs at under-enrolled       |   |
| | S       |     | schools. The site has created or enhanced      |   |
| | chools, |     | choice programs, starting in 2003**--**04 with |   |
| | Fla.**  |     | one new "commuter" school in Zone 1; in        |   |
| |         |     | 2004**--**05, expanded to a total of nine      |   |
| |         |     | schools in Zone 1, and four schools in Zone 2; |   |
| |         |     | and in 2005**--**06 added one additional       |   |
| |         |     | choice program in each of the two zones.       |   |
| +---------+-----+------------------------------------------------+   |
| | **New   | \$  | Has supported four cohorts of students         |   |
| | Haven   | 9.5 | transferring from low-performing schools to    |   |
| | Public  |     | identified higher-performing schools,          |   |
| | S       |     | including Lighthouse Schools, magnets,         |   |
| | chools, |     | charters, and suburban public schools. VPSC    |   |
| | Conn.** |     | funds expanded programs at four Lighthouse     |   |
| |         |     | Schools (K**--**6) in 2002**--**03; five       |   |
| |         |     | Lighthouse Schools in 2003**--**04; and three  |   |
| |         |     | in 2004**--**05 and 2005**--**06.              |   |
| +---------+-----+------------------------------------------------+   |
| | **R     | \$1 | Has supported four cohorts of students         |   |
| | ockford | 0.1 | transferring from low-performing schools       |   |
| | Public  |     | (K**--**8) to identified receiving schools.    |   |
| | School  |     | VPSC funds provide support to receiving        |   |
| | D       |     | schools; the Parent Resource Center; parent    |   |
| | istrict |     | and transportation services; and tutoring      |   |
| | #205,   |     | programs run by community or faith-based       |   |
| | Ill.**  |     | organizations.                                 |   |
| +---------+-----+------------------------------------------------+   |
+----------------------------------------------------------------------+
| +----------+-----+----------------------------------------------+    |
| | **VPSC   | *   | **Description of initiative, from 2002--03   |    |
| | g        | *Aw | to 2005--06**                                |    |
| | rantee** | ard |                                              |    |
| |          | Amt |                                              |    |
| |          | .** |                                              |    |
| |          |     |                                              |    |
| |          | **( |                                              |    |
| |          | \$m |                                              |    |
| |          | )** |                                              |    |
| +----------+-----+----------------------------------------------+    |
| | **C.     |     |                                              |    |
| | INI      |     |                                              |    |
| | TIATIVES |     |                                              |    |
| | CREATING |     |                                              |    |
| | A NEW    |     |                                              |    |
| | CHOICE   |     |                                              |    |
| | OPTION** |     |                                              |    |
| +----------+-----+----------------------------------------------+    |
| | **Ar     | \$  | Has supported four cohorts of students       |    |
| | kansas** | 9.3 | attending an off-campus (residential or      |    |
| |          |     | community) program to receive a rigorous and |    |
| | **De     |     | comprehensive, self-paced education,         |    |
| | partment |     | delivered online and aligned with the        |    |
| | of       |     | state's standards (requires students to take |    |
| | Edu      |     | state assessment). The program covered       |    |
| | cation** |     | K**--**5 in 2002**--**03, K**--**7 in        |    |
| |          |     | 2003**--**04, and K**--**8 starting in       |    |
| |          |     | 2004**--**05.                                |    |
| +----------+-----+----------------------------------------------+    |
| | **Gr     | \$  | Has supported three cohorts of middle and    |    |
| | eenburgh | 2.8 | high school students (7**--**12) attending   |    |
| | Central  |     | new academies of choice within middle and    |    |
| | School   |     | high school. Has also supported two cohorts  |    |
| | District |     | of elementary students having expanded to    |    |
| | No. 7,   |     | the elementary level in 2004**--**05 with    |    |
| | N.Y.**   |     | the implementation of the International      |    |
| |          |     | Baccalaureate Primary Years Programme (IB)   |    |
| |          |     | to offer enhanced programming to all         |    |
| |          |     | K**--**6 students.                           |    |
| +----------+-----+----------------------------------------------+    |
| | **M      | \$  | Has supported four cohorts of students       |    |
| | onadnock | 8.4 | making interdistrict transfers (9**--**12),  |    |
| | Regional |     | attending new programs in their original     |    |
| | School   |     | schools (6**--**12), transferring to an      |    |
| | D        |     | alternative high school (MC2) and a virtual  |    |
| | istrict, |     | high school, or enrolling in college         |    |
| | N.H.**   |     | courses. In 2005**--**06, expanded           |    |
| |          |     | interdistrict choice to the elementary level |    |
| |          |     | and has added new choice programs at two     |    |
| |          |     | high schools and two middle schools.         |    |
| +----------+-----+----------------------------------------------+    |
|                                                                      |
| > Exhibit Reads: Chicago Public Schools received an award of \$10.2  |
| > million to augment a preexisting districtwide choice program.      |
| >                                                                    |
| > Sources: Analysis of site visit data and Grant Performance         |
| > Reports, by COSMOS Corporation, 2007.                              |
+----------------------------------------------------------------------+

**3.1.3 Participating Schools**

***The majority of participating schools were elementary schools.***
Participating schools covered all grade levels: 64 percent spanned the
elementary grades, 15 percent the middle school grades, 11 percent the
high school grades, and 10 percent were classified as "other" or had
missing data (see exhibit 3-3). As a comparison, approximately 58
percent of schools nationwide are elementary schools, 17 percent are
middle schools, 19 percent are high schools, and 6 percent are "other."
Although most of the sites focused their choice initiatives broadly
across all grade levels, four VPSC sites only targeted grades K--8,
which may account for the slightly higher prevalence of participating
elementary schools.

**3.2 Four Types of Choice Arrangements at the VPSC Sites**

The VPSC Program allowed grantees to design choice initiatives to meet
their own needs. Nevertheless, going beyond the unique circumstances at
each site, the initiatives fell into a four-fold typology of choice
arrangements (see exhibit 3-4).

+-----------------------------------------------------------------------+
| **Exhibit 3-3**                                                       |
|                                                                       |
| **Grade Span Distribution of Participating Schools**\*                |
|                                                                       |
| Exhibit Reads: 439 (or 63.7 percent) of the schools were elementary   |
| schools.                                                              |
|                                                                       |
| \*Includes 12 of the 13 sites in the VPSC Program because the 13th    |
| site focused mainly on technical assistance to districts.             |
|                                                                       |
| \*\*"Other" is defined by NCES as any grade span configuration,       |
| including ungraded, not falling within the three categories of        |
| elementary, middle, and high.                                         |
|                                                                       |
| \*\*\*"Not applicable/missing" contains those that either did not     |
| have school-level CCD data available or did not have data available   |
| for the grade span variable.                                          |
|                                                                       |
| Source: NCES, Common Core of Data, 2004--05.                          |
+-----------------------------------------------------------------------+
| **Exhibit 3-4**                                                       |
|                                                                       |
| **Four Types of Choice Arrangements**                                 |
|                                                                       |
|   -----                                                               |
| ----------------------------------- --------- ---- ------------- ---- |
|   *                                                                   |
| *Type of choice arrangement**           **No. of       **No. of       |
|                                                                       |
|                                         VPSC           schools**\*    |
|                                                                       |
|                                         sites**                       |
|                                                                       |
|   *                                                                   |
| *Predesignated sending or receiving     5              95             |
|   s                                                                   |
| chools**                                                              |
|                                                                       |
|   *                                                                   |
| *Same schools are both sending and      5              524            |
|   r                                                                   |
| eceiving schools**                                                    |
|                                                                       |
|   *                                                                   |
| *Within-school options only**           1              5              |
|                                                                       |
|   *                                                                   |
| *Mixture of the first three groups**    2              6              |
|                                                                       |
|   *                                                                   |
| *TOTAL**                                13             630            |
|   -----                                                               |
| ----------------------------------- --------- ---- ------------- ---- |
|                                                                       |
| Exhibit Reads: Five VPSC sites implemented choice arrangements with   |
| predesignated sending or receiving schools.                           |
|                                                                       |
| \*Based on the 630 School Survey respondents. The survey covered 12   |
| of the 13 sites in the VPSC Program because the thirteenth site       |
| focused mainly on technical assistance.                               |
|                                                                       |
| Source: Survey of Schools, COSMOS Corporation, 2006--07.              |
+-----------------------------------------------------------------------+

***First, five initiatives designated specific schools to be either
sending schools or receiving schools but not both.*** In this first
type, students attending predesignated sending schools were eligible to
transfer, and their choices were limited to a predesignated group of
receiving schools. At these sites, VPSC funds mainly supported the
strengthening or capacity-enhancement at the receiving schools.

Under this first arrangement, three of the five sites defined their
sending schools as "low-performing" according to the *NCLB* criteria
regarding schools identified for improvement. However, because the VPSC
legislation did not prescribe a standard for identifying
"higher-performing" schools, the program and its sites defined the
receiving schools simply as those that were not "low-performing"
schools. During the second year of its VPSC initiative at one of these
sites, initially eligible receiving schools later became identified for
improvement. At that point, the site decided these schools would be
ineligible to serve as receiving schools.

The fourth and fifth sites in this first type of arrangement
predesignated sending and receiving schools but did not attempt to limit
either group according to any low- to high-performing criteria. One of
these sites started several new receiving (charter) schools whose
performance could not, by definition, be known at the time when
transfers began. The last of these sites defined all schools in the
system as possible sending schools, and the receiving "schools" were
predesignated off-campus sites.

***Second, five initiatives defined the same schools as both "sending"
and "receiving."*** Under a second type of arrangement, five sites
permitted transfers between all public schools, either districtwide or
within pre-specified zones. In the latter case, students could choose
only among the public schools located within their assigned zone.
Whether delineated by zone or district, students could attend any school
within that area, regardless of a school's prior performance. This type
of choice arrangement gave sites little or no ability to direct the flow
of students from low- to higher-performing schools.

At least one site deliberately defined its geographic zones to include
both schools identified for improvement and higher-performing schools.
Thus, in a large countywide district, "pie-shaped" zones could have
suburban-like, higher-performing schools at their perimeter and
urban-like, low-performing schools at their center. While the site
limited transfers to within-zone choices, the initiative nevertheless
gave students the opportunity to transfer from schools identified for
improvement to higher-performing schools (and from an urban to a more
suburban environment). At the same time, students also could transfer in
the reverse direction.

For both of these first two types of arrangements, the majority of the
student bodies in all three types of schools (sending-only,
receiving-only, or both sending and receiving) were nonwhite students
and were eligible for the Free and Reduced-Price Lunch Program. However,
in comparing the two types of arrangements, those with the nondesignated
(both sending and receiving) schools had lower proportions of low-income
and minority students than the sending-only schools in the first type of
arrangement; in turn, the receiving-only schools tended to have the
lowest proportion of low-income students and Title I schools.

***Third, one site established a within-school initiative, in which
students chose from education programs within the same school.*** In
this third type, all students remained at their original schools. At the
single VPSC site that implemented this third type of arrangement,
students at their high school chose between two different academic
programs that had been put into place with VPSC funds. Middle school
students had a similar choice. At the elementary level, all students had
the same academic program, but the students chose between two forms of
assessment: being graded on a project or taking a test.

***Fourth, two initiatives involved a mixture of the first three
types.*** One VPSC initiative had a mix of choices, including education
programs within the same school, designated receiving schools, and
schools that could be both sending and receiving. The second site under
this last arrangement encompassed 26 school districts in the same state,
with each district defining its own choice options. VPSC activities
provided technical assistance to these districts in implementing public
school choice.

**3.3 Core Activities in the VPSC Choice Initiatives**

Despite the unique circumstances of each of the VPSC initiatives, all
sites focused on two core activities throughout the implementation
process: 1) engaging parents and community members, and 2) building
capacity at schools to attract and accommodate choice enrollees.

**3.3.1 Engaging, Notifying, and Reaching Parents and Community
Members**

***Sites undertook large-scale efforts to engage, notify, and reach
parents and community members.*** From the beginning, the VPSC sites
invested in outreach to parents and communities to ensure that public
school choice initiatives met local needs. Sites started the outreach at
an early stage. Sites even notified parents and community members of the
plans for the choice initiatives to provide an opportunity for everyone
to express preferences for the educational content, student selection
criteria, and design of parent information centers (see exhibit 3-5).

As the sites transitioned from the planning to implementation stage,
several sites created ways to keep parents and community members
actively engaged. For example, sites implemented information centers
that sponsored workshops on parenting skills, health and nutrition, and
computer literacy, as well as public school choice.

+-----------------------------------------------------------------------+
| ###### Exhibit 3-5                                                    |
|                                                                       |
| **Parent Involvement Activities**                                     |
|                                                                       |
| **Reported by School Officials in School Survey**                     |
|                                                                       |
| +-----------------+-----------------------------------------------+   |
| | **Type of       | **Sites' activities**                         |   |
| | involvement**   |                                               |   |
| +-----------------+-----------------------------------------------+   |
| | >               | Market research, surveys of parents,          |   |
| |  **Establishing | community and parent advisors, community      |   |
| | > the           | focus groups and forums                       |   |
| | > initiative**  |                                               |   |
| +-----------------+-----------------------------------------------+   |
| | > **Planning    | Parent advisory committees, principal and     |   |
| | > the           | teacher input, parent representation on       |   |
| | > initiative**  | lottery and parent information center         |   |
| |                 | committees                                    |   |
| +-----------------+-----------------------------------------------+   |
| | >               | Parent participation in daily instruction,    |   |
| |  **Implementing | community support for curriculum and          |   |
| | > the           | materials, parent and student surveys,        |   |
| | > initiative**  | specialists or counselors, workshops on       |   |
| |                 | parenting skills, and technical assistance to |   |
| |                 | schools on parental involvement               |   |
| +-----------------+-----------------------------------------------+   |
| | > **Community   | Face-to-face parent and community outreach,   |   |
| | > outreach      | media campaigns, direct mailing, and parent   |   |
| | > activities**  | information centers                           |   |
| +-----------------+-----------------------------------------------+   |
|                                                                       |
| > Exhibit Reads: Sites engaged parents in establishing the choice     |
| > initiatives through market research, surveys, advisory teams, and   |
| > focus groups.                                                       |
| >                                                                     |
| > Sources: Analysis of site visit data and Grant Performance Reports, |
| > by COSMOS Corporation, 2007.                                        |
+-----------------------------------------------------------------------+

***To notify parents of their choice options and the details of the
enrollment process, VPSC sites used a variety of outreach activities.***
Community outreach activities included face-to-face meetings, enrollment
fairs, open houses at the receiving schools, media announcements, and
letters to parents (see exhibit 3-6). Some sites communicated with
parents and community members in multiple languages. Over half of the
VPSC Program's sites had brochures and applications available in both
English and Spanish, and several reported advertising in local
Spanish-language media outlets. One site reported that running an
advertisement on a local Spanish-language radio station was among its
most successful outreach efforts. Other sites reported printing
materials in Chinese, Hmong, Lao, Polish, Portuguese, Russian, Somali,
and Vietnamese. Public agencies also helped the sites to field phone
calls from non-English speakers.

During 2006--07, a large portion of school administrators at the VPSC
sites indicated their belief that most or all parents and families were
aware of choice options available to them (see exhibit 3-7). At the
average VPSC site, 70 percent of the administrators reported that all or
most of parents and families had a good understanding of their choice
options.[^13]

+-----------------------------------------------------------------------+
| **Exhibit 3-6**                                                       |
|                                                                       |
| **Schools' Reported Efforts to Notify Parents of Their Choice         |
| Options, 2006--07**\*                                                 |
|                                                                       |
|   ------------------------------------------------ -----------------  |
|   **Response category**                            **Average percent  |
|                                                    of schools per     |
|                                                    VPSC site**\*\*    |
|                                                                       |
|   **Individual, face-to-face meetings with school  63.7               |
|   officials**                                                         |
|                                                                       |
|   **Group meetings with school officials**         59.4               |
|                                                                       |
|   **Enrollment fairs or similar events**           60.5               |
|                                                                       |
|   **Open houses at receiving schools**             57.5               |
|                                                                       |
|   **Letter mailed to parents and families**        68.1               |
|                                                                       |
|   **Letter sent home with students**               57.0               |
|                                                                       |
|   **Announcements in community newspapers and      63.6               |
|   other media**                                                       |
|                                                                       |
|   **Contacts made by district's parent information 43.0               |
|   center(s)**                                                         |
|   ------------------------------------------------ -----------------  |
|                                                                       |
| Exhibit Reads: On average, for any VPSC site, 63.7 percent of schools |
| reported having individual, face-to-face meetings between parents and |
| school officials related to choice options.                           |
|                                                                       |
| \*The School Survey covered 12 of the 13 sites in the VPSC Program    |
| because the 13th site focused mainly on technical assistance.         |
|                                                                       |
| \*\*The averages represent the number of schools, not the number of   |
| parents, involved in these activities (the number of parents could    |
| not be determined); schools could appear under more than one          |
| category.                                                             |
|                                                                       |
| Source: Survey of Schools, COSMOS Corporation, 2006--07.              |
+-----------------------------------------------------------------------+
| **Exhibit 3-7**                                                       |
|                                                                       |
| **Parents' and Families' Understanding of Choice**                    |
|                                                                       |
| **Reported by School Officials, 2006--07**\*                          |
|                                                                       |
| +---------------------------------------------+-------------------+   |
| | **School-reported proportion**              | **Average percent |   |
| |                                             | of schools per    |   |
| | **of parents and families**                 | VPSC site**       |   |
| |                                             |                   |   |
| | **having a good understanding of their      |                   |   |
| | choice options**                            |                   |   |
| +---------------------------------------------+-------------------+   |
| | **All**                                     | 25.7              |   |
| +---------------------------------------------+-------------------+   |
| | **Most, over 50 percent**                   | 44.6              |   |
| +---------------------------------------------+-------------------+   |
| | **Some, 20**--**50 percent**                | 21.6              |   |
| +---------------------------------------------+-------------------+   |
| | **Few, less than 20 percent**               | 8.0               |   |
| +---------------------------------------------+-------------------+   |
|                                                                       |
| Exhibit Reads: On average, for any VPSC site, 25.7 percent of schools |
| reported that all parents and families have a good understanding of   |
| their choice options.                                                 |
|                                                                       |
| \*The School Survey covered 12 of the 13 sites in the VPSC Program    |
| because the 13th site mainly focused on technical assistance.         |
|                                                                       |
| Source: Survey of Schools, COSMOS Corporation, 2006--07.              |
+-----------------------------------------------------------------------+

In some cases, sites delayed notifying parents of their choice options
until their states published the names of schools identified as
low-performing. As a consequence, parents and students at these sites
were notified of their eligibility only a few weeks before they had to
make a decision on whether to apply to transfer to another school.
However, states' deadlines for issuing this information have varied over
the VPSC years, so the delayed notification was episodic rather than
chronic at any given site.

Over the course of their grants, sites shifted their outreach strategies
and resources. One site reduced its districtwide marketing efforts but
increased the schools' involvement in the choice notification process.
The same site also phased down its parent resource centers, previously
VPSC-supported. Instead, the site upgraded its Web site to include more
comprehensive choice information directly accessible to parents. It also
encouraged schools to provide information directly to parents.

**3.3.2 Capacity-Enhancing Activities**

***Sites used VPSC funds to enhance the capacity of schools.*** The
sites participating in the VPSC initiatives supported a variety of
capacity-enhancing activities, including: starting new academic
programs; purchasing supplies and equipment; and providing professional
development to staff. Capacity-enhancing activities included efforts to
both accommodate and attract transferring students at receiving schools
(see exhibit 3-8). In addition to improvements to existing schools,
sites increased capacity within the system by opening new schools. For
the most part, the new schools were planned in advance of the VPSC
initiative, although VPSC funds later helped to start new academic
programs at these schools.

Activities to enhance school-level programming included focusing on
professional development programs and education programs at all levels.
The programs at one site included a foreign language program at the
elementary level; an arts and academics program at the middle school
level; and law, geographic information systems, and arts or design
programs at the high school level. Across the sites, curricular topics
covered by the capacity-enhancing activities covered academic subjects,
such as language arts, science, and mathematics as well as
research-based comprehensive programs such as the International
Baccalaureate (IB) program (see exhibit 3-9).

Although some of the new and enhanced education programs were nationally
recognized with a record of improving student performance, in general,
VPSC sites did not rely on scientifically based evidence to guide their
selection of education programs. To date, the VPSC sites lack systematic
documentation of such research for many of their capacity-enhancing
education programs.

Despite this approach, educational programming options implemented at
the VPSC sites have received outside recognition. For example, an
innovative science curriculum at one VPSC site resulted in new
collaboration within a school district. The district utilized

+-----------------------------------------------------------------------+
| ###### Exhibit 3-8                                                    |
|                                                                       |
| **Capacity-Enhancing Activities**                                     |
|                                                                       |
| +-------------------------+---------------------------------------+   |
| | **Type of activity**\*  | **Examples of programs supported**    |   |
| +-------------------------+---------------------------------------+   |
| | > **Starting new        | Distance learning; charter schools;   |   |
| | > schools or reopening  | commuter schools; schools with        |   |
| | > older schools (4)**   | thematic curricula                    |   |
| +-------------------------+---------------------------------------+   |
| | > **Starting new or     | Literacy curriculum; mathematics and  |   |
| | > enhancing existing    | science themes; foreign language      |   |
| | > programs at schools   | programs; environmental science       |   |
| | > (9)**                 | enhancements for all academic         |   |
| |                         | subjects; International Baccalaureate |   |
| +-------------------------+---------------------------------------+   |
| | > **Implementing        | During and after-school tutoring at   |   |
| | > tutorial and other    | school or by community-based          |   |
| | > support programs      | organizations; assistance to          |   |
| | > (6)**                 | supplemental service providers;       |   |
| |                         | hiring of support specialists         |   |
| +-------------------------+---------------------------------------+   |
| | > **Providing           | Professional development workshops    |   |
| | > professional          | and summer institutes, covering       |   |
| | > development to school | choice, the transfer process, or new  |   |
| | > staffs (12)**         | or enhanced curricula                 |   |
| +-------------------------+---------------------------------------+   |
|                                                                       |
| Exhibit Reads: Four VPSC sites have started new schools or reopened   |
| older schools, including a distance learning school; charter schools; |
| commuter schools; and schools with thematic curricula.                |
|                                                                       |
| \*The number in parentheses represents the number of sites conducting |
| each type of activity.                                                |
|                                                                       |
| Sources: Analysis of site visit data and Grant Performance Reports,   |
| COSMOS Corporation, 2007.                                             |
+-----------------------------------------------------------------------+

the Web-based science curriculum in its entire set of K--3 classrooms.
The VPSC initiative provided training and technical assistance in
installing equipment and setting up the classrooms, as well as ongoing
training and assistance to teachers. At another site, a strong
environmental studies program---developed to attract children from
low-performing schools to higher-performing schools---received a number
of honors. For example, the VPSC initiative received the Governor's
Environmental and Economic Leadership Award, the state's highest
environmental recognition.

***Other conditions at the schools were important in encouraging
enrollment in the choice initiatives.*** Existing academic programs, as
well as other conditions at receiving schools, were important in
encouraging enrollment. Other than VPSC-funded activities,
administrators indicated that the preexisting academic programs were a
major factor in students' decisions to transfer. Similarly, interviewed
principals, teachers, and parents overwhelmingly responded that the main
reason students had decided to transfer to specific schools was their
preexisting reputations for high performance. One VPSC site's own parent
survey confirmed the administrators' conclusion that parents tended to
choose schools according to the perceived rigor of academic programs.

+-----------------------------------------------------------------------+
| #### Exhibit 3-9                                                      |
|                                                                       |
| **Illustrative Curriculum Topics at the VPSC Sites**                  |
|                                                                       |
| +----+------------------------------+----------------------------+    |
| | ** | **Capacity-enhancing         | **Illustrative curriculum  |    |
| | VP | activity**                   | topics**                   |    |
| | SC |                              |                            |    |
| | Si |                              |                            |    |
| | te |                              |                            |    |
| | ** |                              |                            |    |
| +====+==============================+============================+    |
| | #  | Individualized, self-paced   | Research-based curriculum: |    |
| | #  | instruction using *Arkansas  | Language Arts/English,     |    |
| | A  | Virtual School* (*ARVS*)     | Math, Science, History,    |    |
| | {# | curriculum, derived from     | Arts, Music, Physical      |    |
| | a  | *K--12, Inc.*-developed      | Education/Health           |    |
| | .u | curriculum, using technology |                            |    |
| | nn | and distance learning for    | Innovative K--3 Elementary |    |
| | um | virtual learning             | Science                    |    |
| | be | environment.                 |                            |    |
| | re |                              |                            |    |
| | d} |                              |                            |    |
| +----+------------------------------+----------------------------+    |
| | #  | New charter schools.         | *International             |    |
| | #  |                              | Baccalaureate* (IB)        |    |
| | B  |                              |                            |    |
| | {# |                              | *Junior Great Books*       |    |
| | b  |                              | (Language Arts),           |    |
| | .u |                              | *Connected Mathematics*;   |    |
| | nn |                              | FOSS (*Full Option Science |    |
| | um |                              | System*), and the *A       |    |
| | be |                              | History of U.S.* series    |    |
| | re |                              |                            |    |
| | d} |                              |                            |    |
| +----+------------------------------+----------------------------+    |
| | *  | Cluster schools implementing | Enhancements to curricula  |    |
| | *C | themes such as science and   |                            |    |
| | ** | mathematics, fine and        | Lab improvements           |    |
| |    | performing arts,             |                            |    |
| |    | international scholars,      | Clubs                      |    |
| |    | literature and writing, and  |                            |    |
| |    | world languages.             | After-school educational   |    |
| |    |                              | programs                   |    |
| +----+------------------------------+----------------------------+    |
| | *  | Receiving schools augmenting | K--12 interdisciplinary    |    |
| | *D | curricula, giving greater    | Environmental Studies      |    |
| | ** | emphasis to                  | Curriculum (enhanced by    |    |
| |    | grade-appropriate and        | hands-on, inquiry based    |    |
| |    | technology-rich              | learning opportunities     |    |
| |    | environmental education      | that integrate technology) |    |
| |    | programs.                    |                            |    |
| +----+------------------------------+----------------------------+    |
| | *  | Site's focus is on technical |                            |    |
| | *E | assistance to districts      |                            |    |
| | ** |                              |                            |    |
| +----+------------------------------+----------------------------+    |
| | *  | Within-school academies with | Humanities and             |    |
| | *F | 250--300 students per        | International Studies      |    |
| | ** | academy and the              | (HAIS)                     |    |
| |    | *International               |                            |    |
| |    | Baccalaureate* (including    | Math, Arts, Science, and   |    |
| |    | *IB* Primary Years           | Technology (MAST)          |    |
| |    | Programme).                  |                            |    |
| |    |                              | Excelsior Humanities       |    |
| |    |                              | Academy                    |    |
| |    |                              |                            |    |
| |    |                              | Woodland Inventors,        |    |
| |    |                              | Technicians, and           |    |
| |    |                              | Scientists (WITS)          |    |
| +----+------------------------------+----------------------------+    |
| | *  | Attractor programs at        | Themes offered are fine    |    |
| | *G | schools, designed to         | arts, computer technology, |    |
| | ** | encourage urban students to  | math and communication     |    |
| |    | select suburban schools.     | technology, aquatics, sign |    |
| |    |                              | language, culinary arts,   |    |
| |    |                              | and environmental science  |    |
| +----+------------------------------+----------------------------+    |
| | *  | Schools are selecting        | Themes include pre-*IB*,   |    |
| | *H | academic themes with proven  | literacy through the arts, |    |
| | ** | track records.               | pre-medicine, *Waterford*  |    |
| |    |                              | early reading (literacy    |    |
| |    |                              | through media use), and    |    |
| |    |                              | college prep               |    |
| +----+------------------------------+----------------------------+    |
| | *  | Schools are offering         | *GEMS* (*Girls in          |    |
| | *I | supplemental programs to     | Engineering, Math, and     |    |
| | ** | students.                    | Science*), *ACE* girls and |    |
| |    |                              | boys programs, the         |    |
| |    |                              | *National Youth Sports     |    |
| |    |                              | Program*, and the          |    |
| |    |                              | *Learning Works* program   |    |
| +----+------------------------------+----------------------------+    |
| | *  | Schools are opening unique   | The Academy of Arts and    |    |
| | *J | programs of study available  | Design HS program          |    |
| | ** | to any student in the area.  |                            |    |
| |    |                              | The Law, Public Safety,    |    |
| |    |                              | and Security (LPSS) HS     |    |
| |    |                              | program                    |    |
| |    |                              |                            |    |
| |    |                              | The GIS (Geographic        |    |
| |    |                              | Information Systems) HS    |    |
| |    |                              | program                    |    |
| |    |                              |                            |    |
| |    |                              | Arts and Academics MS      |    |
| |    |                              | program                    |    |
| |    |                              |                            |    |
| |    |                              | World Language elementary  |    |
| |    |                              | program                    |    |
| +----+------------------------------+----------------------------+    |
| | *  | Schools are developing and   | Focus on Literacy programs |    |
| | *K | implementing themes and      |                            |    |
| | ** | magnet programs.             |                            |    |
| +----+------------------------------+----------------------------+    |
| | *  | Expanding higher-performing  | Arts and Technology        |    |
| | *L | educational options and      |                            |    |
| | ** | opening small learning       | Science and Technology     |    |
| |    | communities.                 |                            |    |
| |    |                              | Young Women's Academy      |    |
| |    |                              |                            |    |
| |    |                              | Young Men's Academy        |    |
| |    |                              |                            |    |
| |    |                              | Elementary language        |    |
| |    |                              | immersion program          |    |
| +----+------------------------------+----------------------------+    |
| | *  | Schools are implementing     | Three faith-based          |    |
| | *M | enhanced programming for all | after-school programs for  |    |
| | ** | students.                    | tutoring and educational   |    |
| |    |                              | experiences                |    |
| +----+------------------------------+----------------------------+    |
|                                                                       |
| > Exhibit Reads: Site A used the ARVS curriculum as its               |
| > capacity-enhancing activity.                                        |
| >                                                                     |
| > Sources: Analysis of site visit data and Grant Performance Reports, |
| > COSMOS Corporation, 2007.                                           |
|                                                                       |
| ####                                                                  |
+-----------------------------------------------------------------------+

Students also may have been attracted by schools' physical facilities or
perceived safety. The VPSC legislation prohibits the use of program
funds for construction, but sites could use funds from other sources to
make physical improvements. At one VPSC site, a newly reopened receiving
school had a combination of new academic activities, supported by VPSC
funds, and a new building and classrooms put into place by the school
district. At another site, the planned receiving schools were new
charter schools with a similar mix of funding sources.

Other participating districts were already in the process of
reconfiguring existing schools, opening new schools, and renovating
schools. These actions did not cover all of the receiving schools.
Nevertheless, the VPSC sites frequently cited these district activities
as reasons for not undertaking specific steps to expand seat capacity at
the schools. Many participating schools were also under-enrolled and
could accommodate all transfer applicants. Furthermore, seat capacity
was generally determined at the school-level, providing flexibility to
allow for reconfigurations within classrooms to accommodate more
students.

Some of the sites actively examined potential options to expand
available seats, but few implemented any specific changes. Some of the
options the sites explored included: hiring more teachers, opening
satellite learning centers, expanding online classes, adding new wings
to existing schools, and building new facilities. One site modified the
method principals used to assess their schools' available seat capacity,
which resulted in more accurate counts, but not necessarily more seats.
Another site expanded its VPSC initiative to include additional schools,
in this manner increasing the total number of seats available each year.

***Overall, however, the sites' capacity-enhancing activities were not
necessarily accompanied by an expansion of seats or classrooms at many
of the receiving schools.*** For instance, none of the sites reported
hiring more teaching staff or taking other steps simply to expand the
number of seats to accommodate higher enrollments at the existing
schools.

**3.4 Initial Implications for Federal Education Policy**

**and Local Education Practice**

The discussion thus far has treated the VPSC Program like many other
federal discretionary programs---tracing VPSC-funded activities at each
site as if the sites were supporting discrete, federally funded
projects. However, from the perspective of the VPSC Program's role in
informing federal education policymaking and local education practice,
the reality at each site requires closer attention. First, as with many
other school districts in the country, nearly all of the 13 sites
already had a broad variety of public school options prior to the VPSC
Program. Second, the VPSC Program permitted sites to enhance existing
options, and sites did not need to start entirely new initiatives.

> **3.4.1 A Variety of Other Choice Initiatives**

***New choice initiatives, such as those funded by the VPSC Program, did
not occur on a blank slate.*** Public school choice has been available
in many school districts for decades. Some of the most common choice
options have been: magnet schools, opportunities to transfer to
different schools in a district, state-initiated choice options, options
for students with at-risk conditions, and, most recently, charter
schools. Likewise, districts with Title I schools have offered choice
options since 1994. Overall, most districts by now have a wide selection
of choice options.

The experiences at the 13 VPSC sites did not differ. All of the 13 sites
had existing choice options prior to the VPSC Program. Some of the sites
had a wide array of such options, with some students eligible to
participate in two or more options. Although sites tried to make their
entire array of choice options work well, they did not necessarily track
students' participation on an option-by-option basis. For federal
policy, the implications are clear. The benefits of investing in a
particular public school choice initiative may not be easily isolated,
given the existing array of choices available to students.

***Choice contexts in different states also can vary markedly.*** Going
one step further, any expanded federal support for public school choice
might more likely be administered on a formula basis through state
departments of education rather than as direct, discretionary awards to
local school districts or other local entities. The VPSC Program's
experience, with three awards going to state departments of education,
potentially preview some of the features of such a scenario. In
particular, the VPSC experience showed that marked differences existed
among states' ongoing choice priorities. They varied from using federal
funds to: provide technical assistance; meet the settlement terms of
significant court rulings; or support an innovative, off-campus choice
option (see appendix C, exhibit C-1, for more detail).

The variations among these three states did not just reflect differences
in choice preferences. The variations reflected considerably different
conditions among states. Thus, any expansion of federal funding through
state allocations may have to anticipate the diversity of contexts among
states.

**3.4.2 Implications of the VPSC Program's Main Condition of Award:**

**Sites Could *Either* Support New Initiatives *or* Enhance or**

> **Expand Existing Ones**

Rather than specifying any particular choice model, the VPSC Program
gave sites the flexibility to define their own procedures. Furthermore,
the program permitted sites either to start a new choice initiative or
to "enhance or expand" an existing one. The flexibility was important
because sites could craft a new initiative but also work within their
existing tapestry of options.

Acknowledging this complication is necessary for interpreting the
findings in the remainder of this report. For some of the sites, the
VPSC funds were only part of the support for the identified initiative.
As a result, the contribution of the VPSC Program may have been
overestimated. At other sites, the VPSC funds not only supported the
identified initiative but also partially supported other choice options
at the same site. As a result, the contribution of VPSC funds may have
been underestimated. Unfortunately, the current analysis was not able to
distinguish the extent of these over- or underestimations.

***Sites could use VPSC funds to partially fund preexisting
initiatives*.** Most of the VPSC-funded initiatives enhanced existing
choice arrangements, complicating any attempts to distinguish "new" from
"old." Enhancements meant that these same arrangements already had other
sources of funding support and technical expertise. For example, four of
the 13 sites used their VPSC funds to support existing, districtwide
choice options (see appendix C, exhibit C-2, for more detail).

***Sites used VPSC funds to support different portions of choice
initiatives.*** Most sites had a wide array of choice initiatives as
part of their choice environment. For instance, some sites directed VPSC
funds to services specifically supporting choice enrollment under a VPSC
choice initiative, but the same sites also used VPSC funds to support
other services related to all of the choice initiatives at the site (see
appendix C, exhibit C-3, for more detail).

This overall pattern---from sites that used VPSC funds as only a partial
source of support for ongoing districtwide options, to sites that used
their funds to support more than a single choice initiative---meant that
the VPSC Program supported parts of an initiative at some sites and more
than a single initiative at other sites (see exhibit 3-10). Because
neither option can be disentangled, the analysis in the remainder of
this report assumes that the VPSC Program was associated with a single
initiative at each site, ignoring the situations where additional
options were partially funded or subject to overlap \[see variations (2)
and (3) in exhibit 3-10\].

***The VPSC initiatives overlapped with Title I choice options.*** The
VPSC Program started in the same year that federal legislation expanded
support for Title I choice options. The legislation allowed spending up
to 20 percent of Title I funds to support transportation costs when
students wanted to transfer out of a Title I school identified for
improvement. Three of the 13 sites defined their VPSC initiatives to
coincide or overlap closely with their Title I choice options (see
appendix C, exhibit C-4, for more detail). However, most VPSC sites
defined schools participating in their VPSC initiatives as a broader set
of schools than simply those designated under Title I as "identified for
improvement." The VPSC funds also could be used to cover the students'
transportation costs or services regardless of a schools' performance
status---i.e., whether identified for improvement, low-performing, or
neither.

+-----------------------------------------------------------------------+
| **Exhibit 3-10**                                                      |
|                                                                       |
| **Variations in Sites' Uses of VPSC Funds to Support Choice Options** |
|                                                                       |
| Exhibit Reads: Under Variation (1), VPSC supports only a part of a    |
| choice option.                                                        |
|                                                                       |
| Source: National evaluation team.                                     |
+-----------------------------------------------------------------------+

**4. PROMOTING EDUCATIONAL EQUITY AND EXCELLENCE**

Over the five years from 2002 to 2007, the VPSC sites enrolled many
students in their choice initiatives. However, sites made less progress
on other program priorities which also were relevant for assessing
educational equity and excellence.

**4.1 Student Participation at the VPSC Sites, 2002--07:**

**Eligibles, Applicants, and Enrollees**

***The exact number of participants in a choice initiative varied
according to the definition of "participation.*"** The evaluation
tracked the following three possible definitions to measure
participation:

1)  *Eligible students*: all students who could potentially participate
    in a VPSC initiative;

2)  *Applicants*: the set of eligible students who applied either to
    attend *another* public school or to participate in an academic
    program within their original school; and

3)  *Enrollees*: those students who successfully applied and enrolled in
    a school or program as a result of a VPSC initiative.

The VPSC sites reported data in each category annually. Their reports
for 2005--06 illustrated the wide variations among the categories for
each VPSC site (see exhibit 4-1).[^14]

All three categories of students can be considered participants in the
VPSC Program, depending on the desired logic. For instance, the logic
favoring a count of *eligible* students as participants, even if not
applying to transfer to another school, is that these students indeed
exercised a choice by deciding to stay at their original school.[^15]
Narrower definitions would limit participants to include only
*applicants*, or even further, to include only *enrollees*. To permit
readers to use their preferred logic, the national evaluation tracked
and reported all three groups of students. The data also permit an
estimate of participation rates, usually defined as the proportion of
enrolling to eligible students.

***In 2005--06 and across 12 of the 13 VPSC sites, 24,921 students
enrolled in the choice initiatives, reflecting an overall participation
rate of 2.8 percent of the students eligible to enroll*.** The amount
and rate of participation also appeared to reflect the type of

+-----------------------------------------------------------------------+
| **Exhibit 4-1**                                                       |
|                                                                       |
| **Student Participation in the VPSC's Initiatives, 2005-06**          |
|                                                                       |
| <table style="width:100%;">                                           |
| <colgroup>                                                            |
| <col style="width: 31%" />                                            |
| <col style="width: 15%" />                                            |
| <col style="width: 2%" />                                             |
| <col style="width: 3%" />                                             |
| <col style="width: 3%" />                                             |
| <col style="width: 15%" />                                            |
| <col style="width: 2%" />                                             |
| <col style="width: 3%" />                                             |
| <col style="width: 15%" />                                            |
| <col style="width: 5%" />                                             |
| </colgroup>                                                           |
| <tbody>                                                               |
| <tr class="odd">                                                      |
| <td rowspan="2"><p><strong>VPSC Program</strong></p>                  |
| <p><strong>sites</strong></p></td>                                    |
| <td colspan="9"><strong>Number of students</strong></td>              |
| </tr>                                                                 |
| <tr class="even">                                                     |
| <td colspan="4"><strong>Eligible</strong></td>                        |
| <td colspan="3"><strong>Applying</strong></td>                        |
| <td colspan="2"><strong>Enrolling</strong></td>                       |
| </tr>                                                                 |
| <tr class="odd">                                                      |
| <td><strong>A</strong></td>                                           |
| <td colspan="2">290,142</td>                                          |
| <td colspan="2"></td>                                                 |
| <td>214</td>                                                          |
| <td colspan="2"></td>                                                 |
| <td>214</td>                                                          |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="even">                                                     |
| <td><strong>B</strong></td>                                           |
| <td colspan="2">10,452</td>                                           |
| <td colspan="2"></td>                                                 |
| <td>170</td>                                                          |
| <td colspan="2"></td>                                                 |
| <td>170</td>                                                          |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="odd">                                                      |
| <td><strong>C</strong></td>                                           |
| <td colspan="2">4,970</td>                                            |
| <td colspan="2"></td>                                                 |
| <td>446</td>                                                          |
| <td colspan="2"></td>                                                 |
| <td>446</td>                                                          |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="even">                                                     |
| <td><strong>D</strong></td>                                           |
| <td colspan="2">4,993</td>                                            |
| <td colspan="2"></td>                                                 |
| <td>114</td>                                                          |
| <td colspan="2"></td>                                                 |
| <td>34</td>                                                           |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="odd">                                                      |
| <td><strong>E</strong></td>                                           |
| <td colspan="2">670</td>                                              |
| <td colspan="2"></td>                                                 |
| <td>NA*</td>                                                          |
| <td colspan="2"></td>                                                 |
| <td>196</td>                                                          |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="even">                                                     |
| <td><strong>F</strong></td>                                           |
| <td colspan="2">320,331</td>                                          |
| <td colspan="2"></td>                                                 |
| <td>NA*</td>                                                          |
| <td colspan="2"></td>                                                 |
| <td>13,068</td>                                                       |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="odd">                                                      |
| <td><strong>G</strong></td>                                           |
| <td colspan="2">46,838</td>                                           |
| <td colspan="2"></td>                                                 |
| <td>1,620</td>                                                        |
| <td colspan="2"></td>                                                 |
| <td>1,494</td>                                                        |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="even">                                                     |
| <td><strong>H</strong></td>                                           |
| <td colspan="2">134,878</td>                                          |
| <td colspan="2"></td>                                                 |
| <td>NA*</td>                                                          |
| <td colspan="2"></td>                                                 |
| <td>3,039</td>                                                        |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="odd">                                                      |
| <td><strong>I</strong></td>                                           |
| <td colspan="2">27,266</td>                                           |
| <td colspan="2"></td>                                                 |
| <td>674</td>                                                          |
| <td colspan="2"></td>                                                 |
| <td>674</td>                                                          |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="even">                                                     |
| <td><strong>J</strong></td>                                           |
| <td colspan="2">45,727</td>                                           |
| <td colspan="2"></td>                                                 |
| <td>5,249</td>                                                        |
| <td colspan="2"></td>                                                 |
| <td>3,783</td>                                                        |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="odd">                                                      |
| <td><strong>K</strong></td>                                           |
| <td colspan="2">1,682</td>                                            |
| <td colspan="2"></td>                                                 |
| <td>1,682</td>                                                        |
| <td colspan="2"></td>                                                 |
| <td>1,682</td>                                                        |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="even">                                                     |
| <td><strong>L</strong></td>                                           |
| <td colspan="2">8,235</td>                                            |
| <td colspan="2"></td>                                                 |
| <td>NA*</td>                                                          |
| <td colspan="2"></td>                                                 |
| <td>121</td>                                                          |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="odd">                                                      |
| <td><strong>M</strong></td>                                           |
| <td colspan="8"><p>Site focused mainly on technical</p>               |
| <p>assistance to districts across the state.</p></td>                 |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="even">                                                     |
| <td></td>                                                             |
| <td colspan="3"></td>                                                 |
| <td colspan="3"></td>                                                 |
| <td colspan="3"></td>                                                 |
| </tr>                                                                 |
| <tr class="odd">                                                      |
| <td><strong>Total number of students</strong></td>                    |
| <td>896,184</td>                                                      |
| <td colspan="2"></td>                                                 |
| <td colspan="2">10,169</td>                                           |
| <td></td>                                                             |
| <td colspan="2">24,921</td>                                           |
| <td></td>                                                             |
| </tr>                                                                 |
| <tr class="even">                                                     |
| <td><strong>Number of sites reporting</strong></td>                   |
| <td>12</td>                                                           |
| <td colspan="2"></td>                                                 |
| <td colspan="2">8</td>                                                |
| <td></td>                                                             |
| <td colspan="2">12</td>                                               |
| <td></td>                                                             |
| </tr>                                                                 |
| </tbody>                                                              |
| </table>                                                              |
|                                                                       |
| Exhibit Reads: VPSC site A reported a total of 290,142 eligible       |
| students.                                                             |
|                                                                       |
| \*Site did not track all applicants.                                  |
|                                                                       |
| Sources: Analysis of site visit data and Grant Performance Reports,   |
| by COSMOS Corporation, 2007.                                          |
+-----------------------------------------------------------------------+

choice arrangement that each site implemented. The five sites that
designated specific sending and receiving schools showed a lower
proportion of enrolling-to-eligible students than other arrangement
types. In 2005--06, the percent of eligible students who enrolled in
these initiatives was 0.3 percent. In contrast, at the five sites with
geography-based arrangements where the same schools were both sending
and receiving schools, 3.8 percent of eligible students enrolled in
2005--06. (The 100 percent participation rate for the "within-school"
option is an artifact of the absence of inter-school transfers.)

***Sites did not track applicants as consistently as they tracked other
participant groups.*** Of the 12 sites reporting eligible and enrolling
students for 2005--06, only eight sites tracked and reported the number
of applicants for their choice initiatives. One site was unable to track
applicants due to an error in coding VPSC applications in the district's
enrollment database. In addition, in the last two years, several sites
reported that 100 percent of their applicants became enrolled, even
though they had reported lower participation rates in earlier years. The
exact reason for the 100 percent rates is not known, but one possibility
is that sites may have changed their reporting procedures in later
years.

***Calculating the number of enrollees also depends in part on whether
the totals include enrollees from prior years.*** The typical VPSC site
tracked and reported the number of students enrolling in their
initiatives each year by counting only the number of students who
started enrollment in that year. Such "first-time enrollees" serve as a
conservative estimate of the overall enrollment in a choice initiative.
Another way of counting enrollment involves including enrollees from
prior years. Thus, the more accurate estimate of enrollment in the VPSC
Program's initiatives, for the four years ending in 2005--06, fell
somewhere between the 24,921 reported in exhibit 4-1 and a cumulative
total of 49,616 first-time students enrolled in the initiatives since
their inception in 2002--03 (see appendix C, exhibit C-5).

Unfortunately, totaling across all years overestimates the total
enrollment, which may have declined due to attrition. A few sites made
rough estimates of the attrition or "dropout" rates over the period of
the VPSC Program, suggesting rates that ranged between 30 and 50
percent. Several other sites estimated their repeat enrollments and
conjectured continuation rates from 50 to 95 percent.

The rough estimates mean that the enrollment is likely to be far lower
than the cumulative total of 49,616 students. However, exactly how much
lower is difficult to determine because, even if the attrition rates
were known, it is extremely difficult to classify the students who leave
or drop out. Some students may have decided to enroll at yet newer
schools through the choice initiative, while others may have returned to
their assigned school and may be considered true "drop-outs." However,
many students may have left the district entirely, not necessarily
having anything to do with their choice experiences. An accurate
estimate for cumulative enrollment would need to distinguish among these
alternatives while also tracking those students that continued to attend
schools of choice through the VPSC Program.

***Overall, the number of enrolling students in the VPSC Program
increased during the earlier years of the program but declined in the
program's fifth year.*** Of the 12 sites enrolling students, only ten
provided eligibility and enrollment data for four consecutive years,
2003--04 to 2006--07 (see exhibit 4-2). These data permitted an
estimation of trends through the fifth VPSC year, 2006--07. The yearly
data captured the number of first-time (i.e., new) enrollees each year,
not the total number that cumulated over all years. For these
"first-time enrollees," the trends showed that the VPSC Program at first
averaged 696 enrollees per site in 2003--04, then reached a peak of
2,459 per site in 2005--06, and then declined to 2,167 per site in
2006--07.

+-----------------------------------------------------------------------+
| ###### Exhibit 4-2                                                    |
|                                                                       |
| **Participation Rates, 2003--07**\*                                   |
|                                                                       |
|   ----                                                                |
| ------------- -------------- --------------- ------------------------ |
|   *                                                                   |
| *Enrollment      **Average                      **Percent of eligible |
|   scho                                                                |
| ol year**     number per                     students enrolled at the |
|                                                                       |
|                    site:**                        ten sites with four |
|                                                                       |
|                                                   years of enrollment |
|                                                    data**             |
|                                                                       |
|                     **Eligible**   **Enrolling**                      |
|                                                                       |
|   **2003--04**      47,165         696             1.5                |
|                                                                       |
|   **2004--05**      54,009         1,592           2.9                |
|                                                                       |
|   **2005--06**      59,781         2,459           4.1                |
|                                                                       |
|   **2006--07**      72,393         2,167           3.0                |
|   ----                                                                |
| ------------- -------------- --------------- ------------------------ |
|                                                                       |
| Exhibit Reads: On average, VPSC Program's choice initiatives had      |
| 47,165 eligible participants per site in 2003--04.                    |
|                                                                       |
| \*The averages were based on ten sites for which participant data     |
| were available for four consecutive years.                            |
|                                                                       |
| \*\*2006--07 data are preliminary.                                    |
|                                                                       |
| Sources: Analysis of site visit data and Grant Performance Reports,   |
| by COSMOS Corporation, 2007.                                          |
+-----------------------------------------------------------------------+

***Participation rates also showed the same pattern over time,
increasing and then declining from 2003--04 to 2006--07.*** For the same
ten sites reporting data from 2003--04 to 2006--07, the participation
rates by first-time enrollees also increased from 1.5 percent to 4.1
percent, from 2003--04 to 2005--06, and then dropped to 3.0 percent in
2006--07 (see exhibit 4-2).

The decline in total enrollment as well as in participation rates may
reflect the actual saturation of the VPSC initiatives because a good
(but unmeasured) portion of the earlier years' enrollees still remained
enrolled in the later years. Their continuing enrollment may possibly
have limited the seats available for first-time enrollees in the final
year. For example, an initiative that retains the same receiving schools
from year to year will likely encounter limited seat capacity after
several years of annual first-time enrollees.

In addition, the supply and demand conditions in choice arrangements,
from year to year, may be extremely dynamic, with the two conditions
adjusting to each other in immeasurable ways. For example, one site
discouraged applications in its last two reporting years, knowing that
the available seats were limited. Shifts in eligibility pools also
occurred over time. For instance, due to annual shifts in the identity
and number of low-performing schools, the pools of eligible students
shifted accordingly at three sites that linked choice options to school
performance. Four other sites expanded choice options to additional
grade levels, and one site expanded its targeted geographic area and
thus increased the number of schools covered. A fuller assessment of the
participation in choice initiatives, over a multiyear period, may
require stronger recordkeeping of all three kinds of participation
(eligibles, applicants,[^16] and enrollees) than the VPSC sites appear
to have implemented.

Regarding the demographic and academic characteristics of the enrollees,
not all of the sites kept precise data, nor did they compare the
enrollees' characteristics with those of non-enrollees---students who
did not participate in the VPSC initiatives. However, four sites did
submit individual-level student data that were used to analyze student
achievement trends (see chapter 5). These data included demographic and
academic characteristics for the year prior to any VPSC participation.
The data also included similar information from selected groups of
non-enrollees (the sites' selection processes also are described in
chapter 5).

Compared to the non-enrollees, the VPSC enrollees tended to: be slightly
more white than nonwhite, have slightly lower participation in the Free
and Reduced-Price Lunch Program, and have slightly better achievement
scores in mathematics and reading. However, the differences were not
statistically significant. Chapter 5 contains all of the details about
these data and their sources (see exhibit 5-3).[^17]

**4.2 Progress on Program Priorities**

The VPSC legislation had four program priorities. Three priorities were:
to provide the widest variety of choices; to encourage transfers from
low- to higher-performing schools; and to provide opportunities for
students to transfer to schools outside of their home districts. The
fourth priority directed sites to use some of their VPSC funds to
support transportation services and costs. These four priorities are
discussed next.

**4.2.1 Widest Variety of Choices**

***The VPSC Program made progress on the first priority of providing the
widest variety of choice.*** Sites expanded the assortment of choice
options in participating schools and offered a large and diverse number
of academic programs to transferring students.

Some sites developed more school choice options than others, with as
many as 12 different options available. The VPSC initiatives also made
efforts, through media campaigns and related activities, to increase
parents' awareness of the variety of education options available to
them.

The sites augmented the quality of choice options by implementing
enhanced education programming at receiving schools. Several VPSC
initiatives broadened the choices available to students through the
improvement and development of thematic programming at participating
schools. The programs implemented as part of the sites' VPSC initiatives
were new to the district (or a particular area of the district) and
often distinct from other existing thematic programming.[^18] These
programs focused on unique academic themes and were often integrated
schoolwide, across all grades and subject areas, especially at the
elementary and middle schools.[^19] Other programs, often implemented at
the high school level, included courses focused on a particular theme
that were separate from the required curricula.

Three sites focused on adding new choice options as part of their VPSC
initiatives. One statewide initiative implemented a virtual school,
offering a completely new type of choice option to students, thereby
directly increasing the variety of choice options within the state. At
another site, magnet schools were the only form of school choice prior
to VPSC. The VPSC initiative, by developing new charter schools,
established a second choice option for students in the district. Lastly,
a third site implemented a new choice school type, *Lighthouse Schools*,
through their VPSC initiative, as an addition to the district's other
choice options.

Two sites extended choice options of urban area students to a wider
array of suburban schools. Another site permitted rural area students to
transfer to a large number of adjoining districts, thereby broadening
the students' education options.

**4.2.2 Transfers from Low- to Higher-Performing Schools**

***Sending schools tended to have higher proportions of minority and
low-income students than receiving schools.*** Over 90 percent of the
sending schools, on average per site, were Title I schools that served
low-income students. Also on average per site, 72 percent of the
students in sending schools were nonwhite, and 74 percent were eligible
for the Free and Reduced-Price Lunch Program (see exhibit 4-3).

+-----------------------------------------------------------------------+
| **Exhibit 4-3**                                                       |
|                                                                       |
| **Demographic Characteristics of Schools, 2006--07**\*                |
|                                                                       |
| +------------------------+-----+----------+----------+-----------+    |
| |                        |     | *        |          |           |    |
| |                        |     | *Average |          |           |    |
| |                        |     | percent  |          |           |    |
| |                        |     | per VPSC |          |           |    |
| |                        |     | site**   |          |           |    |
| +------------------------+-----+----------+----------+-----------+    |
| | **School               | **  | **S      | **R      | **Schools |    |
| | characteristic**       | n** | ending** | eceiving | that are  |    |
| |                        |     |          | schools  | both      |    |
| |                        |     | *        | only**   | sending   |    |
| |                        |     | *schools |          | and       |    |
| |                        |     | only**   |          | re        |    |
| |                        |     |          |          | ceiving** |    |
| +------------------------+-----+----------+----------+-----------+    |
| | **Proportion of        | 499 | 71.7     | 56.9     | 72.6      |    |
| | students:**            |     |          |          |           |    |
| |                        |     |          |          |           |    |
| | > **Race/ethnicity:    |     |          |          |           |    |
| | > nonwhite**           |     |          |          |           |    |
| +------------------------+-----+----------+----------+-----------+    |
| | > **Eligible for Free  | 592 | 74.2     | 59.0     | 57.8      |    |
| | > and Reduced-Price    |     |          |          |           |    |
| | > Lunch Program**      |     |          |          |           |    |
| +------------------------+-----+----------+----------+-----------+    |
| | **Title I schools**    | 565 | 93.1     | 47.3     | 75.3      |    |
| +------------------------+-----+----------+----------+-----------+    |
|                                                                       |
| Exhibit Reads: On average (per site), 71.7 percent of students in     |
| sending schools were nonwhite.                                        |
|                                                                       |
| \*The School Survey covered 12 of the 13 sites in the VPSC Program    |
| because the 13th site focused mainly on technical assistance.         |
|                                                                       |
| Source: Survey of Schools, COSMOS Corporation, 2006--07.              |
+-----------------------------------------------------------------------+

The VPSC legislation favored initiatives that promoted the transfer of
students from low-performing schools to higher-performing schools. The
legislation used *ESEA* Title I provisions to define "low-performing"
schools as schools not making academic yearly progress (AYP) two years
in a row. Some of the sites had no Title I schools but used the
accountability provisions within their states to define "low-performing"
schools. In contrast, the sites did not specify explicit criteria for
defining "higher-performing" schools, only that such schools could not
be "low-performing" schools. Therefore, designated sending schools at
these sites were schools identified for improvement and designated
receiving schools generally had made AYP for two previous years (see
exhibit 4-4).

***Transfers from low- to higher-performing schools comprised only a
portion of the students enrolled in the VPSC initiatives*.** Five of the
VPSC sites created choice

arrangements with predesignated sending and receiving schools. Of these,
only three limited their enrollment to transfers from low- to
higher-performing schools in 2005--06 (see exhibit 4-5, row A).
Similarly, although another five sites permitted transfers throughout a
district or a zone, only two tracked the portion of the transfers from
low- to higher-performing schools (see exhibit 4-5, row B).

Thus, aside from the five sites that either limited their transfers or
tracked the transfers from low- to higher-performing schools, none of
the other eight sites could provide such information (see exhibit 4-5,
row C). At the five sites, the confirmed

+-----------------------------------------------------------------------+
| **Exhibit 4-4**                                                       |
|                                                                       |
| **Reported AYP Status of Schools,**                                   |
|                                                                       |
| **2004--05 through 2006--07**\*                                       |
|                                                                       |
|                                                                       |
|  ----------------------- ----------- ------------ --------- --------- |
| ---- --------- ----------------- --------- ----------- --------- ---- |
|   **AYP status in the                 **No. (and                      |
|                                                                       |
|   past three years\                   percent)**                      |
|                                                                       |
|   (2004--05 through                                                   |
|                                                                       |
|   2006--07)**                                                         |
|                                                                       |
|                                                                       |
|                           **Sending                          **Rec    |
| eiving             **Schools that              **Total**              |
|                           schools                            schoo    |
| ls                 are both sending                                   |
|                           only**                             only*    |
| *                  and                                                |
|                                                                       |
|                    receiving**\*\*                                    |
|                                                                       |
|   **Made AYP**            9                        (21.4)    21       |
|          (77.7)    114               (34.8)    144         (36.3)     |
|                                                                       |
|   **Failed to make AYP 1  22                       (52.4)    4        |
|          (14.8)    90                (27.4)    116         (29.2)     |
|   year**                                                              |
|                                                                       |
|                                                                       |
|   **Failed to make AYP 2  9                        (21.4)    2        |
|          (7.4)     86                (26.2)    97          (24.4)     |
|   consecutive years**                                                 |
|                                                                       |
|                                                                       |
|   **Failed to make AYP    2                        (4.8)     0        |
|          (0.0)     38                (11.6)    40          (10.1)     |
|   all 3 years**                                                       |
|                                                                       |
|                                                                       |
|   **TOTAL**               42                       (100.0)   27       |
|          (100.0)   328               (100.0)   397         (100.0)    |
|                                                                       |
|  ----------------------- ----------- ------------ --------- --------- |
| ---- --------- ----------------- --------- ----------- --------- ---- |
|                                                                       |
| Exhibit Reads: Nine (21.4 percent) sending schools made AYP for       |
| 2004--05 through 2006--07.                                            |
|                                                                       |
| \*The School Survey covered 12 of the 13 sites in the VPSC Program    |
| because the 13th site focused mainly on technical assistance.         |
|                                                                       |
| \*\*One VPSC site with a large, districtwide choice arrangement and   |
| many low-performing schools accounted for more than half of the       |
| schools that did not meet their AYP goals.                            |
|                                                                       |
| Source: Survey of Schools, COSMOS Corporation, 2006--07.              |
+-----------------------------------------------------------------------+

transfers from low- to higher-performing schools (from rows A and B)
represented 1,295 of 5,927, or 21.8 percent, of their total transfers in
2005--06.

The other eight sites either permitted a wider variety of transfers or
had VPSC enrollments that involved no transfers. Among the sites not
tracking transfers, one received a waiver from the Department to omit
such tracking because, to be eligible to apply for transfer, all the
students had to be low-performing (scoring "below proficient" on the
state assessment) as well as of low-income backgrounds (eligible for the
Free and Reduced-Price Lunch Program). At the same time, the actual
transfers at this site were not necessarily transferring from low- to
higher-performing schools. Thus, across the entire VPSC Program, the
actual portion of low- to higher-performing transfers could be larger or
smaller, depending on the nature of the transfers at the seven sites
that did not track or document the pattern of their transfers (row
C).[^20]

+-----------------------------------------------------------------------+
| **Exhibit 4-5**                                                       |
|                                                                       |
| ####                                                                  |
| ## Student Transfers from Low- to Higher-Performing Schools, 2005--06 |
|                                                                       |
| > Exhibit Reads: The choice implementation at three sites targeted    |
| > students transferring from low- to higher-performing schools. The   |
| > sites had a total enrollment of 650 students, all of whom were      |
| > transfers from low- to higher-performing schools.                   |
| >                                                                     |
| > \*Of the 13 VPSC sites, 12 reported enrollment in 2005--06. The     |
| > 13th site has mainly focused on choice-related technical            |
| > assistance.                                                         |
| >                                                                     |
| > Source: Survey of Schools, COSMOS Corporation, 2006--07.            |
+-----------------------------------------------------------------------+

In some cases, sites delayed notifying parents of their choice options
until their states published the names of schools identified as
low-performing. As a consequence, parents and students at these sites
were notified of their eligibility only a few weeks before they had to
make a decision on whether to apply to transfer to another school.
However, states' deadlines for issuing this information have varied over
the VPSC years, so the delayed notification was episodic rather than
chronic at any given site.

**4.2.3 Interdistrict Partnerships**

***Most of the VPSC sites limited their choice initiatives to
within-district options, rather than developing interdistrict
partnerships.*** Although the implementation of formal interdistrict
choice options might have expanded the variety of choices available to
students even further, only five of the 13 VPSC sites created
interdistrict options. At the eight remaining sites, many had existing
transfer understandings with neighboring districts. However, these
transfer options were separate from the VPSC initiatives and were
generally reviewed on a case-by-case basis. Even though the existing
options did not include support for students' transportation costs, the
eight sites nevertheless did not use VPSC funds to promote or support
interdistrict transfers at these sites.

Among the five sites that did offer interdistrict choice options, one
site used VPSC funds to support the opening of a commuter school. The
school provided an intra-district choice option to the district's
students. In addition, the school accepted transfer students from other
nearby districts, thereby providing an interdistrict choice option to
students applying from other districts.

At the other sites with interdistrict choices, the options varied from
interdistrict magnet schools[^21] to other formal transfer agreements
with neighboring districts. Interdistrict initiatives at these sites
included: 1) regional interdistrict magnet school programs, in which
magnet schools accepted students from 26 school districts; and 2) an
urban-suburban transfer plan, which allowed urban students to enroll in
any suburban district school and vice versa. Another partnership
included one urban school district and nine surrounding suburban
districts. Students from the urban district applying to transfer to any
of the suburban school districts were given priority in the application
process. This interdistrict initiative also allowed suburban students to
transfer to the magnet schools within the urban district.

**4.2.4 Support for Transportation Services**

The fourth VPSC priority required that sites use VPSC funds to support
student transportation services or costs. This requirement reflected the
assumption that students would be more willing to travel farther from
home and consider schools from a larger area than if the students did
not have to pay the cost of transportation themselves.

***Relative to enrollment, transportation costs did not increase
proportionately as might have been expected.*** This is partly because
the VPSC initiatives permitted many students who were already attending
distant schools to select schools closer to home. Although nine sites
reported using funds for transportation, overall these costs did not
necessarily increase. Many students already were attending distant
schools, and the VPSC Program now allowed these students to select
schools closer to home. Under these circumstances, sites experienced
minimal or reduced transportation costs.

Moreover, some of the VPSC Program's choice options encouraged students,
who otherwise might have been contemplating a transfer to a more distant
school, to remain at a neighborhood school. For example, one VPSC site
identified neighborhood schools in which enrollment had declined in part
because of their perceived poor quality. The site designated these
under-enrolled schools as "receiving schools" and improved their
education programs to encourage children to attend. As another example,
two of the VPSC sites had recently emerged from court-ordered school
busing, from which students had been assigned to more distant schools as
part of the original desegregation order. The VPSC-funded initiative
gave affected students the choice of returning to their neighborhood
schools.

The sites generally used VPSC funds to supplement the district's
transportation budget, with three sites also implementing services
specifically related to the VPSC initiative. Overall, VPSC funds
supported new technology and personnel to improve transportation
services. These improved services included: a Global Positional System,
mapping software for routing and dispatching school buses,
transportation route coordinators, and support staff.

Two initiatives also supported additional transportation activities
beyond transporting students to school. These activities included busing
students to before- and after-school activities, transporting parents to
visit choice schools, and driving parents and students to VPSC-funded
resource centers (e.g., Parent Information Centers).

In cases in which sites did not require additional funds for
transportation, the sites received waivers from the Department to exempt
them from the original requirement. One of these sites established
off-campus receiving schools, which made transportation unnecessary for
daily attendance. Other sites found it unnecessary to allocate
additional funding toward transportation as other district funds were
sufficient.

In summary, the VPSC sites' experiences demonstrated that transportation
costs need not increase and may even decrease with the implementation of
public school choice, depending upon preexisting enrollment patterns and
the design of the choice initiative itself.

**4.3 Implications for Federal Education Policy and Local Education
Practice:**

**Further Issues of Educational Equity and Excellence**

The findings in this chapter have directed attention to overall student
participation in the VPSC Program and the sites' progress in addressing
the VPSC Program's major priorities. However, an examination of
"educational equity and excellence" can go beyond these issues.

One type of additional examination pertains to potential lessons learned
about the *design* of the various choice arrangements at the 13 VPSC
sites. This is discussed next.

***To meet different equity objectives, different types of choice
arrangements may be needed.*** Concerns over equity typically embrace a
multi-faceted set of objectives:

> 1\) Students, especially disadvantaged students (e.g., those of
> low-income status, or having low academic achievement), need to have
> greater opportunities to obtain quality education;
>
> 2\) Students of all backgrounds should have choices that allow them to
> match their own interests and goals with the variety of settings
> present in school systems; and

3\) Low-performing schools need to improve.

Any specific choice initiative or arrangement may not be able to pursue
all of these objectives simultaneously. In fact, the VPSC Program's
experience raises the possibility that most arrangements may only be
able to focus on one or two of these objectives.

A two-by-two matrix can start to represent these three objectives (see
exhibit 4-6). The matrix also provides a way of assigning ten of the 13
VPSC sites' initiatives (all but three sites could be categorized as
having implemented "within-school options" or "mixtures" of different
options).

The two dimensions of the matrix specify whether eligibility to
participate in a choice arrangement should be broad or restricted, with
regard to either the eligibility of the students or the eligibility of
the schools. These two-by-two combinations result in the four cells
labeled "(A)," "(B)," "(C)," and "(D)."

Cell (A) represents the broadest combination. It includes not only
districtwide initiatives but also zonal initiatives. Although the
designation of zones can have a potentially exclusionary effect, the two
sites with designated zones did not exclude students who might have
wanted to enroll within the zones but who came from sending schools
outside of the zones (see appendix C, exhibit C-6, for more details). As
a result, these initiatives had a "broad" representation of eligible
schools despite their zonal character.

Cell (B) represents a more limited combination, with restricted sending
schools but with all students at these schools eligible to participate.
However, the eligible sending schools do not need to be low-performing
(see appendix C, exhibit C-7, for more details).

Cell (C) also represents a limited combination. However, it differs from
cell (B) because it limits the type of students eligible to participate,
even though the students may be located at a broad array of eligible
sending schools. Finally, cell (D) depicts the most limited combination:
only a certain set of students, from a certain set of sending schools,
are eligible to participate.

+-----------------------------------------------------------------------+
| ###### Exhibit 4-6                                                    |
|                                                                       |
| **Eligibility to Participate in VPSC's Choice Arrangements:**         |
|                                                                       |
| **An Equity Perspective**\*                                           |
|                                                                       |
| +---+---+-----------------------------+-----------------------------+ |
| |   |   | **Eligible sending          |                             | |
| |   |   | schools**                   |                             | |
| +---+---+-----------------------------+-----------------------------+ |
| |   |   | **Broad**                   | **Restricted**              | |
| +---+---+-----------------------------+-----------------------------+ |
| | > | > | ALL STUDENTS AT ALL         | ALL STUDENTS AT SELECTED    | |
| |   |   | SCHOOLS:                    | SENDING SCHOOLS:            | |
| | * | * |                             |                             | |
| | * | * | -   districtwide            | -   students from           | |
| | E | B |     initiatives (2)         |     designated sending      | |
| | l | r |                             |     schools, but not        | |
| | i | o | -   multi-school zones with |     necessarily             | |
| | g | a |     mostly zonal but also   |     low-performing schools  | |
| | i | d |     districtwide            |     (2)                     | |
| | b | * |     eligibility (2)         |                             | |
| | l | * |                             | -   students only from      | |
| | e |   | **(A)**                     |     low-performing sending  | |
| | > |   |                             |     schools (3)             | |
| |   |   |                             |                             | |
| | s |   |                             | **(B)**                     | |
| | t |   |                             |                             | |
| | u |   |                             |                             | |
| | d |   |                             |                             | |
| | e |   |                             |                             | |
| | n |   |                             |                             | |
| | t |   |                             |                             | |
| | s |   |                             |                             | |
| | * |   |                             |                             | |
| | * |   |                             |                             | |
| +---+---+-----------------------------+-----------------------------+ |
| |   | > | SELECTED STUDENTS AT ALL    | SELECTED STUDENTS AT        | |
| |   |   | SCHOOLS:                    | SELECTED SCHOOLS:           | |
| |   | * |                             |                             | |
| |   | * | -   disadvantaged students  | -   none (0)                | |
| |   | R |     coming from any school  |                             | |
| |   | e |     in the district (1)     | > **(D)**                   | |
| |   | s |                             |                             | |
| |   | t | > **(C)**                   |                             | |
| |   | r |                             |                             | |
| |   | i |                             |                             | |
| |   | c |                             |                             | |
| |   | t |                             |                             | |
| |   | e |                             |                             | |
| |   | d |                             |                             | |
| |   | * |                             |                             | |
| |   | * |                             |                             | |
| +---+---+-----------------------------+-----------------------------+ |
|                                                                       |
| > Exhibit Reads: Two districtwide initiatives defined eligibility to  |
| > participate in a choice arrangement broadly by focusing on all      |
| > students at all schools.                                            |
|                                                                       |
| \*The number in parentheses in each cell represents the number of     |
| VPSC arrangements falling into a particular category. Not shown are   |
| the three arrangements that were either "within-school" arrangements  |
| or a mixture of arrangements.                                         |
|                                                                       |
| Source: Analysis of site visit data, by COSMOS Corporation, 2007.     |
+-----------------------------------------------------------------------+

> Within all four cells, the distribution of the ten VPSC initiatives
> shows the limited extent to which the initiatives ended up being
> restrictive, even in their design. Only one initiative confined
> eligibility to disadvantaged students, defined as students eligible
> for the Free and Reduced-Price Lunch Program and with lower academic
> achievement scores \[see cell (C)\]. Similarly, only three initiatives
> confined eligibility to low-performing sending schools, but within
> these schools the sites imposed no restrictions on the eligibility of
> participating students \[see cell (B)\]. The remaining sites all had
> unconstrained *eligibility* requirements, although the bulk of the
> students enrolling in these initiatives may still have been
> disadvantaged students.

The matrix and the VPSC Program's experience suggest that, if future
policymaking and practice are intent on promoting transfers from low- to
higher-performing schools, choice arrangements may need to be more
restricted. For example, the VPSC Program might not just have encouraged
transfers from low- to higher-performing schools but might have
*limited* transfers to those situations. However, limiting choice to the
most relevant condition---the simultaneous combination of a)
low-performing sending schools, and b) disadvantaged students \[see cell
(D)\]---may be infeasible from the standpoint of local school politics
and community cohesiveness. Note that this most restrictive type of
choice arrangement was not found in the VPSC Program.

***To promote excellence, more guidance for capacity-enhancing
activities may be needed.*** No matter what the choice arrangement, an
implicit assumption is that choice students should enroll at a
higher-performing school, if not a school of excellence. To boost the
quality of educational programs at receiving schools, all of the VPSC
sites took steps to enhance the capacity of their receiving schools. At
some of the receiving schools, the capacity-building took place in the
name of emulating educational programs already found to be successful at
other schools in the same district. However, such new adoptions did not
necessarily assure that the receiving school would become a
high-performing school.

One issue raised here is why the receiving schools should have been
changing their academic programs in the first place. Experience with
voucher programs for public school students to attend private schools
seems to suggest the opposite. Beyond potentially reinforcing the
incoming students' orientation and academic support, the receiving
private schools continue to adhere to their existing academic programs
(e.g., see U.S. Department of Education, 2007, and the evaluation of the
D.C. Scholarship Program). In fact, many private schools have had
sufficient success with their academic programs that the schools are
almost rigid in adhering to their programs, even in the face of external
circumstances much more dramatic than participating in a voucher
program.

For the receiving schools in the VPSC initiatives, one would assume that
the greater challenge was to maximize the number of seats available in
already higher-performing or excellent schools at the VPSC sites, not to
modify in any way the schools' existing academic programs. Yet, in the
VPSC Program, every site provided extensive support to change academic
programs at the receiving schools, and these schools appear to have
enthusiastically embraced such activity (see appendix C, exhibit C-8,
for more details).

Under these circumstances, the VPSC Program might have offered more
guidance on the nature of capacity-enhancing activities, including the
presumed need to select programs supported by scientifically based
research. Absent any guidance, some VPSC sites added to the academic
programs of receiving schools that had already been identified as
higher-performing. Other sites started entirely new receiving schools,
charter schools, off-campus programs, or new magnet-like programs. Yet
other sites strengthened programs at under-enrolled schools to appear
more attractive as receiving schools.

**5. STUDENT ACADEMIC ACHIEVEMENT TRENDS**

**CONCURRENT WITH THE VPSC INITIATIVES**

To assess student achievement trends, the analysis in this report traced
student performance from the period prior to VPSC enrollment through the
following years afterwards. These trends were then compared to those of
a similar group of students who had not enrolled in any VPSC choice
initiatives.

Three cautionary notes must accompany the findings in this chapter.
First, any observed student performance trends by the enrolling students
cannot be readily attributed to VPSC initiatives. For example, more
motivated students might have chosen to enroll in the VPSC initiatives
than those who did not, and even though the analysis also includes data
from comparison groups of non-enrolling students, the VPSC Program was
not designed as an experiment that might have ruled out such a
self-selection bias.[^22] Thus, the observed student performance trends
should only be considered as "concurrent" trends. These trends have
occurred during the period of any given VPSC site's choice initiative,
but they have an unknown relationship to the initiative.[^23]

The second cautionary note derives from the measures of student
performance used throughout the analyses---scores on state achievement
tests. The caveat in using these scores is that changed scores from year
to year may reflect either changed student performance or changed
procedures in scoring the states' criterion-referenced tests. At the
same time, the use of state assessment scores occupies a central role in
major federal educational policies and especially the school
accountability provisions under *NCLB*. Thus, the use of these scores in
the following analysis appears justified, though with the appropriate
caution.

The third caution is that because the usable data were only available
from a few of the VPSC sites, the aggregate analysis may not represent
the VPSC Program as a whole.

**5.1 Collection of Individual-Level Student Data**

All of the 13 VPSC sites were charged, from the beginning of their
awards, with collecting and reporting records containing
individual-level student achievement data. In principle, every VPSC site
should have submitted individual-level data, coded to protect the
anonymity of the actual student, that followed a template provided by
the national evaluation (see exhibit 5-1). The template asked sites to
define a student's enrollment and demographic status, along with
achievement scores over a multiple-year period. The ideal period would
have started with the year prior to a student's enrollment in a VPSC
site's initiative and then continued with annual scores for as many
years as possible thereafter.

+-----------------------------------------------------------------------+
| **Exhibit 5-1**                                                       |
|                                                                       |
| **Desired Student Record**                                            |
|                                                                       |
| +------------------------------------------------------------------+  |
| | ***[Variables:]{.underline}* [2001--02]{.underline}              |  |
| | [2002--03]{.underline} [2003--04]{.underline}                    |  |
| | [Etc.]{.underline}**                                             |  |
| |                                                                  |  |
| | **1. Coded ID, but associated *Consistent Across Years***        |  |
| |                                                                  |  |
| | **with the same student**                                        |  |
| |                                                                  |  |
| | **2. Current Grade Level**                                       |  |
| |                                                                  |  |
| | **3. Current School**                                            |  |
| |                                                                  |  |
| | **4. Current District**                                          |  |
| |                                                                  |  |
| | **5. Race/Ethnicity**                                            |  |
| |                                                                  |  |
| | **6. Free and Reduced-Price**                                    |  |
| |                                                                  |  |
| | **Lunch Status**                                                 |  |
| |                                                                  |  |
| | **7. Gender**                                                    |  |
| |                                                                  |  |
| | **8. Achievement Scores**                                        |  |
| |                                                                  |  |
| | **(Math and Reading)**                                           |  |
| |                                                                  |  |
| | **9. Testing Grade**                                             |  |
| |                                                                  |  |
| | **10. Name of Assessment**                                       |  |
| +------------------------------------------------------------------+  |
|                                                                       |
| > Exhibit Reads: A coded student ID consistent across years.          |
|                                                                       |
| Source: National evaluation team.                                     |
+-----------------------------------------------------------------------+

The template also asked sites to provide similar data for a non-VPSC
group of students.[^24] The sites were asked to identify the comparison
groups in either of two ways: a) a matched group of non-VPSC enrollees
selected to mimic the demographic characteristics and baseline academic
performance of the VPSC enrollees, or b) the entire set of students
remaining at the sending schools from which the VPSC enrollees came.
Regardless of the method chosen by a site, all subsequent analyses took
into account any differences in the demographic and baseline
characteristics of the VPSC and non-VPSC students before comparing their
student achievement scores.

Even if sites did not submit the needed data, they had the discretion to
analyze data on their own and to report the results in their annual or
final reports to the Department. Only one of these site-based reports
was sufficiently complete to be reviewed as part of the national
evaluation. The site's findings are discussed later in this chapter.

***The desired databases could contain one or more cohorts of first-time
enrollees and multiple years of annual data for each cohort.*** Because
choice enrollments occur every year, each site also could have submitted
data about more than one cohort of students. For instance, a site that
had started enrolling students in its VPSC initiative in 2003--04 should
have provided a baseline score (2002--03) and three additional years of
data for the cohort enrolling in 2003--04. The site also should have
provided another baseline score (2003--04) and two additional years of
data for the cohort enrolling in 2004--05. In addition, some sites
submitted data covering more than a single year prior to the onset of a
cohort's enrollment in the VPSC initiative.

In response to the template, and given the possibility of multiple
cohorts, sites began submitting data in 2005. However, the absence of
key information in these submissions precluded their use,[^25] and the
national evaluation recommended that the sites be given external
technical assistance (by another contractor, not the national evaluation
team) to collect and report the desired data. This assistance occurred
during the sites' fourth and fifth award years.

***Usable data came from four of the 13 VPSC sites, covering six of 38
potential cohorts of VPSC enrollees and comparison groups of
students.*** By mid-2007, when the analysis for the current report
needed to start, data from only four sites (and only covering six
cohorts) contained the essential information. The sample of sites
covered was therefore smaller than desired. Not only were the four sites
a fraction of the 13 total sites, but the six cohorts (of first-time
enrollees) were only a small fraction of the total number of 38 such
cohorts enrolled by the sites collectively, by 2005--06.

Nevertheless, even though only six cohorts of data were available for
the present analysis,[^26] sites appear to have made progress in
collecting data, and this effort should continue with the extension of
the VPSC Program beyond its initial five-year period. In particular, the
14 new awards made by the program in 2007, also for a five-year period,
include seven sites that had received the original awards in 2002 and
that then subsequently received external technical assistance. For this
reason, the potential for analyzing student achievement data has
improved, and any continuing evaluation of the program can anticipate
the availability of a larger and more representative set of data.

The six cohorts in the analysis came from four sites whose collective
profile was similar to that of the rest of the VPSC sites, as follows.
As with most of the VPSC sites (see exhibit 3-1 earlier), the four all
were urban sites with populations in the middle range (100,000 to 1
million). Similarly, the four sites mimicked the distribution of all of
the VPSC sites with regard to being spread across all three enrollment
size categories, and being nearly evenly split between sites with higher
and lower percentages of nonwhite students and students eligible for the
Free and Reduced-Price Lunch Program. Finally, the four sites also
evenly represented the two main types of choice arrangements
(pre-designated sending and receiving schools; and schools that are both
sending and receiving---see exhibit 3-4 earlier).

Across the six cohorts (see exhibit 5-2), the data coverage and the
starting years for a cohort's first-year enrollment in a VPSC initiative
varied (see exhibit 5-2, columns 2 and 3). This meant that only three
cohorts had two or more annual data points after enrollment, but the
other three cohorts only had one data point. Moreover, one of these
latter three cohorts had four data points prior to VPSC enrollment. The
desire to retain as much of the data submitted by the VPSC sites
accounted for this uneven coverage, but with all of the trends
re-centered around the same "t~1~" (the year of first enrollment),
regardless of the chronological year.

All of the six cohorts had both enrollee and comparison (non-enrollee)
groups. All of the sites chose to define their comparison groups by
using some type of matching procedure. The sites reported these matches
in the following manner.

For sites whose choice initiatives involved predesignated sending and
receiving schools, the comparisons were defined as: a) students
remaining at the sending schools but enrolled there for three years or
less, matched for gender, ethnicity, and English Language Learner (ELL)
status; or b) students remaining at the sending schools, matched on
grade

+-----------------------------------------------------------------------+
| **Exhibit 5-2**                                                       |
|                                                                       |
| **Individual-level Data Submitted by VPSC Sites**                     |
|                                                                       |
| +------+-------+-----------+--------+-------+-------+----------+      |
| | **(  | **    | **(3)**   | *      | **    | **    | **(7)**  |      |
| | 1)** | (2)** |           | *(4)** | (5)** | (6)** |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| |      |       | **Year of | *      |       | **N   |          |      |
| |      |       | first     | *Total |       | umber |          |      |
| |      |       | e         | enr    |       | of    |          |      |
| |      |       | nrollment | ollees |       | reco  |          |      |
| |      |       | (and      | re     |       | rds** |          |      |
| |      |       | number of | ported |       |       |          |      |
| |      |       | years of  | by     |       | *     |          |      |
| |      |       | data      | sites  |       | *used |          |      |
| |      |       | after     | (see   |       | in    |          |      |
| |      |       | first     | ap     |       | anal  |          |      |
| |      |       | enro      | pendix |       | ysis* |          |      |
| |      |       | llment)** | C-5)** |       | *\*\* |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| | **Co | **    |           |        | **    | **V   | *        |      |
| | hort | Years |           |        | Subje | PSC** | *Matched |      |
| | **\* | of\   |           |        | cts** |       | co       |      |
| |      | a     |           |        |       | **e   | mparison |      |
| |      | nnual |           |        |       | nroll | stu      |      |
| |      | d     |           |        |       | ees** | dents**† |      |
| |      | ata** |           |        |       |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| | **A  | 200   | 2005--06  | 170    | Math  | 170   | 161      |      |
| | -1** | 1--02 | (1)       |        |       |       |          |      |
| |      | to    |           |        |       |       |          |      |
| |      | 200   |           |        |       |       |          |      |
| |      | 5--06 |           |        |       |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| |      |       |           |        | Re    | 170   | 161      |      |
| |      |       |           |        | ading |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| | **B  | 200   | 2004--05  | 4,270  | Math  | 1,534 | 2,301    |      |
| | -2** | 2--03 | (2)       |        |       |       |          |      |
| |      | to    |           |        |       |       |          |      |
| |      | 200   |           |        |       |       |          |      |
| |      | 5--06 |           |        |       |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| |      |       |           |        | Re    | 1,526 | 2,295    |      |
| |      |       |           |        | ading |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| | **B  | 200   | 2005--06  | 1,494  | Math  | 950   | 914      |      |
| | -3** | 3--04 | (1)       |        |       |       |          |      |
| |      | to    |           |        |       |       |          |      |
| |      | 200   |           |        |       |       |          |      |
| |      | 5--06 |           |        |       |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| |      |       |           |        | Re    | 938   | 913      |      |
| |      |       |           |        | ading |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| | **C  | 199   | 2002--03  | 501    | Math  | 89    | 722      |      |
| | -4** | 9--00 | (2)\*\*\* |        |       |       |          |      |
| |      | to    |           |        |       |       |          |      |
| |      | 200   |           |        |       |       |          |      |
| |      | 5--06 |           |        |       |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| |      |       |           |        | Re    | 90    | 696      |      |
| |      |       |           |        | ading |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| | **D  | 200   | 2003--04  | 3,844  | Math  | 3,261 | 2,241    |      |
| | -5** | 2--03 | (3)       |        |       |       |          |      |
| |      | to    |           |        |       |       |          |      |
| |      | 200   |           |        |       |       |          |      |
| |      | 5--06 |           |        |       |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| |      |       |           |        | Re    | 2,974 | 2,198    |      |
| |      |       |           |        | ading |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| | **D  | 200   | 2005--06  | 3,783  | Math  | 2,922 | 1,000    |      |
| | -6** | 3--04 | (1)       |        |       |       |          |      |
| |      | to    |           |        |       |       |          |      |
| |      | 200   |           |        |       |       |          |      |
| |      | 5--06 |           |        |       |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| |      |       |           |        | Re    | 2,351 | 634      |      |
| |      |       |           |        | ading |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| |      |       | **TOTAL** | 14,062 | Math  | 8,926 | 7,339    |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
| |      |       |           |        | Re    | 8,049 | 6,897    |      |
| |      |       |           |        | ading |       |          |      |
| +------+-------+-----------+--------+-------+-------+----------+      |
|                                                                       |
| Exhibit Reads: The first cohort comes from VPSC site "A," and student |
| achievement test scores were available from 2001--02 to 2005--06.     |
|                                                                       |
| \*Sites enrolled new students in choice options each year, and each   |
| year's enrollees were considered "first-time" enrollees, whose        |
| achievement scores were tracked prior to VPSC enrollment and annually |
| thereafter. Thus, in theory, a site operating a VPSC initiative for   |
| three consecutive years could have had three cohorts of first-time    |
| enrollees, with each successive cohort having one less annual data    |
| point.                                                                |
|                                                                       |
| \*\*Many of the enrollees originally reported by the sites had        |
| insufficient test data and could not be included in the final         |
| analysis. For instance, the enrollees included young elementary       |
| students who had not taken more than one year of tests at most. Other |
| enrollees from higher grades might not have had three test scores     |
| because they were not tested each year.                               |
|                                                                       |
| \*\*\*This site collected data every other year between 2002 and      |
| 2006, so only three years of data were collected.                     |
|                                                                       |
| †The matched comparison students were those not enrolled in the VPSC  |
| initiatives but who had similar demographic and academic              |
| characteristics as the VPSC enrollees (see text for discussion of     |
| sites' matching procedures).                                          |
|                                                                       |
| Sources: Student-level databases submitted to the national evaluation |
| by the VPSC sites.                                                    |
+-----------------------------------------------------------------------+

level, gender, ethnicity, and eligibility for Free and Reduced-Price
Lunch (FRPL). For sites whose choice initiatives involved districtwide
initiatives involving schools that could be *both* sending and
receiving, the comparisons were defined as: c) students from the same
district, matched for gender, ethnicity, and eligibility for the Free
and Reduced-Price Lunch (FRPL) program; or d) subgroups of comparison
students by grade level, each subgroup having aggregate characteristics
matching as closely as possible the transfer groups in gender and
ethnicity.

As a final note, the analytic preference for multiyear trends (as
opposed to using a "pre-post" design requiring only two data points)
reflected the nature of public school choice initiatives. Unlike the use
of a new curriculum or classroom technology that might take place within
a semester and whose "effects" might be anticipated immediately
following exposure, choice options do not reflect discrete, time-limited
"interventions." Choice involves a change in educational pathways or
school careers, and any likely impact on student achievement may only
occur over a period of time. However, given the sites' difficulties in
reporting achievement data, three data points were considered the
minimum number needed to calculate trends. The analysis therefore
excluded sites that only submitted two years of data.[^27] Similarly,
the paucity of multiyear data (especially following the first year of
enrollment) precluded any examination of more subtle issues, such as
whether students might have suffered an initial disruption but then
performed better, two to three years after changing schools.

**5.2 Methodology for Analyzing Student Achievement Trends**

The analysis[^28] used a meta-analytic strategy to aggregate scores
across sites and produce findings for the VPSC Program as a whole. The
main purpose of the national evaluation has been to assess the VPSC
Program as a whole. The challenge is to arrive at cross-site findings.

For the purpose of considering how to aggregate student achievement data
across sites, one complicating issue is that the program did not specify
how sites should implement their public school choice initiatives.
Instead, sites were encouraged to define choice initiatives that suited
their own local needs and circumstances. As a result, the 13 VPSC sites
all implemented different public school choice initiatives. Furthermore,
most of the sites are located in different states; these states use
different achievement tests to assess student performance.

These conditions precluded any direct aggregation of individual-level
student data *across* the VPSC sites, necessary to satisfy the national
evaluation's need to arrive at findings for the VPSC Program as a whole.
Instead, the site-to-site variations were better suited to a
*meta-analytic* strategy. This strategy called first for determining the
nature of the student achievement trends *within* each VPSC site, and
then using these separate findings as part of a meta-analysis to arrive
at findings *across* the VPSC sites.[^29]

Although the data and other conditions differed from cohort to cohort
(and site to site), the initial, within-cohort analysis followed the
same procedure, ultimately estimating a "mean effect size"[^30] for the
changes among the students in each cohort. In other words, the analysis
first estimated the achievement trends for each individual student. Then
the analysis combined these into a group trend representing the entire
within-site cohort of VPSC enrollees. Finally, the trends were compared
to similarly derived trends for the matched comparison students.

The meta-analytic procedure then combined all of the cohort-specific,
mean effect sizes to estimate a "grand mean" effect size (e.g., Cooper
and Hedges, 1994; and Lipsey and Wilson, 2001). Such a grand mean effect
size represents the aggregate "difference between differences" over time
for all of the cohorts and sites, thus creating the needed program-wide
benchmark for the VPSC Program as a whole.

***The main achievement trends were calculated on the basis of "scale"
scores and "standardized scale" scores.*** Because the key evaluation
question was to determine the student achievement trends occurring
concurrently with enrollment in a VPSC-funded choice initiative, the
main measure of a student's achievement was his or her score on the
annual state assessments.

The specific assessment tests vary from state to state and therefore
from VPSC site to site (except for those few VPSC sites that were
located in the same state). Moreover, the assessments are usually
*criterion-referenced* tests (CRTs) rather than *norm-referenced* tests
(NRTs). Unlike national tests such as the Student Achievement Tests
(SATs) administered by the College Board or the National Assessment of
Educational Progress (NAEP), neither the state assessment tests nor
their scoring metrics are comparable from state to state. Each state
usually reports its students' performance in two ways: 1) by giving a
"scale" score---the specific numeric score attained on the test by a
student; and 2) by indicating whether the student's score exceeded the
state's desired level of proficiency, usually reflected by four or five
categorical groupings of scores, such as "highly proficient,"
"proficient," "below proficient," and "basic."

Because the benchmark scores for achieving the four or five different
proficiency categories may change from year-to-year in the same state,
the present analysis only relied on the "scale" scores. However, the
numerics of the scale scores (e.g., some being very large numbers versus
others being small numbers, depending upon the test scoring procedure)
also led to the need for "standardizing" these scores---converting them
to a similar scale across tests and sites (see equation 1).

*z***-***score* **=**

> *i =* individual student achievement score
>
> *j* = year
>
> *k =* grade
>
> *X* = mean of all students' scale scores for same year and grade level
>
> *s* = standard deviation of all students' scale scores for same year
> and grade level

Any potential biases from "standardizing" scores are not easily known.
Therefore, preliminary analyses were conducted to examine both the
"standardized" and "scale" scores, and the analyses determined that the
results were consistent. The "standardized" scores were then used in the
remainder of this report.[^31]

***The initial preparation of the data included taking into account the
demographic characteristics of the students as well as their baseline
performance.*** Baseline demographic and academic differences are
well-known influences or at least important correlates of student
achievement outcomes. Any analysis of such outcomes, especially efforts
such as the present one to compare the performances of different student
groups, must take these conditions into account in order to make fair
comparisons. As previously discussed, the sites already had matched a
group of non-enrolling students to each group of enrolling students.
Nevertheless, to create as close statistical equivalence as possible,
the demographic and pre-VPSC achievement data were used to adjust the
standardized scores and later achievement trends in the following
manner.

The VPSC sites provided only limited demographic data about the
students. The data on the first condition covered different racial and
ethnic groups, but because the definition of these groups varied from
site to site, the information was condensed into two categories "white"
or "nonwhite." The sites' data also covered a second condition---whether
the student was eligible to participate in the Free and Reduced-Price
Lunch Program (FRPL). Educational analysts commonly use FRPL eligibility
as a proxy for a student coming from a low-income or poverty-level
background. The proxy measure is known to have inaccuracies and to mask
other important differences in students' family backgrounds (e.g., the
education level of the parents), but the measure has been used
extensively in research on student achievement. Equation (2) shows how
these conditions were taken into account in adjusting the standardized
scale scores:

**Equation 2**

*Z*(*SS*) = *b~race~* **\*** *race* **+** *b~FRPL~* **\*** *FRPL* **+**
є~*Z*(*SS*)~

> *Z(SS) =* standardized scale score
>
> *race* = race and ethnicity (white v nonwhite)
>
> *FRPL* = economic status indicator (eligibility for Free and
> Reduced-Price Lunch)
>
> є *=* residual

Following the adjustment for the demographic conditions, the achievement
trends for each student were then estimated using a simple linear or
growth model. These models were then used to incorporate a final step,
which was to account for differences in students' baseline achievement
scores, as these also can affect later outcomes (see Appendix A). Thus,
the final analyses were based on growth models of students' scores that
had been standardized and that had incorporated both demographic and
academic baseline conditions.

Profiles of these baseline demographic and achievement characteristics,
comparing the VPSC enrollees with the non-enrolling students, showed
differences between the groups. As previously discussed in chapter 4,
the enrollees tended to have a higher percentage of white students, a
lower percentage of students eligible for the Free and Reduced-Price
Lunch Program, and better achievement scores, compared to the
non-enrollees. However, none of the differences was statistically
significant, either within or across the six cohorts (see exhibit 5-3).
(For the purpose of comparing the baseline achievement levels, the
"percent proficient and above" was used for descriptive purposes to give
an idea of the level of student performance in the two groups, rather
than the standardized scale scores that were later used in the actual
trend analysis.)

**5.3 Findings on Student Achievement Trends**

The final student achievement trends came from six cohorts that included
VPSC enrollees and matched comparison groups of students, across four
VPSC sites. The cross-site analysis first estimated the student
achievement trends for these two groups separately and then compared the
trends between the two groups.

These two analyses served two different purposes. The first, estimating
trends separately, was needed to establish whether either the enrollee
or matched comparison groups was alone moving in a positive or negative
direction. The goal was to determine whether the enrollees might have
been performing worse, regardless of any relative difference between it
and the comparison students. The second, estimating trends relative to
each other, then captured the comparison between the two groups. (The
two analyses involved two different units of analysis, students in
exhibit 5-4 and cohorts in exhibit 5-5, and the values should not be
compared in these exhibits.)

+-----------------------------------------------------------------------+
| **Exhibit 5-3**                                                       |
|                                                                       |
| **Baseline**\* **Characteristics of VPSC Enrollees and Matched        |
| Comparison Students**                                                 |
|                                                                       |
| +-------------------------------------+-----------+---------------+   |
| |                                     | **        |               |   |
| |                                     | Percent** |               |   |
| |                                     |           |               |   |
| |                                     | **(across |               |   |
| |                                     | six       |               |   |
| |                                     | c         |               |   |
| |                                     | ohorts)** |               |   |
| +-------------------------------------+-----------+---------------+   |
| | **Baseline characteristics**        | **VPSC    | **Matched     |   |
| |                                     | en        | comparison    |   |
| |                                     | rollees** | s             |   |
| |                                     |           | tudents**\*\* |   |
| +-------------------------------------+-----------+---------------+   |
| | **A. Grade span:**                  |           |               |   |
| +-------------------------------------+-----------+---------------+   |
| | > Elementary                        | 45        | 54            |   |
| +-------------------------------------+-----------+---------------+   |
| | > Middle                            | 35        | 31            |   |
| +-------------------------------------+-----------+---------------+   |
| | > High                              | 20        | 15            |   |
| +-------------------------------------+-----------+---------------+   |
| | **B. Race:**                        |           |               |   |
| +-------------------------------------+-----------+---------------+   |
| | > White                             | 56        | 48            |   |
| +-------------------------------------+-----------+---------------+   |
| | > Nonwhite                          | 44        | 52            |   |
| +-------------------------------------+-----------+---------------+   |
| | **C. Free and Reduced-Price Lunch** | 41        | 49            |   |
| +-------------------------------------+-----------+---------------+   |
| | **D. Proficient and above**         |           |               |   |
| +-------------------------------------+-----------+---------------+   |
| | > Reading                           | 74        | 64            |   |
| +-------------------------------------+-----------+---------------+   |
| | > Mathematics                       | 72        | 61            |   |
| +-------------------------------------+-----------+---------------+   |
|                                                                       |
| Exhibit Reads: Of the VPSC enrollees, 45 percent enrolled in the      |
| elementary grades in the baseline year.                               |
|                                                                       |
| \*The baseline year is the year prior to a student's enrollment in a  |
| VPSC initiative.                                                      |
|                                                                       |
| \*\*Students who were not enrolled in the VPSC initiatives but who    |
| had similar demographic and academic characteristics as the enrolling |
| students.                                                             |
|                                                                       |
| Sources: Student-level databases submitted to the national evaluation |
| by the VPSC sites.                                                    |
+-----------------------------------------------------------------------+

***When the VPSC and non-VPSC trends were examined separately (see
exhibit 5-4), the VPSC enrollees' trends were neutral in math and
positive in reading but not statistically significant.*** More
important, the enrollee group's scores were not found to be declining in
any way. In contrast, the non-enrollee group showed a declining trend in
math proficiency that was statistically significant at the 99 percent
confidence level.[^32] The reasons for the decline are unclear. Too
little is known about the type of instruction or educational
opportunities offered to the non-enrollees, either at the sending
schools or for those sites whose non-enrollees came from different
schools throughout a district. Finally, the non-enrollees' trends for
reading were positive but not statistically significant.

+-----------------------------------------------------------------------+
| **Exhibit 5-4**                                                       |
|                                                                       |
| **Performance of VPSC and Matched Comparison Students, Analyzed       |
| Separately**                                                          |
|                                                                       |
| +--------------------+------------+-------+---+----------+------+     |
| | **Category**       | **Mean     | **Sta |   | **I      | **   |     |
| |                    | change per | ndard |   | ntercept | Stan |     |
| |                    | year**     | d     |   | in       | dard |     |
| |                    |            | eviat |   | stan     | dev  |     |
| |                    | **in       | ion** |   | dardized | iati |     |
| |                    | st         |       |   | scores** | on** |     |
| |                    | andardized |       |   |          |      |     |
| |                    | scores**\* |       |   |          |      |     |
| +--------------------+------------+-------+---+----------+------+     |
| |                    |            |       |   |          |      |     |
| +--------------------+------------+-------+---+----------+------+     |
| | **VPSC             |            |       |   |          |      |     |
| | enrollees:**       |            |       |   |          |      |     |
| +--------------------+------------+-------+---+----------+------+     |
| | > **-Math          | 0.00       | 0.35  |   | 0.04     | 1.31 |     |
| | > proficiency**    |            |       |   |          |      |     |
| +--------------------+------------+-------+---+----------+------+     |
| | > **-Reading       | 0.01       | 0.35  |   | 0.04     | 1.29 |     |
| | > proficiency**    |            |       |   |          |      |     |
| +--------------------+------------+-------+---+----------+------+     |
| |                    |            |       |   |          |      |     |
| +--------------------+------------+-------+---+----------+------+     |
| | **Matched          |            |       |   |          |      |     |
| | comparison         |            |       |   |          |      |     |
| | students:**        |            |       |   |          |      |     |
| +--------------------+------------+-------+---+----------+------+     |
| | > **-Math          | -0.02\*\*  | 0.33  |   | 0.04     | 1.29 |     |
| | > proficiency**    |            |       |   |          |      |     |
| +--------------------+------------+-------+---+----------+------+     |
| | > **-Reading       | 0.01       | 0.32  |   | -0.04    | 1.25 |     |
| | > proficiency**    |            |       |   |          |      |     |
| +--------------------+------------+-------+---+----------+------+     |
|                                                                       |
| > Exhibit Reads: VPSC enrollees' math proficiency changed an average  |
| > of 0.00 of a standard score in one year.                            |
| >                                                                     |
| > \*All data are *unweighted*. Some groups' data covered three years  |
| > and others covered four or five. However, this small difference was |
| > not assumed to create undesirable artifacts as might occur if the   |
| > groups varied, for instance, between three and twenty years.        |
| >                                                                     |
| > \*\*p\<.01                                                          |
| >                                                                     |
| > Sources: Student-level databases submitted to the national          |
| > evaluation by the VPSC sites.                                       |
+-----------------------------------------------------------------------+

***When the VPSC and non-VPSC trends were compared, students enrolling
in the VPSC initiatives had better student achievement trends than those
not enrolling.*** The enrollees surpassed the non-enrollees in the
trends for both mathematics and reading, with the differences in
mathematics being statistically significant at the 99.9 percent
confidence level and in reading at the 99 percent confidence level (see
exhibit 5-5). The comparison was based on the meta-analysis of effect
sizes across all six of the cohorts, with the individual effect sizes
for each cohort having already accounted for the demographic and
baseline differences between the VPSC and non-VPSC groups. Because the
units of measure were standardized scores, the trends cannot easily be
translated into everyday educational units, but the effect sizes (.020
and .028 for mathematics and reading respectively) appear to be
modest.[^33]

+-----------------------------------------------------------------------+
| **Exhibit 5-5**                                                       |
|                                                                       |
| **Aggregate Comparison of VPSC and Matched Comparison Groups,**       |
|                                                                       |
| **Across Cohorts (n=6)**                                              |
|                                                                       |
| +-----------------------------+-------------+-------------+-------+   |
| | **VPSC and matched          | **Weighted  | **95        | **z\  |   |
| | comparison cohorts          | mean gain   | percent     | sc    |   |
| | compared:**                 | for VPSC    | confidence  | ore** |   |
| |                             | cohorts**   | interval**  |       |   |
| +-----------------------------+-------------+-------------+-------+   |
| | > **-Mathematics**          | 0.020       | 0.0084 to   | 3.4   |   |
| |                             |             | 0.0311      | 1\*\* |   |
| +-----------------------------+-------------+-------------+-------+   |
| | > **-Reading**              | 0.028       | 0.0061 to   | 2     |   |
| |                             |             | 0.0388      | .70\* |   |
| +-----------------------------+-------------+-------------+-------+   |
|                                                                       |
| Exhibit Reads: The VPSC cohorts outgained the matched comparisons in  |
| mathematics by a standardized score of 0.020.                         |
|                                                                       |
| All data are *weighted* according to each site's variability (more    |
| variability means less weight).                                       |
|                                                                       |
| \*p\<.01; \*\*p\<.001                                                 |
|                                                                       |
| Sources: Student-level databases submitted to the national evaluation |
| by the VPSC sites.                                                    |
+-----------------------------------------------------------------------+

The completion of this meta-analytic procedure required the data to be
subjected to a *test of heterogeneity*, to determine whether excessive
cross-cohort differences existed. If so, such heterogeneity would raise
questions about the confidence or appropriateness of trying to use and
interpret a single average or mean, to represent the collection of
cohorts (the present analysis relied on a "grand mean" effect size to
summarize the trends across cohorts). The results of the heterogeneity
test show that the cohort-specific effect sizes were in fact too
heterogeneous (or different) in reading, and nearly so in mathematics,
to be adequately characterized by the grand mean as a summary
statistic.[^34] The results of the heterogeneity test therefore suggest
caution in calling attention to the enrollee and non-enrollee contrast.

The heterogeneity results also raised a relevant substantive issue: The
high variability among the individual effect sizes might have signaled
some important differences among the actual choice interventions. As
noted earlier, the four sites and six cohorts were evenly divided among
the two main choice arrangements (pre-designated sending, and receiving
schools and schools that are both sending and receiving). The possible
heterogeneity could have been a split between these two types of
arrangements---one arrangement having consistently larger effect sizes
than the other. However, inspection of the data did not show such a
split, neither type of arrangement showing an advantage over the other.
Similarly, the data were inspected for an outlier effect---that one site
or cohort might have been disproportionately influencing the grand mean.
However, the data showed no such outlier effect or any other anomaly of
note.

The caution stemming from the results of the heterogeneity test needs to
be accompanied by several other cautions about interpreting the
differences in achievement trends between the enrollee and non-enrollee
groups. The other reasons for exercising caution are as follows.

First, and most important, in all of the choice initiatives, students
can choose to enroll or not. As a result, an enrollee group may
represent more highly motivated students or differ in other unobserved
ways from a non-enrollee group, accounting for some or all of any
subsequent differences in student achievement. In other words, without
an experimental design, the designated non-enrollee group may not
adequately represent the desired counterfactual condition for drawing a
conclusion about the benefits of choice initiatives.

A possible clue regarding the differences between the enrollees and
non-enrollees in the present analysis is provided by the demographic and
academic contrasts between the two groups, presented earlier in exhibit
5-3. The contrasts showed that the enrollees already were performing
better than the non-enrollees and had a lower proportion of low-income
and minority students, though none of these differences had been
statistically significant. Nevertheless, without restrictions on those
eligible to enroll, a choice initiative may permit higher qualified
students to be among the enrollees. Such a possibility deserves careful
examination in future research.

Second, the trends in the present analysis, while representing a minimum
of three data points, leaned in the direction of having more of the data
points precede rather than follow enrollment in the VPSC initiative (see
exhibit 5-2, column 3). Confidence about the trends concurrent with a
VPSC initiative, much less inferences about any effects, would be
increased if analyses were based on a larger number of data points
*following* enrollment. Again, future research should try to garner a
larger number of post-enrollment data points to reduce the interpretive
problems.

Third, and more generally, the data only came from a small sample of
sites (four of 13) and from only a small sample of the enrolling cohorts
(six of 38 cohorts of enrollees across all of the VPSC sites by
2005--06). Data from more sites and more cohorts would produce a firmer
set of findings about the program as a whole. The ongoing VPSC Program
should put renewed emphasis on obtaining data from additional sites and
cohorts, given the new round of five-year awards made by the program in
2007. The possibilities of such additional data are especially strong,
given that seven of the newly awarded sites were continuations from the
first round of awards.

Finally, any continuing data collection and analysis effort should be
accompanied by a modified procedure for defining the comparison groups,
or non-enrollees. Sites should not try to match any particular group of
non-enrolling students, as the sites did in the current evaluation.
Rather, the more desirable procedure would be for sites to provide data
from a larger but nonselected set of non-enrolling students, such as
those students remaining in the sending schools or even a districtwide
set of students. Analytic procedures such as propensity score matching
or some similar procedure conducted by the evaluation team could then
provide a fairer selection of matched comparison groups than the sites'
procedures in the current evaluation.

Given all the cautions, the findings in this analysis of six cohorts
from four VPSC sites offer early promise regarding the potential
benefits of the VPSC Program. External to the present analysis, the
local evaluator at a fifth site reported findings for two cohorts of
enrollees in the fifth site's choice initiative. The evaluation relied
on cross-section comparisons only, not a trend analysis. The results
were mixed, with the older cohort significantly *out*performing a
matched sample of eligible but non-enrolling students one year after
enrollment, but the later cohort significantly *under*performing its
matched sample one year after enrollment. These mixed results need to be
interpreted with the same cautions as those discussed above.

**6. USEFUL CHOICE PRACTICES AND OVERALL LESSONS LEARNED**

**ABOUT VOLUNTARY PUBLIC SCHOOL CHOICE**

**6.1 Useful Choice Practices for Future Choice Initiatives**

Key VPSC Program sponsors hoped sites could demonstrate useful choice
practices for districts across the country, to help them implement the
Title I choice requirements of the *No Child Left Behind Act*.[^35] The
VPSC sites and the national evaluation have identified an array of such
practices and procedures.

***VPSC sites initiated choice-related practices potentially useful for
other districts.*** The emerging practices covered a range of topics,
including how to: design choice initiatives; involve parents and
families; design capacity-enhancing activities at schools; and develop
technical assistance procedures. Districts across the country can
certainly learn from the program's experiences and adapt specific choice
practices to implement in their own communities (see exhibit 6-1).

As an example of designing choice initiatives, one VPSC site
demonstrated how intradistrict zones can be structured to include a
heterogeneous group of schools. Students therefore have wide choice but
minimize transportation time because they can stay within a zone.

Similarly, as an example of a useful strategy for increasing parent
notification, one VPSC site developed a user-friendly online database.
Parents and students logged into this database and electronically
searched for information about their choice options. Because many
districts across the country have a large variety of choice options, the
amount of information available to parents and students may be
overwhelming. By organizing this information online, the site made it
easier for parents and students to quickly learn about their options.

Another VPSC site distributed hardcopies of materials to notify parents
of their options. Over the course of implementation, the site converted
its materials from a program-specific format (e.g., separate brochures
for each choice option) to a school-specific format (e.g., separate
brochures for each school, describing all of the choice options
available at the school). The latter format allowed parents to collect
information more easily about the programs offered at these schools,
rather than having to search through materials about all programs, some
of which were not relevant to the parents' interests.

+-----------------------------------------------------------------------+
| ###### Exhibit 6-1                                                    |
|                                                                       |
| **Potentially Useful Choice Practices from the VPSC Program**         |
+-----------------------------------------------------------------------+

To further facilitate parents' decisions, one site provided parents with
transportation details at the time parents were reviewing their school
choices. The district's transportation director attended the district's
magnet school fair and provided bus route maps to parents. She worked
with the parents in searching the transportation database to locate the
bus stops closest to their homes. A computerized district map then
identified potential stops and estimated a student's bus ride time for
each school option. For out-of-district students, the district also
provided an "alternate stop" at daycare providers if the student planned
to ride the bus five days a week. The transportation director interacted
with parents and responded to their questions. As a result of all this,
parents were able to incorporate accurate transportation information
into their school selection process.

Sites also engaged in creative capacity-enhancement efforts. For
example, one VPSC site developed a broad-based set of partnerships with
local businesses, community groups, and institutions of higher education
(IHEs). These partners helped the district to strengthen new education
programs. Moreover, the partnership, developed as a result of the VPSC
Program, successfully garnered additional funds to expand existing
educational programs further.

***One of the three state education agencies that received VPSC awards
focused its initiative on providing technical assistance to districts as
they developed and implemented their choice plans.*** This VPSC site
established formal partnerships between mentor districts and mentee
districts. All districts in the state were eligible to apply. The
initiative organized 13 districts, which already had been implementing
choice options, to serve as mentors. Each of the mentor districts
adopted a mentee district that either had no choice plan or was at an
early stage of developing a plan.

The mentor and mentee districts worked together to develop the mentees'
choice plans and to share desired practices. Mentor districts also
assisted mentee districts to develop effective parent involvement
strategies and school choice Web sites. In addition to providing
technical assistance, the mentor districts also used VPSC funds to
expand their own choice initiatives.

The mentors and the mentees both reported having benefited from the
program and plan to continue the collaborative relationship beyond the
end of the VPSC initiative. One complicating factor has been the
stability of the mentor-mentee partnerships. Of the 13 mentor districts,
one mentor withdrew from the initiative after participating for the two
initial years. Another mentor district, despite efforts to recruit a
suitable mentee, has been without a mentee district for three of the
four years of the choice initiative. As of 2006--07, 12 mentor districts
and ten mentee districts were participating, with two mentor districts
not having a mentee.

In addition, the same VPSC site established partnerships with two
universities. The universities used VPSC funds to support choice centers
that provided technical assistance to districts statewide. Therefore,
the initiative's activities targeted not only the participating mentor
and mentee districts, but also all the school districts in the state.

Districts requested technical assistance by calling a choice center or
by completing an online request form at the center's Web site. The
centers' staff made site visits to all mentor and mentee districts to
assess the districts' needs and provide assistance. The choice center
maintained a variety of online resources, including an online parent
survey available to any district to copy. The parent survey, as well as
many other resources, were available in English, Spanish, and Creole.
The Web site also provided information about choice-related parent
resources, educator resources, best practices, online courses in family
involvement, marketing materials, and professional marketing campaign
tips.

Besides these useful practices, the VPSC Program produced important
lessons about public school choice. The final two sections of this
report discuss these lessons.

**6.2 Overall Lessons Learned from the National Evaluation**

**of the VPSC Program**

***The VPSC Program demonstrated a variety of public school choice
options.*** Though limited to awards at only 13 sites, the VPSC Program
showed how four types of public school choice arrangements can work in a
diverse array of communities across the country.

Among the legislative priorities, the arrangements added a wider variety
of choice options, compared to what had previously existed at the sites.
The VPSC sites' experiences, as traced throughout the earlier chapters
of this report, also have produced important insights into the workings
of voluntary public school choice. These insights include the following:

-   Choice arrangements can range from: a) having predesignated sending
    and receiving schools, to b) more flexible arrangements whereby the
    same schools can both send and receive, and finally to c) choices
    among academic programs that take place entirely within the same
    school.

-   Some sites can have any combination of these arrangements
    simultaneously, including options that meet Title I requirements.
    Sites can therefore have a "tapestry" of choice options, serving
    many students under many conditions.

-   Measuring the extent of student participation in choice initiatives
    depends on whether the focus is on eligible, applying, or enrolling
    students; participation rates will vary depending upon the chosen
    measures.

-   Choice operations do not need to follow common stereotypes.

    -   Large metropolitan school districts already may have diverse
        urban-like and suburban-like schools, mitigating the need for
        interdistrict arrangements or transfers;

    -   Similarly, students may not need to travel greater distances to
        transfer to more attractive schools; instead, choice initiatives
        can try to make those schools closer to students' homes more
        attractive.

```{=html}
<!-- -->
```
-   The VPSC sites have developed a variety of procedures potentially
    useful for other local districts implementing choice options. The
    procedures include ways of gaining parent involvement and also ways
    of enhancing the capacity of receiving schools. One VPSC site also
    demonstrated an array of technical assistance practices for helping
    local districts.

With regard to assessing student outcomes, sites' efforts to collect the
needed student achievement data improved over time, suggesting that the
VPSC should continue examining student achievement trends as the program
enters its second five-year phase. This last observation is especially
pertinent because seven of the original awardees in 2002 also were among
those receiving new awards in 2007. Although this final report covers
the first five years of the VPSC Program, its continuation beyond 2007
will provide the opportunity for additional monitoring and lessons
learned about public school choice.

***The evaluation experience clarified desirable features to strengthen
future program-level assessments of voluntary public school choice.***
The lessons learned from the VPSC's cross-site and aggregate
program-level[^36] experiences also help illustrate how future
evaluations can potentially yield more conclusive findings about a
choice program as a whole. Such evaluations would encompass the
following critical features, also discussed throughout this report and
in appendix A.

-   *Careful recordkeeping and tracking of all participants*. Sites
    should keep accurate and aggregate counts of all three types of
    choice participants (eligibles, applicants, and enrollees) and also
    should compile individual student-level data for choice enrollees'
    demographic characteristics and student achievement scores.

-   *More annual data points, covering a longer period after initial
    enrollment.* Analyses of student achievement trends should be based
    on a minimum number of data points covering the years following
    enrollment in a VPSC initiative. For instance, a minimum of three
    years' achievement data after enrollment would produce greater
    confidence that trends had been concurrent with the VPSC initiative
    rather than with conditions prior to the initiative.

-   *Achievement data covering a larger proportion of sites and
    cohorts.* Achievement data from more sites and cohorts would produce
    a firmer set of findings about the program as a whole. The ongoing
    VPSC Program can put renewed emphasis on obtaining such data, given
    the new round of five-year awards made in 2007. The possibilities of
    such additional data are especially strong, given that seven of the
    newly awarded sites were continuations from the first round of
    awards.

-   *A modified procedure for defining comparison students.* Sites
    should not try to match any particular group of non-enrolling
    students, as was done in the current evaluation. Rather, the more
    desirable procedure would be for sites to provide data from a larger
    but non-selected set of non-enrolling students, such as those
    students remaining in the sending schools or even a districtwide set
    of students. Analytic procedures such as propensity score matching
    or some similar procedure would then be used to select a matched
    comparison group from the non-enrollees.

-   *Hypotheses about relationships to site profiles and activities.*
    Evaluations need to specify such hypotheses at the outset of a
    study. With sufficient data, such profiles and activities might
    include queries about the relationship between different types of
    choice arrangements and the nature and extent of choice outcomes.

-   *Inclusion of all choice options at a site as part of the same
    evaluation.* Most sites have a collection or tapestry of options,
    serving slightly different goals. Together, all of the options
    represent a site's public school choice initiative, and evaluating
    any single option alone does not represent the site's underlying
    policy goals (nor do sites necessarily maintain records on an
    option-by-option basis). Evaluations should therefore embrace all of
    the options at a given site if possible.

**6.3 Two Challenges for Improving Voluntary Public School Choice**

Finally, the VPSC Program's first five years also suggest a number of
challenges for future improvement. Two of these challenges especially
deserve attention by federal policymakers and local districts because
the challenges deal directly with the central themes of educational
equity and excellence.

***"Choice," by its nature, may need to cater to a broad array of
potential student and family preferences. However, the broader the
array, the more difficult for a choice initiative to limit participation
to a narrow set of (low-performing) students or schools.***

The VPSC sites did not necessarily confront this potential dilemma. As
previously highlighted in chapter 4 (see exhibit 4-7), some of the sites
designed restricted choice arrangements (e.g., a small set of
predesignated sending and receiving schools). However, the restrictions
did not simultaneously cover "selected students at selected schools." In
contrast, other sites designed "broad" arrangements, either defining
"all students at all schools" or "selected students at all schools" to
be eligible to participate.

Recalling that providing choice for *all* students is an equity
objective, the lesson may be to place importance on acknowledging the
need for a *collection* of choice options at any given site (see
appendix C, exhibit C-9, for more details). Rather than encouraging
sites to focus their attention on specific types of choice initiatives,
the aim would be to assist sites in considering the most appropriate
collection---some options appealing to all students or schools and other
options limited to selected students at selected schools. Only in this
way might the entire set of equity objectives be served.

Similarly, future evaluations of public school choice might embrace all
of the options at a given site, rather than focusing on a single
initiative. Such a complete picture of the entire collection of options
would be needed to provide a fair assessment of any equity outcomes.
Moreover, the broader scope would complement those situations when sites
do not themselves track participation in their options separately (i.e.,
option-by-option).

***The education programs within choice arrangements, not the
arrangements themselves, help set expectations for the desired
educational outcomes for students.*** In a certain way, choice
arrangements, such as those supported by the VPSC Program, represent
administrative arrangements. They aim to give students the opportunity
to enroll in different schools or in different academic programs within
a school. However, with the call for the VPSC sites to emphasize
transfers from low- to higher-performing schools, sites were expected to
go beyond promoting efficient and fair student choice and mobility. The
VPSC Program also aimed to improve student performance.

Although the VPSC initiatives devoted sufficient attention to needed
administrative procedures (e.g., notifying parents, soliciting and
reviewing applications, and making school assignments), a somewhat
surprising but consistent pattern among the VPSC sites was that they
also strengthened education programs at the receiving schools. These
education programs went well beyond the expected orientation or
transitional programs for newly incoming transfer students. The programs
included thematic programs (e.g., environmental studies or the arts),
computer laboratories, curriculum enhancements, and other programs of
study available to any student at the receiving schools.

In principle, other than orientation and transitional programs for
incoming students, the receiving schools in a VPSC choice arrangement
should not have needed to install new or modified education programs. As
with the private schools in the D.C. Scholarship Program discussed
earlier, the preexisting programs at potential receiving schools already
should have made them higher-performing schools, if not schools of
excellence. Therefore, changes would not be expected, and although the
original VPSC legislation permitted funds to be used for educational
programming, the legislation offered no specific guidance about such
programming. As one possible result, the newly implemented programs at
the VPSC sites did not necessarily stem from any systematic inquiry into
parents' desires (see appendix C, exhibit C-10, for more details), nor
did the VPSC sites, as noted earlier, follow any externally driven
selection process, such as relying on scientifically based research to
select the programs.

More importantly, neither the VPSC Program nor any of the sites
attempted to define a "higher-performing" school. Most of the sites
defined such schools as those which were not "low-performing" schools.
In other words, sites did not set any benchmarks for "higher"
performance. Also absent in the VPSC Program has been any discussion of
the standards for school excellence that should be maintained or
monitored during the implementation of a choice initiative (see appendix
C, exhibit C-11, for more details).

These remarks are not intended, in and of themselves, as criticisms of
the VPSC Program. The point is that the conditions make it difficult to
set expectations for the student performance outcomes in a choice
initiative. Students may be more highly motivated and work harder if
they have been able to select their school of enrollment, but their
educational outcomes still will be heavily influenced by the substance
of the education at the receiving schools and their classrooms. Thus,
students' academic outcomes in choice arrangements may still be no
better than the quality of the educational programs available to the
students, regardless of the amount of choice that has occurred. A
potential remedy is for choice arrangements to receive more guidance and
clearer definitions and expectations for designating schools as
"higher-performing" schools. These remedies would benefit future
students in public school choice options.

**REFERENCES**

Borman et al., *Comprehensive School Reform and Student Achievement: A
Meta-Analysis,* CRESPAR/Johns Hopkins University, Baltimore, Md., 2002.

Bryk, Anthony S., and Stephen W. Raudenbush, *Hierarchical Linear
Models: Applications and Data Analysis Methods*, Sage Publications,
Newbury Park, Calif., 1992.

Cochran, W. G., "Some Methods for Strengthening the Common χ^2^ Tests,"
*Biometrics*, 1954, 10:417--451.

Cohen, J., *Statistical Power Analysis in the Behavioral Sciences,*
Lawrence Erlbaum Associates, Inc., Hillsdale, N.J., 1988.

Cooper, Harris, and Larry V. Hedges (eds.), *The Handbook of Research
Synthesis,* Russell Sage Foundation, New York, N.Y., 1994.

Cooper, Harris M., "On the Significance of Effects and the Effects of
Significance," *Journal of Personality and Social Psychology*, 1981,
41(5):1,013--1,018.

Cooper, H., et al., "Making the Most of Summer School: A Meta-Analysis
and Narrative Review," *Monographs of the Society for Research in Child
Development,* Blackwell, Malden, Mass., 2000, 65(1), Serial No. 260.

Cullen, Julie B., Brian Jacob, and Steven Levitt, "The Impact of School
Choice on Student Outcomes: An Analysis of the Chicago Public Schools,"
*Journal of Public Economics*, 2005, 89:729--760.

Hyde, J.S., and M.C. Linn, "Gender Differences in Verbal Ability: A
Meta-Analysis," *Psychological Bulletin,* 1988, 104:53--69.

Lipsey, M. W., and D. B. Wilson, *Practical Meta-Analysis,* Sage,
Thousand Oaks, Calif., 2001.

Rubin, D.B., "Meta-Analysis: Literature Synthesis or Effect-Size Surface
Estimation?" *Journal of Educational Statistics*, 1992, 17(4):363--374.

U.S. Department of Agriculture, "National School Lunch Program,"
*Nutrition Program Facts*, Washington, D.C., December 2004.

U.S. Department of Education, Institute of Education Sciences, National
Center for Education Evaluation and Regional Assistance, *Evaluation of
the DC Opportunity Scholarship Program: Impacts After One Year*, by
Patrick Wolf, Babette Gutmann, Michael Puma, Lou Rizzo, Nada Eissa, and
Marsha Silverberg, Author: Washington, D.C., 2007.

U.S. Department of Education, Office of Elementary and Secondary
Education, "Public School Choice: Draft Non-Regulatory Guidance,"
Washington, D.C., February 2004.

Yin, Robert K., "Rival Explanations as an Alternative to 'Reforms As
Experiments'," in Leonard Bickman (ed.), *Validity and Social
Experimentation: Donald Campbell's Legacy,* Sage, Thousand Oaks, Calif.,
2000, pp. 239--266.

**Appendix A:**

**Detailed Evaluation Methodology**

**Contents**

**Page**

> 1\. Introduction A-3
>
> 2\. Data Collection A-5
>
> Exhibit A-1 Schedule of VPSC Field Visits A-6
>
> Exhibit A-2 VPSC School Survey: Number of Schools Surveyed and\
> Response Rate A-8
>
> 3\. Methods for Analyzing Student Achievement Data A-9
>
> 3.1 Overview A-9
>
> 3.2 Data Management A-9
>
> 3.3 Within-site Analysis A-12
>
> 3.4 Between-site Analysis A-14

**1. INTRODUCTION**

The three evaluation questions were: 1) What are the characteristics of
the VPSC Program's grantees; 2) How and to what extent does the VPSC
Program promote educational equity and excellence; and 3) What academic
achievement is associated with the VPSC Program? To address these
questions, the national evaluation followed a mixed quantitative and
qualitative methods research design, with data coming from a variety of
original and archival sources.

The evaluation was therefore a multisite, multiyear program evaluation,
aimed at evaluating the VPSC Program as a whole. The program covered 13
sites (nine districts, three state education agencies, and one nonprofit
organization[^37]) that received VPSC awards from the U.S. Department of
Education (the Department) in the fall of 2002.

***Timing of the initiatives.*** Some of the sites started enrolling
choice students as early as January 2003, but other sites carried out
planning or other activities and started the following year or even
later. Because all sites began some type of activity in 2002--03, the
school year 2001--02 was considered the base year prior to the
implementation of the VPSC initiatives. The evaluation then traced the
sites' activities for the duration of their five-year awards, through
2006--07.

At the same time, once having started the actual enrollment of students,
sites may have offered choice options to a new set of students every
year. For each cohort of students, the "intervention period" therefore
varied. At the student level, this variation was taken into account by
accommodating different base years for each cohort of students. A
consequence of this staggered pattern was that more years of annual data
were available for the older cohorts than for the younger ones.

***Definition of the initiatives being studied*.** The national
evaluation defined the initiatives under evaluation as the ones being
supported by VPSC funds. The use of VPSC funds not only defined the
initiatives of interest but also helped focus on specific facets of the
initiatives. For instance, for some of the 13 sites, the bulk of the
funds were used to support and enhance parent information centers or
outreach and media campaigns; yet others provided funds for new
educational programs at *receiving schools* to increase their capacity
and attractiveness to serve transferring students. The use of funds also
directed the national evaluation's attention to more specific
activities, while still attending to the overall VPSC choice initiatives
at the 13 sites.

***Observations about the use of lotteries as part of an experimental
design for studying voluntary public school choice.*** The use of random
assignment lotteries, when choice initiatives are oversubscribed, poses
a challenging situation for education analysts. On the one hand, the
existence of the lotteries appears to provide a "natural experiment" for
analysts interested in comparing students participating or not
participating in choice options. On the other hand, in order to conduct
the analysis and fairly interpret the results, a host of obstacles must
still be acknowledged and overcome. A tentative conclusion is that these
barriers outweigh the apparent benefits of using the lotteries as
"natural experiments."

First, the nonparticipants may have previously invested much effort
before entering the lottery---such as visiting prospective schools and
filling out applications for exercising choice transfers---all of which
may have raised their expectations that then were dissipated by their
lottery loss (not unlike applying to college, and then being rejected).
Analysts have not studied the consequences of losing under these
conditions, but analysts cannot assume that the losers' subsequent
status as nonparticipants represents the desired counterfactual or "no
treatment" group. In the choice initiatives' lotteries, those not
selected to participate may in fact have been subject to an alternative
and undesirable treatment: becoming a group of disheartened students
having to remain at their original schools and thereby performing worse
than they might otherwise have performed.

Second, the lottery losers may not remain nonparticipants after their
loss. The ongoing evaluation of the D.C. Opportunity Scholarship Program
that provides public school students with vouchers to attend private
schools (U.S. Department of Education, 2007) has found that 56.6 percent
of the lottery losers actually switched schools after having lost in the
lottery (p. 6). Although only 4 percent actually switched to private
schools, the rest did switch schools, presumably exercising a choice
option of unknown sorts---but again undermining the students' status as
a "no treatment" group. Given such actual participation rates, the U.S.
Department of Education report (2007) has therefore had to regard the
lottery as an "intent-to-treat," whereby a lottery *win* is recognized
but as an *offering* of an opportunity (in this case of a sizable
monetary scholarship) but not necessarily at all synonymous with actual
choice participation (the treatment of interest). In fact, observing the
lottery as an "intent-to-treat" calls further attention to the preferred
counterfactual for making comparisons to students who exercise a choice
option: the performance of students who did not even apply for the
lottery but who *stayed* at their assigned schools. However, the
*stayers* and *movers* are voluntarily formed groups, not subject to
random assignment.

Third, the VPSC sites were not organized to conduct or document their
lotteries as if the sites were conducting research studies. The VPSC
sites followed a number of procedures that minimized the ability to
study their lotteries: a) they required multiple applications from the
same student (one application per school, as in applying to a college or
university) and conducted the lottery among applications, not
applicants; b) they did not employ any follow-up procedure to determine
where lottery winners and losers may have subsequently enrolled; and c)
they neglected to retain lottery records because such records lose their
administrative value immediately following the conclusion of a lottery.

**2. DATA COLLECTION**

***Overview.*** The relevant data included: multiple site visits to the
VPSC sites; surveys covering an average of 50 schools at each VPSC site;
and the collection of archival data about the participating students and
the performance of schools in the district or the site as a whole.

For the site visits, the instruments included field protocols to guide
data collection. The protocols for the VPSC sites covered implementation
at the site, district, and school level. The school survey instrument
was directed to principals of schools participating in the VPSC
initiatives and to administrators of participating districts in one of
the statewide initiatives.

The instruments did not cover the collection and analysis of
quantitative archival data to address the evaluation questions related
to student achievement. Instead, individual-level student data were
compiled and provided to the national evaluation by the VPSC sites.

The completed national evaluation covered the entire five-year VPSC
Program and drew from multiple sources of evidence:

-   Annual (2002--03 through 2005--06):

> --- Original VPSC grant applications (summer 2002 only);
>
> --- Grant performance reports from each VPSC site;
>
> --- Individual records of participating students and
>
> a comparison group; and
>
> --- School-level records of student achievement trends
>
> at VPSC sites and comparison sites.

-   Selected Years (2003--04, 2004--05, and 2006--07):

> --- Field studies of VPSC sites and comparisons; and

--- Surveys of schools participating in the VPSC Program's

initiatives.

***Site visits*.** The site visits covered the VPSC project site as well
as one sending school and one receiving school (or two participating
schools at those sites not designated as specific sending and receiving
schools). The sites chose two schools with a lot of choice activity
(i.e., having many transferring students), so that the site visit team
could observe and learn about the VPSC Program's initiative in action.

During all three rounds of field visits, a two-person team conducted the
visits at each of the VPSC Program sites in the second semester of the
school year (see exhibit A‑1). The field protocols covered interviews
with the VPSC's project director and staff at each site and with other
key participants in the choice initiative, as well as the collection of
documents and archival data in relation to the interviews.

+-----------------------------------------------------------------------+
| **Exhibit A-1**                                                       |
|                                                                       |
| # Schedule of VPSC Field Visits                                       |
|                                                                       |
|   -------                                                             |
| ------------------------ -------------- -------------- -------------- |
|                                                                       |
|                            **2003--04**   **2004--05**   **2006--07** |
|                                                                       |
|   **                                                                  |
| 1. Arkansas Dept. of          March          March 21--23   May 8--10 |
|   Education (Ark.)**              31--April 2                         |
|                                                                       |
|   **2.                                                                |
|  Brighter Choice Charter    May 10--11     April 7--8     Jan. 17--19 |
|   Schools (N.Y.)**                                                    |
|                                                                       |
|   **3.                                                                |
|  Chicago Public Schools     Feb. 25--27    Feb. 23--25    Jan. 23--24 |
|   (Ill.)**                                                            |
|                                                                       |
|   **4. D                                                              |
| esert Sands Unified       Jan. 26--27    Jan. 31--Feb.  Jan. 30--Feb. |
|   School District (Calif.)**                     2              1     |
|                                                                       |
|   **5. Fl                                                             |
| orida Dept. of Education May 19--21     May 2--4       Feb. 27--March |
|   (Fla.)**                                                      1     |
|                                                                       |
|   **6.                                                                |
| Greenburgh Central School  March 9--11    April 4--5     March 20--22 |
|   Dist. No. 7 (N.Y.)**                                                |
|                                                                       |
|   *                                                                   |
| *7. Hillsborough County School Feb. 3--5      Feb. 14--16    May 1--3 |
|   District (Fla.)**                                                   |
|                                                                       |
|   **8.                                                                |
| Miami-Dade County Public   March 17--19   April 27--29   April 18--20 |
|   Schools (Fla.)**                                                    |
|                                                                       |
|   **9.                                                                |
| Minnesota Dept. of         April 21--23   April 11--13   April 24--26 |
|   Education (Minn.)**                                                 |
|                                                                       |
|   **10.                                                               |
|  Monadnock Regional School April 13--15   April 4-6      March 27--29 |
|   District (N.H.)**                                                   |
|                                                                       |
|   **11.                                                               |
|  New Haven Public Schools  June 1--3      May 10--11     March 13--15 |
|   (Conn.)**                                                           |
|                                                                       |
|   **                                                                  |
| 12. Portland Public Schools   March 9--11    May 10--12     Feb. 6--8 |
|   (Oreg.)**                                                           |
|                                                                       |
|   **1                                                                 |
| 3. Rockford School District  May 3--5       April 20--22   Jan. 9--11 |
|   #205 (Ill.)**                                                       |
|   -------                                                             |
| ------------------------ -------------- -------------- -------------- |
|                                                                       |
| > Exhibit Reads: The Arkansas Dept. of Education was visited in the   |
| > 2003--04, 2004--05, and 2006--07 school years.                      |
| >                                                                     |
| > Source: National evaluation team.                                   |
+-----------------------------------------------------------------------+

***School survey*.** The school survey gathered data from schools
participating in the VPSC initiatives,[^38] mainly corroborating the
participation in the VPSC initiatives by the specific schools.

The school survey was based on a closed-ended questionnaire, directed at
the schools' principals. The questionnaire requested information about:

-   Student demographics;

-   School performance;

-   Choice options available to students;

-   Percentages of students taking part in these options;

-   Methods by which choice information was being shared with parents;
    and

-   Whether staff members had been receiving professional development
    related to school choice.

At all but one of the VPSC sites, all of the schools eligible to
participate in the choice initiative were included in the survey. One
statewide initiative was the exception: No schools were surveyed.
Instead, the districts with five or more students participating in the
choice initiative were asked to complete the survey at the district
level.[^39]

The team used the following procedure to obtain a high response rate. At
the request of the sites, the evaluation team first gave the sites'
project directors the questionnaires to distribute to the schools, with
a personalized cover letter from the project director requesting the
schools' participation. Schools were asked to fax the completed
questionnaires directly to the evaluation team, or to return the
completed questionnaires to the site's project director. The sites sent
reminder e-mails to the nonresponding schools approximately one week
following the submission deadline. If necessary, the sites then made
follow-up calls to the schools.

Subsequently, and following the efforts by the VPSC sites' staff (and
with the permission of the site), the evaluation team then made at least
two rounds of follow-up calls directly to the nonresponding schools,
frequently re-faxing the questionnaire to the schools. Using this
procedure, the team achieved high overall response rates (see exhibit
A-2). During the final round of administering the survey, the evaluation
team distributed the survey to 689 schools. Respondents at 630 schools
completed and returned the questionnaire, resulting in an overall
response rate of 91 percent.

***Performance reports.*** The data collection also covered the VPSC
grantees' annual performance reports submitted to the VPSC Program
Office at the Department. Each year the office issues reporting
requirements, based in part on the data collection suggestions by the
national evaluation team. After extensive review of grantees'
performance reports, the evaluation team suggested that the sites
provide more detailed descriptions of their VPSC-funded initiatives in
subsequent annual reports.

For instance, the original reporting requirement had called for sites to
report expenditures in ten categories: personnel; fringe benefits;
travel; equipment; supplies; contractual; construction; other; indirect
costs; and training stipends. However, these categories made it
difficult to determine how funds were used in relation to the choice
activities. For example, staff hired to develop new transportation
routes and staff hired to run a parent information center would both
have been included under "personnel." The evaluation team therefore
suggested that the annual progress report include additional
information, and that sites accompany their reports with a list of the
sub-items used to compile the totals for each category (except for the
"fringe" and "indirect cost" categories). The team further suggested
naming the choice-related function (e.g., outreach; school
capacity-building; professional development; student transportation;
etc.) associated with each sub-item, rather than listing the name of the
vendor or person who incurred the expense.

+-----------------------------------------------------------------------+
| **Exhibit A-2**                                                       |
|                                                                       |
| # VPSC School Survey: Number of Schools Surveyed and Response Rate    |
|                                                                       |
| \*During 2003**--**04, the VPSC Program's initiatives had not yet     |
| scaled-up to include 650 participating schools.                       |
|                                                                       |
| \*\*In 2006**--**07, the site did not participate in the survey, as   |
| the initiative focused on technical assistance (TA), not schools.     |
| Data related to TA were collected during the site visit.              |
|                                                                       |
| Source: National evaluation team.                                     |
+-----------------------------------------------------------------------+

**3. METHODS FOR ANALYZING STUDENT ACHIEVEMENT DATA**

**3.1 Overview**

The complete analysis of the VPSC data required three primary steps with
multiple ancillary procedures within each step. Additionally, there were
many decisions made about the data and the analyses along the way that
may not be completely acknowledged within the body of the report. A
brief overview of the three steps is now followed by a more detailed
account.

The first step involved the process of merging and recoding the data.
Within that step, there were decisions pertaining to how certain
variables would be recoded and treated when values were missing.

The second step involved the primary within-site analyses whereby
individual growth curves were computed for each student who provided
sufficient data. Similar to the first step, there were many decisions.
The decisions within the second step regarded the handling of missing
data, centering scores at a specific time point, residualizing scores
based upon available covariates, and estimating effect sizes.

The third and final step involved the between-sites analysis. Data were
aggregated to form composite scores by subject area (i.e., math or
reading) and site. Once combined within subject and site, the effect
size data were compared via a meta-analytic framework. Decisions about
effect size weighing, outlier detection and handling, and data display
were relevant at this stage.

For the purposes of completeness and clarity, those steps and decisions
are detailed below. The steps are outlined in a sequential format to
guide the reader through the intricacies and decisions as they were made
in the process.

**3.2 Data Management**

***Merge raw data files into single, long format file.*** The VPSC sites
had provided their data in a series of Excel spreadsheets. Each
individual row in a table represented a single student from a single
cohort and data for a single year. Thus, to get data from a site for a
single cohort, data were extracted from the Excel tables and
concatenated to form a multiple-record or long file format. After
combining the data, each student had multiple rows in the data set. Each
row represented a specific time point during the data collection process
and contained the available achievement data. Once the long file was
complete, the student demographics were merged by the student's coded
identification number to form a complete data set for each cohort and
site. It is important to note that data were not combined between sites
or cohorts at this time.

***Recode numerical missing data codes.*** Once the data were arranged
in a suitable format, each value was checked to ensure that the range
and code were appropriate. There were some inter-site differences with
respect to missing data coding. For example, some sites coded missing
values with -9 exclusively while other sites used several different
codes---each code representing a different type of missing value.
Regardless of the code used, each missing value code was recoded to a
"not available" or "NA" specification so that the statistics package
would not treat the value as a legitimate observed value.

***Calculate key variables for subsequent analyses.*** Several variables
that were instrumental in subsequent stages of analysis were computed at
this point. One such variable was a dummy code that represented whether
the student changed schools between years regardless of group
classification (i.e., enrolled or comparison group). This code was
necessary to ensure that a fidelity check could be performed if
necessary. A second variable was computed and held in a separate data
file that represented the number of repeated observations per student.
This variable was instrumental in selecting students who might be
suitable for the growth curve analysis. The new data file contained the
student identification number, cohort, and number of times the student
had complete, valid achievement data. Data from this table formed the
basis for the next step of student selection.

***Identify students who possess sufficient data from both the enrolled
and comparison groups.*** Students were retained for the analysis if
data were provided for at least three (3) time points. The decision to
select only students with at least three time points was based upon the
fact that the aim of the analysis was to provide both individual level
trajectories as well as group aggregated trajectories. Fewer than three
time points would have led to some (in some sites many) students with
only sufficient data for estimating a gain score. That is, two data
points would have been sufficient for a change score based upon two
observed time points. The intercept and slope for that change score
would have been estimated without error since the line of best fit
(i.e., the estimated growth curve trajectory) would be drawn through
both points without any residual. Students who had only a single data
point would not have provided sufficient data for anything but a single,
point-estimate of performance. Thus, for the purposes of completeness
and comparability, only students who provided a minimum of three data
points were retained for subsequent analyses.

***Compute residualized scores from raw, standardized scores provided in
the original data.*** Each observed, raw standardized score was
residualized across all students to account for racial, ethnic, and
socioeconomic factors. Specifically, the variables "race" (a factor
coded according to OMB standards) and "eligibility for the Free and
Reduced-Price Lunch (FRPL) Program" were used to determine predicted
values from a general linear model. Those predicted values were
subtracted from the observed values to form residual scores. The
following equation represents the general formulation of those
residualized scores (*є~ij~*).

*є~ij\ ~***=** *y~ij\ --~ ŷ~ij~*

where the subscripts *i* and *j* represent the student and time point,
respectively. The predicted value (*ŷ~ij~*) was computed with the
following general linear model:

  -----------------------------------------------------------------------
  *ŷ~ij\ ~***=** *b~race~race~i~* **+** *b~frpl~frpl~ij~*

  -----------------------------------------------------------------------

Note that the race variable consists of a summation operator because
race was coded as a polytomous (i.e., multiple categories) discrete
variable. The residualized scores by site will differ somewhat due to
inter-site differences on the demographic characteristics.

***Transform the residualized scores into z-scores.*** Once the scores
were residualized to account for racial and socioeconomic indicators,
the residualized scores were further transformed into z-scores to allow
for comparisons between time and site. The z-score transformation was
essential because the data provided each year by each site contained
different standardized achievement scores. For example, some sites
provided data for some years that ranged from 50 to 200 whereas other
years data from the same students may range from 200 to 800.
Furthermore, grade was relevant to the transformation because the
achievement tests often differ enough by grade to warrant a different
transformed score. To eliminate those intra-site differences, the data
for each year were transformed into z-scores by year and grade. The
general approach to this z-score transformation used the following
standard z-score equation:

  -----------------------------------------------------------------------
  *z*(*є~ijkl~*) = ------------

  -----------------------------------------------------------------------

where the subscripts *k* and *l* represent the year and grade level of
the student, respectively. It is important to note also that the
transformation was done separately for each of the two educational
domains of math and reading. Therefore, for completeness, the equation
above may be further subscripted to indicate the achievement domain. For
clarity and simplicity, that more complicated equation was not included.

***Clean data and ensure values are complete and available for the
within-site analysis.*** The final stage of the data management process
involved a thorough check for missing or miscoded values. Some sites had
so few students at specific grade levels that the z-score transformation
process resulted in either illogical or missing values. In those cases,
there was nothing that could be done to retain those observations and
therefore they were omitted from the remaining analyses. The process of
identifying illogical or missing values was governed by sample size of
grade levels, actual z-score magnitude, and missing data codes. In
almost all cases where data needed to be discarded, the values were
confirmed by hand to ensure that there were no computational or data
handling errors.

**3.3 Within-site Analysis**

The second step in the overall data analysis plan focused specifically
on analyzing the final data set that was cleaned and coded in the
previous step. After careful consideration, there was a decision made
about the proper analytic technique for the within-site data. That
decision was an individual growth curve model. It was reasoned that the
process of reducing the data from the raw repeated observations into
growth curve parameters would offer the best opportunity to capture the
change process. Additionally, the options for computing growth curves
include both OLS and ML approaches. Both have their advantages but only
one seemed to fit best with the goals of reducing the repeated measures
while maintaining as much variability in the parameters. In other words,
there was an explicit goal to avoid any model-based truncation of the
parameters. Those aims led to the selection of an OLS (i.e., individual
regression) procedure instead of the more widely used ML methods found
in software packages such as HLM and SAS. The primary difference between
an HLM/Mixed model approach is that the level-one models allow for
fitting of the growth curve and error covariance structures. Because the
aims of the project were to use the growth curve parameters alone, the
OLS models delivered what we needed and provided a clear, reduced data
structure for secondary analysis. The complete process of within-site
analysis is detailed below.

***Estimate individual growth curve parameter estimation.*** Each
student that had sufficient data was analyzed using a simple linear
model. The linear model was specified for each subject area resulting in
two sets of growth curve parameters for each student. The linear model
equation for the growth curves is as follows:

  -----------------------------------------------------------------------
  *y~ij~* **=** *b~li~time~ij~* **+** *b*~o*i*~ **+** *є~ij~*

  -----------------------------------------------------------------------

where *y~ij~* is the observed value for student *i* at time *j*. The
first parameter of interest is the *b~li~* because that represents the
linear growth parameter over time (*time~ij~* ). Time was centered at
the point when the VPSC Program began at each site. Thus, time was
recoded from a calendar year to integers indicating the point at which
the program had begun (i.e., "0" denoting the inauguration of the VPSC
Program). Interpreting the slope parameter, a positive slope did not
mean that the student necessarily had higher achievement. Because the
scores were transformed at each year for each grade level, the positive
slope indicated that the student increased her relative standing in the
grade compared to previous years. All the values represented relative
achievement performance---again this was necessary to counter the
problems of non-commensurable exam metrics over time and between sites.

Two other parameters are worth mentioning: the intercept (*b*~0*i*~) and
the residual (*є~ij~*). The intercept represents the student achievement
point at which the student began the VPSC Program. Higher values
indicate that the student began at high values relative to peers at
baseline. The residual values show the efficiency of the prediction
models by student *i* and year *j*. Both the intercept and residuals are
important for subsequent analyses and diagnostics.

***Estimate effect size.*** Once the growth curve parameters were
available for each student and for each subject area, effect size
estimates could be estimated. The effect sizes were computed based upon
the rate of change in z-score units from the intercept. Specifically,
the equation for within-site, individual effect sizes was computed with
the following formula:

  -----------------------------------------------------------------------
  *d~within~* = ------------------------------

  -----------------------------------------------------------------------

where *b~li~* and *b*~0*i*~ are the individual growth curve parameters
estimated in the previous step, *time~i~* is the time student *i* spent
in the VPSC Program since its inception, and and are the standard errors
of the linear and intercept growth curve parameters. These effect sizes
are standardized values just like Cohen's *d* but they are weighted
based upon the number of values observed. More repeated measures meant
more weight to the estimated effect size. Effect sizes were then
estimated for random coefficient (often referred to as random effects)
models.

***Aggregate effect sizes within site.*** Once random coefficient effect
sizes were computed, the next step in the within-site analysis involved
combining the effects to summarize the overall site effect of VPSC.
Individual effect sizes were aggregated within site to produce a mean
overall effect by subject area. The most typical way of aggregating
these effects is by taking the measure of central tendency and
estimating the variance about that measure. In short, the mean
represented the average effect. Means, however, are adversely affected
by outlier values and the effect sizes computed in the previous analysis
produced a good number of values that fell well outside a believable
range. Even the weighting process employed above did not protect against
outlier estimates. Therefore, to treat the outliers, windsorized means
were computed for each site and for each subject area. The windsorized
means were based upon the innermost 99 percent of the data. Within-site
but between-group effect sizes were computed by using the standard
method of computing Cohen's *d*:

  -----------------------------------------------------------------------
  *d~between\ ~***=** ------------------------

  -----------------------------------------------------------------------

where *d~wE~* and *d~wC~* were the average, windsorized, within-site
mean effect sizes computed for the enrolled *E* and comparison *C*
groups, respectively. Additionally, the and values were the standard
deviations computed from the windsorized, within-site effect sizes for
the enrolled and comparison. Once the between-group effect sizes were
computed, further analyses were performed comparing the sites.

**3.4 Between-site Analysis**

The final major step in the VPSC analysis consisted of a between-site
analysis. A meta-analytic approach was used for this step for a variety
of reasons given in the main report.

The final student achievement trends came from six cohorts that included
VPSC enrollees and matched comparison groups of students, across four
VPSC sites. The cross-site analysis first estimated the student
achievement trends for these two groups separately and then compared the
trends between the two groups.

These two analyses served two different purposes. The first, estimating
trends separately, was needed to establish whether either the enrollee
or the matched comparison group was alone moving in a positive or
negative direction. The goal was to determine whether the enrollee group
might have been performing worse, regardless of any relative difference
between it and the comparison group. The second, estimating trends
relative to each other, captured the comparison between the two groups.
Thus, the ideal positive result would be a finding that both groups had
positive trends but that the enrollee group performed better than the
matched comparison group. A less ideal finding, from a policy
standpoint, would be that the enrollees performed better, but that both
groups had negative trends. This less ideal situation would produce
ambivalence in deciding, for instance, whether a particular educational
intervention was desirable or not.

The first, or separate group analysis, involved the following estimates:
the average linear slopes across sites (labeled as the "mean
change"),[^40] the standard deviations of those slopes, the average
intercepts across sites, and the standard deviations of the intercepts.
It is important to stress that these estimates all represented
*unweighted* data across the sites.

The second, or two-group comparison, involved the following estimates:
the weighted mean gain of the VPSC enrolled sites relative to the
comparison sites and the associated confidence intervals and statistics
for those values. As a point of emphasis, the second analysis differed
from the first because weighting was now applied (the greater the site's
variability, the lower the weight).

The following steps were used to estimate the overall effects. All
pooled effect sizes were weighted by an inverse variance procedure. For
the random effects meta-analytic model, the DerSimonian-Laird estimate
was used for weighting the effects prior to pooling. The equation below
provides an estimate of the heterogeneity variance that constitutes the
DerSimonian-Laird estimate of weighting pooled effects.

  -----------------------------------------------------------------------
  *τ*^2^ **=** ^------------------------------------^

  -----------------------------------------------------------------------

where *w~i\ ~*= *Var*(*Ө~i~*)^--\ 1^ which is the inverse of the
variance of the measure of interest in the *i*-th cohort and *x*^2^ =
(*Ө*~i~ **--** *μ*)^2^ with *μ* **=** *Ө*~i~ **/** . These pooled
effects, therefore, are weighted based upon the inverse of the variance
of the sampled effects. The weights only become relevant if there is
sufficient heterogeneity of effects between the sites. In the case of
the present analysis, there was sufficient variability to warrant the
weighting by this method.

***Conduct test of homogeneity.*** Summarizing disparate findings from
sites only makes sense if those findings are reasonably homogeneous.
That is, the sites must be viewed as sampled from a similar population
where a measure of central tendency (e.g., a mean) suitably
characterizes the effects for all sites. Testing for homogeneity in
meta-analyses typically involves the *Q-*test---a test developed by
Cochran (1954) to assess the average deviation from the mean. The
*Q-*test can be calculated by taking the weighted sum of the squared
effect size deviations from the mean effect size. Recall that effect
sizes (*d*) were computed as the difference between groups for each
subject area (see earlier effect size equation). The general formula for
the *Q* statistic is:

  -----------------------------------------------------------------------
  *Q* **=** *w~i~*(*T~i~* **--** *T*)^2^

  -----------------------------------------------------------------------

where *T~i~* represents the individual effect size estimates *d*
computed in previous steps, *T* is the average effect size estimate
computed for all studies, and *w~i~* is the fixed-effect weighting
factor for each study based upon the inverse variance of the study. The
term *w~i~* is defined as:

  -----------------------------------------------------------------------
  *w~i~* **=**

  -----------------------------------------------------------------------

where is defined as:

  -----------------------------------------------------------------------
  **= ------------ + ---------------**

  -----------------------------------------------------------------------

The statistic *Q* is chi-squared distributed, with degrees of freedom
(*df*) defined as *k* **--**1 where *k* is the number of cohorts.
Homogeneity is the null hypothesis, so values of *Q* that exceed the
chi-square critical value are indicative of heterogeneity. It should be
noted that the *Q* statistic is underpowered for small meta-analyses
(i.e., very few effects aggregated) and overpowered for large
meta-analyses. Regardless of these limitations, all meta-analyses rely
on the statistic to assess whether homogeneity can be ruled out.

**Appendix B:**

**Instruments Used in the**

**National Evaluation**

**Contents**

**Page**

> 1\. Questionnaire for Individual Schools B-3
>
> 2\. Site Visit Face Sheet B-7
>
> 3\. Interview with VPSC Director and Support Staff B-9
>
> 4\. Interview with Other Participating Persons B-19
>
> 5\. Interview with the Comparison District and Title I Staff B-21

**1. QUESTIONNAIRE FOR INDIVIDUAL SCHOOLS**

**National Evaluation of the Voluntary Public School Choice Program**

**COVER SHEET**

  ----- ------------------ -- ------------------------------------ --------
                                                                   

        **Date:**                                                  

        **Respondent:**                                            

        **Title:**                                                 

        **Phone:**                                                 

        **School Name:**                                           

        **District:**                                              

        **State:**                                                 

                                                                   
  ----- ------------------ -- ------------------------------------ --------

**FOR NATIONAL EVALUATION USE**

ID Number: \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Date Received: \_\_\_\_\_\_\_\_\_\_\_\_\_\_

1

**QUESTIONS**

1\. School's Name, Address, and Grade Levels:

  ---------------- ------------------------------------------------------
  Name:            

  Address:         
  ---------------- ------------------------------------------------------

Grade Levels (*circle lowest and highest*): pre-K K 1 2 3 4 5 6 7 8 9 10
11 12

2\. The following data were calculated based on student enrollment from
which school semester?

(*check ONE only*):

  ------ ----------------------------------------------------------------
         for Spring 2006

         for Fall 2006

         for Spring 2007

         Other date (specify the date)
  ------ ----------------------------------------------------------------

Total No. of Students: \_\_\_\_\_\_\_\_\_\_ (*no.*)

  -------------------------------------------------- -- ---------- -------
  American Indian or Alaska Native:                                (%)

  Asian:                                                           (%)

  Black or African American:                                       (%)

  Hispanic:                                                        (%)

  Native Hawaiian or other Pac. Isld.:                             (%)

  White:                                                           (%)

  Other:                                                           (%)

  Eligible for Free and Reduced Price Lunch:                       (%)

  With IEP:                                                        (%)

  With Limited English Proficiency:                                (%)

  Migrant:                                                         (%)
  -------------------------------------------------- -- ---------- -------

3\. Does this School receive Title I assistance? \_\_\_*Yes
(schoolwide)* \_\_\_*Yes (targeted) \_\_\_No*

If yes, has the school been identified as failing to make Adequate
Yearly Progress, based on student achievement scores for the:

  ------ ----------------------------------------------------------------
         for the 2006-2007 school year

         for the 2005-2006 school year

         for the 2004-2005 school year

         none of the above
  ------ ----------------------------------------------------------------

4\. School Choice Options for Students at this School:

(*check ALL that apply*):

+---+---+---+---+---+---+------------------------+------------------------------+
| a |   |   | s |   |   |                        |                              |
| \ |   |   | t |   |   |                        |                              |
| . |   |   | u |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | w |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | y |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
| b |   |   | s |   |   |                        |                              |
| \ |   |   | t |   |   |                        |                              |
| . |   |   | u |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | u |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | y |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
| c |   |   | s |   |   |                        |                              |
| \ |   |   | t |   |   |                        |                              |
| . |   |   | u |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | y |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | w |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
| d |   |   | s |   |   |                        |                              |
| \ |   |   | t |   |   |                        |                              |
| . |   |   | u |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | y |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | u |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
| e |   |   | * |   |   |                        |                              |
| \ |   |   | * |   |   |                        |                              |
| . |   |   | O |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | - |   |   |                        |                              |
|   |   |   | - |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | . |   |   |                        |                              |
|   |   |   | g |   |   |                        |                              |
|   |   |   | . |   |   |                        |                              |
|   |   |   | , |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | ( |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | x |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | ) |   |   |                        |                              |
|   |   |   | : |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
|   |   |   |   |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
| f |   |   | t |   |   |                        |                              |
| \ |   |   | h |   |   |                        |                              |
| . |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | ( |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | I |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | y |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | u |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | w |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | " |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | " |   |   |                        |                              |
|   |   |   | , |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | ; |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | g |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | u |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | v |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | y |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | . |   |   |                        |                              |
|   |   |   | ) |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
|   |   |   |   |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
|   |   | ( |   |   |   |                        |                              |
|   |   | * |   |   |   |                        |                              |
|   |   | i |   |   |   |                        |                              |
|   |   | f |   |   |   |                        |                              |
|   |   | y |   |   |   |                        |                              |
|   |   | o |   |   |   |                        |                              |
|   |   | u |   |   |   |                        |                              |
|   |   | c |   |   |   |                        |                              |
|   |   | h |   |   |   |                        |                              |
|   |   | e |   |   |   |                        |                              |
|   |   | c |   |   |   |                        |                              |
|   |   | k |   |   |   |                        |                              |
|   |   | e |   |   |   |                        |                              |
|   |   | d |   |   |   |                        |                              |
|   |   | i |   |   |   |                        |                              |
|   |   | t |   |   |   |                        |                              |
|   |   | e |   |   |   |                        |                              |
|   |   | m |   |   |   |                        |                              |
|   |   | s |   |   |   |                        |                              |
|   |   | a |   |   |   |                        |                              |
|   |   | , |   |   |   |                        |                              |
|   |   | b |   |   |   |                        |                              |
|   |   | , |   |   |   |                        |                              |
|   |   | c |   |   |   |                        |                              |
|   |   | , |   |   |   |                        |                              |
|   |   | o |   |   |   |                        |                              |
|   |   | r |   |   |   |                        |                              |
|   |   | d |   |   |   |                        |                              |
|   |   | * |   |   |   |                        |                              |
|   |   | ) |   |   |   |                        |                              |
|   |   | : |   |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
|   |   |   | F |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | 2 |   |   |                        |                              |
|   |   |   | 0 |   |   |                        |                              |
|   |   |   | 0 |   |   |                        |                              |
|   |   |   | 6 |   |   |                        |                              |
|   |   |   | - |   |   |                        |                              |
|   |   |   | 0 |   |   |                        |                              |
|   |   |   | 7 |   |   |                        |                              |
|   |   |   | , |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | b |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | u |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | w |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | y |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | u |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | v |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | ? |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
|   |   |   |   | g |   | (number of students    |                              |
|   |   |   |   | \ |   | transferring **to**    |                              |
|   |   |   |   | . |   | this school)           |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
|   |   |   |   | h |   | (number of students    |                              |
|   |   |   |   | \ |   | transferring **from**  |                              |
|   |   |   |   | . |   | this school)           |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
|   |   |   |   |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
|   |   |   | i |   |   |                        |                              |
|   |   |   | \ |   |   |                        |                              |
|   |   |   | . |   |   |                        |                              |
|   |   |   | D |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | v |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | y |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | x |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | v |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | , |   |   |                        |                              |
|   |   |   |   |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | ? |   |   |                        |                              |
|   |   |   | \ |   |   |                        |                              |
|   |   |   | _ |   |   |                        |                              |
|   |   |   | \ |   |   |                        |                              |
|   |   |   | _ |   |   |                        |                              |
|   |   |   | \ |   |   |                        |                              |
|   |   |   | _ |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
|   |   |   | Y |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | \ |   |   |                        |                              |
|   |   |   | _ |   |   |                        |                              |
|   |   |   | \ |   |   |                        |                              |
|   |   |   | _ |   |   |                        |                              |
|   |   |   | \ |   |   |                        |                              |
|   |   |   | _ |   |   |                        |                              |
|   |   |   | N |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | * |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
|   |   |   | j |   |   |                        |                              |
|   |   |   | \ |   |   |                        |                              |
|   |   |   | . |   |   |                        |                              |
|   |   |   | I |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | y |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | , |   |   |                        |                              |
|   |   |   | w |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | w |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | c |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | h |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | r |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | f |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | s |   |   |                        |                              |
|   |   |   | i |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | a |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | d |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | v |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | l |   |   |                        |                              |
|   |   |   | o |   |   |                        |                              |
|   |   |   | p |   |   |                        |                              |
|   |   |   | m |   |   |                        |                              |
|   |   |   | e |   |   |                        |                              |
|   |   |   | n |   |   |                        |                              |
|   |   |   | t |   |   |                        |                              |
|   |   |   | ? |   |   |                        |                              |
+---+---+---+---+---+---+------------------------+------------------------------+
|   |   |   |   |   |   |                        | (topic 1)                    |
+---+---+---+---+---+---+------------------------+------------------------------+
|   |   |   |   |   |   |                        | (topic 2)                    |
+---+---+---+---+---+---+------------------------+------------------------------+

5\. Notifying Parents about School Choice Options:

What actions did the school take to notify parents/families of their
choice options?

(*check ALL that apply*)

+---+---+-----------------+-----------------------------------------------+
| a |   | individual,     |                                               |
| \ |   | face-to-face    |                                               |
| . |   | meetings with   |                                               |
|   |   | school          |                                               |
|   |   | officials       |                                               |
+---+---+-----------------+-----------------------------------------------+
| b |   | group meetings  |                                               |
| \ |   | with school     |                                               |
| . |   | officials       |                                               |
+---+---+-----------------+-----------------------------------------------+
| c |   | enrollment      |                                               |
| \ |   | fairs or        |                                               |
| . |   | similar events  |                                               |
|   |   | for parents to  |                                               |
|   |   | learn about     |                                               |
|   |   | choice options  |                                               |
+---+---+-----------------+-----------------------------------------------+
| d |   | open houses at  |                                               |
| \ |   | receiving       |                                               |
| . |   | schools         |                                               |
+---+---+-----------------+-----------------------------------------------+
| e |   | letter mailed   |                                               |
| \ |   | to              |                                               |
| . |   | p               |                                               |
|   |   | arents/families |                                               |
+---+---+-----------------+-----------------------------------------------+
| f |   | letter sent     |                                               |
| \ |   | home with       |                                               |
| . |   | students        |                                               |
+---+---+-----------------+-----------------------------------------------+
| g |   | announcements   |                                               |
| \ |   | in community    |                                               |
| . |   | newspapers or   |                                               |
|   |   | other media     |                                               |
+---+---+-----------------+-----------------------------------------------+
| h |   | contacts made   |                                               |
| \ |   | by the          |                                               |
| . |   | district's      |                                               |
|   |   | parent          |                                               |
|   |   | information     |                                               |
|   |   | center(s)       |                                               |
+---+---+-----------------+-----------------------------------------------+
| i |   | other (*please  |                                               |
| \ |   | explain*):      |                                               |
| . |   |                 |                                               |
+---+---+-----------------+-----------------------------------------------+
| j |   |                 |                                               |
| \ |   |                 |                                               |
| . |   |                 |                                               |
| H |   |                 |                                               |
| o |   |                 |                                               |
| w |   |                 |                                               |
| m |   |                 |                                               |
| a |   |                 |                                               |
| n |   |                 |                                               |
| y |   |                 |                                               |
| l |   |                 |                                               |
| a |   |                 |                                               |
| n |   |                 |                                               |
| g |   |                 |                                               |
| u |   |                 |                                               |
| a |   |                 |                                               |
| g |   |                 |                                               |
| e |   |                 |                                               |
| s |   |                 |                                               |
| , |   |                 |                                               |
| o |   |                 |                                               |
| t |   |                 |                                               |
| h |   |                 |                                               |
| e |   |                 |                                               |
| r |   |                 |                                               |
| t |   |                 |                                               |
| h |   |                 |                                               |
| a |   |                 |                                               |
| n |   |                 |                                               |
| E |   |                 |                                               |
| n |   |                 |                                               |
| g |   |                 |                                               |
| l |   |                 |                                               |
| i |   |                 |                                               |
| s |   |                 |                                               |
| h |   |                 |                                               |
| , |   |                 |                                               |
| h |   |                 |                                               |
| a |   |                 |                                               |
| v |   |                 |                                               |
| e |   |                 |                                               |
| b |   |                 |                                               |
| e |   |                 |                                               |
| e |   |                 |                                               |
| n |   |                 |                                               |
| u |   |                 |                                               |
| s |   |                 |                                               |
| e |   |                 |                                               |
| d |   |                 |                                               |
| i |   |                 |                                               |
| n |   |                 |                                               |
| t |   |                 |                                               |
| h |   |                 |                                               |
| e |   |                 |                                               |
| s |   |                 |                                               |
| e |   |                 |                                               |
| n |   |                 |                                               |
| o |   |                 |                                               |
| t |   |                 |                                               |
| i |   |                 |                                               |
| f |   |                 |                                               |
| i |   |                 |                                               |
| c |   |                 |                                               |
| a |   |                 |                                               |
| t |   |                 |                                               |
| i |   |                 |                                               |
| o |   |                 |                                               |
| n |   |                 |                                               |
| p |   |                 |                                               |
| r |   |                 |                                               |
| o |   |                 |                                               |
| c |   |                 |                                               |
| e |   |                 |                                               |
| d |   |                 |                                               |
| u |   |                 |                                               |
| r |   |                 |                                               |
| e |   |                 |                                               |
| s |   |                 |                                               |
| ? |   |                 |                                               |
|   |   |                 |                                               |
| \ |   |                 |                                               |
| _ |   |                 |                                               |
| \ |   |                 |                                               |
| _ |   |                 |                                               |
| \ |   |                 |                                               |
| _ |   |                 |                                               |
| ( |   |                 |                                               |
| * |   |                 |                                               |
| n |   |                 |                                               |
| o |   |                 |                                               |
| . |   |                 |                                               |
| * |   |                 |                                               |
| ) |   |                 |                                               |
+---+---+-----------------+-----------------------------------------------+

6\. In your opinion, what proportion of the parents/families had a good
understanding of their choice

options last year? (*check ONE only*)

+---+---+---------------------------------------------------------------+
| a |   | all parents/families                                          |
| \ |   |                                                               |
| . |   |                                                               |
+---+---+---------------------------------------------------------------+
| b |   | most parents/families (e.g., over 50 percent)                 |
| \ |   |                                                               |
| . |   |                                                               |
+---+---+---------------------------------------------------------------+
| c |   | some parents/families (e.g., between 20-50 percent)           |
| \ |   |                                                               |
| . |   |                                                               |
+---+---+---------------------------------------------------------------+
| d |   | few parents/families (e.g., less than 20 percent)             |
| \ |   |                                                               |
| . |   |                                                               |
+---+---+---------------------------------------------------------------+
| I |   |                                                               |
| f |   |                                                               |
| y |   |                                                               |
| o |   |                                                               |
| u |   |                                                               |
| c |   |                                                               |
| h |   |                                                               |
| e |   |                                                               |
| c |   |                                                               |
| k |   |                                                               |
| e |   |                                                               |
| d |   |                                                               |
| 6 |   |                                                               |
| b |   |                                                               |
| , |   |                                                               |
| 6 |   |                                                               |
| c |   |                                                               |
| , |   |                                                               |
| o |   |                                                               |
| r |   |                                                               |
| 6 |   |                                                               |
| d |   |                                                               |
| , |   |                                                               |
| w |   |                                                               |
| h |   |                                                               |
| a |   |                                                               |
| t |   |                                                               |
| i |   |                                                               |
| s |   |                                                               |
| t |   |                                                               |
| h |   |                                                               |
| e |   |                                                               |
| m |   |                                                               |
| o |   |                                                               |
| s |   |                                                               |
| t |   |                                                               |
| i |   |                                                               |
| m |   |                                                               |
| p |   |                                                               |
| o |   |                                                               |
| r |   |                                                               |
| t |   |                                                               |
| a |   |                                                               |
| n |   |                                                               |
| t |   |                                                               |
| t |   |                                                               |
| h |   |                                                               |
| i |   |                                                               |
| n |   |                                                               |
| g |   |                                                               |
| y |   |                                                               |
| o |   |                                                               |
| u |   |                                                               |
| c |   |                                                               |
| a |   |                                                               |
| n |   |                                                               |
| r |   |                                                               |
| e |   |                                                               |
| c |   |                                                               |
| o |   |                                                               |
| m |   |                                                               |
| m |   |                                                               |
| e |   |                                                               |
| n |   |                                                               |
| d |   |                                                               |
| , |   |                                                               |
| t |   |                                                               |
| o |   |                                                               |
|   |   |                                                               |
| i |   |                                                               |
| m |   |                                                               |
| p |   |                                                               |
| r |   |                                                               |
| o |   |                                                               |
| v |   |                                                               |
| e |   |                                                               |
| p |   |                                                               |
| a |   |                                                               |
| r |   |                                                               |
| e |   |                                                               |
| n |   |                                                               |
| t |   |                                                               |
| s |   |                                                               |
| / |   |                                                               |
| f |   |                                                               |
| a |   |                                                               |
| m |   |                                                               |
| i |   |                                                               |
| l |   |                                                               |
| i |   |                                                               |
| e |   |                                                               |
| s |   |                                                               |
| ' |   |                                                               |
| u |   |                                                               |
| n |   |                                                               |
| d |   |                                                               |
| e |   |                                                               |
| r |   |                                                               |
| s |   |                                                               |
| t |   |                                                               |
| a |   |                                                               |
| n |   |                                                               |
| d |   |                                                               |
| i |   |                                                               |
| n |   |                                                               |
| g |   |                                                               |
| o |   |                                                               |
| f |   |                                                               |
| t |   |                                                               |
| h |   |                                                               |
| e |   |                                                               |
| i |   |                                                               |
| r |   |                                                               |
| c |   |                                                               |
| h |   |                                                               |
| o |   |                                                               |
| i |   |                                                               |
| c |   |                                                               |
| e |   |                                                               |
| o |   |                                                               |
| p |   |                                                               |
| t |   |                                                               |
| i |   |                                                               |
| o |   |                                                               |
| n |   |                                                               |
| s |   |                                                               |
| ? |   |                                                               |
+---+---+---------------------------------------------------------------+
|   |   |                                                               |
+---+---+---------------------------------------------------------------+
|   |   |                                                               |
+---+---+---------------------------------------------------------------+

7\. Has your school started new programs (e.g., magnets, academies,
small learning communities, new academic subjects) to be more
attractive, either to reduce the number of students transferring out or
to increase the number transferring in? (*check ALL that apply*)

+---+---+----------------------------------------------------------------+
| a |   | becoming a charter school                                      |
| \ |   |                                                                |
| . |   |                                                                |
+---+---+----------------------------------------------------------------+
| b |   | starting new magnets, academies, or small learning communities |
| \ |   |                                                                |
| . |   |                                                                |
+---+---+----------------------------------------------------------------+
| c |   | starting other new academic programs or subjects               |
| \ |   |                                                                |
| . |   |                                                                |
+---+---+----------------------------------------------------------------+
| d |   | making other changes in school administration (e.g., changing  |
| \ |   | school hours)                                                  |
| . |   |                                                                |
|   |   |                                                                |
| e |   |                                                                |
| \ |   |                                                                |
| . |   |                                                                |
+---+---+----------------------------------------------------------------+
|   |   | other                                                          |
+---+---+----------------------------------------------------------------+
| f |   | no new programs                                                |
| \ |   |                                                                |
| . |   |                                                                |
+---+---+----------------------------------------------------------------+
| B |   |                                                                |
| r |   |                                                                |
| i |   |                                                                |
| e |   |                                                                |
| f |   |                                                                |
| l |   |                                                                |
| y |   |                                                                |
| d |   |                                                                |
| e |   |                                                                |
| s |   |                                                                |
| c |   |                                                                |
| r |   |                                                                |
| i |   |                                                                |
| b |   |                                                                |
| e |   |                                                                |
| t |   |                                                                |
| h |   |                                                                |
| e |   |                                                                |
| n |   |                                                                |
| e |   |                                                                |
| w |   |                                                                |
| p |   |                                                                |
| r |   |                                                                |
| o |   |                                                                |
| g |   |                                                                |
| r |   |                                                                |
| a |   |                                                                |
| m |   |                                                                |
| s |   |                                                                |
| a |   |                                                                |
| n |   |                                                                |
| d |   |                                                                |
| t |   |                                                                |
| h |   |                                                                |
| e |   |                                                                |
| m |   |                                                                |
| a |   |                                                                |
| i |   |                                                                |
| n |   |                                                                |
| c |   |                                                                |
| h |   |                                                                |
| a |   |                                                                |
| n |   |                                                                |
| g |   |                                                                |
| e |   |                                                                |
| s |   |                                                                |
| i |   |                                                                |
| n |   |                                                                |
| s |   |                                                                |
| c |   |                                                                |
| h |   |                                                                |
| o |   |                                                                |
| o |   |                                                                |
| l |   |                                                                |
| o |   |                                                                |
| p |   |                                                                |
| e |   |                                                                |
| r |   |                                                                |
| a |   |                                                                |
| t |   |                                                                |
| i |   |                                                                |
| o |   |                                                                |
| n |   |                                                                |
| a |   |                                                                |
| n |   |                                                                |
| d |   |                                                                |
| a |   |                                                                |
| d |   |                                                                |
| m |   |                                                                |
| i |   |                                                                |
| n |   |                                                                |
| i |   |                                                                |
| s |   |                                                                |
| t |   |                                                                |
| r |   |                                                                |
| a |   |                                                                |
| t |   |                                                                |
| i |   |                                                                |
| o |   |                                                                |
| n |   |                                                                |
| . |   |                                                                |
+---+---+----------------------------------------------------------------+
|   |   |                                                                |
+---+---+----------------------------------------------------------------+
|   |   |                                                                |
+---+---+----------------------------------------------------------------+
|   |   |                                                                |
+---+---+----------------------------------------------------------------+
|   |   |                                                                |
+---+---+----------------------------------------------------------------+
|   |   |                                                                |
+---+---+----------------------------------------------------------------+
|   |   |                                                                |
+---+---+----------------------------------------------------------------+

4

**2. SITE VISIT FACE SHEET**

(Data collection: 2006--07)

**Face Sheet To Be Completed for**

**Each VPSC Site at the Time of the Site Visit**

  ------------------------- --------------------- -- ---------------------
  **Date of Site Visit:**                            

  **Site Visit Team:**                               
  ------------------------- --------------------- -- ---------------------

<table style="width:100%;">
<colgroup>
<col style="width: 33%" />
<col style="width: 8%" />
<col style="width: 2%" />
<col style="width: 0%" />
<col style="width: 12%" />
<col style="width: 2%" />
<col style="width: 0%" />
<col style="width: 12%" />
<col style="width: 2%" />
<col style="width: 0%" />
<col style="width: 8%" />
<col style="width: 3%" />
<col style="width: 12%" />
</colgroup>
<tbody>
<tr class="odd">
<td><strong>Name of Site:</strong></td>
<td colspan="12"></td>
</tr>
<tr class="even">
<td><p><strong>Lead Organization and</strong></p>
<p><strong>Dept. for VPSC Initiative:</strong></p></td>
<td colspan="12"></td>
</tr>
<tr class="odd">
<td><strong>Partnering Organizations:</strong></td>
<td colspan="12"></td>
</tr>
<tr class="even">
<td><p><strong>Type of Jurisdiction</strong></p>
<p><strong>(e.g., rural, suburban, urban):</strong></p></td>
<td colspan="12"></td>
</tr>
<tr class="odd">
<td><strong>Student Population:</strong></td>
<td colspan="2"></td>
<td colspan="3"></td>
<td colspan="7"></td>
</tr>
<tr class="even">
<td><strong>Ethnicity:</strong></td>
<td colspan="2"></td>
<td colspan="3"></td>
<td colspan="3"></td>
<td colspan="4"></td>
</tr>
<tr class="odd">
<td></td>
<td><strong>White</strong></td>
<td colspan="2"></td>
<td><strong>African American</strong></td>
<td colspan="2"></td>
<td><strong>Hispanic</strong></td>
<td colspan="2"></td>
<td><strong>Asian</strong></td>
<td></td>
<td><strong>Other</strong></td>
</tr>
</tbody>
</table>

**3. INTERVIEW WITH VPSC DIRECTOR AND SUPPORT STAFF**

(Data collection: 2006-07)

**1. Definition of the VPSC Initiative**

1.1 What activities or staff are being supported with VPSC funds? \[*Use
the revised proposal budget as a starting point, but be prepared for
changes to have occurred.*\]

1.2 Is it reasonable to define the initiative according to how the funds
are being spent? If not, provide some other rationale for defining the
initiative. \[*Also check the revised proposal budget.*\]

1.3 Describe the VPSC initiative in 2-3 short sentences.

1.4 How participatory was the *planning process* that was used in
developing the VPSC initiative? \[P*robe mainly for parent and school
participation, and describe the nature of the participation.*\]

1.5 What were the main problems encountered in *planning* (not
implementing) the VPSC initiative?

1.6 What have been the main problems encountered in *implementing* the
VPSC initiative?

**2. Identity of the Schools Involved in the VPSC Initiative**

2.1 Define the breadth of the initiative across its participating
schools:

+---+---------------------------------------------------------+-------+---+
|   |                                                         |       |   |
+---+---------------------------------------------------------+-------+---+
|   | 2.1.1 Academic years covered by implementation (not     |       |   |
|   | including planning year):                               |       |   |
+---+---------------------------------------------------------+-------+---+
|   | 2.1.2 No. and grade level of sending schools:           |       |   |
+---+---------------------------------------------------------+-------+---+
|   | 2.1.3 No. and grade level of receiving schools or       |       |   |
|   | sites, by district: \[*Also see Qs. 7.1 and 8.1         |       |   |
|   | below.*\]                                               |       |   |
+---+---------------------------------------------------------+-------+---+
|   | 2.1.4 No. and grade level of remaining schools, by      |       |   |
|   | district:                                               |       |   |
|   |                                                         |       |   |
|   | > Within this number, no. and grade level of schools    |       |   |
|   | > that might become sending schools in the near future: |       |   |
|   |                                                         |       |   |
|   | \[*Define criteria used.*\]                             |       |   |
+---+---------------------------------------------------------+-------+---+
|   |                                                         |       |   |
+---+---------------------------------------------------------+-------+---+
|   |                                                         |       |   |
+---+---------------------------------------------------------+-------+---+

> 2.2 Describe any "zone" pattern and how zones were defined. \[*If
> possible, obtain a map of the schools and of the zone pattern.*\]

2.3 How were the sending and receiving schools identified to be part of
the initiative? \[*If related to school performance, obtain
documentation of such performance.*\]

2.4 Were any logically eligible schools "exempted" from becoming sending
or receiving schools? \[*If YES, please describe.*\]

2.5 What are the grades for which choice options can be exercised (e.g.,
the entering grade for each school, but not the other grades)?

2.6 About how many students are eligible to transfer, due to the VPSC
initiative, by grade level (and school, if possible)?

2.7 About how many seats will be created at receiving schools, due to
the VPSC initiative, by grade level (and school or site, if possible)?
\[*Also see responses to Qs. 7.1 and 8.1 below.*\]

2.8 How did the district estimate the number of seats available at
receiving schools? \[*Also comment on the quality of the procedures and
data used by the district.*\]

2.9 What is the schedule of tuition transfers, if any, that accompanied
the student transfer?

**3. Student Assignment Criteria and Procedures**

3.1 What criteria have been used to define students eligible for
transfer, and what source of data is used to review whether students are
eligible?

3.2 How and when do the eligible students indicate whether they want to
transfer, and how many potential receiving schools may they identify?
\[*Obtain an application form and review a sample of the completed
forms.*\]

  -- --------------------------------------------------- -------------- --
                                                                        

     3.2.1 Do they have to indicate, explicitly, that                   
     they wish to stay at their present school?                         

     3.2.2 How does the date when the VPSC choice                       
     applications are due coincide with the application                 
     dates for other choice options---e.g., to charters,                
     magnets, or any other special schools?                             

     3.2.3 Identify the location(s) where the                           
     applications must be submitted and the rationale                   
     for selecting the location(s):                                     

                                                                        

                                                                        
  -- --------------------------------------------------- -------------- --

> 3.3 What criteria are used in deciding to approve a transfer?
> \[*Confirm the criteria through documentation and discussions with
> others, including parents, if possible.*\]

+---+-----------------------------------------+---------+---+----------+---+
|   |                                         |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | **Check whether the following student   | **Pr    |   | **Cri    |   |
|   | assignment criteria**                   | esent** |   | terion** |   |
|   |                                         |         |   |          |   |
|   | **are used; if YES, state the           |         |   |          |   |
|   | criterion:**                            |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | 3.3.1 *Proximity* preferences (e.g., 50 |         |   |          |   |
|   | percent of the seats are reserved for   |         |   |          |   |
|   | students who live near enough to walk   |         |   |          |   |
|   | to school: *\[Note whether these        |         |   |          |   |
|   | families still have to fill out a       |         |   |          |   |
|   | choice application.\]*                  |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | 3.3.2 *Racial fairness* guidelines      |         |   |          |   |
|   | (e.g., the ratio of Whites and          |         |   |          |   |
|   | non-Whites cannot vary more than 10     |         |   |          |   |
|   | percentage points from that of the      |         |   |          |   |
|   | district as a whole):                   |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | 3.3.3 *Socioeconomic fairness*          |         |   |          |   |
|   | guidelines (e.g., the ratio of students |         |   |          |   |
|   | eligible for free or reduced lunch      |         |   |          |   |
|   | cannot vary more than 10 percentage     |         |   |          |   |
|   | points from that of the district as a   |         |   |          |   |
|   | whole):                                 |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | 3.3.4 *Sibling* preferences:            |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | 3.3.5 *Special provisions*, if any, for |         |   |          |   |
|   | students requiring bilingual or special |         |   |          |   |
|   | education programs:                     |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | 3.3.6 *Substitution* rules when         |         |   |          |   |
|   | available seats, especially those       |         |   |          |   |
|   | designated by proximity, race, or       |         |   |          |   |
|   | socioeconomics, are undersubscribed:    |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | 3.3.7 *Lottery* or *random assignment*  |         |   |          |   |
|   | when seats are oversubscribed (and      |         |   |          |   |
|   | especially whether separate lotteries   |         |   |          |   |
|   | in effect exist for different subgroups |         |   |          |   |
|   | of seats---e.g., Whites and             |         |   |          |   |
|   | non-Whites): \[*If used, describe the   |         |   |          |   |
|   | lottery or random assignment            |         |   |          |   |
|   | procedure.*\]                           |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | 3.3.8 Whether and how a transferring    |         |   |          |   |
|   | student will receive *supplemental      |         |   |          |   |
|   | services*:                              |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | 3.3.9 Whether and how an eligible but   |         |   |          |   |
|   | non-transferring student will receive   |         |   |          |   |
|   | *supplemental services*:                |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | 3.3.10 Whether the program maintains a  |         |   |          |   |
|   | *waiting list*, and how late in the     |         |   |          |   |
|   | semester students can still transfer,   |         |   |          |   |
|   | if a seat becomes vacant:               |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   | 3.3.11 Whether the students originally  |         |   |          |   |
|   | enrolled in the receiving schools can   |         |   |          |   |
|   | be displaced:                           |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+
|   |                                         |         |   |          |   |
+---+-----------------------------------------+---------+---+----------+---+

3.4 Have there been any complaints regarding the student assignment
process:

  -- ------------------------------------------------- ----------------- --
                                                                         

     3.4.1 By students and their families? \[*If YES,                    
     describe.*\]                                                        

     3.4.2 By sending schools' staff? \[*If YES,                         
     describe.*\]                                                        

     3.4.3 By receiving schools' staff? \[*If YES,                       
     describe.*\]                                                        

     3.4.4 By any other relevant parties? \[*If YES,                     
     describe.*\]                                                        

                                                                         
  -- ------------------------------------------------- ----------------- --

> 3.5 How has the district's student population changed, relative to the
> population of school-age children?

**4. Parent Notification Procedures**

4.1 How did parents participate (and how many participated) in the
planning of the choice program and especially in the design of the
parent information center?

4.2 When were parents first notified of the choice options supported by
the VPSC initiative? \[*Obtain a copy of the written notification.*\]

4.3 What feedback information is collected to assure that all parents
are being properly informed about their choice options?

**5. Parent Information Center(s) (PICs)**

5.1 What district-wide enrollment does the PIC cover, and how many
students is it actually serving? \[*Depending upon the choice program,
the number of applicants for transfer may only be a portion of the total
student enrollment---obtain both numbers.*\]

5.2 What are the physical characteristics of the PIC's location?

  -- ------------------------------------------------- ----------------- --
                                                                         

     5.2.1 Access conditions (e.g., parking and public                   
     transportation):                                                    

     5.2.2 Physical facility (e.g., in a high school;                    
     working atmosphere):                                                

     5.2.3 Hours and days of operation:                                  

     5.2.4 No. of different languages spoken by staff,                   
     and match to student population:                                    

                                                                         

                                                                         
  -- ------------------------------------------------- ----------------- --

5.3 Enumerate the PIC's staff, by title and FTE. Also note the
supervisory relationships and to whom the PIC director reports in the
district.

5.4 Enumerate the outreach activities undertaken by the PIC during the
past school year. Also, give some estimate of the frequency of each type
of activity:

  -- ------------------------------------------------- ----------------- --
                                                                         

     5.4.1 Home visits or individual conferences with  How often?        
     school (or choice) staff:                                           

     5.4.2 Use of mass media:                                            

     5.4.3 Use of community or neighborhood events or                    
     facilities:                                                         

     5.4.4 Other:                                                        

                                                                         
  -- ------------------------------------------------- ----------------- --

5.5 How have these outreach activities been adjusted, if at all, from
year to year?

5.6 How does the PIC know that its outreach activities are sufficient in
reaching all eligible families? \[*Cite the actual data used by the PIC
in developing this knowledge.*\]

5.7 What is the nature of the parent survey (sample and instrument), if
any, conducted by the PIC, and how are the survey results used?

5.8 How does the PIC work with individual schools? \[*If YES to any of
the following, please describe*\]:

  -- ------------------------------------------- ----------------------- --
                                                                         

     5.8.1 The PIC maintains up-to-date                                  
     information about the schools' enrollments:                         

     5.8.2 The PIC arranges tours of the                                 
     schools:                                                            

     5.8.3 The PIC encourages schools to develop                         
     their own marketing materials:                                      

     5.8.4 The PIC helps the schools to                                  
     disseminate information about themselves:                           

                                                                         

                                                                         
  -- ------------------------------------------- ----------------------- --

**6. Transportation Support within VPSC Initiative**

6.1 Were VPSC funds used to support new transportation services? \[*If
YES, please describe.*\]

  -- ------------------------------------------- ----------------------- --
                                                                         

     6.1.1 Purchase of buses:                                            

     6.1.2 Hiring of drivers:                                            

     6.1.3 Increases in drivers' salaries:                               

     6.1.4 Revamping of technological                                    
     infrastructure (e.g., computerized bus                              
     routes):                                                            

                                                                         

                                                                         
  -- ------------------------------------------- ----------------------- --

6.2 Were school hours changed (e.g., staggered daily times or alternate
school calendars) to control transportation costs? \[*If so, describe
and note whether the changes were related to VPSC or not.*\]

6.3 Whether VPSC funds were used to support new transportation services
or not, collect data prior to and during the VPSC Program on: \[*If
possible, obtain these data for individual schools, especially the
sending and receiving schools.*\]

  -- ------------------------------------------- ----------------------- --
                                                                         

     6.3.1 The number of buses:                                          

     6.3.2 The number of students riding buses:                          

     6.3.3 Average and range of trip times:                              

     6.3.4 The number of schools served each day                         
     by a single bus:                                                    

                                                                         
  -- ------------------------------------------- ----------------------- --

6.4 How important are the augmented transportation services to the
entire VPSC initiative (e.g., did parents pay for or otherwise support a
portion of the existing services, before)?

**7. Capacity-Building (*New* Schools or Sites to Serve as Receiving
Schools)**

7.1 How many *new* schools or sites (and how many seats), if any, were
opened as receiving schools, with VPSC support? \[*Also see Q. 8.1.*\]

7.2 What are the substantive educational themes at these schools, and in
what way do they reflect a diversification or replication of themes
already covered by the existing schools?

7.3 How were the themes chosen, and did parents participate in the
process?

**8. Capacity-Building (New or Expanded Programs at *Existing* Schools
or Sites)**

8.1 How many existing schools or sites have been expanded or added as
receiving schools (and how many seats), with VPSC support? \[*The total
of the schools or sites and seats in 7.1 and 8.1 should coincide with
the responses to Qs. 1.4.3 and 2.7 above.*\]

8.2 What are the substantive educational themes at these schools, and in
what way do they reflect a diversification or replication of themes
already covered?

8.3 Describe the specific actions taken to increase the capacity of the
existing receiving schools: \[*Check each of the following, and if YES,
describe.*\]

  -- --------------------------------------------- -------------------- --
                                                                        

     8.3.1 New educational programs (e.g., magnets                      
     or academies within the school) at the                             
     existing schools:                                                  

     8.3.2 Administrative changes to increase the                       
     school's enrollment capacity:                                      

     8.3.3 Supplemental services to assist the                          
     transferring students:                                             

     8.3.4 Professional development to assist                           
     educators in providing instruction to the                          
     transferring students:                                             

     8.3.5 Other:                                                       

                                                                        
  -- --------------------------------------------- -------------------- --

**9. Capacity-Building (Sending Schools)**

9.1 Do sending schools lose funds in relation to out-transfers? \[*If
so, describe the procedures and the amount of funds involved*.\]

9.2 Are the sending schools under a formal sanction category (e.g., in
need of improvement, in corrective action, or restructuring), and how
long have they been in the category?

9.3 Whether in a formal sanction category or not, what actions will be
(or have been) taken to

improve the future performance of the sending schools (check all of the
following)?

  -- --------------------------------------------- -------------------- --
                                                                        

     9.3.1 Professional development or other                            
     educational assistance:                                            

     9.3.2 Adoption of new educational practices:                       

     9.3.3 Restructuring of the school:                                 

     9.3.4 Elimination of the school:                                   

                                                                        
  -- --------------------------------------------- -------------------- --

**10. Relationship to Title I Provisions of *NCLB***

10.1 Has the district started to offer the school choice options under
the new provisions of Title I, and if so, how do these arrangements
relate to those of the VPSC initiative?

  -- --------------------------------------------- -------------------- --
                                                                        

     10.1.1 Are different schools involved in                           
     each, or are VPSC funds used to complement                         
     the use of Title I funds for the same                              
     schools?                                                           

                                                                        
  -- --------------------------------------------- -------------------- --

**11. Transferring Students and Comparisons with non-Transferring
Students**

11.1 For each relevant academic year in implementing the VPSC
initiative, obtain the following data (preferably noting the individual
sending and receiving schools):

  -- --------------------------------------------- -------------------- --
                                                                        

     11.1.1 How many students were eligible to                          
     apply for transfer?                                                

     11.1.2 How many applied, and how many were                         
     transferred?                                                       

     11.1.3 What were the main experiences in                           
     dealing with under- or over-subscription?                          

     11.1.4 What were the main lessons learned                          
     about the transfer process, and did these                          
     lessons lead to modifications in the                               
     procedures used the following year?                                

                                                                        

                                                                        
  -- --------------------------------------------- -------------------- --

11.2 Based on the analysis of individual student records \[*such
analysis may be done by the VPSC site*\], how do the transferring
students compare to those eligible but not transferring, on the
following characteristics?

  -- ----------------------------------------------- ------------------ --
                                                                        

     11.2.1 Annual academic performance: \[*Start                       
     with the year prior to the initiative*.\]                          

     11.2.2 Demographic characteristics (White vs.                      
     non-White; school lunch vs. non-school lunch):                     

     11.2.3 Other characteristics that might account                    
     for academic differences between the two groups                    
     (e.g., number of siblings; level of parents'                       
     education; nature of parents' employment and                       
     income):                                                           

                                                                        
  -- ----------------------------------------------- ------------------ --

**12. Sending and Receiving Schools and Their Changes Over Time**

12.1 For each sending and receiving school and for each academic year of
the initiative, track:

+---+-----------------------------------------------+------------------+---+
|   |                                               |                  |   |
+---+-----------------------------------------------+------------------+---+
|   | 12.1.1 The total enrollments, by grade level: |                  |   |
+---+-----------------------------------------------+------------------+---+
|   | 12.1.2 The proportion of White and non-White  |                  |   |
|   | students:                                     |                  |   |
+---+-----------------------------------------------+------------------+---+
|   | 12.1.3 The proportion of students eligible    |                  |   |
|   | for free or reduced lunch:                    |                  |   |
+---+-----------------------------------------------+------------------+---+
|   | 12.1.4 The academic performance (mathematics  |                  |   |
|   | and reading) of all the students at the       |                  |   |
|   | school:                                       |                  |   |
|   |                                               |                  |   |
|   | > \[*Use all test data available, even for    |                  |   |
|   | > grades not necessarily affected by the VPSC |                  |   |
|   | > initiative, not just the transferring       |                  |   |
|   | > students*.\]                                |                  |   |
+---+-----------------------------------------------+------------------+---+
|   | 12.1.5 Obtain district-wide averages for the  |                  |   |
|   | above, omitting the sending and receiving     |                  |   |
|   | schools, to permit contextual comparisons:    |                  |   |
+---+-----------------------------------------------+------------------+---+
|   |                                               |                  |   |
+---+-----------------------------------------------+------------------+---+

12.2 At the receiving schools, what problems have emerged that might
have been associated with the transfers or transfer process, and what
remedies have been put into place?

12.3 At the sending schools, what problems have emerged that might have
been associated with the transfers or transfer process, and what
remedies have been put into place?

12.4 What is the evidence that the sending schools have taken more
serious measures to improve their performance or otherwise to become
more competitive?

12.5 Is there any evidence that *potential* sending schools are taking
more serious measures to improve their performance or otherwise to
become more competitive?

**13. Contextual Conditions**

13.1 What other choice options and programs are available in the
district (not covered by the VPSC initiative), and how many students are
involved in such options or programs?

13.2 Have these other choice options and programs changed during the
period of the VPSC initiative and might they have played a role in the
preceding early outcomes?

13.3 Have there been any notable changes by the surrounding districts
that might have played a role in the preceding early outcomes?

13.4 Have there been any changes in the community (e.g., residential
relocation patterns) that might have played a role in the preceding
early outcomes?

13.5 How has the district's "share" of the student population changed,
relative to that of private schools? \[*Obtain annual data by grade
level, if possible*.\]

**4. INTERVIEW WITH OTHER PARTICIPATING PERSONS**

(Data collection: 2006-07)

1\. What is your connection to the VPSC initiative \[e.g., principal or
teacher at participating school, parent of participating student,
other\]?

2\. How were you made aware of the VPSC initiative?

3\. Can you briefly describe your understanding of the VPSC initiative?

4.  What changes have you seen or experienced as a result of the VPSC
    initiative?

5.  How has *student enrollment* at your school \[or your child's
    school\] changed during the past year and to what extent have these
    changes been associated with the VPSC initiative, compared to other
    conditions?

6\. What changes in the *academic program* have been made as part of the
VPSC initiative, and how well are these changes working?

7\. What changes in *school administration* (e.g., redefining school
hours or the school year) have been made as part of the VPSC initiative,
and how well are these changes working?

8\. How smooth or disruptive was the timing or sequence of key events in
the VPSC initiative---e.g., application deadlines or parent notification
dates?

9\. How well do you think the VPSC initiative has been received by
teachers, principals, and parents, and do you think they are
sufficiently informed about any new role(s) expected of them under the
initiative?

10\. How has student behavior and academic performance changed, and to
what extent are these changes associated with VPSC?

11\. Did VPSC provide any other benefits or create any other problems
that have not been addressed by the preceding questions?

**5. INTERVIEW WITH THE COMPARISON DISTRICT AND TITLE I STAFF**

(Data collection: 2006-07)

**Procedures**

One-day site visits will be made to comparison sites in spring of 2007
(February-May). Where the comparison sites are within the jurisdiction
already covered by the VPSC Program's site (e.g., another "zone" in a
VPSC district or another district within a statewide VPSC initiative),
the needed data should be collected from the VPSC and related district
staff.

Where the comparison sites are outside of the VPSC jurisdiction, initial
contact with the sites should emphasize that our National Evaluation
is: 1) covering public school choice, 2) selecting some sites where
minimal choice options other than Title I, magnets, and charters are in
operation, and 3) making site visits to sites that are near a VPSC
Program's site but that did not receive a VPSC award. To minimize
burden, the site visit will involve interviews with: the Title I
coordinator and any related district staff (e.g., overseeing magnets and
charters).

***Preparation for Site Visits***. Prior to any site visit, field teams
should collect and analyze data about the targeted area. This
information will have come from several sources, including: the VPSC
Program's site that is aware of (and helped to select) the comparison
site, and materials gathered from the district or state Web sites.

***Assembling of Evidence and Preliminary Reports Immediately Following
Site Visits***. Field teams are urged to begin the formal analysis and
report-writing process as soon as a site visit has ended, though
additional data may still have to be collected. Assembling data and
drafting narratives proceeds more efficiently and with much higher
quality if this time sequence is followed. Teams should reserve the day
or two *after* the site visit for this activity, avoiding other
commitments.

***Outline of Report***. The report should follow the same heading
structure as the topics of inquiry described next.

**Topics of Inquiry**

The topics below not only cover the "agenda" to be followed by the field
team but also give explicit probes and examples regarding the type of
evidence that is being sought. As a result, the protocol should provide
guidance on how to know what to look or listen for, and how to recognize
relevant evidence when it is encountered.

**1. Public School Choice Options at This Site**

Enumerate the public school choice options at this site, giving the
number of schools and students involved by type of option:

1.1 School identified for improvement under Title I:

1.2 Magnet Schools (no. of schools [and programs]{.underline}):

1.3 Charter Schools:

1.4 Other choice arrangements, including unsafe school choice and
desegregation (*describe*):

\[*If options were identified under 1.2, 1.3, or 1.4 above, be sure to
cover Q. 3 below.*\]

**2. New Title I Choice Arrangements**

How has the district started to offer the school choice options under
the new provisions of Title I? (*probe for*):

2.1 Use of designation as a Title I school identified for improvement in
defining sending and receiving schools:

2.2 Expanded transportation services:

2.3 Provision of supplemental services:

2.4 Expanded or improved parent notification procedures:

2.5 Other (describe):

**3. Other, non-Title I Choice Arrangements**

Describe the procedures used for the other choice options identified in
Q. 1 above (*address the following questions for [each]{.underline} type
of option separately*):

3.1 How are eligible students defined?

3.2 How are students/parents notified of their choices? \[*Be sure to
cover all the communication modes---e.g., see Q.5 in the School Survey
instrument.*\]

3.3 What criteria are used in selecting students to exercise their
choice?

\[*Be sure to cover all criteria---e.g., see Q3.3 in the VPSC field*

*instrument.*\]

3.4 What proportion of eligible students were able to exercise their
choice?

\[*Also determine the proportion receiving their 1st, 2nd, etc.
choices;*

*obtain data for the past 2-3 years, if possible.*\]

3.5 How have transportation services been affected, if at all, by the
choice

arrangement? \[*Be sure to cover all facets of these services---e.g.,
see Q.6 in the*

*VPSC instrument.*\]

3.6 Are other choice options likely to be implemented in the future?

**4. Trends Associated with Choice Options**

Determine whether the district maintains records of individual students
who have transferred or of the performance of sending and receiving
schools. The following questions may be addressed once, even if the
district has more than one choice arrangement.

4.1 Are there any data showing trends in academic performance by the
students who have exercised their choice? \[*Probe for data for the past
2-3 years, if possible.*\]

4.2 Are there any data showing trends in academic performance by the
schools involved in the choice arrangement? \[*Probe for data for the
past 2-3 years, if possible.*\]

4.3 What are the district's policies, views, and preferences regarding
school choice options?

4.4 What interest has been expressed by parents, communities, or
students regarding school choice options at this district?

**5. Contextual Conditions**

5.1 Have there been any notable changes by the surrounding districts
that might be relevant to either the choice options or their outcomes at
this district?

5.2 Have there been any changes in the community (e.g., residential
relocation patterns) that might be relevant to either the choice options
or their outcomes at this district?

5.3 How has the district's "share" of the student population changed,
relative to that of private schools in the area? \[*Obtain annual data
by grade level, if possible*\]

5.4 Is the district implementing other approaches to school improvement
as an alternative to choice?

**Appendix C:**

**Supplemental Data About**

**Individual VPSC Sites**

**Contents**

**Page**

> Exhibit C-1 State Education Agencies Used VPSC Funds to
>
> Support Widely Different Choice Priorities C-3
>
> Exhibit C-2 VPSC Initiatives Coincided With Existing, Districtwide
> Choice\
> Options C-3
>
> Exhibit C-3 Sites Split Their VPSC Funds to Support
>
> a Variety of Activities, Not Limited to a Single Choice\
> Initiative C-4
>
> Exhibit C-4 VPSC Initiatives Could Be Separate From or Coincide With\
> Title I Choice C-4
>
> Exhibit C-5 Five Years of VPSC Participants C-5
>
> Exhibit C-6 Choice Initiatives Limited to Zones Within a District
>
> Nevertheless Involved a Broad Array of Sending Schools C-6
>
> Exhibit C-7 Eligible Sending Schools Were Restricted\
> but Not Necessarily to Low-Performing Schools C-6
>
> Exhibit C-8 Capacity-Building Occurred at Receiving Schools
>
> That Already Were (Presumably) Higher-Performing\
> Schools C-6
>
> Exhibit C-9 Local Sites May Need to Focus on a "Tapestry" of Choice\
> Options, Rather than on Any Single Initiative C-7
>
> Exhibit C-10 "Attractor" Programs at Receiving Schools May Not\
> Have Been Needed to Attract Incoming Students C-7
>
> Exhibit C-11 Explicit Efforts May Be Needed to Maintain the
>
> Performance of Receiving Schools C-7

+-----------------------------------------------------------------------+
| **Exhibit C-5**                                                       |
|                                                                       |
| **Five Years of VPSC Participants**                                   |
+-----------------------------------------------------------------------+

![](media/image3.png){width="8.543055555555556in" height="11.0in"}

[^1]: ^?^ The *No Child Left Behind Act of 2001* (P.L.107-110) amended
    the *Elementary and Secondary Education Act of 1965*.

[^2]: ^?^ If all schools in a district have not made adequate yearly
    progress (AYP) for two or more consecutive years, the district must,
    "to the extent practicable," establish a cooperative agreement for a
    transfer with other districts in the surrounding area. 34 C.F.R.
    200.44(h)(1).

[^3]: ^?^ The nonprofit organization operated charter schools and
    qualified as a school district, to be eligible for a VPSC award.

[^4]: Confidence levels represent the probability that observed
    differences do not differ by chance alone but represent actual
    differences between two groups. Statistically, the analyses were
    based on the use of random-effects models, which maximize the
    appropriate use of all of the sites' data submitted for each of the
    six cohorts. Unlike fixed-effects models, the random-effects models
    assume that the available data come from but a sample of a fuller
    universe of students and that the sample therefore is likely to
    contain additional variance due to sampling errors (Bryk and
    Raudenbush, 1992).

[^5]: To overcome self-selection and other biases, one frequently raised
    possibility has been to use data from sites' lotteries as if they
    were "natural experiments" (sites often use lotteries to select
    participants when the number of students exceeds the available
    choice opportunities). However, the lottery losers may not actually
    serve as the needed counterfactuals for lottery winners (see fuller
    discussion in appendix A).

[^6]: ^?^ The *No Child Left Behind Act of 2001* (P.L. 107-110) amended
    the *Elementary and Secondary Education Act of 1965*.

[^7]: ^?^ If all schools in a district have not made adequate yearly
    progress (AYP) for two or more consecutive years, the district must,
    "to the extent practicable," establish a cooperative agreement for a
    transfer with other districts in the surrounding area. 34 C.F.R.
    200.44(h)(1).

[^8]: Each site engaged a "local" evaluator to assess the specific
    site's progress and accomplishments.

[^9]: The survey covered 12 of the 13 sites in the VPSC Program because
    the 13th site focused mainly on technical assistance to districts
    statewide.

[^10]: At this site, the district was the appropriate respondent because
    many of the participating students were not assigned to specific
    schools, either prior to or as part of the VPSC initiative.

[^11]: The nonprofit organization operated charter schools and qualified
    as a school district, to be eligible for a VPSC award.

[^12]: The Free and Reduced-Price Lunch Program is part of the National
    School Lunch Program administered by the U.S. Department of
    Agriculture. The program provides nutritionally balanced, low-cost
    or free lunches to more than 26 million students each school day.
    Children from families with incomes at or below 185 percent of the
    poverty level are eligible for free and reduced-price meals. For the
    period July 1, 2004, through June 30, 2005, 185 percent of the
    poverty level was \$34,873 for a family of four (USDA, 2004).

[^13]: Conducting a formal survey of parents' perspectives was beyond
    the scope of the evaluation. However, parents were informally
    interviewed during the site visits. On those occasions, parents
    voiced no complaints about the choice options.

[^14]: See appendix C, exhibit C-5, for data covering all five years,
    2002--07.

[^15]: In 2005--06, the number of eligible students in the VPSC Program
    was more than 35 times the number of enrollees, making eligibility
    the high end estimate of participation in the program. In some cases
    (see rows A, F, and H in exhibit 4-1), individual sites defined a
    large pool of students as eligible, even though the initiative could
    only accommodate much smaller numbers of enrollees.

[^16]: The enormity of the task should not be underestimated. At one
    site, applicants complete multiple applications, one for each school
    of interest. The site then makes selections among applications, not
    applicants, and the receiving schools act like colleges or
    universities who may have accepted applicants that have also been
    simultaneously accepted at another school. The desired recordkeeping
    would not only have to distinguish between the number of applicants
    and the number of applications but would also have to determine
    whether and where the student actually finally enrolled.

[^17]: A similar pattern was found when the enrollees' characteristics
    were compared to districtwide averages at each of the four sites.
    The parallel finding was important because non-enrollees were chosen
    to match the enrollees (see chapter 5), whereas the districtwide
    averages represented all of the students at the four sites.

[^18]: At sites where geographic areas were used to focus VPSC
    activities, thematic programming was new to that area although
    offered elsewhere in the district, and as a result, more accessible
    to students in that area.

[^19]: Examples of themes and programs include: math and science;
    International Baccalaureate program; world languages; fine and
    performing arts; literature and writing; humanities and
    international studies; environmental studies; dual-language;
    Spanish; communications technology; culinary arts; medical skills;
    hospitality and tourism; Advanced Placement classes; and media
    industry.

[^20]: The total number can nevertheless be used to estimate a range
    within which the current estimate of 21.8 percent falls: Assuming
    all of the undocumented transfers were from low- to
    higher-performing schools, the proportion would rise from 21.8 to
    81.4 percent; conversely, assuming that none of the undocumented
    were from low- to higher-performing schools, the proportion would
    dip to 5.2 percent. The range would therefore be 5.2 to 81.4
    percent.

[^21]: In 2004--05, one site's interdistrict option ended with the
    closing of the magnet school.

[^22]: ^?^ Without random assignment, any number of important but
    unobserved variables could account for differences between two
    groups. For instance, Cullen, Jacob, and Levitt (2005) studied
    school choice in the Chicago Public Schools from 1993 to 1995. The
    study found that "the leading explanation" for significant
    differences in graduation rates (students electing to transfer out
    of their neighborhood schools had higher rates) was that the
    transferring students were "superior along unobservable dimensions"
    (p. 755). High on the list of such dimensions was student motivation
    and the education background of students' parents. The study
    concluded that, aside from transfers to career academies
    (emphasizing vocational skills and integrating school with work),
    "\...systematic choice within a public school district does not seem
    to benefit those who participate" (p. 755).

[^23]: ^?^ To overcome self-selection and other biases, one frequently
    raised possibility has been to use data from sites' lotteries as if
    they were "natural experiments" (sites often use lotteries to select
    participants when the number of students exceeds the available
    choice opportunities). However, the lottery losers may not actually
    serve as the needed counterfactuals for lottery winners (see fuller
    discussion in appendix A).

[^24]: ^?^ However, whether the non-VPSC students later enrolled in a
    VPSC initiative, or otherwise transferred schools in subsequent
    years, is unknown. Ideally, had the template been followed, the
    variable "current school" would have provided information on a
    student's matriculation status in ensuing years.

[^25]: ^?^ For example, one site only provided two years of achievement
    scores, which were insufficient to calculate the needed trends, as
    described later in the text; a second site submitted cross-sectional
    data, only providing one year's scores for each successive cohort of
    "first-time enrollees;" a third site's state changed its assessment
    tests in the middle of the trend period, also then limiting the
    comparable data to only two points in time.

[^26]: A fifth site submitted data covering four cohorts, but the
    submission came well after the analysis was underway and could not
    be included in the analysis.

[^27]: ^?^ The known volatility of achievement scores was an additional
    numeric reason for requiring three or more years of data.
    Statistically, trends based on two data points assume no measurement
    error, which is unrealistic.

[^28]: ^?^ The full methodology is found in appendix A, section 3.

[^29]: ^?^ Investigators have used meta-analyses to examine many topics
    in education, including: gender differences in verbal ability (Hyde
    and Linn, 1988); the achievement benefits of summer school (Cooper
    et al., 2000); and comprehensive school reform (Borman et al.,
    2002).

[^30]: ^?^ Effect sizes are a measure of the magnitude of differences
    between two conditions---for instance, the cross-sectional
    differences in performance between two groups, or the change over
    time for one group, or both.

[^31]: The analysis avoided another potential complication that would
    have arisen if a state changed its assessment test during the
    multiyear period under study. No such changes occurred with the six
    cohorts under study.

[^32]: Confidence levels represent the probability that observed
    differences do not differ by chance alone but represent actual
    differences between two groups. Statistically, the analyses were
    based on the use of random-effects models, which maximize the
    appropriate use of all of the sites' data submitted for each of the
    six cohorts. Unlike fixed-effects models, the random-effects models
    assume that the available data come from but a sample of a fuller
    universe of students and that the sample therefore is likely to
    contain additional variance due to sampling errors (Bryk and
    Raudenbush, 1992).

[^33]: The values for effect sizes can range from negative infinity
    (extremely negative difference) to positive infinity (extremely
    positive difference), with most values falling between -4 and 4.
    However, the values are difficult to interpret with respect to their
    importance. Cohen (1988) offers one interpretation: that values from
    0 to .30 represent "small" effects, from .30 to .60 "medium"
    effects, and from .60 to 1.00 "large" effects. In education, a large
    effect may be considered a difference of a grade level in
    performance (e.g., fourth-graders performing at a fifth-grade
    level). Nevertheless, attempted interpretations of effect size
    values for any given analysis, as with standardized scores more
    generally, are heavily affected by the specific sample sizes,
    variances, and other parameters in the specific data set being
    analyzed (e.g., Cooper, 1981; and Rubin, 1992), and the present
    analysis was therefore unable to provide a precise interpretation of
    the effect sizes or standardized scores reported in this chapter.

[^34]: The conventional test of heterogeneity is a *Q*-test. A
    significant *Q* statistic indicates that the individual effect sizes
    from each cohort are sufficiently different to be poorly
    characterized by a single summary statistic, such as the grand mean.
    In the present analysis, the results were nearly significant (hence
    possibly too heterogeneous) for mathematics (*Q* = 10.2, 5 df,
    p\<.10) and significant (hence undesirably heterogeneous) for
    reading (*Q* = 20.56, 5 df, p\<.001).

[^35]: For instance, see the statement by Sen. Joseph Lieberman
    (D-Conn.), Congressional Record, 107th Congress, *No Child Left
    Behind Act of 2001*, Conference Report, U.S. Senate, Dec. 18, 2001,
    S13413--S13414.

[^36]: The lessons pertain to multisite evaluations, like the present
    one, in which the goal is to evaluate a program as a whole, not the
    workings at individual sites (the latter would still be the
    responsibility of separate *project*-level evaluators).

[^37]: ^?^The nonprofit organization operated charter schools and
    qualified as a school district, to be eligible for a VPSC award.

[^38]: Participating schools were those with students who were either
    eligible for or enrolled in a VPSC-funded initiative.

[^39]: At this site, the district was the appropriate respondent because
    many of the participating students were assigned to districts but
    not to specific schools, either prior to or as part of the VPSC
    initiative.

[^40]: The values represent the average change, per year, in
    standardized scale scores for each of the groups. Some groups' data
    covered three years and others covered four or five. However, this
    small difference was not assumed to create undesirable artifacts as
    might occur if the groups varied, for instance, between three and
    twenty years.