0
Study of the 

Voluntary Public School Choice Program 

Final Report 

For 

U.S. Department of Education 

Office of Planning, Evaluation and Policy Development 

Prepared by 

Robert K. Yin 

Pirkko S. Ahonen 

COSMOS Corporation 

Bethesda, Md. 

 2008 
This report was prepared for the U.S. Department of Education under Contract Number ED04CO0047 (Task 

Order No. 001) with COSMOS Corporation. Reeba Daniel served as the contracting officer’s representative. 

The views expressed herein do not necessarily represent the positions or policies of the U.S. Department of 

Education. No official endorsement by the U.S. Department of Education is intended or should be inferred. 

U.S. Department of Education 

Margaret Spellings 

Secretary 

Office of Planning, Evaluation and Policy Development 

Bill Evers 

Assistant Secretary 

Policy and Program Studies Service 

Alan Ginsburg 

Director 

Program and Analytic Studies Division 

David Goodwin 

Director 

November 2008 

This report is in the public domain, except for the photograph on the front cover, which is used with 

permission and copyright, 2007, Getty Images. Authorization to reproduce the report in whole or in part is 

granted. While permission to reprint this publication is not necessary, the suggested citation is: U.S. 

Department of Education, Office of Planning, Evaluation and Policy Development, Policy and Program 

Studies Service, Study of the Voluntary Public School Choice Program: Final Report, Washington, D.C., 

2008. 

This report is available on the Department’s Web site at 

http://www.ed.gov/about/offices/list/opepd/ppss/reports.html. 

On request, this publication is available in alternate formats, such as Braille, large print, or computer diskette. 

For more information, please contact the Department’s Alternate Format Center at 202-260-0852 or 202-260-

0818. 
iii

Contents 

Page 

LIST OF EXHIBITS ........................................................................................................... iv 

ACKNOWLEDGMENTS.................................................................................................. vii

EXECUTIVE SUMMARY .................................................................................................. ix 

1. The Voluntary Public School Choice (VPSC) Program and Its Evaluation ..................... 1 

2. Summary of Evaluation Methodology.............................................................................. 5 

2.1 Evaluation Design for a Program Evaluation ......................................................... 5 

2.2 Data Collection ....................................................................................................... 6 

2.3 Analytic Priorities................................................................................................... 8 

 

3. Characteristics of the VPSC Program’s Sites ................................................................. 11 

3.1 Overview of VPSC Sites ...................................................................................... 11 

3.2 Four Types of Choice Arrangements at the VPSC Sites ...................................... 14 

3.3 Core Activities in the VPSC Choice Initiatives.................................................... 17 

3.4 Initial Implications for Federal Education Policy and 

 Local Education Practice.................................................................................... 23 

4. Promoting Educational Equity and Excellence............................................................... 27

4.1 Student Participation at the VPSC Sites, 2002–07: 

 Eligibles, Applicants, and Enrollees .................................................................. 27 

4.2 Progress on Program Priorities ............................................................................. 31 

4.3 Implications for Federal Education Policy and Local Education Practice: 

 Further Issues of Educational Equity and Excellence........................................ 37 

5. Student Academic Achievement Trends Concurrent with the VPSC Initiatives............ 41 

5.1 Collection of Individual-Level Student Data....................................................... 42 

5.2 Methodology for Analyzing Student Achievement Trends ................................. 46 

5.3 Findings on Student Achievement Trends ........................................................... 49 

6. Useful Choice Practices and Overall Lessons Learned about 

 Voluntary Public School Choice................................................................................. 55 

6.1 Useful Choice Practices for Future Choice Initiatives......................................... 55 

6.2 Overall Lessons Learned from the National Evaluation of the VPSC Program.. 58 

6.3 Two Challenges for Improving Voluntary Public School Choice ....................... 61 

REFERENCES .................................................................................................................. 63 
iv

APPENDIXES

Page 

A. Detailed Evaluation Methodology ..................................................................................A-1 

B. Instruments Used in the National Evaluation..................................................................B-1 

C. Supplemental Data about Individual VPSC Sites ...........................................................C-1 

EXHIBITS 

1-1 Location of the VPSC Grantees (2002 Awards).............................................................2 

2-1 Other Public School Choice Options at the VPSC Sites .................................................7 

3-1 Community Characteristics...........................................................................................12 

3-2 Brief Descriptions of the VPSC Sites’ Choice Initiatives ...........................................13 

3-3 Grade Span Distribution of Participating Schools ........................................................15 

3-4 Four Types of Choice Arrangements............................................................................15 

3-5 Parent Involvement Activities Reported by School Officials in School Survey ..........18 

3-6 Schools’ Reported Efforts to Notify Parents of Their Choice Options, 2006–07.........19 

3-7 Parents’ and Families’ Understanding of Choice 

 Reported by School Officials, 2006–07..................................................................19 

3-8 Capacity-Enhancing Activities .....................................................................................21 

3-9 Illustrative Curriculum Topics at the VPSC Sites ........................................................22 

3-10 Variations in Sites’ Uses of VPSC Funds to Support Choice Options.........................26 

4-1 Student Participation in the VPSC’s Initiatives, 2005–06 ............................................28 

4-2 Participation Rates, 2003–07 ........................................................................................30 

4-3 Demographic Characteristics of Schools, 2006–07 ......................................................33 

4-4 Reported AYP Status of Schools, 2004–05 through 2006–07......................................34 

4-5 Student Transfers from Low- to Higher-Performing Schools, 2005–06 ......................35 

4-6 Eligibility to Participate in VPSC’s Choice Arrangements 

 An Equity Perspective.............................................................................................39 

5-1 Desired Student Record ................................................................................................42 

5-2 Individual-level Data Submitted by VPSC Sites .........................................................45 

5-3 Baseline Characteristics of VPSC Enrollees and Matched Comparison Students........50 

5-4 Performance of VPSC and Matched Comparison Students, 

 Analyzed Separately................................................................................................51 

5-5 Aggregate Comparison of VPSC and Matched Comparison Groups, 

 Across Cohorts........................................................................................................52 

6-1 Potentially Useful Choice Practices from the VPSC Program .....................................56 

A-1 Schedule of VPSC Field Visits.....................................................................................A-6 

A-2 VPSC School Survey: Number of Schools Surveyed and 

 Response Rate ........................................................................................................A-8 
v

C-1 State Education Agencies Used VPSC Funds to 

 Support Widely Different Choice Priorities...........................................................C-3 

C-2 VPSC Initiatives Coincided With Existing, Districtwide Choice Options ..................C-3 

C-3 Sites Split Their VPSC Funds to Support a Variety of Activities, 

 Not Limited to a Single Choice Initiative ..............................................................C-4 

C-4 VPSC Initiatives Could Be Separate From or Coincide With Title I Choice .............C-4 

C-5 Five Years of VPSC Participants.................................................................................C-5 

C-6 Choice Initiatives Limited to Zones Within a District 

 Nevertheless Involved a Broad Array of Sending Schools....................................C-6 

C-7 Eligible Sending Schools Were Restricted, 

 but Not Necessarily to Low-Performing Schools..................................................C-6 

C-8 Capacity-building Occurred at Receiving Schools 

 That Already Were (Presumably) Higher-performing Schools ............................C-6 

C-9 Local Sites May Need to Focus on a “Tapestry” of Choice 

 Options, Rather Than on Any Single Initiative .....................................................C-7 

C-10 “Attractor” Programs at Receiving Schools May Not 

 Have Been Needed to Attract Incoming Students .................................................C-7 

C-11 Explicit Efforts May Be Needed to Maintain the 

 Performance of Receiving Schools ........................................................................C-7 
vi
vii

ACKNOWLEDGMENTS 

 This report has benefited from the advice and comments of many expert reviewers, 

who have given generously of their time. A Technical Working Group (TWG) met four 

times during the course of the project (March 2003, July 2005, April 2006, and December 

2007), each time providing helpful feedback on the evaluation’s progress. At its final 

meeting, the TWG discussed an earlier version of this final report. The following is a list 

of the TWG members: 

 Frank Brown, professor of education, University of North Carolina 

 Peter Cookson, dean of graduate school, Lewis and Clark College 

David Heistad, executive director, Testing, Evaluation, and Student Information, 

Minneapolis Public Schools 

 Valerie Lee, professor of education, University of Michigan 

 Janelle T. Scott, assistant professor of education, New York University 

 Paul Teske, dean, Graduate School of Public Affairs, University of 

Colorado at Denver 

 Patrick Wolf, professor and endowed chair in school choice, University of 

 Arkansas 

 Todd Ziebarth, senior policy analyst, National Alliance for Public Charter 

 Schools 

 Patrick McKnight of George Mason University’s Department of Psychology 

conducted the analyses described in section 3 of appendix A. These analyses support the 

findings in chapter 5. In addition, Clive Belfield of the Teachers College at Columbia 

University served as an independent consultant throughout the entire project, reviewing 

drafts of reports, including this final report. The final report also benefited from reviews of 

earlier versions by David Goodwin, Daphne Kaplan, and Reeba Daniel at the U.S. 

Department of Education (the Department), along with comments by two anonymous 

reviewers. 

 The bulk of the data came from the 13 sites, with Iris Lane and John Fiegel of the 

Department’s VPSC Program Office graciously facilitating the process. Without the 

cooperation of all these parties, the evaluation could not have been conducted. 

 All this assistance notwithstanding, the authors are responsible for this final report and 

any errors it may contain. 
viii
ix

EXECUTIVE SUMMARY

The Voluntary Public School Choice (VPSC) Program 

And Its Evaluation 

 The No Child Left Behind Act of 2001 (NCLB)1 expanded public school choice 

opportunities for students. First, the new accountability requirements in Title I of the 

Elementary and Secondary Education Act (ESEA) (Title I, Section 1116(b)) required 

districts to offer public school choice to students in Title I schools that are identified for 

improvement, corrective action, or restructuring as a result of not meeting state definitions 

for adequate yearly progress (U.S. Department of Education, 2004).2

 Second, the law created the VPSC Program (Title V, Subpart 3, Section 5241 of the 

ESEA), which is the subject of the current report. The program supports the emergence and 

growth of choice initiatives across the country, by assisting states and local school districts 

in developing innovative strategies to expand choice options for students. 

 The VPSC Program functions independently from the choice provisions in Title I and 

provides funds to a relatively small number of sites across the country. In October 2002, 

the U.S. Department of Education (the Department) awarded five-year competitive grants 

to 13 applicants. Awards ranged in size from $2.8 million to $17.8 million for an average 

award of $9.2 million for five years, or approximately $1.8 million per year. The VPSCfunded locations included: the state of Arkansas; Albany, N.Y.; Chicago, Ill.; the state of 

Florida; Hartsdale, N.Y.; Hillsborough County, Fla.; La Quinta, Calif.; Miami, Fla.; the 

state of Minnesota; Swanzey, N.H.; New Haven, Conn.; Portland, Oreg.; and Rockford, Ill. 

This report contains the final assessment of the first five years of the VPSC 

Program (2002–07). The evaluation was charged with assessing the VPSC Program’s 

progress in meeting the goals and fulfilling the intent of the original legislation. The 

evaluation addressed three central questions: 

 1) What are the characteristics of the VPSC Program’s sites? 

 2) How and to what extent does the VPSC Program promote educational 

 equity and excellence? 

 3) What academic achievement is associated with the VPSC Program? 

 1 The No Child Left Behind Act of 2001 (P.L.107-110) amended the Elementary and Secondary 

Education Act of 1965. 2 If all schools in a district have not made adequate yearly progress (AYP) for two or more consecutive 

years, the district must, “to the extent practicable,” establish a cooperative agreement for a transfer with other 

districts in the surrounding area. 34 C.F.R. 200.44(h)(1). 
x

To address the third question, only a few of the VPSC sites provided the needed data. As a 

result, the evaluation’s findings for this third question may come from too small a sample 

of sites or of VPSC choice enrollees to cover the VPSC Program’s experience adequately. 

 In addition to the three central questions, the analysis also focused on four priorities in 

the original VPSC legislation: a) providing the widest possible choice to students in 

participating schools; b) promoting transfers of students from low- to higher-performing 

schools; c) forming interdistrict partnerships to allow students to transfer to a school in 

another district from that of their original school; and d) requiring sites to use funds to 

support transportation services for students (on the assumption that this would allow 

students to attend more attractive but also more distant schools). 

The evaluation covers choice initiatives at 13 sites that received VPSC grant awards 

from the U.S. Department of Education in the fall of 2002. The 2002 grantees were nine 

districts, three state education agencies, and one nonprofit organization.3 However, the 

findings of this report draw conclusions about the VPSC Program as a whole, not the 

activities at any given VPSC site. At the same time, the evaluation does not cover a second 

round of 14 five-year grants made by the VPSC Program in the summer of 2007. 

 The relevant evaluation data came from several sources: multiple site visits to the 

VPSC sites; three rounds of surveys covering an average of 50 participating schools at each 

VPSC site; reviews of grantees’ annual reports; and the collection of archival data about 

student performance. The archival data tracked individual student achievement trends, 

covering one or more academic years prior to the start of a VPSC initiative and as many 

academic years as possible during the implementation of the initiative. The trends were 

then compared to those for a group of students not enrolled in a VPSC choice initiative. 

Brief Summary of Key Findings

 Briefly summarized, at the end of five years, the VPSC Program, though limited to 

grant awards at only 13 sites, showed a variety of different public school choice 

arrangements working in a diverse array of communities across the country. Among the 

four legislative priorities, 

• The VPSC Program made progress on the first priority in providing the 

widest variety of choice. 

• Transfers from low- to higher-performing schools comprised only a 

portion of the students enrolled in the VPSC initiatives, with only three 

 3

 The nonprofit organization operated charter schools and qualified as a school district, to be eligible for a 

VPSC award. 
xi

of the 13 sites limiting their enrollment to transfers from low- to higherperforming schools. 

• Most of the VPSC sites limited their choice initiatives to within-district 

options, rather than developing interdistrict options. 

• Relative to enrollment, transportation costs did not increase 

proportionately as might have been expected, because the VPSC 

initiatives permitted many students who were already attending distant 

schools to select schools closer to home. 

 Regarding student achievement trends, data from a highly limited set of 

VPSC sites and students enrolling in their choice initiatives showed improved 

trends in math and reading, compared to matched groups of non-enrolling 

students. 

 The gains were statistically significant, but the findings need to be tempered 

by several cautions: 

− the voluntary nature of choosing to enroll in a VPSC initiative and the 

possibility that the VPSC enrollees were more highly motivated students 

than non-enrollees, thereby accounting for some or all of the differences 

in student achievement; 

− the procedure for selecting the matched groups of non-enrolling 

students; 

− the length of time covered by the trends (more annual data points came 

from the years before rather than after enrollment); and 

− the fact that the data came from only four of the 13 VPSC sites and only 

six of 38 cohorts of annual enrollees across all of the VPSC sites. 

 Given all the cautions, the findings in this analysis of six cohorts from four VPSC 

sites offer early promise regarding the potential benefits of the VPSC Program. 

Overview of the VPSC’s Sites

 The 13 VPSC sites were located across the country. Ten were located in 

predominantly urban communities, two in areas that cover both urban and rural regions, 

and one in an entirely rural area. 

 Participating schools covered all grade levels: 64 percent spanned the elementary 

grades, 15 percent the middle school grades, 11 percent the high school grades, and 10 

percent were classified as “other” or had missing data. As a comparison, approximately 58 
xii

percent of schools nationwide are elementary schools, 17 percent are middle schools, 19 

percent are high schools, and 6 percent are “other.” 

 The VPSC's choice initiatives varied in the timing of their implementation. Some of 

the sites started enrolling choice students as early as January 2003, while other sites started 

enrollment the following year or even later. Because all sites began some type of planning 

if not enrollment activity in 2002–03, the school year 2001–02 is considered the base year 

prior to the implementation of VPSC initiatives. The evaluation traced the sites’ activities 

for the duration of their five-year awards through 2006–07, although the majority of sites 

received no-cost extensions to support their operations for another year. 

 Once they started enrollment, the VPSC sites offered choice options to a new set of 

students every year. Therefore, for each cohort of “first-time enrollees,” the “intervention 

period” varied. At the student level, this variation was taken into account by defining 

different base years for each cohort of students. A consequence of this staggered pattern is 

that more years of annual data were available for the older cohorts than for the younger 

ones. 

As with many other school districts in the country, nearly all of the 13 sites already 

had a broad variety of public school options prior to the VPSC Program. The program 

permitted sites to enhance or expand existing options, and sites did not need to start entirely 

new initiatives. Sites tried to make their entire array of choice options work well, but they 

did not necessarily track students’ participation on an option-by-option basis. 

Acknowledging this complication also is necessary for interpreting the findings in this 

report. For some of the sites, the VPSC funds were only part of the support for the 

identified initiative. As a result, the contribution of the VPSC Program may have been 

overestimated. At other sites, the VPSC funds not only supported the identified initiative 

but also partially supported other choice options at the same site. As a result, the 

contribution of the VPSC funds may have been underestimated. Unfortunately, the 

analysis was not able to distinguish the extent of these over- or underestimations. 

 The VPSC Program also started in the same year that federal legislation expanded 

support for Title I choice options. The legislation allowed spending up to 20 percent of 

Title I funds to support transportation costs for students transferring away from schools 

designated under Title I as “identified for improvement.” Three of the 13 sites defined 

their VPSC initiatives to coincide or overlap closely with their Title I choice options. 

However, most VPSC sites defined schools participating in their VPSC initiatives as a 

broader set of schools than simply those designated under Title I as identified for 

improvement. 
xiii

Four Types of Choice Arrangements at the VPSC Sites

 

 The sites varied greatly in the design of their choice initiatives, differing in the 

number of students served, the number of participating public schools, and the capacity to 

accommodate transfers. In addition, the sites differed by how they defined choice zones 

and managed the flow of students among participating schools. Despite this variation, sites 

also pursued some common paths. 

 First, although unique, the VPSC’s choice initiatives tended to fall under four major 

categories, based on how sites defined choice arrangements and directed the flow of 

transferring students: 

• Five of the sites designated specific schools to be either sending schools 

or receiving schools but not both. 

• Five sites defined initiatives whereby the same schools could be both 

sending and receiving. 

• One site established a within-school initiative, in which students chose 

from education programs within the same school and did not transfer 

between schools. 

• Two sites had initiatives involving a mixture of the first three types. 

 Second, all sites focused on two core activities throughout the implementation 

process: 1) engaging parents and community members, and 2) building capacity at schools 

to attract and accommodate choice transfers. 

 Parent and community activities included a rich array of outreach, marketing, and 

communication efforts. Sites also engaged parents and community representatives in 

developing and implementing choice initiatives. 

 Capacity-enhancing activities included starting new academic programs or subjects, 

purchasing supplies and equipment for schools, and providing professional development to 

teaching staff. However, the sites’ capacity-enhancing activities were not necessarily 

accompanied by an expansion of seats or classrooms at many of the receiving schools. For 

instance, none of the sites reported hiring more teaching staff or taking other steps simply 

to expand the number of seats to accommodate higher enrollments at the existing schools. 

Student Participation at the VPSC Sites 

In 2005–06, across 12 of the 13 VPSC sites, 24,921 students enrolled in choice 

initiatives, reflecting an overall participation rate of 2.8 percent of the students eligible to 
xiv

enroll. This number of students represented those who had enrolled for the first time that 

year (“first-time enrollees”). The cumulative first-time enrollees, starting from the 

inception of the VPSC Program in 2002–03 until 2005–06, included 49,616 students. 

Thus, the most accurate estimate of participation falls somewhere between 24,921 and 

49,616. (However, the cumulative enrollment overestimates the total enrollment because 

of an attrition factor that was not tracked by the sites.) 

Overall, the number of enrolling students in the VPSC Program increased during 

the earlier years of the program but declined in the program’s fifth year. Of the VPSC 

sites’ enrolling students, ten provided eligibility and enrollment data for four consecutive 

years (2003–04 to 2006–07). These data permitted the estimation of trends through the 

fifth VPSC year (2006–07). The yearly data captured the number of first-time (i.e., new) 

enrollees each year, not the total number across all years. For these “first-time enrollees,” 

the trends showed that the VPSC Program at first averaged 696 enrollees per site in 2003–

04, then reached a peak of 2,459 per site in 2005–06, and then declined to 2,167 per site in 

2006–07. 

 The decline may have reflected the actual saturation of the VPSC initiatives because a 

good (but unmeasured) portion of the earlier years’ enrollees still remained enrolled in the 

later years, possibly limiting the seats available for first-time enrollees in the final year. 

Participation rates also showed the same pattern, increasing from 2003–04 to 2005–

06, and then declining in 2006–07. For the same ten sites reporting data from 2003–04 to 

2006–07, the participation rates by first-time enrollees also increased from 1.5 percent to 

4.1 percent, from 2003–04 to 2005–06, and then dropped to 3.0 percent in 2006–07. 

 

Progress on Program Priorities

 The VPSC legislation had four program priorities. Three priorities were: to provide 

the widest variety of choices; to encourage transfers from low- to higher-performing 

schools; and to provide opportunities for students to transfer to schools outside of their 

home districts. The fourth priority directed sites to use some of their VPSC funds to 

support transportation services and costs. Findings on these four priorities are discussed 

next. 

The VPSC Program made progress on the first priority of providing the widest 

variety of choice. Sites expanded the assortment of choice options in participating schools 

and offered a diverse number of academic programs to transferring students. The VPSC 

initiatives also made efforts, through media campaigns and related activities, to increase 

parents’ awareness of the variety of education options available to them. 

Transfers from low- to higher-performing schools comprised only a portion of the 

students enrolled in VPSC initiatives. Only three VPSC sites limited their enrollment to 
xv

transfers from low- to higher-performing schools. Another five sites permitted transfers 

within an entire district or zone, but only two of the sites tracked the portion of the transfers 

from low- to higher-performing schools. Aside from the five sites that either limited or 

tracked their transfers from low- to higher-performing schools, none of the other eight sites 

could provide such information. At the five sites, the confirmed transfers from low- to 

higher-performing schools represented 1,295 of 5,927, or 21.8 percent, of their total 

transfers from 2005–06. 

 The other eight VPSC sites either permitted a wider variety of transfers or had VPSC 

enrollments involving no transfers. Among the sites not tracking the low- to higherperforming transfers, one received a waiver from the Department to omit such tracking 

because all of the site’s enrollees were low-performing students (scoring “below proficient” 

on the state assessment), but they were not necessarily transferring from low- to higherperforming schools. Thus, across the entire VPSC Program, the actual portion of low- to 

higher-performing transfers could be larger or smaller, depending on the number of 

transfers at the eight sites that did not track or document the pattern of their transfers. 

 Most of the VPSC sites limited their choice initiatives to within-district options, 

rather than developing interdistrict partnerships. Although the implementation of formal 

interdistrict choice options might have expanded the variety of choices available to students 

even further, only five of the 13 VPSC sites created formal interdistrict options. At the 

eight remaining sites, many had existing transfer agreements with neighboring districts. 

However, these transfer options were separate from VPSC initiatives and were generally 

reviewed on a case-by-case basis. The majority of the VPSC initiatives did not directly 

promote student transfers to other school districts, and VPSC funds were not used to 

support interdistrict transfers at these sites. 

Relative to enrollment, transportation costs did not increase proportionately as 

might have been expected. This is partly because the VPSC initiatives permitted many 

students who were already attending distant schools to select schools closer to home. As an 

example, two of the VPSC sites had recently emerged from court-ordered school busing, 

from which students had been assigned to more distant schools as part of the original 

desegregation order. The VPSC-funded initiative gave affected students the choice of 

enrolling at neighborhood schools. Under these circumstances, sites experienced minimal 

or reduced transportation costs. 

Findings on Student Achievement Trends

 The final student achievement trends came from six cohorts of VPSC enrollees and 

six corresponding groups of non-VPSC enrollees, across four of the 13 VPSC sites. The 

cross-site analysis first estimated the student achievement trends for these two groups 

separately and then compared the trends between the two groups. 
xvi

 These two analyses served two different purposes. The first, estimating trends 

separately, was needed to establish whether either the enrollee or matched comparison 

group was alone moving in a positive or negative direction. The goal was to determine 

whether the enrollee group might have been performing worse, regardless of any relative 

difference between it and the comparison students. The second, estimating trends relative 

to each other, then captured the comparison between the two groups. (The two analyses 

involved two different units of analysis, students and cohorts, and the values should not be 

compared across these two analyses.) 

When the VPSC and non-VPSC trends were examined separately, the VPSC 

enrollees’ trends were neutral in math and positive in reading, but not statistically 

significant. More important, the enrollee group’s scores were not found to be declining in 

any way. In contrast, the non-enrollee group showed a declining trend in math proficiency 

that was statistically significant at the 99 percent confidence level.4 The non-enrollees’ 

trends for reading were positive but not significant. 

When the VPSC and non-VPSC trends were compared, students enrolling in the 

VPSC initiatives had better student achievement trends than those not enrolling. The 

enrollees surpassed the non-enrollees in the trends for both mathematics and reading, with 

the differences in mathematics being statistically significant at the 99.9 percent confidence 

level and in reading at the 99 percent level. The comparison was based on a meta-analysis 

of effect sizes across all six of the cohorts, with the individual effect sizes for each cohort 

having already accounted for the demographic and baseline differences between the VPSC 

and non-VPSC groups. Because the units of measure were standardized scores, the trends 

cannot easily be translated into everyday educational units, but the effect sizes (.020 and 

.028 for mathematics and reading respectively) appear to be modest. 

 These findings need to be accompanied by several cautions about interpreting the 

differences in achievement trends between the enrolling and non-enrolling groups. The 

main cautions are as follows. 

 First, in all of the choice initiatives, students can choose to enroll or not. As a result, 

an enrollee group may represent more highly motivated students or differ in other 

unobserved ways from a non-enrollee group, accounting for some or all of any subsequent 

differences in student achievement.5

 4

 Confidence levels represent the probability that observed differences do not differ by chance alone but 

represent actual differences between two groups. Statistically, the analyses were based on the use of randomeffects models, which maximize the appropriate use of all of the sites’ data submitted for each of the six 

cohorts. Unlike fixed-effects models, the random-effects models assume that the available data come from 

but a sample of a fuller universe of students and that the sample therefore is likely to contain additional 

variance due to sampling errors (Bryk and Raudenbush, 1992). 5

 To overcome self-selection and other biases, one frequently raised possibility has been to use data from 

sites’ lotteries as if they were “natural experiments” (sites often use lotteries to select participants when the 

number of students exceeds the available choice opportunities). However, the lottery losers may not actually 

serve as the needed counterfactuals for lottery winners (see fuller discussion in appendix A). 
xvii

 Second, the trends in the present analysis leaned in the direction of having more of the 

data points precede rather than follow enrollment in the VPSC initiative. Confidence about 

the trends concurrent with a VPSC initiative, much less inferences about any effects, would 

be increased if analyses were based on a larger number of data points following enrollment. 

 Third, the data only came from a small sample of sites (four of 13) and from only a 

small sample of the enrolling students (six of 38 cohorts of annual enrollees across all of 

the VPSC sites by 2005-06). Data from more sites and cohorts would produce a firmer set 

of findings about the program as a whole. The ongoing VPSC Program can put renewed 

emphasis on obtaining such data, given the new round of five-year awards made in 2007. 

The possibilities of such additional data are especially strong, given that seven of the newly 

awarded sites were continuations from the first round of awards. 

 Given the cautions, the findings in this analysis of six cohorts from four VPSC sites 

offer early promise regarding the potential benefits of the VPSC Program. 

 Recommendations for improving future evaluations as the VPSC Program progresses 

with its 2007 cohort of grantees are detailed at the end of the full report. Aside from taking 

steps to overcome the earlier cautions about interpreting student achievement findings, two 

specific procedures could improve the robustness of future analyses: 

● Careful recordkeeping and tracking of all participants. Sites should 

keep accurate and aggregate counts of all three types of choice 

participants (eligibles, applicants, and enrollees) and also should 

compile individual student-level data for choice enrollees’ 

demographic characteristics and student achievement scores. 

● A modified procedure for defining comparison students. Sites should 

not try to match any particular group of non-enrolling students, as the 

sites did in the current evaluation. Rather, the more desirable 

procedure would be for sites to provide data from a larger but 

nonselected set of non-enrolling students, such as those students 

remaining in the sending schools or even a districtwide set of 

students. Analytic procedures such as propensity score matching or 

some similar procedure conducted by the evaluation team could then 

provide a fairer selection of matched comparison group than the sites’ 

procedures in the current evaluation. 
xviii
1 

1. THE VOLUNTARY PUBLIC SCHOOL CHOICE (VPSC) PROGRAM 

AND ITS EVALUATION 

 The No Child Left Behind Act of 2001 (NCLB)

6

 expanded public school choice 

opportunities for students. First, the new accountability requirements in Title I of the 

Elementary and Secondary Education Act (ESEA) (Title I, Section 1116(b)) required 

districts to offer public school choice to students in Title I schools that are identified for 

improvement, corrective action, or restructuring as a result of not meeting state definitions 

for adequate yearly progress (U.S. Department of Education, 2004).7

 

 Second, and the subject of the present report, Congress created the VPSC Program 

(Title V, Subpart 3, Section 5241 of the ESEA) to support the emergence and growth of 

choice initiatives across the country. The purpose of the program is to assist state and local 

school districts in the development of innovative strategies to expand options for students, 

and to encourage transfers of students from low- to higher-performing schools. 

 The VPSC Program functions independently from the choice provisions in Title I and 

provides funds to a relatively small number of sites across the country. The program has 

made two rounds of awards, only the first of which is the subject of the present evaluation. 

The first round occurred in the fall of 2002, when the U.S. Department of Education (the 

Department) awarded five-year grants to 13 applicants (see exhibit 1-1). The second round 

occurred in the summer of 2007, when the Department made another set of five-year grants 

to 14 applicants, seven of which had received awards in the first round. 

 The VPSC Program was designed and initiated against a backdrop of increasing 

interest in public school choice. Such initiatives give students the option of enrolling at a 

public school other than the one to which they are assigned as a result of their residential 

location. Their choices can include: all the other public schools in a system; some of the 

schools (e.g., schools in a geographic region within the system); or specific schools (e.g., 

magnet schools, charter schools, or schools identified under Title I choice). Despite the 

variation in options, all choice is limited to public schools only. 

 The evaluation of the VPSC Program, focusing on the first round of awards only, 

addressed three questions to assess the VPSC Program’s progress in meeting the goals and 

fulfilling the intent of the original legislation. The three central questions Congress asked were: 

 6

 The No Child Left Behind Act of 2001 (P.L. 107-110) amended the Elementary and Secondary 

Education Act of 1965. 7

 If all schools in a district have not made adequate yearly progress (AYP) for two or more consecutive 

years, the district must, “to the extent practicable,” establish a cooperative agreement for a transfer with other 

districts in the surrounding area. 34 C.F.R. 200.44(h)(1). 
2 

Exhibit 1-1 

Location of the VPSC Grantees 

(2002 Awards) 

Exhibit Reads: One of the 13 VPSC grantees, Portland Public Schools in Oregon, received a fiveyear VPSC award of $6,467,122. 

1) What are the characteristics of the VPSC Program’s sites? 

2) How and to what extent does the VPSC Program promote educational 

equity and excellence? 

3) What academic achievement is associated with the VPSC Program? 

 Question one relates to basic descriptive information about the program sites and their 

implementation strategies, including activities related to community outreach and capacitybuilding within participating schools. 

State of Florida 

Department of 

Education 

Award: $17,817,910

School District of 

Hillsborough County, Fla.

Award: $10,226,759

Rockford Public School 

District #205, Ill.

Award: $10,095,054

Portland Public 

Schools, Oreg.

Award: $6,467,122

State of Minnesota 

Department of 

Education

Award: $11,751,052

Chicago Public 

Schools, Ill.

Award: $10,225,138

State of Arkansas 

Department of 

Education 

Award: $9,276,886

Miami-Dade 

County Public 

Schools, Fla.

Award: $11,717,901

Brighter Choice 

Charter Schools, N.Y.

Award: $3,411,567

Greenburgh 

Central School 

District No. 7, N.Y. 

Award: $2,822,155

Monadnock 

Regional School 

District, N.H.

Award: $8,396,542

Desert Sands 

Unified School 

District, Calif.

Award: $7,947,309

New Haven Public 

Schools, Conn.

Award: $9,509,901

State of Florida 

Department of 

Education 

Award: $17,817,910

School District of 

Hillsborough County, Fla.

Award: $10,226,759

Rockford Public School 

District #205, Ill.

Award: $10,095,054

Portland Public 

Schools, Oreg.

Award: $6,467,122

State of Minnesota 

Department of 

Education

Award: $11,751,052

Chicago Public 

Schools, Ill.

Award: $10,225,138

State of Arkansas 

Department of 

Education 

Award: $9,276,886

Miami-Dade 

County Public 

Schools, Fla.

Award: $11,717,901

Brighter Choice 

Charter Schools, N.Y.

Award: $3,411,567

Greenburgh 

Central School 

District No. 7, N.Y. 

Award: $2,822,155

Monadnock 

Regional School 

District, N.H.

Award: $8,396,542

Desert Sands 

Unified School 

District, Calif.

Award: $7,947,309

New Haven Public 

Schools, Conn.

Award: $9,509,901
3 

 Question two relates to the extent and nature of student participation in the choice 

initiatives funded by the program, as well as sites’ pursuit of the stated goals of the VPSC 

legislation. The legislation stipulated four priorities in the selection of sites: 

a) Provide the widest possible choice to students in participating schools; 

b) Promote transfers of students from low-performing to higher-performing 

 schools; 

c) Include interdistrict partnerships to allow students to transfer to a school 

 in another district from that of their original school; and 

d) Require sites to use funds to support transportation services for students 

(on the assumption that this would allow students to attend more 

attractive but also more distant schools). 

 Question three pertains to Congress’s interest in having the evaluation investigate the 

achievement outcomes associated with the VPSC Program. It asks whether students who 

participated in the VPSC Program had greater academic outcomes in reading and math 

compared to similar students that had not participated in the program. 

This report covers the three evaluation questions and related topics. After a brief 

description of the methodology in chapter 2, the report is organized as follows: 

 Chapter 3 addresses the first evaluation question by providing a detailed analysis of 

the characteristics of the VPSC Program’s grantees. This chapter also categorizes the sites 

by type of choice arrangement, documents important program practices, and describes 

other choice initiatives at the VPSC sites. 

 Chapter 4 addresses the second evaluation question and discusses how and to what 

extent the program has promoted educational equity and excellence. The chapter presents 

trends in eligibility, applicants, and enrollees at the sites. This chapter also examines the 

degree to which the program has provided students with: a wide variety of choice; 

opportunities to transfer from low- to higher-performing schools; interdistrict choice 

options; and transportation support. 

 Chapter 5 addresses the third evaluation question by presenting and discussing 

student achievement trends concurrent with the VPSC Program. 

 Finally, Chapter 6 of the report identifies useful choice practices emerging from the 

VPSC Program and discusses implications for the program and future research. 
4 
5 

2. SUMMARY OF EVALUATION METHODOLOGY 

 The purpose of the national evaluation was to assess the experience of the VPSC 

Program. The evaluation followed a mixed quantitative and qualitative methods research 

design, with data coming from a variety of original and archival sources (for a full 

description of the evaluation methodology, see appendix A: Detailed Evaluation 

Methodology). 

2.1 Evaluation Design for a Program Evaluation 

 The evaluation covered the 13 sites that received VPSC awards from the Department 

in the fall of 2002. However, the analysis and findings in this report aimed at drawing 

conclusions about the VPSC Program as a whole, not the activities at any given VPSC 

site.8

 

 Because choice initiatives can involve schools and students in a variety of roles, the 

evaluation defined the schools’ roles in the following manner: 

(a) Sending schools: schools from which students transfer; 

(b) Receiving schools: schools to which students transfer; 

(c) Both sending and receiving: schools that have students both 

transferring in and out as part of the same choice initiative; and 

(d) Within-school initiatives: initiatives with no inter-school transfers, 

because students choose among different education programs within 

the same school. 

Similarly, the evaluation defined the students’ roles in the following manner: 

(a) Eligible students: all students who could potentially participate in a 

VPSC initiative; 

(b) Applicants: the set of eligible students who applied either to attend 

another public school or to participate in an academic program within 

their original school; and 

(c) Enrollees: those students who successfully applied and enrolled in a 

school or program as a result of a VPSC initiative. 

 8

 Each site engaged a “local” evaluator to assess the specific site’s progress and accomplishments. 
6 

 The evaluation accounted for the varied timing of the VPSC’s choice initiatives in the 

following manner. After receiving their five-year awards in the fall of 2002, some of the 

sites started enrolling choice students as early as January 2003. Other sites carried out 

planning activities first and then started enrollments the following year or later. Because 

all sites began some type of activity in 2002–03, the school year 2001–02 is considered the 

base year prior to the implementation of the VPSC initiatives. The evaluation traced the 

sites’ activities for the duration of their five-year awards, through 2006–07. 

 Sites also offered choice options to a new set of students every year. Therefore, for 

each cohort of students, the “intervention period” varied. At the student level, this 

variation was taken into account by defining different base years for each cohort of 

students. A consequence of this staggered pattern is that more years of annual data were 

available for the older cohorts than for the younger ones. 

Finally, the evaluation design called for the collection of data from comparison 

groups for every VPSC site. The groups mainly consisted of students who were not 

enrolled in the VPSC initiatives but who had similar demographic and academic 

characteristics as the enrolling students. The VPSC sites provided data about these 

comparison groups. (See chapter 5 for additional details.) 

 From a design standpoint, two complications diminished the evaluation’s ability to 

assess the VPSC Program. First, all of the sites already had one or more choice options 

ongoing at the onset of the VPSC initiative. These other options included magnet and 

charter schools, Title I public school choice, and a variety of other choice options offered 

by the district or state. Overall, even when sites used VPSC funds exclusively to support a 

new VPSC Program initiative, the sites had other unrelated choice options operating at the 

same time (see exhibit 2-1). 

 Second, the VPSC Program permitted sites to expand existing arrangements rather 

than to establish new ones. The majority of the sites chose to expand existing initiatives. 

Separating the VPSC-funded portion of a choice initiative from other aspects of the 

initiative has not been easily delineated. 

2.2 Data Collection 

The data used in this evaluation came from a variety of sources. These included: 

multiple site visits to the VPSC sites; surveys covering an average of 50 participating 

schools at each VPSC site; and the collection of archival data about student achievement as 

well as reviews of the grantees’ reports to the Department. 

 

The team conducted three rounds of site visits to every grantee site. The site visits 

covered the VPSC project site as well as one sending school and one receiving school (or 

two participating schools at those sites not designating specific sending and receiving 
7 

Exhibit 2-1 

Other Public School Choice Options at the VPSC Sites 

Other choice options

Number of sites with other public 

school choice options in addition 

to the VPSC-funded initiative* 

Magnet schools 13 

Charter schools 11 

Title I choice 11 

Interdistrict options 9 

Other district options 9 

Other state options 5 

Total for all 13 sites 58 

Average for each site 4 or more**

Exhibit Reads: All 13 sites had magnet schools in addition to the VPSC-funded initiative. 

*Individual VPSC sites can appear under more than one type of choice option. 

**Each of the 13 sites had an average of four or more public school choice options in addition to the VPSCfunded initiative. 

Sources: Analysis of site visit data and Grant Performance Reports, by COSMOS Corporation, 2007. 

schools). The sites chose two schools that had a lot of choice activity (i.e., many 

transferring students), so that the site visit team could observe the VPSC initiative in action 

at the school level. 

 A two-person team conducted all three rounds of field visits. The team’s data 

collection activities were guided by field protocols (see appendix B). These protocols 

included interviews with the VPSC site’s project director, staff, and other key participants 

in the choice initiative. The protocols also called for the collection of documents and 

archival data related to the interviews. 

 The team also conducted three rounds of surveys with participating schools at VPSC 

sites. The surveys were brief, mainly gathering data to corroborate the schools’ 

participation in initiatives but not to investigate the schools’ other conditions in any depth. 9

 

 9

 The survey covered 12 of the 13 sites in the VPSC Program because the 13th site focused mainly on 

technical assistance to districts statewide. 
8 

 The surveys were based on a closed-ended questionnaire (see appendix B) sent to 

school principals. The questionnaire requested information about: student demographics; 

school performance; the choice options available to students; the percentages of students 

taking part in these options; the methods by which choice information was being shared 

with parents; and whether staff members were receiving professional development related 

to school choice. 

 The survey included all of the schools that were eligible to participate in the choice 

initiative. One statewide initiative was an exception: no schools were surveyed in this 

initiative; instead, the districts with five or more students participating in the choice initiative 

were asked to complete the survey at the district level.10

 In the survey’s third and final round, the evaluation team distributed questionnaires to 

689 schools. Respondents at 630 schools completed and returned the questionnaire, 

resulting in an overall response rate of 91 percent. The response rates for the earlier two 

rounds were 75 and 91 percent, respectively (see appendix A, exhibit A-2). 

 The data collection also included archival data. The data covered individual student 

achievement scores for one or more academic years prior to the start of a VPSC initiative 

through as many academic years as possible during the implementation of the initiative. 

The data covered both the students enrolled in the VPSC choice initiatives and matched 

comparison groups of students. 

 The archival data also came from reviews of annual performance reports (2003–06) 

submitted by the VPSC grantees to the VPSC Program Office at the Department. Each 

year, the office issues reporting requirements, based in part on the data collection 

suggestions by the national evaluation team. However, the review did not include the 

grantees’ final performance reports, which were to be submitted after the conclusion of the 

national evaluation. 

2.3 Analytic Priorities 

2.3.1 Cross-site Analysis 

The general analytic strategy in dealing with evaluation data was to examine the 

extent of any relationship between a VPSC initiative and concurrent educational outcomes, 

including equity, excellence, and student achievement. 

 The analysis began by focusing on within-site conditions. A separate database, 

consisting of narrative and numeric material, brought together all of the data about a single 

 10 At this site, the district was the appropriate respondent because many of the participating students were 

not assigned to specific schools, either prior to or as part of the VPSC initiative. 
9 

VPSC initiative and addressed such issues as: (1) distinguishing ongoing choice options 

from the VPSC initiative at each site; and (2) calling attention to contextual conditions 

potentially related to the VPSC initiative. The creation of this database was itself a 

qualitative task and aimed at strengthening the possible association between VPSCsupported actions at a site and later outcomes. An important part of this procedure was to 

search for and understand the role of possible rival explanations (Yin, 2000). For instance, 

ongoing district or school policies apart from the VPSC initiative may be strongly 

associated with the observed outcomes. 

 The primary focus of the national evaluation was then to conduct a cross-site analysis 

to arrive at findings for the VPSC Program as a whole, rather than to assess the 

accomplishments at any given site. The cross-site analysis identified patterns across the 

VPSC sites. For instance, findings and lessons from the VPSC Program revealed different 

types of choice initiatives, with the 13 sites arrayed into subgroups based on the design of 

their VPSC-funded choice arrangement (see chapter 3). The subgroups helped provide 

greater insight into the associated outcomes from choice initiatives and also served as 

practical examples to be considered for implementation by other districts in the future. 

 The practical examples were part of an effort that came from an expressed need, by 

the Department, for the national evaluation to report new information about useful practices 

for conducting choice initiatives. From the national evaluation, such information on choice 

initiatives would serve at least two audiences: 1) districts implementing the choice 

provisions of Title I, and 2) districts wanting to start or strengthen their own public school 

choice initiatives independent of the Title I requirements. 

2.3.2 Student-level Analysis 

 The evaluation collected and analyzed student achievement trends at the VPSC sites. 

The evaluation invested extensive efforts in getting sites to collect the needed student 

achievement data. Data cleaning and clarification of the data files submitted by the sites 

also consumed time and resources. Chapter 5 describes the methodological techniques and 

the findings (also see section 3, “Methods for Analyzing Student Achievement Data” in 

appendix A for the detailed analysis plan). 
10 
11 

3. CHARACTERISTICS OF THE VPSC PROGRAM’S SITES 

 The Department awarded grants to three types of organizations in the fall of 2002: 

1) nine local or regional school districts; 2) three state education agencies; and 3) one 

nonprofit, charter school organization.11 Each type of organization engaged in a unique 

choice initiative. Despite this variation, sites pursued some common paths. First, the 

school choice initiatives tended to fall under four major categories based on how sites 

defined choice arrangements and directed the flow of enrolling students. Second, all sites 

focused on two core activities throughout the implementation process: 1) engaging parents 

and community members; and 2) building capacity at schools to attract and accommodate 

choice enrollment. 

3.1 Overview of VPSC Sites 

3.1.1 Sites’ Characteristics 

The 13 VPSC sites were located across the country. Ten were located in 

predominantly urban communities, two in areas that cover both urban and rural regions, 

and one in an entirely rural area (see exhibit 3-1). Ten of the locales represented a total 

population of over 100,000 people each. The public school student populations were 

mostly diverse and of low-income backgrounds. Nonwhite students comprised over 60 

percent of the student population at seven school systems represented by the sites. 

Similarly, over 60 percent of the students were eligible for the Free and Reduced-Price 

Lunch Program12 at seven sites. 

 3.1.2 Brief Descriptions of Sites’ Choice Initiatives 

 The VPSC sites varied greatly in the design of their choice initiatives. The sites 

differed widely in the number of students served, the number of participating public 

schools, and the capacity to accommodate transfers (see exhibit 3-2 for brief descriptions). 

They differed by how they defined choice zones and managed the flow of students among 

participating schools. Furthermore, four sites used VPSC funds to support a preexisting 

 11 The nonprofit organization operated charter schools and qualified as a school district, to be eligible for 

a VPSC award. 12 The Free and Reduced-Price Lunch Program is part of the National School Lunch Program 

administered by the U.S. Department of Agriculture. The program provides nutritionally balanced, low-cost 

or free lunches to more than 26 million students each school day. Children from families with incomes at or 

below 185 percent of the poverty level are eligible for free and reduced-price meals. For the period July 1, 

2004, through June 30, 2005, 185 percent of the poverty level was $34,873 for a family of four (USDA, 

2004). 
12 

or preplanned expansion of districtwide choice. Nearly all of the others used the funds to 

support a specific function within an environment that already had other existing choice 

options. Only two sites had minimal choice options prior to VPSC because they had no 

Title I schools and only a preexisting magnet program. 

Exhibit 3-1 

Community Characteristics 

Characteristic No. of sites

Type of community: 

 Urban 10 

 Rural 1 

 Mixed 2 

Population: 

Under 100,000 3 

 100,000 to 1 million 6 

 Over 1 million 4 

Public school enrollment: 

 No. of students enrolled 

 Under 25,000 4 

 25,000–100,000 4 

 Over 100,000 5 

 Percent nonwhite 

 Under 30 1 

 30–60 5 

 Over 60 7 

 Percent eligible for the Free and 

Reduced-Price Lunch Program 

 Under 30 1 

 30–60 5 

 Over 60 7 

Exhibit Reads: Ten of the VPSC sites are located in urban areas. 

Source: NCES, Common Core of Data, 2004–05. 
13 

Exhibit 3-2 

Brief Descriptions of the VPSC Sites’ Choice Initiatives 

VPSC grantee

Award 

Amt. 

($m) Description of initiative, from 2002–03 to 2005–06

A. PREEXISTING OR PREPLANNED DISTRICTWIDE INITIATIVES 

Chicago Public 

Schools, Ill.

$10.2 Has augmented preexisting districtwide choice by using VPSC funds to support schools in 

neighborhood learning clusters (NLCs) of four-to-six K–8 schools each. Four clusters 

started in 2003–04, three started in 2004–05, and three started in 2005–06. There is a new 

school in each cluster, and existing schools develop magnet themes; clusters have 

coordinators; and schools receive VPSC funds. 

School District 

of Hillsborough 

County, Fla.

$10.2 Has supported two cohorts of students (K–12) starting in 2004–05, enrolling in a districtwide 

controlled choice initiative, involving seven regions (and an urban “zone” within each region) 

and the creation or expansion of “attractor” programs at existing schools to maintain or 

increase student diversity. Plans for the initiative were in place well before VPSC, which 

only partially supports the initiative. 

Minnesota 

Department of 

Education

$11.8 Has supported three cohorts of transfer students in a preexisting program allowing MPS 

students qualifying for the Free and Reduced-Price Lunch Program to transfer to nearly 60 

schools in eight surrounding suburban districts, and suburban or MPS students to attend 12 

schools in MPS (K–12). VPSC funds partially support Parent Information Centers, some 

transportation, and related support services. 

Portland Public 

Schools, Oreg.

$6.5 Has supported three cohorts of transfer students through the augmentation of a districtwide, 

previously available choice program (K–12). VPSC funds help support new enrollment and 

transfer policies; selection, lottery, and transition services; and collaborative curricula 

planning. 

B. INITIATIVES ENHANCING OTHER EXISTING OPTIONS

Brighter 

Choice Charter 

Schools, N.Y.

$3.4 Has supported the first cohort of students by opening three new charter middle schools in 

Albany in 2005–06. The site has continued to support charter school development and to 

coordinate supplemental educational services. 

Desert Sands 

Unified School 

District, Calif.

$7.9 Has supported four cohorts of students transferring from low-performing sending schools to 

higher-performing receiving schools (K–12). Funds help augment curricula at receiving 

schools with environmental studies theme to make them more attractive to transferring 

students. 

Florida 

Department of 

Education

$17.8 Has provided technical assistance to school districts as they develop and implement their 

choice plans. Funds assist a subset of mentor districts (already successful at choice 

options) and mentee districts (needing to expand options), and postsecondary institutions 

(to support school choice information and technical assistance centers). 

Miami-Dade 

County Public 

Schools, Fla.

$11.7 Has supported three cohorts of students (K–12) in two of eight transportation zones in the 

district. Provides funds to create choice programs at under-enrolled schools. The site has 

created or enhanced choice programs, starting in 2003–04 with one new “commuter” school 

in Zone 1; in 2004–05, expanded to a total of nine schools in Zone 1, and four schools in 

Zone 2; and in 2005–06 added one additional choice program in each of the two zones. 

New Haven 

Public Schools, 

Conn.

$9.5 Has supported four cohorts of students transferring from low-performing schools to identified 

higher-performing schools, including Lighthouse Schools, magnets, charters, and suburban 

public schools. VPSC funds expanded programs at four Lighthouse Schools (K–6) in 2002–

03; five Lighthouse Schools in 2003–04; and three in 2004–05 and 2005–06. 

Rockford 

Public School 

District #205, 

Ill. 

$10.1 Has supported four cohorts of students transferring from low-performing schools (K–8) to 

identified receiving schools. VPSC funds provide support to receiving schools; the Parent 

Resource Center; parent and transportation services; and tutoring programs run by 

community or faith-based organizations. 

(Exhibit 3-2, Section C begins on the next page) 
14 

VPSC grantee Award 

Amt. 

($m) Description of initiative, from 2002–03 to 2005–06

C. INITIATIVES CREATING A NEW CHOICE OPTION

Arkansas 

Department of 

Education

$9.3 Has supported four cohorts of students attending an off-campus (residential or 

community) program to receive a rigorous and comprehensive, self-paced education, 

delivered online and aligned with the state’s standards (requires students to take 

state assessment). The program covered K–5 in 2002–03, K–7 in 2003–04, and K–8 

starting in 2004–05. 

Greenburgh 

Central School 

District No. 7, 

N.Y. 

$2.8 Has supported three cohorts of middle and high school students (7–12) attending new 

academies of choice within middle and high school. Has also supported two cohorts 

of elementary students having expanded to the elementary level in 2004–05 with the 

implementation of the International Baccalaureate Primary Years Programme (IB) to 

offer enhanced programming to all K–6 students. 

Monadnock 

Regional School 

District, N.H.

$8.4 Has supported four cohorts of students making interdistrict transfers (9–12), attending 

new programs in their original schools (6–12), transferring to an alternative high 

school (MC2) and a virtual high school, or enrolling in college courses. In 2005–06, 

expanded interdistrict choice to the elementary level and has added new choice 

programs at two high schools and two middle schools. 

Exhibit Reads: Chicago Public Schools received an award of $10.2 million to augment a preexisting 

districtwide choice program. 

Sources: Analysis of site visit data and Grant Performance Reports, by COSMOS Corporation, 2007. 

3.1.3 Participating Schools 

The majority of participating schools were elementary schools. Participating 

schools covered all grade levels: 64 percent spanned the elementary grades, 15 percent the 

middle school grades, 11 percent the high school grades, and 10 percent were classified as 

“other” or had missing data (see exhibit 3-3). As a comparison, approximately 58 percent 

of schools nationwide are elementary schools, 17 percent are middle schools, 19 percent 

are high schools, and 6 percent are “other.” Although most of the sites focused their 

choice initiatives broadly across all grade levels, four VPSC sites only targeted grades 

K–8, which may account for the slightly higher prevalence of participating elementary 

schools. 

3.2 Four Types of Choice Arrangements at the VPSC Sites

The VPSC Program allowed grantees to design choice initiatives to meet their own 

needs. Nevertheless, going beyond the unique circumstances at each site, the initiatives 

fell into a four-fold typology of choice arrangements (see exhibit 3-4). 

(Exhibit 3-2, continued) 
15 

Exhibit 3-3 

Grade Span Distribution of Participating Schools*

Exhibit Reads: 439 (or 63.7 percent) of the schools were elementary schools. 

*Includes 12 of the 13 sites in the VPSC Program because the 13th site focused mainly on technical 

assistance to districts. 

**“Other” is defined by NCES as any grade span configuration, including ungraded, not falling within the 

three categories of elementary, middle, and high. 

***“Not applicable/missing” contains those that either did not have school-level CCD data available or did 

not have data available for the grade span variable. 

Source: NCES, Common Core of Data, 2004–05. 

School level Number Percent 

Elementary school 439 63.7 

Middle school 105 15.2 

High school 73 10.6 

Other** 19 2.8 

Not applicable/missing*** 53 7.7 

TOTAL 689 100.0 

Exhibit 3-4 

Four Types of Choice Arrangements 

Type of choice arrangement 

No. of VPSC 

sites No. of schools*

Predesignated sending or receiving schools 5 95 

Same schools are both sending and receiving schools 5 524 

Within-school options only 1 5 

Mixture of the first three groups 2 6 

TOTAL 13 630 

Exhibit Reads: Five VPSC sites implemented choice arrangements with predesignated sending or 

receiving schools. 

*Based on the 630 School Survey respondents. The survey covered 12 of the 13 sites in the VPSC Program 

because the thirteenth site focused mainly on technical assistance. 

Source: Survey of Schools, COSMOS Corporation, 2006–07. 
16 

 First, five initiatives designated specific schools to be either sending schools or 

receiving schools but not both. In this first type, students attending predesignated sending 

schools were eligible to transfer, and their choices were limited to a predesignated group of 

receiving schools. At these sites, VPSC funds mainly supported the strengthening or 

capacity-enhancement at the receiving schools. 

 Under this first arrangement, three of the five sites defined their sending schools as 

“low-performing” according to the NCLB criteria regarding schools identified for 

improvement. However, because the VPSC legislation did not prescribe a standard for 

identifying “higher-performing” schools, the program and its sites defined the receiving 

schools simply as those that were not “low-performing” schools. During the second year 

of its VPSC initiative at one of these sites, initially eligible receiving schools later became 

identified for improvement. At that point, the site decided these schools would be 

ineligible to serve as receiving schools. 

 The fourth and fifth sites in this first type of arrangement predesignated sending and 

receiving schools but did not attempt to limit either group according to any low- to highperforming criteria. One of these sites started several new receiving (charter) schools 

whose performance could not, by definition, be known at the time when transfers began. 

The last of these sites defined all schools in the system as possible sending schools, and the 

receiving “schools” were predesignated off-campus sites. 

 Second, five initiatives defined the same schools as both “sending” and 

“receiving.” Under a second type of arrangement, five sites permitted transfers between 

all public schools, either districtwide or within pre-specified zones. In the latter case, 

students could choose only among the public schools located within their assigned zone. 

Whether delineated by zone or district, students could attend any school within that area, 

regardless of a school’s prior performance. This type of choice arrangement gave sites 

little or no ability to direct the flow of students from low- to higher-performing schools. 

 At least one site deliberately defined its geographic zones to include both schools 

identified for improvement and higher-performing schools. Thus, in a large countywide 

district, “pie-shaped” zones could have suburban-like, higher-performing schools at their 

perimeter and urban-like, low-performing schools at their center. While the site limited 

transfers to within-zone choices, the initiative nevertheless gave students the opportunity to 

transfer from schools identified for improvement to higher-performing schools (and from 

an urban to a more suburban environment). At the same time, students also could transfer 

in the reverse direction. 

 For both of these first two types of arrangements, the majority of the student bodies in 

all three types of schools (sending-only, receiving-only, or both sending and receiving) 

were nonwhite students and were eligible for the Free and Reduced-Price Lunch Program. 

However, in comparing the two types of arrangements, those with the nondesignated (both 
17 

sending and receiving) schools had lower proportions of low-income and minority students 

than the sending-only schools in the first type of arrangement; in turn, the receiving-only 

schools tended to have the lowest proportion of low-income students and Title I schools.

 Third, one site established a within-school initiative, in which students chose from 

education programs within the same school. In this third type, all students remained at 

their original schools. At the single VPSC site that implemented this third type of 

arrangement, students at their high school chose between two different academic programs 

that had been put into place with VPSC funds. Middle school students had a similar 

choice. At the elementary level, all students had the same academic program, but the 

students chose between two forms of assessment: being graded on a project or taking a 

test. 

 Fourth, two initiatives involved a mixture of the first three types. One VPSC 

initiative had a mix of choices, including education programs within the same school, 

designated receiving schools, and schools that could be both sending and receiving. The 

second site under this last arrangement encompassed 26 school districts in the same state, 

with each district defining its own choice options. VPSC activities provided technical 

assistance to these districts in implementing public school choice. 

3.3 Core Activities in the VPSC Choice Initiatives 

 Despite the unique circumstances of each of the VPSC initiatives, all sites focused on 

two core activities throughout the implementation process: 1) engaging parents and 

community members, and 2) building capacity at schools to attract and accommodate 

choice enrollees. 

 3.3.1 Engaging, Notifying, and Reaching Parents and Community Members 

Sites undertook large-scale efforts to engage, notify, and reach parents and 

community members. From the beginning, the VPSC sites invested in outreach to parents 

and communities to ensure that public school choice initiatives met local needs. Sites 

started the outreach at an early stage. Sites even notified parents and community members 

of the plans for the choice initiatives to provide an opportunity for everyone to express 

preferences for the educational content, student selection criteria, and design of parent 

information centers (see exhibit 3-5). 

As the sites transitioned from the planning to implementation stage, several sites

created ways to keep parents and community members actively engaged. For example, 

sites implemented information centers that sponsored workshops on parenting skills, health 

and nutrition, and computer literacy, as well as public school choice. 
18 

Exhibit 3-5 

Parent Involvement Activities 

Reported by School Officials in School Survey 

Type of involvement Sites’ activities

Establishing the initiative Market research, surveys of parents, community and parent advisors, 

community focus groups and forums 

Planning the initiative Parent advisory committees, principal and teacher input, parent representation 

on lottery and parent information center committees 

Implementing the 

initiative

Parent participation in daily instruction, community support for curriculum and 

materials, parent and student surveys, specialists or counselors, workshops 

on parenting skills, and technical assistance to schools on parental 

involvement 

Community outreach 

activities 

Face-to-face parent and community outreach, media campaigns, direct mailing, 

and parent information centers 

Exhibit Reads: Sites engaged parents in establishing the choice initiatives through market research, 

surveys, advisory teams, and focus groups. 

Sources: Analysis of site visit data and Grant Performance Reports, by COSMOS Corporation, 2007. 

 To notify parents of their choice options and the details of the enrollment process, 

VPSC sites used a variety of outreach activities. Community outreach activities included 

face-to-face meetings, enrollment fairs, open houses at the receiving schools, media 

announcements, and letters to parents (see exhibit 3-6). Some sites communicated with 

parents and community members in multiple languages. Over half of the VPSC Program’s 

sites had brochures and applications available in both English and Spanish, and several 

reported advertising in local Spanish-language media outlets. One site reported that 

running an advertisement on a local Spanish-language radio station was among its most 

successful outreach efforts. Other sites reported printing materials in Chinese, Hmong, 

Lao, Polish, Portuguese, Russian, Somali, and Vietnamese. Public agencies also helped 

the sites to field phone calls from non-English speakers. 

 During 2006–07, a large portion of school administrators at the VPSC sites indicated 

their belief that most or all parents and families were aware of choice options available to 

them (see exhibit 3-7). At the average VPSC site, 70 percent of the administrators reported 

that all or most of parents and families had a good understanding of their choice options.13

 13 Conducting a formal survey of parents’ perspectives was beyond the scope of the evaluation. 

However, parents were informally interviewed during the site visits. On those occasions, parents voiced no 

complaints about the choice options. 
19 

Exhibit 3-6 

Schools’ Reported Efforts to Notify Parents of Their Choice Options, 2006–07*

Response category 

Average percent of 

schools per VPSC 

site**

Individual, face-to-face meetings with school officials 63.7 

Group meetings with school officials 59.4 

Enrollment fairs or similar events 60.5 

Open houses at receiving schools 57.5 

Letter mailed to parents and families 68.1 

Letter sent home with students 57.0 

Announcements in community newspapers and other media 63.6 

Contacts made by district’s parent information center(s) 43.0 

Exhibit Reads: On average, for any VPSC site, 63.7 percent of schools reported having individual, 

face-to-face meetings between parents and school officials related to choice options. 

*The School Survey covered 12 of the 13 sites in the VPSC Program because the 13th site focused mainly on 

technical assistance. 

**The averages represent the number of schools, not the number of parents, involved in these activities (the 

number of parents could not be determined); schools could appear under more than one category. 

Source: Survey of Schools, COSMOS Corporation, 2006–07. 

Exhibit 3-7 

Parents’ and Families’ Understanding of Choice 

Reported by School Officials, 2006–07*

School-reported proportion 

of parents and families 

having a good understanding of their choice options

Average percent of 

schools per VPSC site 

All 25.7 

Most, over 50 percent 44.6 

Some, 20–50 percent 21.6 

Few, less than 20 percent 8.0 

Exhibit Reads: On average, for any VPSC site, 25.7 percent of schools reported that all parents 

and families have a good understanding of their choice options. 

*The School Survey covered 12 of the 13 sites in the VPSC Program because the 13th site mainly focused on 

technical assistance. 

Source: Survey of Schools, COSMOS Corporation, 2006–07. 
20 

 In some cases, sites delayed notifying parents of their choice options until their states 

published the names of schools identified as low-performing. As a consequence, parents 

and students at these sites were notified of their eligibility only a few weeks before they 

had to make a decision on whether to apply to transfer to another school. However, states’ 

deadlines for issuing this information have varied over the VPSC years, so the delayed 

notification was episodic rather than chronic at any given site. 

 Over the course of their grants, sites shifted their outreach strategies and resources. 

One site reduced its districtwide marketing efforts but increased the schools’ involvement 

in the choice notification process. The same site also phased down its parent resource 

centers, previously VPSC-supported. Instead, the site upgraded its Web site to include 

more comprehensive choice information directly accessible to parents. It also encouraged 

schools to provide information directly to parents. 

3.3.2 Capacity-Enhancing Activities 

Sites used VPSC funds to enhance the capacity of schools. The sites participating in 

the VPSC initiatives supported a variety of capacity-enhancing activities, including: 

starting new academic programs; purchasing supplies and equipment; and providing 

professional development to staff. Capacity-enhancing activities included efforts to both 

accommodate and attract transferring students at receiving schools (see exhibit 3-8). In 

addition to improvements to existing schools, sites increased capacity within the system by 

opening new schools. For the most part, the new schools were planned in advance of the 

VPSC initiative, although VPSC funds later helped to start new academic programs at 

these schools. 

 Activities to enhance school-level programming included focusing on professional 

development programs and education programs at all levels. The programs at one site 

included a foreign language program at the elementary level; an arts and academics 

program at the middle school level; and law, geographic information systems, and arts or 

design programs at the high school level. Across the sites, curricular topics covered by the 

capacity-enhancing activities covered academic subjects, such as language arts, science, 

and mathematics as well as research-based comprehensive programs such as the 

International Baccalaureate (IB) program (see exhibit 3-9). 

Although some of the new and enhanced education programs were nationally 

recognized with a record of improving student performance, in general, VPSC sites did not 

rely on scientifically based evidence to guide their selection of education programs. To 

date, the VPSC sites lack systematic documentation of such research for many of their 

capacity-enhancing education programs. 

Despite this approach, educational programming options implemented at the VPSC 

sites have received outside recognition. For example, an innovative science curriculum at 

one VPSC site resulted in new collaboration within a school district. The district utilized 
21 

Exhibit 3-8 

Capacity-Enhancing Activities 

Type of activity* Examples of programs supported 

Starting new schools or reopening 

older schools (4)

Distance learning; charter schools; commuter schools; schools 

with thematic curricula 

Starting new or enhancing existing 

programs at schools (9)

Literacy curriculum; mathematics and science themes; foreign 

language programs; environmental science enhancements 

for all academic subjects; International Baccalaureate 

Implementing tutorial and other 

support programs (6) 

During and after-school tutoring at school or by communitybased organizations; assistance to supplemental service 

providers; hiring of support specialists 

Providing professional development 

to school staffs (12) 

Professional development workshops and summer institutes, 

covering choice, the transfer process, or new or enhanced 

curricula 

Exhibit Reads: Four VPSC sites have started new schools or reopened older schools, including a 

distance learning school; charter schools; commuter schools; and schools with thematic curricula. 

*The number in parentheses represents the number of sites conducting each type of activity. 

Sources: Analysis of site visit data and Grant Performance Reports, COSMOS Corporation, 2007. 

the Web-based science curriculum in its entire set of K–3 classrooms. The VPSC initiative 

provided training and technical assistance in installing equipment and setting up the 

classrooms, as well as ongoing training and assistance to teachers. At another site, a strong 

environmental studies program—developed to attract children from low-performing 

schools to higher-performing schools—received a number of honors. For example, the 

VPSC initiative received the Governor’s Environmental and Economic Leadership Award, 

the state’s highest environmental recognition. 

Other conditions at the schools were important in encouraging enrollment in the 

choice initiatives. Existing academic programs, as well as other conditions at receiving 

schools, were important in encouraging enrollment. Other than VPSC-funded activities, 

administrators indicated that the preexisting academic programs were a major factor in 

students’ decisions to transfer. Similarly, interviewed principals, teachers, and parents 

overwhelmingly responded that the main reason students had decided to transfer to specific 

schools was their preexisting reputations for high performance. One VPSC site’s own 

parent survey confirmed the administrators’ conclusion that parents tended to choose 

schools according to the perceived rigor of academic programs. 
22 

Exhibit 3-9 

Illustrative Curriculum Topics at the VPSC Sites

VPSC 

Site Capacity-enhancing activity Illustrative curriculum topics 

A Individualized, self-paced instruction using Arkansas Virtual 

School (ARVS) curriculum, derived from K–12, Inc.-

developed curriculum, using technology and distance 

learning for virtual learning environment. 

Research-based curriculum: Language Arts/English, 

Math, Science, History, Arts, Music, Physical 

Education/Health 

Innovative K–3 Elementary Science 

B New charter schools. International Baccalaureate (IB) 

Junior Great Books (Language Arts), Connected 

Mathematics; FOSS (Full Option Science System), and 

the A History of U.S. series 

C Cluster schools implementing themes such as science and 

mathematics, fine and performing arts, international 

scholars, literature and writing, and world languages. 

Enhancements to curricula 

Lab improvements 

Clubs 

After-school educational programs 

D Receiving schools augmenting curricula, giving greater 

emphasis to grade-appropriate and technology-rich 

environmental education programs. 

K–12 interdisciplinary Environmental Studies Curriculum 

(enhanced by hands-on, inquiry based learning 

opportunities that integrate technology) 

E Site’s focus is on technical assistance to districts 

F Within-school academies with 250–300 students per academy 

and the International Baccalaureate (including IB Primary 

Years Programme). 

Humanities and International Studies (HAIS) 

Math, Arts, Science, and Technology (MAST) 

Excelsior Humanities Academy 

Woodland Inventors, Technicians, and Scientists (WITS) 

G Attractor programs at schools, designed to encourage urban 

students to select suburban schools. 

Themes offered are fine arts, computer technology, math 

and communication technology, aquatics, sign 

language, culinary arts, and environmental science 

H Schools are selecting academic themes with proven track 

records.

Themes include pre-IB, literacy through the arts, premedicine, Waterford early reading (literacy through 

media use), and college prep 

I Schools are offering supplemental programs to students. GEMS (Girls in Engineering, Math, and Science), ACE 

girls and boys programs, the National Youth Sports 

Program, and the Learning Works program 

J Schools are opening unique programs of study available to 

any student in the area. 

The Academy of Arts and Design HS program 

The Law, Public Safety, and Security (LPSS) HS program 

The GIS (Geographic Information Systems) HS program 

Arts and Academics MS program 

World Language elementary program 

K Schools are developing and implementing themes and 

magnet programs. 

Focus on Literacy programs 

L Expanding higher-performing educational options and 

opening small learning communities. 

Arts and Technology 

Science and Technology 

Young Women’s Academy 

Young Men’s Academy 

Elementary language immersion program 

M Schools are implementing enhanced programming for all 

students. 

Three faith-based after-school programs for tutoring and 

educational experiences 

Exhibit Reads: Site A used the ARVS curriculum as its capacity-enhancing activity. 

Sources: Analysis of site visit data and Grant Performance Reports, COSMOS Corporation, 2007. 
23 

 Students also may have been attracted by schools’ physical facilities or perceived 

safety. The VPSC legislation prohibits the use of program funds for construction, but sites 

could use funds from other sources to make physical improvements. At one VPSC site, a 

newly reopened receiving school had a combination of new academic activities, supported 

by VPSC funds, and a new building and classrooms put into place by the school district. 

At another site, the planned receiving schools were new charter schools with a similar mix 

of funding sources. 

Other participating districts were already in the process of reconfiguring existing 

schools, opening new schools, and renovating schools. These actions did not cover all of 

the receiving schools. Nevertheless, the VPSC sites frequently cited these district 

activities as reasons for not undertaking specific steps to expand seat capacity at the 

schools. Many participating schools were also under-enrolled and could accommodate all 

transfer applicants. Furthermore, seat capacity was generally determined at the schoollevel, providing flexibility to allow for reconfigurations within classrooms to accommodate 

more students. 

 Some of the sites actively examined potential options to expand available seats, but 

few implemented any specific changes. Some of the options the sites explored included: 

hiring more teachers, opening satellite learning centers, expanding online classes, adding 

new wings to existing schools, and building new facilities. One site modified the method 

principals used to assess their schools’ available seat capacity, which resulted in more 

accurate counts, but not necessarily more seats. Another site expanded its VPSC initiative 

to include additional schools, in this manner increasing the total number of seats available 

each year. 

Overall, however, the sites’ capacity-enhancing activities were not necessarily 

accompanied by an expansion of seats or classrooms at many of the receiving schools. 

For instance, none of the sites reported hiring more teaching staff or taking other steps 

simply to expand the number of seats to accommodate higher enrollments at the existing 

schools. 

3.4 Initial Implications for Federal Education Policy 

 and Local Education Practice 

The discussion thus far has treated the VPSC Program like many other federal 

discretionary programs—tracing VPSC-funded activities at each site as if the sites were 

supporting discrete, federally funded projects. However, from the perspective of the VPSC 

Program’s role in informing federal education policymaking and local education practice, 

the reality at each site requires closer attention. First, as with many other school districts 

in the country, nearly all of the 13 sites already had a broad variety of public school 

options prior to the VPSC Program. Second, the VPSC Program permitted sites to 

enhance existing options, and sites did not need to start entirely new initiatives. 
24 

3.4.1 A Variety of Other Choice Initiatives 

New choice initiatives, such as those funded by the VPSC Program, did not occur 

on a blank slate. Public school choice has been available in many school districts for 

decades. Some of the most common choice options have been: magnet schools, 

opportunities to transfer to different schools in a district, state-initiated choice options, 

options for students with at-risk conditions, and, most recently, charter schools. Likewise, 

districts with Title I schools have offered choice options since 1994. Overall, most 

districts by now have a wide selection of choice options. 

 The experiences at the 13 VPSC sites did not differ. All of the 13 sites had existing 

choice options prior to the VPSC Program. Some of the sites had a wide array of such 

options, with some students eligible to participate in two or more options. Although sites 

tried to make their entire array of choice options work well, they did not necessarily track 

students’ participation on an option-by-option basis. For federal policy, the implications 

are clear. The benefits of investing in a particular public school choice initiative may not 

be easily isolated, given the existing array of choices available to students. 

Choice contexts in different states also can vary markedly. Going one step further, 

any expanded federal support for public school choice might more likely be administered 

on a formula basis through state departments of education rather than as direct, 

discretionary awards to local school districts or other local entities. The VPSC Program’s 

experience, with three awards going to state departments of education, potentially preview 

some of the features of such a scenario. In particular, the VPSC experience showed that 

marked differences existed among states’ ongoing choice priorities. They varied from 

using federal funds to: provide technical assistance; meet the settlement terms of 

significant court rulings; or support an innovative, off-campus choice option (see appendix 

C, exhibit C-1, for more detail). 

 The variations among these three states did not just reflect differences in choice 

preferences. The variations reflected considerably different conditions among states. 

Thus, any expansion of federal funding through state allocations may have to anticipate the 

diversity of contexts among states. 

3.4.2 Implications of the VPSC Program’s Main Condition of Award: 

 Sites Could Either Support New Initiatives or Enhance or 

 Expand Existing Ones

Rather than specifying any particular choice model, the VPSC Program gave sites the 

flexibility to define their own procedures. Furthermore, the program permitted sites either 

to start a new choice initiative or to “enhance or expand” an existing one. The flexibility 

was important because sites could craft a new initiative but also work within their existing 

tapestry of options. 
25 

 Acknowledging this complication is necessary for interpreting the findings in the 

remainder of this report. For some of the sites, the VPSC funds were only part of the 

support for the identified initiative. As a result, the contribution of the VPSC Program 

may have been overestimated. At other sites, the VPSC funds not only supported the 

identified initiative but also partially supported other choice options at the same site. As a 

result, the contribution of VPSC funds may have been underestimated. Unfortunately, the 

current analysis was not able to distinguish the extent of these over- or underestimations. 

Sites could use VPSC funds to partially fund preexisting initiatives. Most of the 

VPSC-funded initiatives enhanced existing choice arrangements, complicating any 

attempts to distinguish “new” from “old.” Enhancements meant that these same 

arrangements already had other sources of funding support and technical expertise. For 

example, four of the 13 sites used their VPSC funds to support existing, districtwide choice 

options (see appendix C, exhibit C-2, for more detail). 

Sites used VPSC funds to support different portions of choice initiatives. Most sites 

had a wide array of choice initiatives as part of their choice environment. For instance, 

some sites directed VPSC funds to services specifically supporting choice enrollment 

under a VPSC choice initiative, but the same sites also used VPSC funds to support other 

services related to all of the choice initiatives at the site (see appendix C, exhibit C-3, for 

more detail). 

 This overall pattern—from sites that used VPSC funds as only a partial source of 

support for ongoing districtwide options, to sites that used their funds to support more than 

a single choice initiative—meant that the VPSC Program supported parts of an initiative at 

some sites and more than a single initiative at other sites (see exhibit 3-10). Because 

neither option can be disentangled, the analysis in the remainder of this report assumes that 

the VPSC Program was associated with a single initiative at each site, ignoring the 

situations where additional options were partially funded or subject to overlap [see 

variations (2) and (3) in exhibit 3-10]. 

 The VPSC initiatives overlapped with Title I choice options. The VPSC Program 

started in the same year that federal legislation expanded support for Title I choice options. 

The legislation allowed spending up to 20 percent of Title I funds to support transportation 

costs when students wanted to transfer out of a Title I school identified for improvement. 

Three of the 13 sites defined their VPSC initiatives to coincide or overlap closely with 

their Title I choice options (see appendix C, exhibit C-4, for more detail). However, most 

VPSC sites defined schools participating in their VPSC initiatives as a broader set of 

schools than simply those designated under Title I as “identified for improvement.” The 

VPSC funds also could be used to cover the students’ transportation costs or services 

regardless of a schools’ performance status—i.e., whether identified for improvement, lowperforming, or neither. 
26 

Exhibit 3-10 

Variations in Sites’ Uses of VPSC Funds to Support Choice Options 

Exhibit Reads: Under Variation (1), VPSC supports only a part of a choice option. 

Source: National evaluation team. 

 (1) 

VPSC 

VPSC supports 

only a part of a 

choice option. 

VPSC VPSC 

 (2) 

VPSC supports a single 

option and also parts of 

other options. 

VPSC OTHER

 (3)

VPSC supports an initiative 

that overlaps with other 

choice options (e.g., Title I). 

VPSC 

 (4) 

The VPSC initiative 

is separate from 

other choice 

options. 
27 

4. PROMOTING EDUCATIONAL EQUITY AND EXCELLENCE 

 Over the five years from 2002 to 2007, the VPSC sites enrolled many students in their 

choice initiatives. However, sites made less progress on other program priorities which 

also were relevant for assessing educational equity and excellence. 

4.1 Student Participation at the VPSC Sites, 2002–07: 

 Eligibles, Applicants, and Enrollees 

 The exact number of participants in a choice initiative varied according to the 

definition of “participation.” The evaluation tracked the following three possible 

definitions to measure participation: 

1) Eligible students: all students who could potentially participate in a 

VPSC initiative; 

2) Applicants: the set of eligible students who applied either to attend 

another public school or to participate in an academic program within 

their original school; and 

3) Enrollees: those students who successfully applied and enrolled in a 

school or program as a result of a VPSC initiative. 

The VPSC sites reported data in each category annually. Their reports for 2005–06 

illustrated the wide variations among the categories for each VPSC site (see exhibit 4-1).14 

 All three categories of students can be considered participants in the VPSC Program, 

depending on the desired logic. For instance, the logic favoring a count of eligible students 

as participants, even if not applying to transfer to another school, is that these students 

indeed exercised a choice by deciding to stay at their original school.15 Narrower 

definitions would limit participants to include only applicants, or even further, to include 

only enrollees. To permit readers to use their preferred logic, the national evaluation 

tracked and reported all three groups of students. The data also permit an estimate of 

participation rates, usually defined as the proportion of enrolling to eligible students. 

 In 2005–06 and across 12 of the 13 VPSC sites, 24,921 students enrolled in the 

choice initiatives, reflecting an overall participation rate of 2.8 percent of the students 

eligible to enroll. The amount and rate of participation also appeared to reflect the type of 

 14 See appendix C, exhibit C-5, for data covering all five years, 2002–07. 

15 In 2005–06, the number of eligible students in the VPSC Program was more than 35 times the number of 

enrollees, making eligibility the high end estimate of participation in the program. In some cases (see rows A, F, 

and H in exhibit 4-1), individual sites defined a large pool of students as eligible, even though the initiative could 

only accommodate much smaller numbers of enrollees. 
28 

Exhibit 4-1 

Student Participation in the VPSC’s Initiatives, 2005-06 

Number of students

VPSC Program 

sites 

Eligible Applying Enrolling

A 290,142 214 214 

B 10,452 170 170 

C 4,970 446 446 

D 4,993 114 34 

E 670 NA* 196 

F 320,331 NA* 13,068 

G 46,838 1,620 1,494 

H 134,878 NA* 3,039 

I 27,266 674 674 

J 45,727 5,249 3,783 

K 1,682 1,682 1,682 

L 8,235 NA* 121

M Site focused mainly on technical 

assistance to districts across the state. 

Total number of 

students 

896,184 10,169 24,921

Number of sites 

reporting 

12 8 12 

Exhibit Reads: VPSC site A reported a total of 290,142 eligible students. 

*Site did not track all applicants. 

Sources: Analysis of site visit data and Grant Performance Reports, by COSMOS Corporation, 2007. 

choice arrangement that each site implemented. The five sites that designated specific 

sending and receiving schools showed a lower proportion of enrolling-to-eligible students 

than other arrangement types. In 2005–06, the percent of eligible students who enrolled in 

these initiatives was 0.3 percent. In contrast, at the five sites with geography-based 

arrangements where the same schools were both sending and receiving schools, 3.8 percent 

of eligible students enrolled in 2005–06. (The 100 percent participation rate for the 

“within-school” option is an artifact of the absence of inter-school transfers.) 

Sites did not track applicants as consistently as they tracked other participant groups. 

Of the 12 sites reporting eligible and enrolling students for 2005–06, only eight sites 
29 

tracked and reported the number of applicants for their choice initiatives. One site was 

unable to track applicants due to an error in coding VPSC applications in the district’s 

enrollment database. In addition, in the last two years, several sites reported that 100 

percent of their applicants became enrolled, even though they had reported lower 

participation rates in earlier years. The exact reason for the 100 percent rates is not known, 

but one possibility is that sites may have changed their reporting procedures in later years. 

 Calculating the number of enrollees also depends in part on whether the totals 

include enrollees from prior years. The typical VPSC site tracked and reported the 

number of students enrolling in their initiatives each year by counting only the number of 

students who started enrollment in that year. Such “first-time enrollees” serve as a 

conservative estimate of the overall enrollment in a choice initiative. Another way of 

counting enrollment involves including enrollees from prior years. Thus, the more 

accurate estimate of enrollment in the VPSC Program’s initiatives, for the four years 

ending in 2005–06, fell somewhere between the 24,921 reported in exhibit 4-1 and a 

cumulative total of 49,616 first-time students enrolled in the initiatives since their 

inception in 2002–03 (see appendix C, exhibit C-5). 

 Unfortunately, totaling across all years overestimates the total enrollment, which may 

have declined due to attrition. A few sites made rough estimates of the attrition or 

“dropout” rates over the period of the VPSC Program, suggesting rates that ranged 

between 30 and 50 percent. Several other sites estimated their repeat enrollments and 

conjectured continuation rates from 50 to 95 percent. 

 The rough estimates mean that the enrollment is likely to be far lower than the 

cumulative total of 49,616 students. However, exactly how much lower is difficult to 

determine because, even if the attrition rates were known, it is extremely difficult to 

classify the students who leave or drop out. Some students may have decided to enroll at 

yet newer schools through the choice initiative, while others may have returned to their 

assigned school and may be considered true “drop-outs.” However, many students may 

have left the district entirely, not necessarily having anything to do with their choice 

experiences. An accurate estimate for cumulative enrollment would need to distinguish 

among these alternatives while also tracking those students that continued to attend schools 

of choice through the VPSC Program. 

Overall, the number of enrolling students in the VPSC Program increased during 

the earlier years of the program but declined in the program’s fifth year. Of the 12 sites 

enrolling students, only ten provided eligibility and enrollment data for four consecutive 

years, 2003–04 to 2006–07 (see exhibit 4-2). These data permitted an estimation of trends 

through the fifth VPSC year, 2006–07. The yearly data captured the number of first-time 

(i.e., new) enrollees each year, not the total number that cumulated over all years. For 

these “first-time enrollees,” the trends showed that the VPSC Program at first averaged 

696 enrollees per site in 2003–04, then reached a peak of 2,459 per site in 2005–06, and 

then declined to 2,167 per site in 2006–07. 
30 

 Participation rates also showed the same pattern over time, increasing and then 

declining from 2003–04 to 2006–07. For the same ten sites reporting data from 2003–04 

to 2006–07, the participation rates by first-time enrollees also increased from 1.5 percent to 

4.1 percent, from 2003–04 to 2005–06, and then dropped to 3.0 percent in 2006–07 (see 

exhibit 4-2). 

The decline in total enrollment as well as in participation rates may reflect the actual 

saturation of the VPSC initiatives because a good (but unmeasured) portion of the earlier 

years’ enrollees still remained enrolled in the later years. Their continuing enrollment may 

possibly have limited the seats available for first-time enrollees in the final year. For 

example, an initiative that retains the same receiving schools from year to year will likely 

encounter limited seat capacity after several years of annual first-time enrollees. 

 In addition, the supply and demand conditions in choice arrangements, from year to 

year, may be extremely dynamic, with the two conditions adjusting to each other in 

immeasurable ways. For example, one site discouraged applications in its last two 

reporting years, knowing that the available seats were limited. Shifts in eligibility pools also 

occurred over time. For instance, due to annual shifts in the identity and number of lowperforming schools, the pools of eligible students shifted accordingly at three sites that linked 

Exhibit 4-2 

Participation Rates, 2003–07*

Average number per site:

Enrollment school 

year Eligible Enrolling

Percent of eligible 

students enrolled at the 

ten sites with four years 

of enrollment data 

2003–04 47,165 696 1.5 

2004–05 54,009 1,592 2.9 

2005–06 59,781 2,459 4.1 

 2006–07 72,393 2,167 3.0 

Exhibit Reads: On average, VPSC Program’s choice initiatives had 47,165 eligible participants per 

site in 2003–04. 

*The averages were based on ten sites for which participant data were available for four consecutive years. 

**2006–07 data are preliminary. 

Sources: Analysis of site visit data and Grant Performance Reports, by COSMOS Corporation, 2007. 

Average number per site

**
31 

choice options to school performance. Four other sites expanded choice options to additional 

grade levels, and one site expanded its targeted geographic area and thus increased the number 

of schools covered. A fuller assessment of the participation in choice initiatives, over a 

multiyear period, may require stronger recordkeeping of all three kinds of participation 

(eligibles, applicants,16 and enrollees) than the VPSC sites appear to have implemented. 

 

 Regarding the demographic and academic characteristics of the enrollees, not all of 

the sites kept precise data, nor did they compare the enrollees’ characteristics with those of 

non-enrollees—students who did not participate in the VPSC initiatives. However, four 

sites did submit individual-level student data that were used to analyze student 

achievement trends (see chapter 5). These data included demographic and academic 

characteristics for the year prior to any VPSC participation. The data also included similar 

information from selected groups of non-enrollees (the sites’ selection processes also are 

described in chapter 5). 

 Compared to the non-enrollees, the VPSC enrollees tended to: be slightly more white 

than nonwhite, have slightly lower participation in the Free and Reduced-Price Lunch 

Program, and have slightly better achievement scores in mathematics and reading. 

However, the differences were not statistically significant. Chapter 5 contains all of the 

details about these data and their sources (see exhibit 5-3).17

4.2 Progress on Program Priorities

 The VPSC legislation had four program priorities. Three priorities were: to provide 

the widest variety of choices; to encourage transfers from low- to higher-performing 

schools; and to provide opportunities for students to transfer to schools outside of their 

home districts. The fourth priority directed sites to use some of their VPSC funds to 

support transportation services and costs. These four priorities are discussed next. 

 4.2.1 Widest Variety of Choices 

 The VPSC Program made progress on the first priority of providing the widest 

variety of choice. Sites expanded the assortment of choice options in participating schools 

and offered a large and diverse number of academic programs to transferring students. 

 16 The enormity of the task should not be underestimated. At one site, applicants complete multiple 

applications, one for each school of interest. The site then makes selections among applications, not 

applicants, and the receiving schools act like colleges or universities who may have accepted applicants that 

have also been simultaneously accepted at another school. The desired recordkeeping would not only have to 

distinguish between the number of applicants and the number of applications but would also have to 

determine whether and where the student actually finally enrolled. 17 A similar pattern was found when the enrollees’ characteristics were compared to districtwide 

averages at each of the four sites. The parallel finding was important because non-enrollees were chosen to 

match the enrollees (see chapter 5), whereas the districtwide averages represented all of the students at the 

four sites. 
32 

 Some sites developed more school choice options than others, with as many as 12 

different options available. The VPSC initiatives also made efforts, through media 

campaigns and related activities, to increase parents’ awareness of the variety of education 

options available to them. 

 The sites augmented the quality of choice options by implementing enhanced 

education programming at receiving schools. Several VPSC initiatives broadened the 

choices available to students through the improvement and development of thematic 

programming at participating schools. The programs implemented as part of the sites’ 

VPSC initiatives were new to the district (or a particular area of the district) and often 

distinct from other existing thematic programming.18 These programs focused on unique 

academic themes and were often integrated schoolwide, across all grades and subject areas, 

especially at the elementary and middle schools.19 Other programs, often implemented at 

the high school level, included courses focused on a particular theme that were separate 

from the required curricula. 

 Three sites focused on adding new choice options as part of their VPSC initiatives. 

One statewide initiative implemented a virtual school, offering a completely new type of 

choice option to students, thereby directly increasing the variety of choice options within 

the state. At another site, magnet schools were the only form of school choice prior to 

VPSC. The VPSC initiative, by developing new charter schools, established a second 

choice option for students in the district. Lastly, a third site implemented a new choice 

school type, Lighthouse Schools, through their VPSC initiative, as an addition to the 

district’s other choice options. 

 Two sites extended choice options of urban area students to a wider array of suburban 

schools. Another site permitted rural area students to transfer to a large number of 

adjoining districts, thereby broadening the students’ education options. 

 4.2.2 Transfers from Low- to Higher-Performing Schools 

 Sending schools tended to have higher proportions of minority and low-income 

students than receiving schools. Over 90 percent of the sending schools, on average per 

site, were Title I schools that served low-income students. Also on average per site, 72 

percent of the students in sending schools were nonwhite, and 74 percent were eligible for 

the Free and Reduced-Price Lunch Program (see exhibit 4-3). 

 18 At sites where geographic areas were used to focus VPSC activities, thematic programming was new 

to that area although offered elsewhere in the district, and as a result, more accessible to students in that area. 19 Examples of themes and programs include: math and science; International Baccalaureate program; 

world languages; fine and performing arts; literature and writing; humanities and international studies; 

environmental studies; dual-language; Spanish; communications technology; culinary arts; medical skills; 

hospitality and tourism; Advanced Placement classes; and media industry. 
33 

Exhibit 4-3 

Demographic Characteristics of Schools, 2006–07*

 Average percent per VPSC site 

School characteristic n 

Sending 

schools only 

Receiving 

schools only 

Schools that are 

both sending and 

receiving 

Proportion of students: 

Race/ethnicity: nonwhite 499 71.7 56.9 72.6 

Eligible for Free and ReducedPrice Lunch Program 592 74.2 59.0 57.8 

Title I schools 565 93.1 47.3 75.3 

Exhibit Reads: On average (per site), 71.7 percent of students in sending schools were nonwhite.

*The School Survey covered 12 of the 13 sites in the VPSC Program because the 13th site focused mainly on 

technical assistance. 

Source: Survey of Schools, COSMOS Corporation, 2006–07. 

 The VPSC legislation favored initiatives that promoted the transfer of students from 

low-performing schools to higher-performing schools. The legislation used ESEA Title I 

provisions to define “low-performing” schools as schools not making academic yearly 

progress (AYP) two years in a row. Some of the sites had no Title I schools but used the 

accountability provisions within their states to define “low-performing” schools. In 

contrast, the sites did not specify explicit criteria for defining “higher-performing” schools, 

only that such schools could not be “low-performing” schools. Therefore, designated 

sending schools at these sites were schools identified for improvement and designated 

receiving schools generally had made AYP for two previous years (see exhibit 4-4). 

 Transfers from low- to higher-performing schools comprised only a portion of the 

students enrolled in the VPSC initiatives. Five of the VPSC sites created choice 

arrangements with predesignated sending and receiving schools. Of these, only three 

limited their enrollment to transfers from low- to higher-performing schools in 2005–06 

(see exhibit 4-5, row A). Similarly, although another five sites permitted transfers 

throughout a district or a zone, only two tracked the portion of the transfers from low- to 

higher-performing schools (see exhibit 4-5, row B). 

 Thus, aside from the five sites that either limited their transfers or tracked the 

transfers from low- to higher-performing schools, none of the other eight sites could 

provide such information (see exhibit 4-5, row C). At the five sites, the confirmed 
34 

transfers from low- to higher-performing schools (from rows A and B) represented 1,295 

of 5,927, or 21.8 percent, of their total transfers in 2005–06. 

 The other eight sites either permitted a wider variety of transfers or had VPSC 

enrollments that involved no transfers. Among the sites not tracking transfers, one 

received a waiver from the Department to omit such tracking because, to be eligible to 

apply for transfer, all the students had to be low-performing (scoring “below proficient” on 

the state assessment) as well as of low-income backgrounds (eligible for the Free and 

Reduced-Price Lunch Program). At the same time, the actual transfers at this site were not 

necessarily transferring from low- to higher-performing schools. Thus, across the entire 

VPSC Program, the actual portion of low- to higher-performing transfers could be larger or 

smaller, depending on the nature of the transfers at the seven sites that did not track or 

document the pattern of their transfers (row C).20 

 20 The total number can nevertheless be used to estimate a range within which the current estimate of 

21.8 percent falls: Assuming all of the undocumented transfers were from low- to higher-performing 

schools, the proportion would rise from 21.8 to 81.4 percent; conversely, assuming that none of the 

undocumented were from low- to higher-performing schools, the proportion would dip to 5.2 percent. The 

range would therefore be 5.2 to 81.4 percent. 

Exhibit 4-4 

Reported AYP Status of Schools, 

2004–05 through 2006–07*

 No. (and percent) 

AYP status in the past three years 

(2004–05 through 2006–07) 

Sending 

schools only 

Receiving 

schools only 

Schools that are 

both sending 

and receiving** Total 

Made AYP 9 (21.4) 21 (77.7) 114 (34.8) 144 (36.3)

Failed to make AYP 1 year 22 (52.4) 4 (14.8) 90 (27.4) 116 (29.2)

Failed to make AYP 2 consecutive 

years 9 (21.4) 2 (7.4) 86 (26.2) 97 (24.4)

Failed to make AYP all 3 years 2 (4.8) 0 (0.0) 38 (11.6) 40 (10.1)

TOTAL 42 (100.0) 27 (100.0) 328 (100.0) 397 (100.0)

Exhibit Reads: Nine (21.4 percent) sending schools made AYP for 2004–05 through 2006–07. 

*The School Survey covered 12 of the 13 sites in the VPSC Program because the 13th site focused mainly on 

technical assistance. 

**One VPSC site with a large, districtwide choice arrangement and many low-performing schools accounted 

for more than half of the schools that did not meet their AYP goals. 

Source: Survey of Schools, COSMOS Corporation, 2006–07. 
35 

Exhibit 4-5 

Student Transfers from Low- to Higher-Performing Schools, 2005–06 

Exhibit Reads: The choice implementation at three sites targeted students transferring from low- to 

higher-performing schools. The sites had a total enrollment of 650 students, all of whom were 

transfers from low- to higher-performing schools. 

*Of the 13 VPSC sites, 12 reported enrollment in 2005–06. The 13th site has mainly focused on choicerelated technical assistance. 

Source: Survey of Schools, COSMOS Corporation, 2006–07. 

 

Transfers from low- to higherperforming schools 

Sites’ choice implementation 

No. of 

sites*

Total 

enrollment No. Percent 

A. Only targeted students transferring from 

low- to higher-performing schools 3 650 650 1100 

B. Supported various enrollments, and 

tracked the students transferring from 

low- to higher-performing schools 

2 5,277 645 111 

C. Supported various enrollments, but did not 

track students transferring from low- to 

higher-performing schools 

7 18,994 Sites did not track transfers from 

low- to higher-performing schools

TOTAL 12 24,921 1,295 

 (minimum) unknown

 In some cases, sites delayed notifying parents of their choice options until their states 

published the names of schools identified as low-performing. As a consequence, parents 

and students at these sites were notified of their eligibility only a few weeks before they 

had to make a decision on whether to apply to transfer to another school. However, states’ 

deadlines for issuing this information have varied over the VPSC years, so the delayed 

notification was episodic rather than chronic at any given site. 

 4.2.3 Interdistrict Partnerships 

 Most of the VPSC sites limited their choice initiatives to within-district options, 

rather than developing interdistrict partnerships. Although the implementation of formal 

interdistrict choice options might have expanded the variety of choices available to 

students even further, only five of the 13 VPSC sites created interdistrict options. At the 

eight remaining sites, many had existing transfer understandings with neighboring districts. 

However, these transfer options were separate from the VPSC initiatives and were 

generally reviewed on a case-by-case basis. Even though the existing options did not 
36 

include support for students’ transportation costs, the eight sites nevertheless did not use 

VPSC funds to promote or support interdistrict transfers at these sites. 

 Among the five sites that did offer interdistrict choice options, one site used VPSC 

funds to support the opening of a commuter school. The school provided an intra-district 

choice option to the district’s students. In addition, the school accepted transfer students 

from other nearby districts, thereby providing an interdistrict choice option to students 

applying from other districts. 

 At the other sites with interdistrict choices, the options varied from interdistrict 

magnet schools21 to other formal transfer agreements with neighboring districts. 

Interdistrict initiatives at these sites included: 1) regional interdistrict magnet school 

programs, in which magnet schools accepted students from 26 school districts; and 2) an 

urban-suburban transfer plan, which allowed urban students to enroll in any suburban 

district school and vice versa. Another partnership included one urban school district and 

nine surrounding suburban districts. Students from the urban district applying to transfer 

to any of the suburban school districts were given priority in the application process. This 

interdistrict initiative also allowed suburban students to transfer to the magnet schools 

within the urban district. 

 4.2.4 Support for Transportation Services 

 The fourth VPSC priority required that sites use VPSC funds to support student 

transportation services or costs. This requirement reflected the assumption that students 

would be more willing to travel farther from home and consider schools from a larger area 

than if the students did not have to pay the cost of transportation themselves. 

 Relative to enrollment, transportation costs did not increase proportionately as 

might have been expected. This is partly because the VPSC initiatives permitted many 

students who were already attending distant schools to select schools closer to home. 

Although nine sites reported using funds for transportation, overall these costs did not 

necessarily increase. Many students already were attending distant schools, and the VPSC 

Program now allowed these students to select schools closer to home. Under these 

circumstances, sites experienced minimal or reduced transportation costs. 

 Moreover, some of the VPSC Program’s choice options encouraged students, who 

otherwise might have been contemplating a transfer to a more distant school, to remain at a 

neighborhood school. For example, one VPSC site identified neighborhood schools in 

which enrollment had declined in part because of their perceived poor quality. The site 

designated these under-enrolled schools as “receiving schools” and improved their 

education programs to encourage children to attend. As another example, two of the 

VPSC sites had recently emerged from court-ordered school busing, from which students 

 21 In 2004–05, one site’s interdistrict option ended with the closing of the magnet school. 
37 

had been assigned to more distant schools as part of the original desegregation order. The 

VPSC-funded initiative gave affected students the choice of returning to their 

neighborhood schools. 

 The sites generally used VPSC funds to supplement the district’s transportation 

budget, with three sites also implementing services specifically related to the VPSC 

initiative. Overall, VPSC funds supported new technology and personnel to improve 

transportation services. These improved services included: a Global Positional System, 

mapping software for routing and dispatching school buses, transportation route 

coordinators, and support staff. 

 Two initiatives also supported additional transportation activities beyond transporting 

students to school. These activities included busing students to before- and after-school 

activities, transporting parents to visit choice schools, and driving parents and students to 

VPSC-funded resource centers (e.g., Parent Information Centers). 

 In cases in which sites did not require additional funds for transportation, the sites 

received waivers from the Department to exempt them from the original requirement. One 

of these sites established off-campus receiving schools, which made transportation 

unnecessary for daily attendance. Other sites found it unnecessary to allocate additional 

funding toward transportation as other district funds were sufficient. 

In summary, the VPSC sites’ experiences demonstrated that transportation costs need 

not increase and may even decrease with the implementation of public school choice, 

depending upon preexisting enrollment patterns and the design of the choice initiative 

itself. 

4.3 Implications for Federal Education Policy and Local Education Practice: 

 Further Issues of Educational Equity and Excellence 

The findings in this chapter have directed attention to overall student participation in 

the VPSC Program and the sites’ progress in addressing the VPSC Program’s major 

priorities. However, an examination of “educational equity and excellence” can go beyond 

these issues. 

 One type of additional examination pertains to potential lessons learned about the 

design of the various choice arrangements at the 13 VPSC sites. This is discussed next. 

To meet different equity objectives, different types of choice arrangements may be 

needed. Concerns over equity typically embrace a multi-faceted set of objectives: 
38 

1) Students, especially disadvantaged students (e.g., those of lowincome status, or having low academic achievement), need to have 

greater opportunities to obtain quality education; 

2) Students of all backgrounds should have choices that allow them 

to match their own interests and goals with the variety of settings 

present in school systems; and 

3) Low-performing schools need to improve. 

Any specific choice initiative or arrangement may not be able to pursue all of these 

objectives simultaneously. In fact, the VPSC Program’s experience raises the possibility 

that most arrangements may only be able to focus on one or two of these objectives. 

 A two-by-two matrix can start to represent these three objectives (see exhibit 4-6). 

The matrix also provides a way of assigning ten of the 13 VPSC sites’ initiatives (all but 

three sites could be categorized as having implemented “within-school options” or 

“mixtures” of different options). 

 The two dimensions of the matrix specify whether eligibility to participate in a choice 

arrangement should be broad or restricted, with regard to either the eligibility of the 

students or the eligibility of the schools. These two-by-two combinations result in the four 

cells labeled “(A),” “(B),” “(C),” and “(D).” 

 Cell (A) represents the broadest combination. It includes not only districtwide 

initiatives but also zonal initiatives. Although the designation of zones can have a 

potentially exclusionary effect, the two sites with designated zones did not exclude 

students who might have wanted to enroll within the zones but who came from sending 

schools outside of the zones (see appendix C, exhibit C-6, for more details). As a result, 

these initiatives had a “broad” representation of eligible schools despite their zonal 

character. 

 Cell (B) represents a more limited combination, with restricted sending schools but 

with all students at these schools eligible to participate. However, the eligible sending 

schools do not need to be low-performing (see appendix C, exhibit C-7, for more details). 

 Cell (C) also represents a limited combination. However, it differs from cell (B) 

because it limits the type of students eligible to participate, even though the students may 

be located at a broad array of eligible sending schools. Finally, cell (D) depicts the most 

limited combination: only a certain set of students, from a certain set of sending schools, 

are eligible to participate. 
39 

Exhibit 4-6 

Eligibility to Participate in VPSC’s Choice Arrangements: 

An Equity Perspective*

Eligible sending schools

Broad Restricted 

Broad 

ALL STUDENTS AT ALL SCHOOLS: 

- districtwide initiatives (2) 

- multi-school zones with mostly zonal 

but also districtwide eligibility (2) 

(A) 

ALL STUDENTS AT SELECTED SENDING 

SCHOOLS: 

- students from designated sending 

schools, but not necessarily lowperforming schools (2) 

- students only from low-performing 

sending schools (3) 

(B) 

Eligible students 

Restricted 

SELECTED STUDENTS AT ALL SCHOOLS:

- disadvantaged students coming from 

any school in the district (1) 

 (C) 

SELECTED STUDENTS AT SELECTED 

SCHOOLS: 

- none (0) 

 (D) 

Exhibit Reads: Two districtwide initiatives defined eligibility to participate in a choice arrangement 

broadly by focusing on all students at all schools. 

*The number in parentheses in each cell represents the number of VPSC arrangements falling into a 

particular category. Not shown are the three arrangements that were either “within-school” arrangements 

or a mixture of arrangements. 

Source: Analysis of site visit data, by COSMOS Corporation, 2007. 

 Within all four cells, the distribution of the ten VPSC initiatives shows the limited 

extent to which the initiatives ended up being restrictive, even in their design. Only one 

initiative confined eligibility to disadvantaged students, defined as students eligible for the 

Free and Reduced-Price Lunch Program and with lower academic achievement scores [see 

cell (C)]. Similarly, only three initiatives confined eligibility to low-performing sending 

schools, but within these schools the sites imposed no restrictions on the eligibility of 

participating students [see cell (B)]. The remaining sites all had unconstrained eligibility 

requirements, although the bulk of the students enrolling in these initiatives may still have 

been disadvantaged students. 

 The matrix and the VPSC Program’s experience suggest that, if future policymaking 

and practice are intent on promoting transfers from low- to higher-performing schools, 
40 

choice arrangements may need to be more restricted. For example, the VPSC Program 

might not just have encouraged transfers from low- to higher-performing schools but might 

have limited transfers to those situations. However, limiting choice to the most relevant 

condition—the simultaneous combination of a) low-performing sending schools, and b) 

disadvantaged students [see cell (D)]—may be infeasible from the standpoint of local 

school politics and community cohesiveness. Note that this most restrictive type of choice 

arrangement was not found in the VPSC Program. 

To promote excellence, more guidance for capacity-enhancing activities may be 

needed. No matter what the choice arrangement, an implicit assumption is that choice 

students should enroll at a higher-performing school, if not a school of excellence. To 

boost the quality of educational programs at receiving schools, all of the VPSC sites took 

steps to enhance the capacity of their receiving schools. At some of the receiving schools, 

the capacity-building took place in the name of emulating educational programs already 

found to be successful at other schools in the same district. However, such new adoptions 

did not necessarily assure that the receiving school would become a high-performing 

school. 

 One issue raised here is why the receiving schools should have been changing their 

academic programs in the first place. Experience with voucher programs for public school 

students to attend private schools seems to suggest the opposite. Beyond potentially 

reinforcing the incoming students’ orientation and academic support, the receiving private 

schools continue to adhere to their existing academic programs (e.g., see U.S. Department 

of Education, 2007, and the evaluation of the D.C. Scholarship Program). In fact, many 

private schools have had sufficient success with their academic programs that the schools 

are almost rigid in adhering to their programs, even in the face of external circumstances 

much more dramatic than participating in a voucher program. 

 For the receiving schools in the VPSC initiatives, one would assume that the greater 

challenge was to maximize the number of seats available in already higher-performing or 

excellent schools at the VPSC sites, not to modify in any way the schools’ existing 

academic programs. Yet, in the VPSC Program, every site provided extensive support to 

change academic programs at the receiving schools, and these schools appear to have 

enthusiastically embraced such activity (see appendix C, exhibit C-8, for more details). 

Under these circumstances, the VPSC Program might have offered more guidance on 

the nature of capacity-enhancing activities, including the presumed need to select programs 

supported by scientifically based research. Absent any guidance, some VPSC sites added 

to the academic programs of receiving schools that had already been identified as higherperforming. Other sites started entirely new receiving schools, charter schools, off-campus 

programs, or new magnet-like programs. Yet other sites strengthened programs at underenrolled schools to appear more attractive as receiving schools. 
41 

5. STUDENT ACADEMIC ACHIEVEMENT TRENDS 

CONCURRENT WITH THE VPSC INITIATIVES

 To assess student achievement trends, the analysis in this report traced student 

performance from the period prior to VPSC enrollment through the following years 

afterwards. These trends were then compared to those of a similar group of students who 

had not enrolled in any VPSC choice initiatives. 

 Three cautionary notes must accompany the findings in this chapter. First, any 

observed student performance trends by the enrolling students cannot be readily attributed 

to VPSC initiatives. For example, more motivated students might have chosen to enroll in 

the VPSC initiatives than those who did not, and even though the analysis also includes 

data from comparison groups of non-enrolling students, the VPSC Program was not 

designed as an experiment that might have ruled out such a self-selection bias.22 Thus, the 

observed student performance trends should only be considered as “concurrent” trends. 

These trends have occurred during the period of any given VPSC site’s choice initiative, 

but they have an unknown relationship to the initiative.23

 The second cautionary note derives from the measures of student performance used 

throughout the analyses—scores on state achievement tests. The caveat in using these 

scores is that changed scores from year to year may reflect either changed student 

performance or changed procedures in scoring the states’ criterion-referenced tests. At the 

same time, the use of state assessment scores occupies a central role in major federal 

educational policies and especially the school accountability provisions under NCLB. 

Thus, the use of these scores in the following analysis appears justified, though with the 

appropriate caution. 

 The third caution is that because the usable data were only available from a few of the 

VPSC sites, the aggregate analysis may not represent the VPSC Program as a whole. 

 22 Without random assignment, any number of important but unobserved variables could account for 

differences between two groups. For instance, Cullen, Jacob, and Levitt (2005) studied school choice in the 

Chicago Public Schools from 1993 to 1995. The study found that “the leading explanation” for significant 

differences in graduation rates (students electing to transfer out of their neighborhood schools had higher 

rates) was that the transferring students were “superior along unobservable dimensions” (p. 755). High on 

the list of such dimensions was student motivation and the education background of students’ parents. The 

study concluded that, aside from transfers to career academies (emphasizing vocational skills and integrating 

school with work), “...systematic choice within a public school district does not seem to benefit those who 

participate” (p. 755). 23 To overcome self-selection and other biases, one frequently raised possibility has been to use data 

from sites’ lotteries as if they were “natural experiments” (sites often use lotteries to select participants when 

the number of students exceeds the available choice opportunities). However, the lottery losers may not 

actually serve as the needed counterfactuals for lottery winners (see fuller discussion in appendix A). 
42 

5.1 Collection of Individual-Level Student Data

 All of the 13 VPSC sites were charged, from the beginning of their awards, with 

collecting and reporting records containing individual-level student achievement data. In 

principle, every VPSC site should have submitted individual-level data, coded to protect 

the anonymity of the actual student, that followed a template provided by the national 

evaluation (see exhibit 5-1). The template asked sites to define a student’s enrollment and 

demographic status, along with achievement scores over a multiple-year period. The ideal 

period would have started with the year prior to a student’s enrollment in a VPSC site’s 

initiative and then continued with annual scores for as many years as possible thereafter. 

Exhibit 5-1 

Desired Student Record 

Variables: 2001–02 2002–03 2003–04 Etc.

 1. Coded ID, but associated Consistent Across Years 

 with the same student

 2. Current Grade Level 9 9 9 9

 3. Current School 9 9 9 9

 4. Current District 9 9 9 9

 5. Race/Ethnicity 9 9 9 9

 6. Free and Reduced-Price 

 Lunch Status 9 9 9 9

 7. Gender 9 9 9 9

 8. Achievement Scores 9 9 9 9

 (Math and Reading) 

 9. Testing Grade 9 9 9 9

 10. Name of Assessment 9 9 9 9

Exhibit Reads: A coded student ID consistent across years. 

Source: National evaluation team. 
43 

 The template also asked sites to provide similar data for a non-VPSC group of 

students.24 The sites were asked to identify the comparison groups in either of two ways: 

a) a matched group of non-VPSC enrollees selected to mimic the demographic 

characteristics and baseline academic performance of the VPSC enrollees, or b) the entire 

set of students remaining at the sending schools from which the VPSC enrollees came. 

Regardless of the method chosen by a site, all subsequent analyses took into account any 

differences in the demographic and baseline characteristics of the VPSC and non-VPSC 

students before comparing their student achievement scores. 

 Even if sites did not submit the needed data, they had the discretion to analyze data 

on their own and to report the results in their annual or final reports to the Department. 

Only one of these site-based reports was sufficiently complete to be reviewed as part of the 

national evaluation. The site’s findings are discussed later in this chapter. 

The desired databases could contain one or more cohorts of first-time enrollees and 

multiple years of annual data for each cohort. Because choice enrollments occur every 

year, each site also could have submitted data about more than one cohort of students. For 

instance, a site that had started enrolling students in its VPSC initiative in 2003–04 should 

have provided a baseline score (2002–03) and three additional years of data for the cohort 

enrolling in 2003–04. The site also should have provided another baseline score (2003–

04) and two additional years of data for the cohort enrolling in 2004–05. In addition, some 

sites submitted data covering more than a single year prior to the onset of a cohort’s 

enrollment in the VPSC initiative. 

 In response to the template, and given the possibility of multiple cohorts, sites began 

submitting data in 2005. However, the absence of key information in these submissions 

precluded their use,25 and the national evaluation recommended that the sites be given 

external technical assistance (by another contractor, not the national evaluation team) to 

collect and report the desired data. This assistance occurred during the sites’ fourth and 

fifth award years. 

Usable data came from four of the 13 VPSC sites, covering six of 38 potential 

cohorts of VPSC enrollees and comparison groups of students. By mid-2007, when the 

analysis for the current report needed to start, data from only four sites (and only covering 

six cohorts) contained the essential information. The sample of sites covered was therefore 

smaller than desired. Not only were the four sites a fraction of the 13 total sites, but the six 

 24 However, whether the non-VPSC students later enrolled in a VPSC initiative, or otherwise transferred 

schools in subsequent years, is unknown. Ideally, had the template been followed, the variable “current 

school” would have provided information on a student’s matriculation status in ensuing years. 25 For example, one site only provided two years of achievement scores, which were insufficient to 

calculate the needed trends, as described later in the text; a second site submitted cross-sectional data, only 

providing one year’s scores for each successive cohort of “first-time enrollees;” a third site’s state changed 

its assessment tests in the middle of the trend period, also then limiting the comparable data to only two 

points in time. 
44 

cohorts (of first-time enrollees) were only a small fraction of the total number of 38 such 

cohorts enrolled by the sites collectively, by 2005–06. 

 Nevertheless, even though only six cohorts of data were available for the present 

analysis,26 sites appear to have made progress in collecting data, and this effort should 

continue with the extension of the VPSC Program beyond its initial five-year period. In 

particular, the 14 new awards made by the program in 2007, also for a five-year period, 

include seven sites that had received the original awards in 2002 and that then 

subsequently received external technical assistance. For this reason, the potential for 

analyzing student achievement data has improved, and any continuing evaluation of the 

program can anticipate the availability of a larger and more representative set of data. 

 The six cohorts in the analysis came from four sites whose collective profile was 

similar to that of the rest of the VPSC sites, as follows. As with most of the VPSC sites 

(see exhibit 3-1 earlier), the four all were urban sites with populations in the middle range 

(100,000 to 1 million). Similarly, the four sites mimicked the distribution of all of the 

VPSC sites with regard to being spread across all three enrollment size categories, and 

being nearly evenly split between sites with higher and lower percentages of nonwhite 

students and students eligible for the Free and Reduced-Price Lunch Program. Finally, the 

four sites also evenly represented the two main types of choice arrangements (predesignated sending and receiving schools; and schools that are both sending and 

receiving—see exhibit 3-4 earlier). 

 Across the six cohorts (see exhibit 5-2), the data coverage and the starting years for a 

cohort’s first-year enrollment in a VPSC initiative varied (see exhibit 5-2, columns 2 and 

3). This meant that only three cohorts had two or more annual data points after enrollment, 

but the other three cohorts only had one data point. Moreover, one of these latter three 

cohorts had four data points prior to VPSC enrollment. The desire to retain as much of the 

data submitted by the VPSC sites accounted for this uneven coverage, but with all of the 

trends re-centered around the same “t1” (the year of first enrollment), regardless of the 

chronological year. 

 All of the six cohorts had both enrollee and comparison (non-enrollee) groups. All of 

the sites chose to define their comparison groups by using some type of matching 

procedure. The sites reported these matches in the following manner. 

 For sites whose choice initiatives involved predesignated sending and receiving 

schools, the comparisons were defined as: a) students remaining at the sending schools but 

enrolled there for three years or less, matched for gender, ethnicity, and English Language 

Learner (ELL) status; or b) students remaining at the sending schools, matched on grade 

 26 A fifth site submitted data covering four cohorts, but the submission came well after the analysis was 

underway and could not be included in the analysis. 
45 

Exhibit 5-2 

Individual-level Data Submitted by VPSC Sites 

(1) (2) (3) (4) (5) (6) (7) 

 

Number of records 

used in analysis**

Cohort* 

Years of 

annual 

data 

Year of first 

enrollment 

(and number 

of years of 

data after 

first 

enrollment) 

Total 

enrollees 

reported 

by sites 

(see 

appendix 

C-5) Subjects 

VPSC 

enrollees 

Matched 

comparison 

students†

Math 170 161 A-1 2001–02 to 

2005–06 2005–06 (1) 170 Reading 170 161 

Math 1,534 2,301 B-2 2002–03 to 

2005–06 2004–05 (2) 4,270 Reading 1,526 2,295 

Math 950 914 

B-3 2003–04 to 

2005–06 2005–06 (1) 1,494 Reading 938 913 

Math 89 722 

C-4 1999–00 to 

2005–06 2002–03 (2)*** 501 Reading 90 696 

Math 3,261 2,241 D-5 2002–03 to 

2005–06 2003–04 (3) 3,844 Reading 2,974 2,198 

Math 2,922 1,000 

D-6 2003–04 to 

2005–06 2005–06 (1) 3,783 Reading 2,351 634 

Math 8,926 7,339 TOTAL 14,062 Reading 8,049 6,897 

Exhibit Reads: The first cohort comes from VPSC site “A,” and student achievement test scores 

were available from 2001–02 to 2005–06. 

*Sites enrolled new students in choice options each year, and each year’s enrollees were considered “firsttime” enrollees, whose achievement scores were tracked prior to VPSC enrollment and annually thereafter. 

Thus, in theory, a site operating a VPSC initiative for three consecutive years could have had three cohorts 

of first-time enrollees, with each successive cohort having one less annual data point. 

**Many of the enrollees originally reported by the sites had insufficient test data and could not be included 

in the final analysis. For instance, the enrollees included young elementary students who had not taken 

more than one year of tests at most. Other enrollees from higher grades might not have had three test 

scores because they were not tested each year. 

***This site collected data every other year between 2002 and 2006, so only three years of data were 

collected. 

†The matched comparison students were those not enrolled in the VPSC initiatives but who had similar 

demographic and academic characteristics as the VPSC enrollees (see text for discussion of sites’ matching 

procedures). 

Sources: Student-level databases submitted to the national evaluation by the VPSC sites. 
46 

level, gender, ethnicity, and eligibility for Free and Reduced-Price Lunch (FRPL). For 

sites whose choice initiatives involved districtwide initiatives involving schools that could 

be both sending and receiving, the comparisons were defined as: c) students from the 

same district, matched for gender, ethnicity, and eligibility for the Free and Reduced-Price 

Lunch (FRPL) program; or d) subgroups of comparison students by grade level, each 

subgroup having aggregate characteristics matching as closely as possible the transfer 

groups in gender and ethnicity. 

 As a final note, the analytic preference for multiyear trends (as opposed to using a 

“pre-post” design requiring only two data points) reflected the nature of public school 

choice initiatives. Unlike the use of a new curriculum or classroom technology that might 

take place within a semester and whose “effects” might be anticipated immediately 

following exposure, choice options do not reflect discrete, time-limited “interventions.” 

Choice involves a change in educational pathways or school careers, and any likely impact 

on student achievement may only occur over a period of time. However, given the sites’ 

difficulties in reporting achievement data, three data points were considered the minimum 

number needed to calculate trends. The analysis therefore excluded sites that only 

submitted two years of data.27 Similarly, the paucity of multiyear data (especially 

following the first year of enrollment) precluded any examination of more subtle issues, 

such as whether students might have suffered an initial disruption but then performed 

better, two to three years after changing schools. 

 

5.2 Methodology for Analyzing Student Achievement Trends

 The analysis28 used a meta-analytic strategy to aggregate scores across sites and 

produce findings for the VPSC Program as a whole. The main purpose of the national 

evaluation has been to assess the VPSC Program as a whole. The challenge is to arrive at 

cross-site findings. 

 For the purpose of considering how to aggregate student achievement data across 

sites, one complicating issue is that the program did not specify how sites should 

implement their public school choice initiatives. Instead, sites were encouraged to define 

choice initiatives that suited their own local needs and circumstances. As a result, the 13 

VPSC sites all implemented different public school choice initiatives. Furthermore, most 

of the sites are located in different states; these states use different achievement tests to 

assess student performance. 

 27 The known volatility of achievement scores was an additional numeric reason for requiring three or 

more years of data. Statistically, trends based on two data points assume no measurement error, which is 

unrealistic. 28 The full methodology is found in appendix A, section 3. 
47 

 These conditions precluded any direct aggregation of individual-level student data 

across the VPSC sites, necessary to satisfy the national evaluation’s need to arrive at 

findings for the VPSC Program as a whole. Instead, the site-to-site variations were better 

suited to a meta-analytic strategy. This strategy called first for determining the nature of 

the student achievement trends within each VPSC site, and then using these separate 

findings as part of a meta-analysis to arrive at findings across the VPSC sites.29

 Although the data and other conditions differed from cohort to cohort (and site to 

site), the initial, within-cohort analysis followed the same procedure, ultimately estimating 

a “mean effect size”30 for the changes among the students in each cohort. In other words, 

the analysis first estimated the achievement trends for each individual student. Then the 

analysis combined these into a group trend representing the entire within-site cohort of 

VPSC enrollees. Finally, the trends were compared to similarly derived trends for the 

matched comparison students. 

 The meta-analytic procedure then combined all of the cohort-specific, mean effect 

sizes to estimate a “grand mean” effect size (e.g., Cooper and Hedges, 1994; and Lipsey 

and Wilson, 2001). Such a grand mean effect size represents the aggregate “difference 

between differences” over time for all of the cohorts and sites, thus creating the needed 

program-wide benchmark for the VPSC Program as a whole. 

The main achievement trends were calculated on the basis of “scale” scores and 

“standardized scale” scores. Because the key evaluation question was to determine the 

student achievement trends occurring concurrently with enrollment in a VPSC-funded 

choice initiative, the main measure of a student’s achievement was his or her score on the 

annual state assessments. 

 The specific assessment tests vary from state to state and therefore from VPSC site to 

site (except for those few VPSC sites that were located in the same state). Moreover, the 

assessments are usually criterion-referenced tests (CRTs) rather than norm-referenced

tests (NRTs). Unlike national tests such as the Student Achievement Tests (SATs) 

administered by the College Board or the National Assessment of Educational Progress 

(NAEP), neither the state assessment tests nor their scoring metrics are comparable from 

state to state. Each state usually reports its students’ performance in two ways: 1) by 

giving a “scale” score—the specific numeric score attained on the test by a student; and 2) 

by indicating whether the student’s score exceeded the state’s desired level of proficiency, 

usually reflected by four or five categorical groupings of scores, such as “highly 

proficient,” “proficient,” “below proficient,” and “basic.” 

 29 Investigators have used meta-analyses to examine many topics in education, including: gender 

differences in verbal ability (Hyde and Linn, 1988); the achievement benefits of summer school (Cooper et 

al., 2000); and comprehensive school reform (Borman et al., 2002). 30 Effect sizes are a measure of the magnitude of differences between two conditions—for instance, the 

cross-sectional differences in performance between two groups, or the change over time for one group, or 

both. 
48 

xijk – Xjk

sjk

 

 Because the benchmark scores for achieving the four or five different proficiency 

categories may change from year-to-year in the same state, the present analysis only relied 

on the “scale” scores. However, the numerics of the scale scores (e.g., some being very 

large numbers versus others being small numbers, depending upon the test scoring 

procedure) also led to the need for “standardizing” these scores—converting them to a 

similar scale across tests and sites (see equation 1). 

Equation 1 

 

z-score = 

i = individual student achievement score 

j = year 

k = grade 

X = mean of all students’ scale scores for same year and grade level 

s = standard deviation of all students’ scale scores for same year and grade level 

Any potential biases from “standardizing” scores are not easily known. Therefore, 

preliminary analyses were conducted to examine both the “standardized” and “scale” 

scores, and the analyses determined that the results were consistent. The “standardized” 

scores were then used in the remainder of this report.31 

The initial preparation of the data included taking into account the demographic 

characteristics of the students as well as their baseline performance. Baseline 

demographic and academic differences are well-known influences or at least important 

correlates of student achievement outcomes. Any analysis of such outcomes, especially 

efforts such as the present one to compare the performances of different student groups, 

must take these conditions into account in order to make fair comparisons. As previously 

discussed, the sites already had matched a group of non-enrolling students to each group of 

enrolling students. Nevertheless, to create as close statistical equivalence as possible, the 

demographic and pre-VPSC achievement data were used to adjust the standardized scores 

and later achievement trends in the following manner. 

 The VPSC sites provided only limited demographic data about the students. The data 

on the first condition covered different racial and ethnic groups, but because the definition 

of these groups varied from site to site, the information was condensed into two categories 

“white” or “nonwhite.” The sites’ data also covered a second condition—whether the 

student was eligible to participate in the Free and Reduced-Price Lunch Program (FRPL). 

Educational analysts commonly use FRPL eligibility as a proxy for a student coming from 

 31 The analysis avoided another potential complication that would have arisen if a state changed its 

assessment test during the multiyear period under study. No such changes occurred with the six cohorts 

under study. 
49 

a low-income or poverty-level background. The proxy measure is known to have 

inaccuracies and to mask other important differences in students’ family backgrounds (e.g., 

the education level of the parents), but the measure has been used extensively in research 

on student achievement. Equation (2) shows how these conditions were taken into account 

in adjusting the standardized scale scores: 

Equation 2 

Z(SS) = brace * race + bFRPL * FRPL + єZ(SS)

Z(SS) = standardized scale score 

race = race and ethnicity (white v nonwhite) 

FRPL = economic status indicator (eligibility for Free and Reduced-Price Lunch) 

є = residual 

 Following the adjustment for the demographic conditions, the achievement trends for 

each student were then estimated using a simple linear or growth model. These models 

were then used to incorporate a final step, which was to account for differences in students’ 

baseline achievement scores, as these also can affect later outcomes (see Appendix A). 

Thus, the final analyses were based on growth models of students’ scores that had been 

standardized and that had incorporated both demographic and academic baseline 

conditions. 

 Profiles of these baseline demographic and achievement characteristics, comparing 

the VPSC enrollees with the non-enrolling students, showed differences between the 

groups. As previously discussed in chapter 4, the enrollees tended to have a higher 

percentage of white students, a lower percentage of students eligible for the Free and 

Reduced-Price Lunch Program, and better achievement scores, compared to the nonenrollees. However, none of the differences was statistically significant, either within or 

across the six cohorts (see exhibit 5-3). (For the purpose of comparing the baseline 

achievement levels, the “percent proficient and above” was used for descriptive purposes 

to give an idea of the level of student performance in the two groups, rather than the 

standardized scale scores that were later used in the actual trend analysis.) 

5.3 Findings on Student Achievement Trends

 The final student achievement trends came from six cohorts that included VPSC 

enrollees and matched comparison groups of students, across four VPSC sites. The crosssite analysis first estimated the student achievement trends for these two groups separately 

and then compared the trends between the two groups. 

 These two analyses served two different purposes. The first, estimating trends 

separately, was needed to establish whether either the enrollee or matched comparison 
50 

groups was alone moving in a positive or negative direction. The goal was to determine 

whether the enrollees might have been performing worse, regardless of any relative 

difference between it and the comparison students. The second, estimating trends relative 

to each other, then captured the comparison between the two groups. (The two analyses 

involved two different units of analysis, students in exhibit 5-4 and cohorts in exhibit 5-5, 

and the values should not be compared in these exhibits.) 

Exhibit 5-3 

Baseline* Characteristics of VPSC Enrollees and Matched Comparison Students 

 Percent 

(across six cohorts) 

Baseline characteristics 

VPSC 

enrollees 

Matched 

comparison 

students**

A. Grade span: 

Elementary 45 54 

Middle 35 31 

High 20 15 

B. Race: 

White 56 48 

Nonwhite 44 52 

C. Free and Reduced-Price Lunch 41 49 

D. Proficient and above 

Reading 74 64 

Mathematics 72 61 

Exhibit Reads: Of the VPSC enrollees, 45 percent enrolled in the elementary grades in the 

baseline year. 

*The baseline year is the year prior to a student’s enrollment in a VPSC initiative. 

**Students who were not enrolled in the VPSC initiatives but who had similar demographic and academic 

characteristics as the enrolling students. 

Sources: Student-level databases submitted to the national evaluation by the VPSC sites. 
51 

When the VPSC and non-VPSC trends were examined separately (see exhibit 5-4), 

the VPSC enrollees’ trends were neutral in math and positive in reading but not 

statistically significant. More important, the enrollee group’s scores were not found to be 

declining in any way. In contrast, the non-enrollee group showed a declining trend in math 

proficiency that was statistically significant at the 99 percent confidence level.32 The 

reasons for the decline are unclear. Too little is known about the type of instruction or 

educational opportunities offered to the non-enrollees, either at the sending schools or for 

those sites whose non-enrollees came from different schools throughout a district. Finally, 

the non-enrollees’ trends for reading were positive but not statistically significant. 

Exhibit 5-4 

Performance of VPSC and Matched Comparison Students, Analyzed Separately 

Category 

Mean change 

per year 

in standardized 

scores*

Standard 

deviation

Intercept in 

standardized 

scores 

Standard 

deviation

 

VPSC enrollees: 

-Math proficiency 0.00 0.35 0.04 1.31 

-Reading proficiency 0.01 0.35 0.04 1.29 

 

Matched comparison 

students: 

-Math proficiency -0.02** 0.33 0.04 1.29 

-Reading proficiency 0.01 0.32 -0.04 1.25 

Exhibit Reads: VPSC enrollees’ math proficiency changed an average of 0.00 of a standard score 

in one year. 

*All data are unweighted. Some groups’ data covered three years and others covered four or five. However, 

this small difference was not assumed to create undesirable artifacts as might occur if the groups varied, for 

instance, between three and twenty years. 

**p<.01 

Sources: Student-level databases submitted to the national evaluation by the VPSC sites. 

 32 Confidence levels represent the probability that observed differences do not differ by chance alone but 

represent actual differences between two groups. Statistically, the analyses were based on the use of randomeffects models, which maximize the appropriate use of all of the sites’ data submitted for each of the six 

cohorts. Unlike fixed-effects models, the random-effects models assume that the available data come from 

but a sample of a fuller universe of students and that the sample therefore is likely to contain additional 

variance due to sampling errors (Bryk and Raudenbush, 1992). 
52 

When the VPSC and non-VPSC trends were compared, students enrolling in the 

VPSC initiatives had better student achievement trends than those not enrolling. The 

enrollees surpassed the non-enrollees in the trends for both mathematics and reading, with 

the differences in mathematics being statistically significant at the 99.9 percent confidence 

level and in reading at the 99 percent confidence level (see exhibit 5-5). The comparison 

was based on the meta-analysis of effect sizes across all six of the cohorts, with the 

individual effect sizes for each cohort having already accounted for the demographic and 

baseline differences between the VPSC and non-VPSC groups. Because the units of 

measure were standardized scores, the trends cannot easily be translated into everyday 

educational units, but the effect sizes (.020 and .028 for mathematics and reading 

respectively) appear to be modest.33

 

 33 The values for effect sizes can range from negative infinity (extremely negative difference) to positive 

infinity (extremely positive difference), with most values falling between -4 and 4. However, the values are 

difficult to interpret with respect to their importance. Cohen (1988) offers one interpretation: that values 

from 0 to .30 represent “small” effects, from .30 to .60 “medium” effects, and from .60 to 1.00 “large” 

effects. In education, a large effect may be considered a difference of a grade level in performance (e.g., 

fourth-graders performing at a fifth-grade level). Nevertheless, attempted interpretations of effect size values 

for any given analysis, as with standardized scores more generally, are heavily affected by the specific 

sample sizes, variances, and other parameters in the specific data set being analyzed (e.g., Cooper, 1981; and 

Rubin, 1992), and the present analysis was therefore unable to provide a precise interpretation of the effect 

sizes or standardized scores reported in this chapter. 

Exhibit 5-5 

Aggregate Comparison of VPSC and Matched Comparison Groups, 

Across Cohorts (n=6) 

VPSC and matched comparison 

cohorts compared:

Weighted mean 

gain for VPSC 

cohorts 

95 percent 

confidence 

interval 

z 

score 

-Mathematics 0.020 0.0084 to 0.0311 3.41**

-Reading 0.028 0.0061 to 0.0388 2.70*

Exhibit Reads: The VPSC cohorts outgained the matched comparisons in mathematics by a 

standardized score of 0.020. 

All data are weighted according to each site’s variability (more variability means less weight). 

*p<.01; **p<.001 

Sources: Student-level databases submitted to the national evaluation by the VPSC sites. 
53 

 The completion of this meta-analytic procedure required the data to be subjected to a 

test of heterogeneity, to determine whether excessive cross-cohort differences existed. If 

so, such heterogeneity would raise questions about the confidence or appropriateness of 

trying to use and interpret a single average or mean, to represent the collection of cohorts 

(the present analysis relied on a “grand mean” effect size to summarize the trends across 

cohorts). The results of the heterogeneity test show that the cohort-specific effect sizes 

were in fact too heterogeneous (or different) in reading, and nearly so in mathematics, to 

be adequately characterized by the grand mean as a summary statistic.34 The results of the 

heterogeneity test therefore suggest caution in calling attention to the enrollee and nonenrollee contrast. 

 The heterogeneity results also raised a relevant substantive issue: The high variability 

among the individual effect sizes might have signaled some important differences among 

the actual choice interventions. As noted earlier, the four sites and six cohorts were evenly 

divided among the two main choice arrangements (pre-designated sending, and receiving 

schools and schools that are both sending and receiving). The possible heterogeneity could 

have been a split between these two types of arrangements—one arrangement having 

consistently larger effect sizes than the other. However, inspection of the data did not 

show such a split, neither type of arrangement showing an advantage over the other. 

Similarly, the data were inspected for an outlier effect—that one site or cohort might have 

been disproportionately influencing the grand mean. However, the data showed no such 

outlier effect or any other anomaly of note. 

 The caution stemming from the results of the heterogeneity test needs to be 

accompanied by several other cautions about interpreting the differences in achievement 

trends between the enrollee and non-enrollee groups. The other reasons for exercising 

caution are as follows. 

 First, and most important, in all of the choice initiatives, students can choose to enroll 

or not. As a result, an enrollee group may represent more highly motivated students or 

differ in other unobserved ways from a non-enrollee group, accounting for some or all of 

any subsequent differences in student achievement. In other words, without an 

experimental design, the designated non-enrollee group may not adequately represent the 

desired counterfactual condition for drawing a conclusion about the benefits of choice 

initiatives. 

 A possible clue regarding the differences between the enrollees and non-enrollees in 

the present analysis is provided by the demographic and academic contrasts between the 

two groups, presented earlier in exhibit 5-3. The contrasts showed that the enrollees 

 34 The conventional test of heterogeneity is a Q-test. A significant Q statistic indicates that the 

individual effect sizes from each cohort are sufficiently different to be poorly characterized by a single 

summary statistic, such as the grand mean. In the present analysis, the results were nearly significant (hence 

possibly too heterogeneous) for mathematics (Q = 10.2, 5 df, p<.10) and significant (hence undesirably 

heterogeneous) for reading (Q = 20.56, 5 df, p<.001). 
54 

already were performing better than the non-enrollees and had a lower proportion of lowincome and minority students, though none of these differences had been statistically 

significant. Nevertheless, without restrictions on those eligible to enroll, a choice initiative 

may permit higher qualified students to be among the enrollees. Such a possibility 

deserves careful examination in future research. 

 Second, the trends in the present analysis, while representing a minimum of three data 

points, leaned in the direction of having more of the data points precede rather than follow 

enrollment in the VPSC initiative (see exhibit 5-2, column 3). Confidence about the trends 

concurrent with a VPSC initiative, much less inferences about any effects, would be 

increased if analyses were based on a larger number of data points following enrollment. 

Again, future research should try to garner a larger number of post-enrollment data points 

to reduce the interpretive problems. 

 Third, and more generally, the data only came from a small sample of sites (four of 

13) and from only a small sample of the enrolling cohorts (six of 38 cohorts of enrollees 

across all of the VPSC sites by 2005–06). Data from more sites and more cohorts would 

produce a firmer set of findings about the program as a whole. The ongoing VPSC 

Program should put renewed emphasis on obtaining data from additional sites and cohorts, 

given the new round of five-year awards made by the program in 2007. The possibilities 

of such additional data are especially strong, given that seven of the newly awarded sites 

were continuations from the first round of awards. 

 Finally, any continuing data collection and analysis effort should be accompanied by 

a modified procedure for defining the comparison groups, or non-enrollees. Sites should 

not try to match any particular group of non-enrolling students, as the sites did in the 

current evaluation. Rather, the more desirable procedure would be for sites to provide data 

from a larger but nonselected set of non-enrolling students, such as those students 

remaining in the sending schools or even a districtwide set of students. Analytic 

procedures such as propensity score matching or some similar procedure conducted by the 

evaluation team could then provide a fairer selection of matched comparison groups than 

the sites’ procedures in the current evaluation. 

 Given all the cautions, the findings in this analysis of six cohorts from four VPSC 

sites offer early promise regarding the potential benefits of the VPSC Program. External to 

the present analysis, the local evaluator at a fifth site reported findings for two cohorts of 

enrollees in the fifth site’s choice initiative. The evaluation relied on cross-section 

comparisons only, not a trend analysis. The results were mixed, with the older cohort 

significantly outperforming a matched sample of eligible but non-enrolling students one 

year after enrollment, but the later cohort significantly underperforming its matched 

sample one year after enrollment. These mixed results need to be interpreted with the 

same cautions as those discussed above. 
55 

6. USEFUL CHOICE PRACTICES AND OVERALL LESSONS LEARNED 

ABOUT VOLUNTARY PUBLIC SCHOOL CHOICE 

6.1 Useful Choice Practices for Future Choice Initiatives 

 Key VPSC Program sponsors hoped sites could demonstrate useful choice practices 

for districts across the country, to help them implement the Title I choice requirements of 

the No Child Left Behind Act.

35 The VPSC sites and the national evaluation have 

identified an array of such practices and procedures. 

 VPSC sites initiated choice-related practices potentially useful for other districts. 

The emerging practices covered a range of topics, including how to: design choice 

initiatives; involve parents and families; design capacity-enhancing activities at schools; 

and develop technical assistance procedures. Districts across the country can certainly 

learn from the program’s experiences and adapt specific choice practices to implement in 

their own communities (see exhibit 6-1). 

 As an example of designing choice initiatives, one VPSC site demonstrated how 

intradistrict zones can be structured to include a heterogeneous group of schools. Students 

therefore have wide choice but minimize transportation time because they can stay within a 

zone. 

 Similarly, as an example of a useful strategy for increasing parent notification, one 

VPSC site developed a user-friendly online database. Parents and students logged into this 

database and electronically searched for information about their choice options. Because 

many districts across the country have a large variety of choice options, the amount of 

information available to parents and students may be overwhelming. By organizing this 

information online, the site made it easier for parents and students to quickly learn about 

their options. 

 Another VPSC site distributed hardcopies of materials to notify parents of their 

options. Over the course of implementation, the site converted its materials from a 

program-specific format (e.g., separate brochures for each choice option) to a schoolspecific format (e.g., separate brochures for each school, describing all of the choice 

options available at the school). The latter format allowed parents to collect information 

more easily about the programs offered at these schools, rather than having to search 

through materials about all programs, some of which were not relevant to the parents’ 

interests. 

 35 For instance, see the statement by Sen. Joseph Lieberman (D-Conn.), Congressional Record, 107th 

Congress, No Child Left Behind Act of 2001, Conference Report, U.S. Senate, Dec. 18, 2001, S13413–

S13414. 
56 

Exhibit 6-1 

Potentially Useful Choice Practices from the VPSC Program 

Design of Choice Initiatives

Maximizing Choice Options Within Small Geographic Areas: Defining subdistrict “zones” to include 

the most heterogeneous group of schools possible, so students can have wide choice but still stay within a 

zone and therefore minimize transportation costs. 

Empowering Schools to “Own” a Choice Initiative: Developing a formal process for soliciting 

schools’ participation, to encourage them to take greater ownership over a choice initiative. 

Parent Involvement

Gaining “Input” From Families: Using large-scale surveys, media campaigns, and public forums to 

raise public awareness and participation in the choice initiative. 

Operating Parent Information Centers: 1) Designing and staffing multiple parent information centers 

to serve a large community; and 2) establishing a centralized, one-stop family resource center that handles all 

school- and choice-related procedures. 

Designing User-Friendly Communication About Choice Options: Creating easily searchable 

electronic or hardcopy materials, presenting students and their families with different kinds of choice options. 

Parent and Community Outreach Event: Organizing annual community event, showcasing all 

schools in the district, to publicize school choice options; successfully attracting families to attend by offering 

free transportation, free food, student performances, and translation services. 

Offering Parents Transportation Details to Facilitate School Choice: Allowing parents to 

incorporate precise transportation information into their school selection process by providing parents current 

bus route and stop maps, locations of stops closest to home, and estimates of student’s waiting and bus ride 

time, for each school option. 

Working Outside the System: Creating a “community movement” and mobilizing families to be aware 

of the performance of their schools, leading to the promotion of new choices. 

Capacity Enhancement

Defining an Array of Educational Programs to Promote Student Diversity: Attending to the 

offerings at different schools to assure that the programs attract diverse student populations. 

Developing Formal Community Partnerships: Forming agreements with over 30 local institutions, 

including universities, businesses, and nonprofit agencies, to provide support for new educational programs 

at schools in the choice initiative. 

Expanding Distant Learning at “Off-Campus” Locations: Developing a K–8 curriculum for offcampus, online learning that is aligned with state standards and state assessments. 

Supporting Teacher Training Through a Master’s of Choice Program: Collaborating with a 

university to offer a M.Ed. degree program with a school choice concentration. 
57 

 To further facilitate parents’ decisions, one site provided parents with transportation 

details at the time parents were reviewing their school choices. The district’s 

transportation director attended the district’s magnet school fair and provided bus route 

maps to parents. She worked with the parents in searching the transportation database to 

locate the bus stops closest to their homes. A computerized district map then identified 

potential stops and estimated a student’s bus ride time for each school option. For out-ofdistrict students, the district also provided an “alternate stop” at daycare providers if the 

student planned to ride the bus five days a week. The transportation director interacted 

with parents and responded to their questions. As a result of all this, parents were able to 

incorporate accurate transportation information into their school selection process. 

 Sites also engaged in creative capacity-enhancement efforts. For example, one VPSC 

site developed a broad-based set of partnerships with local businesses, community groups, 

and institutions of higher education (IHEs). These partners helped the district to 

strengthen new education programs. Moreover, the partnership, developed as a result of 

the VPSC Program, successfully garnered additional funds to expand existing educational 

programs further. 

 One of the three state education agencies that received VPSC awards focused its 

initiative on providing technical assistance to districts as they developed and 

implemented their choice plans. This VPSC site established formal partnerships between 

mentor districts and mentee districts. All districts in the state were eligible to apply. The 

initiative organized 13 districts, which already had been implementing choice options, to 

serve as mentors. Each of the mentor districts adopted a mentee district that either had no 

choice plan or was at an early stage of developing a plan. 

 The mentor and mentee districts worked together to develop the mentees’ choice 

plans and to share desired practices. Mentor districts also assisted mentee districts to 

develop effective parent involvement strategies and school choice Web sites. In addition 

to providing technical assistance, the mentor districts also used VPSC funds to expand 

their own choice initiatives. 

 The mentors and the mentees both reported having benefited from the program and 

plan to continue the collaborative relationship beyond the end of the VPSC initiative. One 

complicating factor has been the stability of the mentor-mentee partnerships. Of the 13 

mentor districts, one mentor withdrew from the initiative after participating for the two 

initial years. Another mentor district, despite efforts to recruit a suitable mentee, has been 

without a mentee district for three of the four years of the choice initiative. As of 2006–07, 

12 mentor districts and ten mentee districts were participating, with two mentor districts 

not having a mentee. 

 In addition, the same VPSC site established partnerships with two universities. The 

universities used VPSC funds to support choice centers that provided technical assistance 
58 

to districts statewide. Therefore, the initiative’s activities targeted not only the 

participating mentor and mentee districts, but also all the school districts in the state. 

 Districts requested technical assistance by calling a choice center or by completing an 

online request form at the center’s Web site. The centers’ staff made site visits to all 

mentor and mentee districts to assess the districts’ needs and provide assistance. The 

choice center maintained a variety of online resources, including an online parent survey 

available to any district to copy. The parent survey, as well as many other resources, were 

available in English, Spanish, and Creole. The Web site also provided information about 

choice-related parent resources, educator resources, best practices, online courses in family 

involvement, marketing materials, and professional marketing campaign tips. 

 Besides these useful practices, the VPSC Program produced important lessons about 

public school choice. The final two sections of this report discuss these lessons. 

6.2 Overall Lessons Learned from the National Evaluation 

of the VPSC Program

The VPSC Program demonstrated a variety of public school choice options. 

Though limited to awards at only 13 sites, the VPSC Program showed how four types of 

public school choice arrangements can work in a diverse array of communities across the 

country. 

 Among the legislative priorities, the arrangements added a wider variety of choice 

options, compared to what had previously existed at the sites. The VPSC sites’ 

experiences, as traced throughout the earlier chapters of this report, also have produced 

important insights into the workings of voluntary public school choice. These insights 

include the following: 

• Choice arrangements can range from: a) having predesignated sending 

and receiving schools, to b) more flexible arrangements whereby the 

same schools can both send and receive, and finally to c) choices 

among academic programs that take place entirely within the same 

school. 

• Some sites can have any combination of these arrangements 

simultaneously, including options that meet Title I requirements. Sites 

can therefore have a “tapestry” of choice options, serving many 

students under many conditions. 

• Measuring the extent of student participation in choice initiatives 

depends on whether the focus is on eligible, applying, or enrolling 
59 

students; participation rates will vary depending upon the chosen 

measures. 

• Choice operations do not need to follow common stereotypes. 

- Large metropolitan school districts already may have diverse 

urban-like and suburban-like schools, mitigating the need for 

interdistrict arrangements or transfers; 

- Similarly, students may not need to travel greater distances to 

transfer to more attractive schools; instead, choice initiatives 

can try to make those schools closer to students’ homes more 

attractive. 

• The VPSC sites have developed a variety of procedures potentially 

useful for other local districts implementing choice options. The 

procedures include ways of gaining parent involvement and also ways 

of enhancing the capacity of receiving schools. One VPSC site also 

demonstrated an array of technical assistance practices for helping 

local districts. 

 With regard to assessing student outcomes, sites’ efforts to collect the needed student 

achievement data improved over time, suggesting that the VPSC should continue 

examining student achievement trends as the program enters its second five-year phase. 

This last observation is especially pertinent because seven of the original awardees in 2002 

also were among those receiving new awards in 2007. Although this final report covers 

the first five years of the VPSC Program, its continuation beyond 2007 will provide the 

opportunity for additional monitoring and lessons learned about public school choice. 

The evaluation experience clarified desirable features to strengthen future 

program-level assessments of voluntary public school choice. The lessons learned from 

the VPSC’s cross-site and aggregate program-level36 experiences also help illustrate how 

future evaluations can potentially yield more conclusive findings about a choice program as 

a whole. Such evaluations would encompass the following critical features, also discussed 

throughout this report and in appendix A. 

• Careful recordkeeping and tracking of all participants. Sites should 

keep accurate and aggregate counts of all three types of choice 

participants (eligibles, applicants, and enrollees) and also should 

 36 The lessons pertain to multisite evaluations, like the present one, in which the goal is to evaluate a 

program as a whole, not the workings at individual sites (the latter would still be the responsibility of 

separate project-level evaluators). 
60 

compile individual student-level data for choice enrollees’ 

demographic characteristics and student achievement scores. 

• More annual data points, covering a longer period after initial 

enrollment. Analyses of student achievement trends should be based 

on a minimum number of data points covering the years following 

enrollment in a VPSC initiative. For instance, a minimum of three 

years’ achievement data after enrollment would produce greater 

confidence that trends had been concurrent with the VPSC initiative 

rather than with conditions prior to the initiative. 

• Achievement data covering a larger proportion of sites and cohorts. 

Achievement data from more sites and cohorts would produce a firmer 

set of findings about the program as a whole. The ongoing VPSC 

Program can put renewed emphasis on obtaining such data, given the 

new round of five-year awards made in 2007. The possibilities of 

such additional data are especially strong, given that seven of the 

newly awarded sites were continuations from the first round of 

awards. 

• A modified procedure for defining comparison students. Sites should 

not try to match any particular group of non-enrolling students, as was 

done in the current evaluation. Rather, the more desirable procedure 

would be for sites to provide data from a larger but non-selected set of 

non-enrolling students, such as those students remaining in the 

sending schools or even a districtwide set of students. Analytic 

procedures such as propensity score matching or some similar 

procedure would then be used to select a matched comparison group 

from the non-enrollees. 

• Hypotheses about relationships to site profiles and activities. 

Evaluations need to specify such hypotheses at the outset of a study. 

With sufficient data, such profiles and activities might include queries 

about the relationship between different types of choice arrangements 

and the nature and extent of choice outcomes. 

• Inclusion of all choice options at a site as part of the same evaluation. 

Most sites have a collection or tapestry of options, serving slightly 

different goals. Together, all of the options represent a site’s public 

school choice initiative, and evaluating any single option alone does 

not represent the site’s underlying policy goals (nor do sites 

necessarily maintain records on an option-by-option basis). 

Evaluations should therefore embrace all of the options at a given site 

if possible. 
61 

6.3 Two Challenges for Improving Voluntary Public School Choice 

 Finally, the VPSC Program’s first five years also suggest a number of challenges for 

future improvement. Two of these challenges especially deserve attention by federal 

policymakers and local districts because the challenges deal directly with the central 

themes of educational equity and excellence. 

 

 “Choice,” by its nature, may need to cater to a broad array of potential student and 

family preferences. However, the broader the array, the more difficult for a choice 

initiative to limit participation to a narrow set of (low-performing) students or schools. 

 

 The VPSC sites did not necessarily confront this potential dilemma. As previously 

highlighted in chapter 4 (see exhibit 4-7), some of the sites designed restricted choice 

arrangements (e.g., a small set of predesignated sending and receiving schools). However, 

the restrictions did not simultaneously cover “selected students at selected schools.” In 

contrast, other sites designed “broad” arrangements, either defining “all students at all 

schools” or “selected students at all schools” to be eligible to participate. 

 Recalling that providing choice for all students is an equity objective, the lesson may 

be to place importance on acknowledging the need for a collection of choice options at any 

given site (see appendix C, exhibit C-9, for more details). Rather than encouraging sites to 

focus their attention on specific types of choice initiatives, the aim would be to assist sites 

in considering the most appropriate collection—some options appealing to all students or 

schools and other options limited to selected students at selected schools. Only in this way 

might the entire set of equity objectives be served. 

 Similarly, future evaluations of public school choice might embrace all of the options 

at a given site, rather than focusing on a single initiative. Such a complete picture of the 

entire collection of options would be needed to provide a fair assessment of any equity 

outcomes. Moreover, the broader scope would complement those situations when sites do 

not themselves track participation in their options separately (i.e., option-by-option). 

The education programs within choice arrangements, not the arrangements 

themselves, help set expectations for the desired educational outcomes for students. In a 

certain way, choice arrangements, such as those supported by the VPSC Program, 

represent administrative arrangements. They aim to give students the opportunity to enroll 

in different schools or in different academic programs within a school. However, with the 

call for the VPSC sites to emphasize transfers from low- to higher-performing schools, 

sites were expected to go beyond promoting efficient and fair student choice and mobility. 

The VPSC Program also aimed to improve student performance. 
62 

 Although the VPSC initiatives devoted sufficient attention to needed administrative 

procedures (e.g., notifying parents, soliciting and reviewing applications, and making 

school assignments), a somewhat surprising but consistent pattern among the VPSC sites 

was that they also strengthened education programs at the receiving schools. These 

education programs went well beyond the expected orientation or transitional programs for 

newly incoming transfer students. The programs included thematic programs (e.g., 

environmental studies or the arts), computer laboratories, curriculum enhancements, and 

other programs of study available to any student at the receiving schools. 

 In principle, other than orientation and transitional programs for incoming students, 

the receiving schools in a VPSC choice arrangement should not have needed to install new 

or modified education programs. As with the private schools in the D.C. Scholarship 

Program discussed earlier, the preexisting programs at potential receiving schools already 

should have made them higher-performing schools, if not schools of excellence. 

Therefore, changes would not be expected, and although the original VPSC legislation 

permitted funds to be used for educational programming, the legislation offered no specific 

guidance about such programming. As one possible result, the newly implemented 

programs at the VPSC sites did not necessarily stem from any systematic inquiry into 

parents’ desires (see appendix C, exhibit C-10, for more details), nor did the VPSC sites, as 

noted earlier, follow any externally driven selection process, such as relying on 

scientifically based research to select the programs. 

 More importantly, neither the VPSC Program nor any of the sites attempted to define 

a “higher-performing” school. Most of the sites defined such schools as those which were 

not “low-performing” schools. In other words, sites did not set any benchmarks for 

“higher” performance. Also absent in the VPSC Program has been any discussion of the 

standards for school excellence that should be maintained or monitored during the 

implementation of a choice initiative (see appendix C, exhibit C-11, for more details). 

 These remarks are not intended, in and of themselves, as criticisms of the VPSC 

Program. The point is that the conditions make it difficult to set expectations for the 

student performance outcomes in a choice initiative. Students may be more highly 

motivated and work harder if they have been able to select their school of enrollment, but 

their educational outcomes still will be heavily influenced by the substance of the 

education at the receiving schools and their classrooms. Thus, students’ academic 

outcomes in choice arrangements may still be no better than the quality of the educational 

programs available to the students, regardless of the amount of choice that has occurred. A 

potential remedy is for choice arrangements to receive more guidance and clearer 

definitions and expectations for designating schools as “higher-performing” schools. 

These remedies would benefit future students in public school choice options. 
63 

REFERENCES 

Borman et al., Comprehensive School Reform and Student Achievement: A Meta-Analysis,

CRESPAR/Johns Hopkins University, Baltimore, Md., 2002. 

Bryk, Anthony S., and Stephen W. Raudenbush, Hierarchical Linear Models: 

Applications and Data Analysis Methods, Sage Publications, Newbury Park, Calif., 

1992. 

Cochran, W. G., “Some Methods for Strengthening the Common χ2 Tests,” Biometrics, 

1954, 10:417–451. 

Cohen, J., Statistical Power Analysis in the Behavioral Sciences, Lawrence Erlbaum 

Associates, Inc., Hillsdale, N.J., 1988. 

Cooper, Harris, and Larry V. Hedges (eds.), The Handbook of Research Synthesis, Russell 

Sage Foundation, New York, N.Y., 1994. 

Cooper, Harris M., “On the Significance of Effects and the Effects of Significance,” 

Journal of Personality and Social Psychology, 1981, 41(5):1,013–1,018. 

Cooper, H., et al., “Making the Most of Summer School: A Meta-Analysis and Narrative 

Review,” Monographs of the Society for Research in Child Development, Blackwell, 

Malden, Mass., 2000, 65(1), Serial No. 260. 

Cullen, Julie B., Brian Jacob, and Steven Levitt, “The Impact of School Choice on Student 

Outcomes: An Analysis of the Chicago Public Schools,” Journal of Public 

Economics, 2005, 89:729–760. 

Hyde, J.S., and M.C. Linn, “Gender Differences in Verbal Ability: A Meta-Analysis,” 

Psychological Bulletin, 1988, 104:53–69. 

Lipsey, M. W., and D. B. Wilson, Practical Meta-Analysis, Sage, Thousand Oaks, Calif., 

2001. 

Rubin, D.B., “Meta-Analysis: Literature Synthesis or Effect-Size Surface Estimation?” 

Journal of Educational Statistics, 1992, 17(4):363–374. 

U.S. Department of Agriculture, “National School Lunch Program,” Nutrition Program 

Facts, Washington, D.C., December 2004. 
64 

U.S. Department of Education, Institute of Education Sciences, National Center for 

Education Evaluation and Regional Assistance, Evaluation of the DC Opportunity 

Scholarship Program: Impacts After One Year, by Patrick Wolf, Babette Gutmann, 

Michael Puma, Lou Rizzo, Nada Eissa, and Marsha Silverberg, Author: Washington, 

D.C., 2007. 

U.S. Department of Education, Office of Elementary and Secondary Education, “Public 

School Choice: Draft Non-Regulatory Guidance,” Washington, D.C., February 

2004. 

Yin, Robert K., “Rival Explanations as an Alternative to ‘Reforms As Experiments’,” in 

Leonard Bickman (ed.), Validity and Social Experimentation: Donald Campbell’s 

Legacy, Sage, Thousand Oaks, Calif., 2000, pp. 239–266. 
A-1

Appendix A: 

 

Detailed Evaluation Methodology
A-2

 

Contents 

 Page 

1. Introduction.......................................................................................................A-3 

2. Data Collection .................................................................................................A-5 

Exhibit A-1 Schedule of VPSC Field Visits................................................A-6 

Exhibit A-2 VPSC School Survey: Number of Schools Surveyed and 

 Response Rate......................................................................................A-8 

3. Methods for Analyzing Student Achievement Data .........................................A-9 

3.1 Overview................................................................................................A-9 

3.2 Data Management ..................................................................................A-9 

3.3 Within-site Analysis.............................................................................A-12 

3.4 Between-site Analysis..........................................................................A-14 
A-3

1. INTRODUCTION 

 The three evaluation questions were: 1) What are the characteristics of the VPSC 

Program’s grantees; 2) How and to what extent does the VPSC Program promote 

educational equity and excellence; and 3) What academic achievement is associated with 

the VPSC Program? To address these questions, the national evaluation followed a 

mixed quantitative and qualitative methods research design, with data coming from a 

variety of original and archival sources. 

 The evaluation was therefore a multisite, multiyear program evaluation, aimed at 

evaluating the VPSC Program as a whole. The program covered 13 sites (nine districts, 

three state education agencies, and one nonprofit organization37) that received VPSC 

awards from the U.S. Department of Education (the Department) in the fall of 2002. 

Timing of the initiatives. Some of the sites started enrolling choice students as 

early as January 2003, but other sites carried out planning or other activities and started 

the following year or even later. Because all sites began some type of activity in 2002–

03, the school year 2001–02 was considered the base year prior to the implementation of 

the VPSC initiatives. The evaluation then traced the sites’ activities for the duration of 

their five-year awards, through 2006–07. 

 At the same time, once having started the actual enrollment of students, sites may 

have offered choice options to a new set of students every year. For each cohort of 

students, the “intervention period” therefore varied. At the student level, this variation 

was taken into account by accommodating different base years for each cohort of 

students. A consequence of this staggered pattern was that more years of annual data 

were available for the older cohorts than for the younger ones. 

 Definition of the initiatives being studied. The national evaluation defined the 

initiatives under evaluation as the ones being supported by VPSC funds. The use of 

VPSC funds not only defined the initiatives of interest but also helped focus on specific 

facets of the initiatives. For instance, for some of the 13 sites, the bulk of the funds were 

used to support and enhance parent information centers or outreach and media 

campaigns; yet others provided funds for new educational programs at receiving schools

to increase their capacity and attractiveness to serve transferring students. The use of 

funds also directed the national evaluation’s attention to more specific activities, while 

still attending to the overall VPSC choice initiatives at the 13 sites. 

 

Observations about the use of lotteries as part of an experimental design for 

studying voluntary public school choice. The use of random assignment lotteries, when 

choice initiatives are oversubscribed, poses a challenging situation for education analysts. 

On the one hand, the existence of the lotteries appears to provide a “natural experiment” 

for analysts interested in comparing students participating or not participating in choice 

 37The nonprofit organization operated charter schools and qualified as a school district, to be eligible 

for a VPSC award. 
A-4

options. On the other hand, in order to conduct the analysis and fairly interpret the 

results, a host of obstacles must still be acknowledged and overcome. A tentative 

conclusion is that these barriers outweigh the apparent benefits of using the lotteries as 

“natural experiments.” 

 First, the nonparticipants may have previously invested much effort before entering 

the lottery—such as visiting prospective schools and filling out applications for 

exercising choice transfers—all of which may have raised their expectations that then 

were dissipated by their lottery loss (not unlike applying to college, and then being 

rejected). Analysts have not studied the consequences of losing under these conditions, 

but analysts cannot assume that the losers’ subsequent status as nonparticipants 

represents the desired counterfactual or “no treatment” group. In the choice initiatives’ 

lotteries, those not selected to participate may in fact have been subject to an alternative 

and undesirable treatment: becoming a group of disheartened students having to remain 

at their original schools and thereby performing worse than they might otherwise have 

performed. 

 Second, the lottery losers may not remain nonparticipants after their loss. The 

ongoing evaluation of the D.C. Opportunity Scholarship Program that provides public 

school students with vouchers to attend private schools (U.S. Department of Education, 

2007) has found that 56.6 percent of the lottery losers actually switched schools after 

having lost in the lottery (p. 6). Although only 4 percent actually switched to private 

schools, the rest did switch schools, presumably exercising a choice option of unknown 

sorts—but again undermining the students’ status as a “no treatment” group. Given such 

actual participation rates, the U.S. Department of Education report (2007) has therefore 

had to regard the lottery as an “intent-to-treat,” whereby a lottery win is recognized but as 

an offering of an opportunity (in this case of a sizable monetary scholarship) but not 

necessarily at all synonymous with actual choice participation (the treatment of interest). 

In fact, observing the lottery as an “intent-to-treat” calls further attention to the preferred 

counterfactual for making comparisons to students who exercise a choice option: the 

performance of students who did not even apply for the lottery but who stayed at their 

assigned schools. However, the stayers and movers are voluntarily formed groups, not 

subject to random assignment. 

 Third, the VPSC sites were not organized to conduct or document their lotteries as if 

the sites were conducting research studies. The VPSC sites followed a number of 

procedures that minimized the ability to study their lotteries: a) they required multiple 

applications from the same student (one application per school, as in applying to a 

college or university) and conducted the lottery among applications, not applicants; b) 

they did not employ any follow-up procedure to determine where lottery winners and 

losers may have subsequently enrolled; and c) they neglected to retain lottery records 

because such records lose their administrative value immediately following the 

conclusion of a lottery. 
A-5

2. DATA COLLECTION 

 Overview. The relevant data included: multiple site visits to the VPSC sites; 

surveys covering an average of 50 schools at each VPSC site; and the collection of 

archival data about the participating students and the performance of schools in the 

district or the site as a whole. 

For the site visits, the instruments included field protocols to guide data collection. 

The protocols for the VPSC sites covered implementation at the site, district, and school 

level. The school survey instrument was directed to principals of schools participating in 

the VPSC initiatives and to administrators of participating districts in one of the statewide 

initiatives. 

 The instruments did not cover the collection and analysis of quantitative archival 

data to address the evaluation questions related to student achievement. Instead, 

individual-level student data were compiled and provided to the national evaluation by 

the VPSC sites. 

 The completed national evaluation covered the entire five-year VPSC Program and 

drew from multiple sources of evidence: 

• Annual (2002–03 through 2005–06): 

— Original VPSC grant applications (summer 2002 only); 

— Grant performance reports from each VPSC site; 

— Individual records of participating students and 

 a comparison group; and 

— School-level records of student achievement trends 

 at VPSC sites and comparison sites. 

• Selected Years (2003–04, 2004–05, and 2006–07): 

— Field studies of VPSC sites and comparisons; and 

— Surveys of schools participating in the VPSC Program’s 

 initiatives. 

 

 Site visits. The site visits covered the VPSC project site as well as one sending 

school and one receiving school (or two participating schools at those sites not designated 

as specific sending and receiving schools). The sites chose two schools with a lot of 

choice activity (i.e., having many transferring students), so that the site visit team could 

observe and learn about the VPSC Program’s initiative in action. 

 During all three rounds of field visits, a two-person team conducted the visits at 

each of the VPSC Program sites in the second semester of the school year (see exhibit 

A-1). The field protocols covered interviews with the VPSC’s project director and staff 
A-6

at each site and with other key participants in the choice initiative, as well as the 

collection of documents and archival data in relation to the interviews. 

Exhibit A-1 

Schedule of VPSC Field Visits 

 

2003–04 2004–05 2006–07

1. Arkansas Dept. of Education (Ark.) March 31–April 2 March 21–23 May 8–10 

2. Brighter Choice Charter Schools (N.Y.) May 10–11 April 7–8 Jan. 17–19 

3. Chicago Public Schools (Ill.) Feb. 25–27 Feb. 23–25 Jan. 23–24 

4. Desert Sands Unified School District (Calif.) Jan. 26–27 Jan. 31–Feb. 2 Jan. 30–Feb. 1 

5. Florida Dept. of Education (Fla.) May 19–21 May 2–4 Feb. 27–March 1

6. Greenburgh Central School Dist. No. 7 (N.Y.) March 9–11 April 4–5 March 20–22 

7. Hillsborough County School District (Fla.) Feb. 3–5 Feb. 14–16 May 1–3 

8. Miami-Dade County Public Schools (Fla.) March 17–19 April 27–29 April 18–20 

9. Minnesota Dept. of Education (Minn.) April 21–23 April 11–13 April 24–26 

10. Monadnock Regional School District (N.H.) April 13–15 April 4-6 March 27–29 

11. New Haven Public Schools (Conn.) June 1–3 May 10–11 March 13–15 

12. Portland Public Schools (Oreg.) March 9–11 May 10–12 Feb. 6–8 

13. Rockford School District #205 (Ill.) May 3–5 April 20–22 Jan. 9–11 

Exhibit Reads: The Arkansas Dept. of Education was visited in the 2003–04, 2004–05, and 2006–

07 school years. 

Source: National evaluation team. 

School survey. The school survey gathered data from schools participating in the 

VPSC initiatives,38 mainly corroborating the participation in the VPSC initiatives by the 

specific schools. 

 The school survey was based on a closed-ended questionnaire, directed at the 

schools’ principals. The questionnaire requested information about: 

• Student demographics; 

• School performance; 

 38 Participating schools were those with students who were either eligible for or enrolled in a VPSCfunded initiative. 
A-7

• Choice options available to students; 

• Percentages of students taking part in these options; 

• Methods by which choice information was being shared with parents; 

and 

• Whether staff members had been receiving professional development 

related to school choice. 

 At all but one of the VPSC sites, all of the schools eligible to participate in the choice 

initiative were included in the survey. One statewide initiative was the exception: No 

schools were surveyed. Instead, the districts with five or more students participating in the 

choice initiative were asked to complete the survey at the district level.39 

 The team used the following procedure to obtain a high response rate. At the request of 

the sites, the evaluation team first gave the sites’ project directors the questionnaires to 

distribute to the schools, with a personalized cover letter from the project director requesting 

the schools’ participation. Schools were asked to fax the completed questionnaires directly to 

the evaluation team, or to return the completed questionnaires to the site’s project director. 

The sites sent reminder e-mails to the nonresponding schools approximately one week 

following the submission deadline. If necessary, the sites then made follow-up calls to the 

schools. 

 Subsequently, and following the efforts by the VPSC sites’ staff (and with the 

permission of the site), the evaluation team then made at least two rounds of follow-up calls 

directly to the nonresponding schools, frequently re-faxing the questionnaire to the schools. 

Using this procedure, the team achieved high overall response rates (see exhibit A-2). 

During the final round of administering the survey, the evaluation team distributed the survey 

to 689 schools. Respondents at 630 schools completed and returned the questionnaire, 

resulting in an overall response rate of 91 percent. 

 Performance reports. The data collection also covered the VPSC grantees’ annual 

performance reports submitted to the VPSC Program Office at the Department. Each 

year the office issues reporting requirements, based in part on the data collection 

suggestions by the national evaluation team. After extensive review of grantees’ 

performance reports, the evaluation team suggested that the sites provide more detailed 

descriptions of their VPSC-funded initiatives in subsequent annual reports. 

 For instance, the original reporting requirement had called for sites to report 

expenditures in ten categories: personnel; fringe benefits; travel; equipment; supplies; 

contractual; construction; other; indirect costs; and training stipends. However, these 

categories made it difficult to determine how funds were used in relation to the choice 

activities. For example, staff hired to develop new transportation routes and staff hired to 

 39 At this site, the district was the appropriate respondent because many of the participating students 

were assigned to districts but not to specific schools, either prior to or as part of the VPSC initiative. 
A-8

run a parent information center would both have been included under “personnel.” The 

evaluation team therefore suggested that the annual progress report include additional 

information, and that sites accompany their reports with a list of the sub-items used to 

compile the totals for each category (except for the “fringe” and “indirect cost” 

categories). The team further suggested naming the choice-related function (e.g., 

outreach; school capacity-building; professional development; student transportation; 

etc.) associated with each sub-item, rather than listing the name of the vendor or person 

who incurred the expense. 

Exhibit A-2 

VPSC School Survey: Number of Schools Surveyed and Response Rate 

*During 2003–04, the VPSC Program’s initiatives had not yet scaled-up to include 650 participating 

schools. 

**In 2006–07, the site did not participate in the survey, as the initiative focused on technical assistance 

(TA), not schools. Data related to TA were collected during the site visit. 

Source: National evaluation team. 

2003–04* 2004–05 2006–07

Surveys completed Surveys completed Surveys completed 

Site

No. of 

schools 

in VPSC No.

Rate 

(Percent)

No. of 

schools 

in VPSC No.

Rate 

(Percent)

No. of 

schools 

in VPSC No.

Rate 

(Percent)

A 12 8 67 12 9 75 24 14 58 

B No participating schools 5 5 100 

C 19 19 100 46 46 100 65 59 91 

D 13 13 100 13 13 100 19 19 100 

E No participating schools 100 96 96 Not participating in survey**

F 6 6 100 6 6 100 5 5 100 

G 205 113 55 202 182 90 211 202 96 

H 56 56 100 58 52 90 62 57 92 

I 107 92 86 147 129 88 133 127 96 

J 6 6 100 6 6 100 6 6 100 

K 24 22 92 19 19 100 42 27 64 

L 81 58 72 94 81 86 86 79 92 

M 21 17 81 21 20 95 31 30 97 

TOTAL 550 410 75 724 659 91 689 630 91 
A-9

3. METHODS FOR ANALYZING STUDENT ACHIEVEMENT DATA 

3.1 Overview 

The complete analysis of the VPSC data required three primary steps with multiple 

ancillary procedures within each step. Additionally, there were many decisions made 

about the data and the analyses along the way that may not be completely acknowledged 

within the body of the report. A brief overview of the three steps is now followed by a 

more detailed account. 

 The first step involved the process of merging and recoding the data. Within that 

step, there were decisions pertaining to how certain variables would be recoded and 

treated when values were missing. 

 The second step involved the primary within-site analyses whereby individual 

growth curves were computed for each student who provided sufficient data. Similar to 

the first step, there were many decisions. The decisions within the second step regarded 

the handling of missing data, centering scores at a specific time point, residualizing 

scores based upon available covariates, and estimating effect sizes. 

 The third and final step involved the between-sites analysis. Data were aggregated 

to form composite scores by subject area (i.e., math or reading) and site. Once combined 

within subject and site, the effect size data were compared via a meta-analytic 

framework. Decisions about effect size weighing, outlier detection and handling, and 

data display were relevant at this stage. 

 For the purposes of completeness and clarity, those steps and decisions are detailed 

below. The steps are outlined in a sequential format to guide the reader through the 

intricacies and decisions as they were made in the process. 

3.2 Data Management 

Merge raw data files into single, long format file. The VPSC sites had provided 

their data in a series of Excel spreadsheets. Each individual row in a table represented a 

single student from a single cohort and data for a single year. Thus, to get data from a 

site for a single cohort, data were extracted from the Excel tables and concatenated to 

form a multiple-record or long file format. After combining the data, each student had 

multiple rows in the data set. Each row represented a specific time point during the data 

collection process and contained the available achievement data. Once the long file was 

complete, the student demographics were merged by the student’s coded identification 

number to form a complete data set for each cohort and site. It is important to note that 

data were not combined between sites or cohorts at this time. 
A-10

Recode numerical missing data codes. Once the data were arranged in a suitable 

format, each value was checked to ensure that the range and code were appropriate. 

There were some inter-site differences with respect to missing data coding. For example, 

some sites coded missing values with -9 exclusively while other sites used several 

different codes—each code representing a different type of missing value. Regardless of 

the code used, each missing value code was recoded to a “not available” or “NA” 

specification so that the statistics package would not treat the value as a legitimate 

observed value. 

Calculate key variables for subsequent analyses. Several variables that were 

instrumental in subsequent stages of analysis were computed at this point. One such 

variable was a dummy code that represented whether the student changed schools 

between years regardless of group classification (i.e., enrolled or comparison group). 

This code was necessary to ensure that a fidelity check could be performed if necessary. 

A second variable was computed and held in a separate data file that represented the 

number of repeated observations per student. This variable was instrumental in selecting 

students who might be suitable for the growth curve analysis. The new data file 

contained the student identification number, cohort, and number of times the student had 

complete, valid achievement data. Data from this table formed the basis for the next step 

of student selection. 

 

 Identify students who possess sufficient data from both the enrolled and 

comparison groups. Students were retained for the analysis if data were provided for at 

least three (3) time points. The decision to select only students with at least three time 

points was based upon the fact that the aim of the analysis was to provide both individual 

level trajectories as well as group aggregated trajectories. Fewer than three time points 

would have led to some (in some sites many) students with only sufficient data for 

estimating a gain score. That is, two data points would have been sufficient for a change 

score based upon two observed time points. The intercept and slope for that change score 

would have been estimated without error since the line of best fit (i.e., the estimated 

growth curve trajectory) would be drawn through both points without any residual. 

Students who had only a single data point would not have provided sufficient data for 

anything but a single, point-estimate of performance. Thus, for the purposes of 

completeness and comparability, only students who provided a minimum of three data 

points were retained for subsequent analyses. 

Compute residualized scores from raw, standardized scores provided in the 

original data. Each observed, raw standardized score was residualized across all students 

to account for racial, ethnic, and socioeconomic factors. Specifically, the variables 

“race” (a factor coded according to OMB standards) and “eligibility for the Free and 

Reduced-Price Lunch (FRPL) Program” were used to determine predicted values from a 

general linear model. Those predicted values were subtracted from the observed values 

to form residual scores. The following equation represents the general formulation of 

those residualized scores (єij). 

єij = yij – ŷij
A-11

Σ

skl

where the subscripts i and j represent the student and time point, respectively. The 

predicted value (ŷij) was computed with the following general linear model: 

ŷij = braceracei + bfrplfrplij

Note that the race variable consists of a summation operator because race was coded as a 

polytomous (i.e., multiple categories) discrete variable. The residualized scores by site 

will differ somewhat due to inter-site differences on the demographic characteristics. 

 Transform the residualized scores into z-scores. Once the scores were residualized 

to account for racial and socioeconomic indicators, the residualized scores were further 

transformed into z-scores to allow for comparisons between time and site. The z-score 

transformation was essential because the data provided each year by each site contained 

different standardized achievement scores. For example, some sites provided data for 

some years that ranged from 50 to 200 whereas other years data from the same students 

may range from 200 to 800. Furthermore, grade was relevant to the transformation 

because the achievement tests often differ enough by grade to warrant a different 

transformed score. To eliminate those intra-site differences, the data for each year were 

transformed into z-scores by year and grade. The general approach to this z-score 

transformation used the following standard z-score equation: 

z(єijkl) = ————

where the subscripts k and l represent the year and grade level of the student, 

respectively. It is important to note also that the transformation was done separately for 

each of the two educational domains of math and reading. Therefore, for completeness, 

the equation above may be further subscripted to indicate the achievement domain. For 

clarity and simplicity, that more complicated equation was not included. 

 Clean data and ensure values are complete and available for the within-site 

analysis. The final stage of the data management process involved a thorough check for 

missing or miscoded values. Some sites had so few students at specific grade levels that 

the z-score transformation process resulted in either illogical or missing values. In those 

cases, there was nothing that could be done to retain those observations and therefore 

they were omitted from the remaining analyses. The process of identifying illogical or 

missing values was governed by sample size of grade levels, actual z-score magnitude, 

and missing data codes. In almost all cases where data needed to be discarded, the values 

were confirmed by hand to ensure that there were no computational or data handling 

errors. 

єij – єijkl
A-12

3.3 Within-site Analysis 

 The second step in the overall data analysis plan focused specifically on analyzing 

the final data set that was cleaned and coded in the previous step. After careful 

consideration, there was a decision made about the proper analytic technique for the 

within-site data. That decision was an individual growth curve model. It was reasoned 

that the process of reducing the data from the raw repeated observations into growth 

curve parameters would offer the best opportunity to capture the change process. 

Additionally, the options for computing growth curves include both OLS and ML 

approaches. Both have their advantages but only one seemed to fit best with the goals of 

reducing the repeated measures while maintaining as much variability in the parameters. 

In other words, there was an explicit goal to avoid any model-based truncation of the 

parameters. Those aims led to the selection of an OLS (i.e., individual regression) 

procedure instead of the more widely used ML methods found in software packages such 

as HLM and SAS. The primary difference between an HLM/Mixed model approach is 

that the level-one models allow for fitting of the growth curve and error covariance 

structures. Because the aims of the project were to use the growth curve parameters 

alone, the OLS models delivered what we needed and provided a clear, reduced data 

structure for secondary analysis. The complete process of within-site analysis is detailed 

below. 

Estimate individual growth curve parameter estimation. Each student that had 

sufficient data was analyzed using a simple linear model. The linear model was specified 

for each subject area resulting in two sets of growth curve parameters for each student. 

The linear model equation for the growth curves is as follows: 

yij = blitimeij + boi + єij

where yij is the observed value for student i at time j. The first parameter of interest is 

the bli because that represents the linear growth parameter over time (timeij ). Time was 

centered at the point when the VPSC Program began at each site. Thus, time was 

recoded from a calendar year to integers indicating the point at which the program had 

begun (i.e., “0” denoting the inauguration of the VPSC Program). Interpreting the slope 

parameter, a positive slope did not mean that the student necessarily had higher 

achievement. Because the scores were transformed at each year for each grade level, the 

positive slope indicated that the student increased her relative standing in the grade 

compared to previous years. All the values represented relative achievement 

performance—again this was necessary to counter the problems of non-commensurable 

exam metrics over time and between sites. 

 Two other parameters are worth mentioning: the intercept (b0i) and the residual 

(єij). The intercept represents the student achievement point at which the student began 

the VPSC Program. Higher values indicate that the student began at high values relative 

to peers at baseline. The residual values show the efficiency of the prediction models by 
A-13

s

2

wE

s

2

wC

dwE – dwC

s

2

wE s

2

+ wC

2 

blitimei – boi

2

(SE2 *timei) + (SE2 *timei) 0i li

student i and year j. Both the intercept and residuals are important for subsequent 

analyses and diagnostics. 

Estimate effect size. Once the growth curve parameters were available for each 

student and for each subject area, effect size estimates could be estimated. The effect 

sizes were computed based upon the rate of change in z-score units from the intercept. 

Specifically, the equation for within-site, individual effect sizes was computed with the 

following formula: 

 

dwithin = —————————— 

where bli and b0i are the individual growth curve parameters estimated in the previous 

step, timei is the time student i spent in the VPSC Program since its inception, and 

and are the standard errors of the linear and intercept growth curve parameters. 

These effect sizes are standardized values just like Cohen’s d but they are weighted based 

upon the number of values observed. More repeated measures meant more weight to the 

estimated effect size. Effect sizes were then estimated for random coefficient (often 

referred to as random effects) models. 

 Aggregate effect sizes within site. Once random coefficient effect sizes were 

computed, the next step in the within-site analysis involved combining the effects to 

summarize the overall site effect of VPSC. Individual effect sizes were aggregated 

within site to produce a mean overall effect by subject area. The most typical way of 

aggregating these effects is by taking the measure of central tendency and estimating the 

variance about that measure. In short, the mean represented the average effect. Means, 

however, are adversely affected by outlier values and the effect sizes computed in the 

previous analysis produced a good number of values that fell well outside a believable 

range. Even the weighting process employed above did not protect against outlier 

estimates. Therefore, to treat the outliers, windsorized means were computed for each 

site and for each subject area. The windsorized means were based upon the innermost 99 

percent of the data. Within-site but between-group effect sizes were computed by using 

the standard method of computing Cohen’s d: 

dbetween = ———————— 

where dwE and dwC were the average, windsorized, within-site mean effect sizes 

computed for the enrolled E and comparison C groups, respectively. Additionally, the 

and values were the standard deviations computed from the windsorized, within-site 

0i SE2

SE2

li
A-14

effect sizes for the enrolled and comparison. Once the between-group effect sizes were 

computed, further analyses were performed comparing the sites. 

3.4 Between-site Analysis 

 The final major step in the VPSC analysis consisted of a between-site analysis. A 

meta-analytic approach was used for this step for a variety of reasons given in the main 

report. 

 The final student achievement trends came from six cohorts that included VPSC 

enrollees and matched comparison groups of students, across four VPSC sites. The 

cross-site analysis first estimated the student achievement trends for these two groups 

separately and then compared the trends between the two groups. 

 

 These two analyses served two different purposes. The first, estimating trends 

separately, was needed to establish whether either the enrollee or the matched 

comparison group was alone moving in a positive or negative direction. The goal was to 

determine whether the enrollee group might have been performing worse, regardless of 

any relative difference between it and the comparison group. The second, estimating 

trends relative to each other, captured the comparison between the two groups. Thus, the 

ideal positive result would be a finding that both groups had positive trends but that the 

enrollee group performed better than the matched comparison group. A less ideal 

finding, from a policy standpoint, would be that the enrollees performed better, but that 

both groups had negative trends. This less ideal situation would produce ambivalence in 

deciding, for instance, whether a particular educational intervention was desirable or not. 

 The first, or separate group analysis, involved the following estimates: the average 

linear slopes across sites (labeled as the “mean change”),40 the standard deviations of 

those slopes, the average intercepts across sites, and the standard deviations of the 

intercepts. It is important to stress that these estimates all represented unweighted data 

across the sites. 

 The second, or two-group comparison, involved the following estimates: the 

weighted mean gain of the VPSC enrolled sites relative to the comparison sites and the 

associated confidence intervals and statistics for those values. As a point of emphasis, 

the second analysis differed from the first because weighting was now applied (the 

greater the site’s variability, the lower the weight). 

 The following steps were used to estimate the overall effects. All pooled effect 

sizes were weighted by an inverse variance procedure. For the random effects metaanalytic model, the DerSimonian-Laird estimate was used for weighting the effects prior 

 40 The values represent the average change, per year, in standardized scale scores for each of the 

groups. Some groups’ data covered three years and others covered four or five. However, this small 

difference was not assumed to create undesirable artifacts as might occur if the groups varied, for instance, 

between three and twenty years. 
A-15

Σi Σi

^ 

x

2 – (n – 1)

wi – 

^ 

1 

^ 

wi w ^ 

i

^ wi

^ 

to pooling. The equation below provides an estimate of the heterogeneity variance that 

constitutes the DerSimonian-Laird estimate of weighting pooled effects. 

τ

2

 = ————————————

where wi = Var(Өi)

– 1 which is the inverse of the variance of the measure of interest in 

the i-th cohort and x

2

 = (Өi – μ)

2 with μ = Өi / . These pooled 

effects, therefore, are weighted based upon the inverse of the variance of the sampled 

effects. The weights only become relevant if there is sufficient heterogeneity of effects 

between the sites. In the case of the present analysis, there was sufficient variability to 

warrant the weighting by this method. 

Conduct test of homogeneity. Summarizing disparate findings from sites only 

makes sense if those findings are reasonably homogeneous. That is, the sites must be 

viewed as sampled from a similar population where a measure of central tendency (e.g., a 

mean) suitably characterizes the effects for all sites. Testing for homogeneity in metaanalyses typically involves the Q-test—a test developed by Cochran (1954) to assess the 

average deviation from the mean. The Q-test can be calculated by taking the weighted 

sum of the squared effect size deviations from the mean effect size. Recall that effect 

sizes (d) were computed as the difference between groups for each subject area (see 

earlier effect size equation). The general formula for the Q statistic is: 

Q = wi(Ti – T)

2

where Ti represents the individual effect size estimates d computed in previous steps, T is 

the average effect size estimate computed for all studies, and wi is the fixed-effect 

weighting factor for each study based upon the inverse variance of the study. The term wi

is defined as: 

 wi = 

where is defined as: 

= ———— + ————— 

σ

2

i 

^ 

Σi Σi wi

Σi wi

2

Σ

ne + nc

nenc 2(ne + nc) 

d2

Σi

σ

2 

i 

^ 

σ

2 

i 

^ 
A-16

The statistic Q is chi-squared distributed, with degrees of freedom (df) defined as k

–1 where k is the number of cohorts. Homogeneity is the null hypothesis, so values of Q

that exceed the chi-square critical value are indicative of heterogeneity. It should be 

noted that the Q statistic is underpowered for small meta-analyses (i.e., very few effects 

aggregated) and overpowered for large meta-analyses. Regardless of these limitations, 

all meta-analyses rely on the statistic to assess whether homogeneity can be ruled out. 
B-1 

Appendix B: 

Instruments Used in the 

National Evaluation 
B-2 

Contents 

Page 

 1. Questionnaire for Individual Schools ...................................................... B-3 

 2. Site Visit Face Sheet ................................................................................ B-7 

 3. Interview with VPSC Director and Support Staff.................................... B-9 

 4. Interview with Other Participating Persons ........................................... B-19 

 5. Interview with the Comparison District and Title I Staff ...................... B-21 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-3

1. QUESTIONNAIRE FOR INDIVIDUAL SCHOOLS 

National Evaluation of the Voluntary Public School Choice Program

COVER SHEET 

FOR NATIONAL EVALUATION USE

 ID Number: _________________ 

 Date Received: ______________ 

1 

 

 

Date: 

Respondent:

Title:

Phone:

School Name:

District:

State:

 

 School Survey, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-4

QUESTIONS 

1. School’s Name, Address, and Grade Levels: 

Name: 

Address: 

 Grade Levels (circle lowest and highest): pre-K K 1 2 3 4 5 6 7 8 9 10 11 12 

2. The following data were calculated based on student enrollment from which school semester? 

 (check ONE only): 

for Spring 2006 

 for Fall 2006 

 for Spring 2007 

 Other date (specify the date)

 

 Total No. of Students: __________ (no.) 

American Indian or Alaska Native: (%) 

Asian: (%) 

Black or African American: (%) 

Hispanic: (%) 

Native Hawaiian or other Pac. Isld.: (%) 

White: (%) 

Other: (%) 

Eligible for Free and Reduced Price Lunch:

 

(%) 

With IEP: (%) 

With Limited English Proficiency: (%) 

Migrant: (%) 

3. Does this School receive Title I assistance? ___Yes (schoolwide) ___Yes (targeted) ___No

 If yes, has the school been identified as failing to make Adequate Yearly Progress, based on 

student achievement scores for the: 

for the 2006-2007 school year 

 for the 2005-2006 school year

 for the 2004-2005 school year

none of the above 

School Survey, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 

2 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-5

School Survey, 2006-07

4. School Choice Options for Students at this School: 

 (check ALL that apply): 

a. students within the district may transfer to this school 

b. students outside the district may transfer to this school 

c. 

 

students may transfer from this school to other schools within the district 

d. students may transfer from this school to other schools outside the district 

e. 

 

Other – e.g., open enrollment (please 

explain): 

 

f. there are no school choice options (If you answered “f”, please stop; do not 

respond to the remaining survey items.) 

(if you checked items a, b, c, or d): 

 For 2006-07, about how many students have transferred ? 

 g. (number of students transferring to this school) 

 h. (number of students transferring from this school) 

 

 

i. Did teachers receive any extra staff or professional development, 

 in relation to the transfer process? ___Yes ___No

 j. If yes, what were the main topics of the staff or professional development? 

 (topic 1) 

 (topic 2) 

5. Notifying Parents about School Choice Options: 

 What actions did the school take to notify parents/families of their choice options? 

 (check ALL that apply) 

a. individual, face-to-face meetings with school officials 

b. group meetings with school officials 

c. enrollment fairs or similar events for parents to learn about choice options 

d. open houses at receiving schools 

e. letter mailed to parents/families 

f. letter sent home with students 

g. announcements in community newspapers or other media 

h. contacts made by the district’s parent information center(s) 

i. other (please explain): 

j. How many languages, other than English, have been used in these notification procedures? 

 ___ (no.) 3

COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-6

6. In your opinion, what proportion of the parents/families had a good understanding of their choice 

options last year? (check ONE only) 

a. all parents/families 

b. most parents/families (e.g., over 50 percent) 

c. some parents/families (e.g., between 20-50 percent) 

d. few parents/families (e.g., less than 20 percent) 

If you checked 6b, 6c, or 6d, what is the most important thing you can 

recommend, to 

improve parents/families’ understanding of their choice options? 

7. Has your school started new programs (e.g., magnets, academies, small learning communities, new 

academic subjects) to be more attractive, either to reduce the number of students transferring out or to 

increase the number transferring in? (check ALL that apply) 

a. becoming a charter school 

b. starting new magnets, academies, or small learning communities 

c. starting other new academic programs or subjects 

d. making other changes in school administration (e.g., changing school hours) 

e. other 

f. no new programs 

Briefly describe the new programs and the main changes in school operation and 

administration. 

4 

School Survey, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-7

2. SITE VISIT FACE SHEET

(Data collection: 2006–07) 

Face Sheet To Be Completed for 

Each VPSC Site at the Time of the Site Visit 

Date of Site Visit: 

Site Visit Team: 

Name of Site: 

Lead Organization 

and 

Dept. for VPSC 

Initiative: 

Partnering 

Organizations: 

Type of Jurisdiction 

(e.g., rural, suburban, 

urban): 

Student Population: 

Ethnicity: 

 White African 

American 

 Hispanic Asian Other 

Site Visit Face Sheet, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-8
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-9

3. INTERVIEW WITH VPSC DIRECTOR AND SUPPORT STAFF 

(Data collection: 2006-07) 

1. Definition of the VPSC Initiative

 1.1 What activities or staff are being supported with VPSC funds? [Use the revised proposal budget as 

a starting point, but be prepared for changes to have occurred.] 

 1.2 Is it reasonable to define the initiative according to how the funds are being spent? If not, provide 

some other rationale for defining the initiative. [Also check the revised proposal budget.] 

 1.3 Describe the VPSC initiative in 2-3 short sentences. 

 1.4 How participatory was the planning process that was used in developing the VPSC initiative? 

[Probe mainly for parent and school participation, and describe the nature of the participation.] 

 1.5 What were the main problems encountered in planning (not implementing) the VPSC initiative? 

 1.6 What have been the main problems encountered in implementing the VPSC initiative? 

2. Identity of the Schools Involved in the VPSC Initiative

 2.1 Define the breadth of the initiative across its participating schools: 

2.1.1 Academic years covered by implementation (not 

including planning year): 

 

 2.1.2 No. and grade level of sending schools: 

2.1.3 No. and grade level of receiving schools or sites, 

by district: [Also see Qs. 7.1 and 8.1 below.] 

 

2.1.4 No. and grade level of remaining schools, by 

district: 

Within this number, no. and grade level of 

schools that might become sending schools in 

the near future: 

 [Define criteria used.] 

Interview with the VPSC Project Director and Support Staff, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-10

2.2 Describe any “zone” pattern and how zones were defined. [If possible, obtain a map of the schools 

and of the zone pattern.] 

 2.3 How were the sending and receiving schools identified to be part of the initiative? [If related to 

school performance, obtain documentation of such performance.] 

 2.4 Were any logically eligible schools “exempted” from becoming sending or receiving schools? [If 

YES, please describe.] 

 2.5 What are the grades for which choice options can be exercised (e.g., the entering grade for each 

school, but not the other grades)? 

 

 2.6 About how many students are eligible to transfer, due to the VPSC initiative, by grade level (and 

school, if possible)? 

 2.7 About how many seats will be created at receiving schools, due to the VPSC initiative, by grade 

level (and school or site, if possible)? [Also see responses to Qs. 7.1 and 8.1 below.] 

 2.8 How did the district estimate the number of seats available at receiving schools? [Also comment on 

the quality of the procedures and data used by the district.] 

 2.9 What is the schedule of tuition transfers, if any, that accompanied the student transfer? 

3. Student Assignment Criteria and Procedures

 3.1 What criteria have been used to define students eligible for transfer, and what source of data is used 

to review whether students are eligible? 

 3.2 How and when do the eligible students indicate whether they want to transfer, and how many 

potential receiving schools may they identify? [Obtain an application form and review a sample of 

the completed forms.] 

 3.2.1 Do they have to indicate, explicitly, that they 

wish to stay at their present school? 

 

 3.2.2 How does the date when the VPSC choice 

applications are due coincide with the 

application dates for other choice options—

e.g., to charters, magnets, or any other special 

schools? 

 

 3.2.3 Identify the location(s) where the applications 

must be submitted and the rationale for 

selecting the location(s): 

 

 Interview with the VPSC Project Director and Support Staff, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-11

3.3 What criteria are used in deciding to approve a transfer? [Confirm the criteria through 

documentation and discussions with others, including parents, if possible.] 

Check whether the following student assignment criteria 

are used; if YES, state the criterion:

Present Criterion

3.3.1 Proximity preferences (e.g., 50 percent of the seats are 

reserved for students who live near enough to walk to 

school: [Note whether these families still have to fill 

out a choice application.]

 

3.3.2 Racial fairness guidelines (e.g., the ratio of Whites 

and non-Whites cannot vary more than 10 percentage 

points from that of the district as a whole): 

 

3.3.3 Socioeconomic fairness guidelines (e.g., the ratio of 

students eligible for free or reduced lunch cannot 

vary more than 10 percentage points from that of the 

district as a whole): 

 

3.3.4 Sibling preferences: 

3.3.5 Special provisions, if any, for students requiring 

bilingual or special education programs: 

 

3.3.6 Substitution rules when available seats, especially 

those designated by proximity, race, or 

socioeconomics, are undersubscribed: 

 

3.3.7 Lottery or random assignment when seats are 

oversubscribed (and especially whether separate 

lotteries in effect exist for different subgroups of 

seats—e.g., Whites and non-Whites): [If used, 

describe the lottery or random assignment 

procedure.] 

 

3.3.8 Whether and how a transferring student will receive 

supplemental services: 

 

3.3.9 Whether and how an eligible but non-transferring 

student will receive supplemental services: 

 

3.3.10 Whether the program maintains a waiting list, and 

how late in the semester students can still transfer, if 

a seat becomes vacant: 

 

3.3.11 Whether the students originally enrolled in the 

receiving schools can be displaced: 

 

 Interview with the VPSC Project Director and Support Staff, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-12

 

 3.4 Have there been any complaints regarding the student assignment process: 

 3.4.1 By students and their families? [If YES, 

describe.] 

 

 3.4.2 By sending schools’ staff? [If YES, describe.] 

 3.4.3 By receiving schools’ staff? [If YES, describe.] 

 3.4.4 By any other relevant parties? [If YES, 

describe.] 

 

3.5 How has the district’s student population changed, relative to the population of school-age 

children? 

 

4. Parent Notification Procedures

 4.1 How did parents participate (and how many participated) in the planning of the choice program and 

especially in the design of the parent information center? 

 4.2 When were parents first notified of the choice options supported by the VPSC initiative? [Obtain a 

copy of the written notification.] 

 

 4.3 What feedback information is collected to assure that all parents are being properly informed about 

their choice options? 

5. Parent Information Center(s) (PICs) 

 5.1 What district-wide enrollment does the PIC cover, and how many students is it actually serving? 

[Depending upon the choice program, the number of applicants for transfer may only be a portion 

of the total student enrollment—obtain both numbers.] 

 5.2 What are the physical characteristics of the PIC’s location? 

 5.2.1 Access conditions (e.g., parking and public 

transportation): 

 

 5.2.2 Physical facility (e.g., in a high school; working 

atmosphere): 

 

 5.2.3 Hours and days of operation: 

5.2.4 No. of different languages spoken by staff, 

and match to student population: 

 

 Interview with the VPSC Project Director and Support Staff, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-13

 5.3 Enumerate the PIC’s staff, by title and FTE. Also note the supervisory relationships and to whom 

the PIC director reports in the district. 

 5.4 Enumerate the outreach activities undertaken by the PIC during the past school year. Also, give 

some estimate of the frequency of each type of activity: 

 

5.4.1 Home visits or individual conferences with 

school (or choice) staff: 

How often? 

 5.4.2 Use of mass media: 

 5.4.3 Use of community or neighborhood events or 

facilities: 

 

 5.4.4 Other: 

 5.5 How have these outreach activities been adjusted, if at all, from year to year? 

 5.6 How does the PIC know that its outreach activities are sufficient in reaching all eligible families? 

[Cite the actual data used by the PIC in developing this knowledge.] 

 5.7 What is the nature of the parent survey (sample and instrument), if any, conducted by the PIC, and 

how are the survey results used? 

 5.8 How does the PIC work with individual schools? [If YES to any of the following, please describe]: 

 5.8.1 The PIC maintains up-to-date 

information about the schools’ 

enrollments: 

 

 5.8.2 The PIC arranges tours of the schools: 

 5.8.3 The PIC encourages schools to develop 

their own marketing materials: 

 

 5.8.4 The PIC helps the schools to disseminate 

information about themselves: 

 

 Interview with the VPSC Project Director and Support Staff, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-14

6. Transportation Support within VPSC Initiative

 6.1 Were VPSC funds used to support new transportation services? [If YES, please describe.] 

 6.1.1 Purchase of buses: 

 6.1.2 Hiring of drivers: 

 6.1.3 Increases in drivers’ salaries: 

 6.1.4 Revamping of technological 

infrastructure (e.g., computerized bus 

routes): 

 

 

 6.2 Were school hours changed (e.g., staggered daily times or alternate school calendars) to control 

transportation costs? [If so, describe and note whether the changes were related to VPSC or not.] 

 

 6.3 Whether VPSC funds were used to support new transportation services or not, collect data prior to 

and during the VPSC Program on: [If possible, obtain these data for individual schools, especially 

the sending and receiving schools.] 

 6.3.1 The number of buses: 

 6.3.2 The number of students riding buses: 

 6.3.3 Average and range of trip times: 

 6.3.4 The number of schools served each day 

by a single bus: 

 

 6.4 How important are the augmented transportation services to the entire VPSC initiative (e.g., did 

parents pay for or otherwise support a portion of the existing services, before)? 

 

7. Capacity-Building (New Schools or Sites to Serve as Receiving Schools)

 7.1 How many new schools or sites (and how many seats), if any, were opened as receiving schools, 

with VPSC support? [Also see Q. 8.1.] 

 7.2 What are the substantive educational themes at these schools, and in what way do they reflect a 

diversification or replication of themes already covered by the existing schools? 

 7.3 How were the themes chosen, and did parents participate in the process? 

 Interview with the VPSC Project Director and Support Staff, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-15

8. Capacity-Building (New or Expanded Programs at Existing Schools or Sites)

 8.1 How many existing schools or sites have been expanded or added as receiving schools (and how 

many seats), with VPSC support? [The total of the schools or sites and seats in 7.1 and 8.1 should 

coincide with the responses to Qs. 1.4.3 and 2.7 above.] 

 8.2 What are the substantive educational themes at these schools, and in what way do they reflect a 

diversification or replication of themes already covered? 

 8.3 Describe the specific actions taken to increase the capacity of the existing receiving schools: 

[Check each of the following, and if YES, describe.] 

 8.3.1 New educational programs (e.g., magnets or 

academies within the school) at the 

existing schools: 

 

 8.3.2 Administrative changes to increase the 

school’s enrollment capacity: 

 

 8.3.3 Supplemental services to assist the 

transferring students: 

 

 8.3.4 Professional development to assist educators 

in providing instruction to the transferring 

students: 

 

 8.3.5 Other: 

 

9. Capacity-Building (Sending Schools)

 9.1 Do sending schools lose funds in relation to out-transfers? [If so, describe the procedures and the 

amount of funds involved.] 

 9.2 Are the sending schools under a formal sanction category (e.g., in need of improvement, in 

corrective action, or restructuring), and how long have they been in the category? 

 Interview with the VPSC Project Director and Support Staff, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-16

9.3 Whether in a formal sanction category or not, what actions will be (or have been) taken to 

 improve the future performance of the sending schools (check all of the following)? 

 

 9.3.1 Professional development or other 

educational assistance: 

 

 9.3.2 Adoption of new educational practices: 

 9.3.3 Restructuring of the school: 

 9.3.4 Elimination of the school: 

10. Relationship to Title I Provisions of NCLB

 10.1 Has the district started to offer the school choice options under the new provisions of Title I, and if 

so, how do these arrangements relate to those of the VPSC initiative? 

10.1.1 Are different schools involved in each, or 

are VPSC funds used to complement the 

use of Title I funds for the same schools? 

 

11. Transferring Students and Comparisons with non-Transferring Students

11.1 For each relevant academic year in implementing the VPSC initiative, obtain the following data 

(preferably noting the individual sending and receiving schools): 

 

 11.1.1 How many students were eligible to apply 

for transfer? 

 

 11.1.2 How many applied, and how many were 

transferred? 

 

 11.1.3 What were the main experiences in dealing 

with under- or over-subscription? 

 

11.1.4 What were the main lessons learned 

about the transfer process, and did 

these lessons lead to modifications in 

the procedures used the following 

year? 

 

 Interview with the VPSC Project Director and Support Staff, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-17

Interview with the VPSC Project Director and Support Staff, 2006-07 

 11.2 Based on the analysis of individual student records [such analysis may be done by the VPSC site], 

how do the transferring students compare to those eligible but not transferring, on the following 

characteristics? 

11.2.1 Annual academic performance: [Start with 

the year prior to the initiative.] 

 

11.2.2 Demographic characteristics (White vs. nonWhite; school lunch vs. non-school lunch): 

 

11.2.3 Other characteristics that might account for 

academic differences between the two 

groups (e.g., number of siblings; level of 

parents’ education; nature of parents’ 

employment and income): 

 

12. Sending and Receiving Schools and Their Changes Over Time 

 12.1 For each sending and receiving school and for each academic year of the initiative, track: 

 12.1.1 The total enrollments, by grade level: 

 12.1.2 The proportion of White and non-White 

students: 

 

 12.1.3 The proportion of students eligible for free 

or reduced lunch: 

 

 12.1.4 The academic performance (mathematics and 

reading) of all the students at the school: 

 [Use all test data available, even for grades 

not necessarily affected by the VPSC 

initiative, not just the transferring students.] 

 

 12.1.5 Obtain district-wide averages for the above, 

omitting the sending and receiving schools, 

to permit contextual comparisons: 

 

 12.2 At the receiving schools, what problems have emerged that might have been associated with the 

transfers or transfer process, and what remedies have been put into place? 

 12.3 At the sending schools, what problems have emerged that might have been associated with the 

transfers or transfer process, and what remedies have been put into place? 

 

COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-18

 12.4 What is the evidence that the sending schools have taken more serious measures to improve their 

performance or otherwise to become more competitive? 

 12.5 Is there any evidence that potential sending schools are taking more serious measures to improve 

their performance or otherwise to become more competitive? 

13. Contextual Conditions

 13.1 What other choice options and programs are available in the district (not covered by the VPSC 

initiative), and how many students are involved in such options or programs? 

 13.2 Have these other choice options and programs changed during the period of the VPSC initiative 

and might they have played a role in the preceding early outcomes? 

 13.3 Have there been any notable changes by the surrounding districts that might have played a role in 

the preceding early outcomes? 

 13.4 Have there been any changes in the community (e.g., residential relocation patterns) that might 

have played a role in the preceding early outcomes? 

 13.5 How has the district’s “share” of the student population changed, relative to that of private schools? 

[Obtain annual data by grade level, if possible.] 

 Interview with the VPSC Project Director and Support Staff, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-19

4. INTERVIEW WITH OTHER PARTICIPATING PERSONS

(Data collection: 2006-07) 

 1. What is your connection to the VPSC initiative [e.g., principal or teacher at participating school, 

parent of participating student, other]? 

 2. How were you made aware of the VPSC initiative? 

 3. Can you briefly describe your understanding of the VPSC initiative? 

4. What changes have you seen or experienced as a result of the VPSC initiative? 

5. How has student enrollment at your school [or your child’s school] changed during the past year 

and to what extent have these changes been associated with the VPSC initiative, compared to other 

conditions? 

 6. What changes in the academic program have been made as part of the VPSC initiative, and how 

well are these changes working? 

 7. What changes in school administration (e.g., redefining school hours or the school year) have been 

made as part of the VPSC initiative, and how well are these changes working? 

 8. How smooth or disruptive was the timing or sequence of key events in the VPSC initiative—e.g., 

application deadlines or parent notification dates? 

 9. How well do you think the VPSC initiative has been received by teachers, principals, and parents, 

and do you think they are sufficiently informed about any new role(s) expected of them under the 

initiative? 

 10. How has student behavior and academic performance changed, and to what extent are these 

changes associated with VPSC? 

 11. Did VPSC provide any other benefits or create any other problems that have not been addressed by 

the preceding questions? 

Interview with Other Participating Persons, 2006-07 

COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-20
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-21

5. INTERVIEW WITH THE COMPARISON DISTRICT AND TITLE I STAFF 

(Data collection: 2006-07) 

Procedures

 One-day site visits will be made to comparison sites in spring of 2007 (February-May). Where 

the comparison sites are within the jurisdiction already covered by the VPSC Program’s site (e.g., 

another “zone” in a VPSC district or another district within a statewide VPSC initiative), the needed 

data should be collected from the VPSC and related district staff. 

 Where the comparison sites are outside of the VPSC jurisdiction, initial contact with the sites 

should emphasize that our National Evaluation is: 1) covering public school choice, 2) selecting some 

sites where minimal choice options other than Title I, magnets, and charters are in operation, and 3) 

making site visits to sites that are near a VPSC Program’s site but that did not receive a VPSC award. 

To minimize burden, the site visit will involve interviews with: the Title I coordinator and any related 

district staff (e.g., overseeing magnets and charters). 

Preparation for Site Visits. Prior to any site visit, field teams should collect and analyze data 

about the targeted area. This information will have come from several sources, including: the VPSC 

Program’s site that is aware of (and helped to select) the comparison site, and materials gathered from 

the district or state Web sites. 

Assembling of Evidence and Preliminary Reports Immediately Following Site Visits. Field 

teams are urged to begin the formal analysis and report-writing process as soon as a site visit has 

ended, though additional data may still have to be collected. Assembling data and drafting narratives 

proceeds more efficiently and with much higher quality if this time sequence is followed. Teams 

should reserve the day or two after the site visit for this activity, avoiding other commitments. 

Outline of Report. The report should follow the same heading structure as the topics of inquiry 

described next.

Topics of Inquiry

 The topics below not only cover the “agenda” to be followed by the field team but also give 

explicit probes and examples regarding the type of evidence that is being sought. As a result, the 

protocol should provide guidance on how to know what to look or listen for, and how to recognize 

relevant evidence when it is encountered. 

Interview with the Comparison Site, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-22

1. Public School Choice Options at This Site

 Enumerate the public school choice options at this site, giving the number of schools and students 

involved by type of option: 

 1.1 School identified for improvement under Title I: 

 1.2 Magnet Schools (no. of schools and programs): 

 1.3 Charter Schools: 

 1.4 Other choice arrangements, including unsafe school choice and desegregation (describe): 

[If options were identified under 1.2, 1.3, or 1.4 above, be sure to cover Q. 3 below.] 

2. New Title I Choice Arrangements

 How has the district started to offer the school choice options under the new provisions of Title I? (probe 

for): 

 2.1 Use of designation as a Title I school identified for improvement in defining sending and receiving 

schools: 

 2.2 Expanded transportation services: 

 2.3 Provision of supplemental services: 

 2.4 Expanded or improved parent notification procedures: 

 2.5 Other (describe): 

3. Other, non-Title I Choice Arrangements

 Describe the procedures used for the other choice options identified in Q. 1 above (address the following 

questions for each type of option separately): 

 3.1 How are eligible students defined? 

 3.2 How are students/parents notified of their choices? [Be sure to cover all the communication modes—

e.g., see Q.5 in the School Survey instrument.] 

 3.3 What criteria are used in selecting students to exercise their choice? 

 [Be sure to cover all criteria—e.g., see Q3.3 in the VPSC field 

 instrument.] 

Interview with the Comparison Site, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-23

3.4 What proportion of eligible students were able to exercise their choice? 

 [Also determine the proportion receiving their 1st, 2nd, etc. choices; 

 obtain data for the past 2-3 years, if possible.] 

 3.5 How have transportation services been affected, if at all, by the choice 

 arrangement? [Be sure to cover all facets of these services—e.g., see Q.6 in the 

 VPSC instrument.] 

 3.6 Are other choice options likely to be implemented in the future? 

4. Trends Associated with Choice Options

 Determine whether the district maintains records of individual students who have transferred or of the 

performance of sending and receiving schools. The following questions may be addressed once, even if the 

district has more than one choice arrangement. 

 4.1 Are there any data showing trends in academic performance by the students who have exercised their 

choice? [Probe for data for the past 2-3 years, if possible.] 

 4.2 Are there any data showing trends in academic performance by the schools involved in the choice 

arrangement? [Probe for data for the past 2-3 years, if possible.] 

 4.3 What are the district’s policies, views, and preferences regarding school choice options? 

 4.4 What interest has been expressed by parents, communities, or students regarding school choice 

options at this district? 

5. Contextual Conditions

 5.1 Have there been any notable changes by the surrounding districts that might be relevant to either the 

choice options or their outcomes at this district? 

 5.2 Have there been any changes in the community (e.g., residential relocation patterns) that might be 

relevant to either the choice options or their outcomes at this district? 

 5.3 How has the district’s “share” of the student population changed, relative to that of private schools in 

the area? [Obtain annual data by grade level, if possible] 

 5.4 Is the district implementing other approaches to school improvement as an alternative to choice? 

Interview with the Comparison Site, 2006-07 COSMOS Corporation 

Phone: 301-215-9100 

Fax: 301-215-6969 
OMB Clearance #1875-0224 

Expiration Date: 10-31-09 

B-24
 

C-1

Appendix C: 

Supplemental Data About 

Individual VPSC Sites 
 

C-2

Contents 

Page 

Exhibit C-1 State Education Agencies Used VPSC Funds to 

 Support Widely Different Choice Priorities.............................C-3 

Exhibit C-2 VPSC Initiatives Coincided With Existing, Districtwide Choice 

 Options .....................................................................................C-3 

Exhibit C-3 Sites Split Their VPSC Funds to Support 

 a Variety of Activities, Not Limited to a Single Choice 

 Initiative ...................................................................................C-4 

Exhibit C-4 VPSC Initiatives Could Be Separate From or Coincide With 

 Title I Choice ..........................................................................C-4 

Exhibit C-5 Five Years of VPSC Participants ...................................................C-5 

Exhibit C-6 Choice Initiatives Limited to Zones Within a District 

 Nevertheless Involved a Broad Array of Sending Schools......C-6 

Exhibit C-7 Eligible Sending Schools Were Restricted 

 but Not Necessarily to Low-Performing Schools ....................C-6 

Exhibit C-8 Capacity-Building Occurred at Receiving Schools 

 That Already Were (Presumably) Higher-Performing 

 Schools .....................................................................................C-6 

Exhibit C-9 Local Sites May Need to Focus on a “Tapestry” of Choice 

 Options, Rather than on Any Single Initiative.........................C-7 

Exhibit C-10 “Attractor” Programs at Receiving Schools May Not 

 Have Been Needed to Attract Incoming Students....................C-7 

Exhibit C-11 Explicit Efforts May Be Needed to Maintain the 

 Performance of Receiving Schools ..........................................C-7 
 

C-3

 

Exhibit C-1 

State Education Agencies Used VPSC Funds to 

Support Widely Different Choice Priorities 

State departments of education received three of the 13 VPSC awards. 

One did not use the VPSC funds to support any particular choice options directly 

but instead used the funds to provide technical assistance to its districts, to plan and 

implement their choice options. The technical assistance appeared to be an 

appropriate priority, given the state’s historically wide array of choice options on top of 

options implemented by individual districts. 

A second state used the funds to support an urban-suburban interdistrict choice 

arrangement. Such an arrangement fulfilled the state’s commitment, stemming from 

the settlement of a racial discrimination lawsuit, to provide opportunities for lowincome students in the state’s largest urban district to transfer to suburban schools. 

The third state initiated a new option whereby students did not need to attend 

their regular schools but received Web-based curricula and instruction at off-campus 

locations. The state already had experienced some legislative controversy over 

supporting off-campus schooling, and the new option provided an opportunity to 

strengthen such schooling as well as to test it by permitting students to take the 

annual state assessments. 

Exhibit C-2 

VPSC Initiatives Coincided With 

Existing, Districtwide Choice Options 

Four of the 13 VPSC sites used the VPSC funds to enhance preexisting choice 

options whereby students could choose to attend any school in the district. 

At three districts, the VPSC funds helped the districts to strengthen the academic 

programs at under-enrolled schools, making them more attractive, as well as to 

formalize the selection and assignment procedures in exercising choice options. The 

fourth district already had been actively planning its new districtwide option, and the 

VPSC funds helped to support the implementation of the option—e.g., the kick-off 

marketing campaign to solicit choice applications, and the development of “attractor” 

programs and associated professional development at selected schools. 

At none of the four sites was it possible to disentangle the influence of VPSC 

funds from districts’ original designs and other sources of funding support. 
 

C-4

Exhibit C-4 

VPSC Initiatives Could Be Separate From or 

Coincide With Title I Choice 

Two VPSC sites defined their VPSC initiatives to coincide directly with their 

fulfillment of Title I choice requirements, defining sending schools as Title I schools 

“identified for improvement” and receiving schools as higher-performing schools. At a 

third site, the specific set of low-performing VPSC sending schools included but were 

not limited to the site’s Title I schools. The remainder of the VPSC sites defined and 

administered their VPSC initiatives totally apart from their Title I choice options. 

Exhibit C-3 

Sites Split Their VPSC Funds to Support a 

Variety of Activities, Not Limited to a Single Choice Initiative 

Some of the different combinations of activities at the VPSC sites included: 

● Supporting after-school services for students in the VPSC initiative but also 

supporting marketing and parent information centers catering to 

participants in all choice initiatives; 

● Supporting program development for schools targeted to be part of the 

VPSC initiative but also supporting marketing campaigns and materials 

related to all other choice options; and 

● Assisting providers of new supplemental educational services for two years 

until new charter schools had been approved and were able to open for 

enrollment. 
C-5 

Exhibit C-5 

Five Years of VPSC Participants 

Eligible Applying 

VPSC 

Site 

Data 

for 4 

Years 

2006– 

07 

2005– 

06 

2004– 

05 

2003– 

04 

2002– 

03 

2006–

07 

2005–

06 

2004–

05 

2003–

04 

2002–

03 

2006–

07 

2005–

06 

A n.a. 290,142 318,231 279,800 206,700 n.a. 214 198 774 717 n.a. 214 

B √ 2,594 670 0 0 0 n.a. n.a. 0 0 0 266 196 

C* √ 311,481 320,331 300,000 332,055 0 n.a. n.a. 4,831 4,487 0 11,371 13,068 

D √ 10,292 10,452 10,452 6,775 6,425 161 170 608 545 37 161 170 

E n.a. n.a. 

F √ 1,795 1,682 1,842 720 0 1,795 1,682 1,842 720 0 1,795 1,682 

G √ 163,159 46,838 28,349 0 0 2,042 1,620 9,924 0 0 1,969 1,494 

H √ 150,000 134,878 108,845 45,971 0 2,312 n.a. 3,884 162 0 901 3,039 

I** √ 25,333 27,266 29,531 29,000 0 827 674 1,037 3,417 0 827 674 

J n.a. 8,235 4,073 3,941 n.a. n.a. n.a. 167 126 19 n.a. 121 

K*** √ 8,444 4,970 4,849 1,725 4,905 1,205 446 741 190 501 1,205 446 L √ 45,970 45,727 51,000 48,900 0 4,818 5,249 5,007 4,936 0 3,144 3,783 

M √ 4,863 4,993 5,224 6,500 1,660 76 114 149 364 117 35 34 

 Total 723,931 896,184 862,396 755,387 219,690 13,236 10,169 28,388 15,721 1,391 21,674 24,921 

Total Number of 

Sites Reporting 10 12 11 10 4 8 8 11 10 5 10 12 

Mean for the 10 

sites with 4 years 

of enrollment data 

72,393 59,781 54,009 47,165 n.a. n.a. 2,167 2,459 

Percent of eligible 

students enrolled 

at the 10 sites 

 3.0 4.1 

Exhibit Reads: Site A for 2005–06, reported 290,142 eligible, 214 applying, and 214 enrolling VPSC participants. *For 2005–06 and 2006–07, the site may have counted transfers from earlier years along with first-time enrollees. **The number of eligible students reflects the site’s students, K–8, who are eligible for the Free and Reduced-Price Lunch Program. T

underestimate: suburban students who have the option to move to the site’s magnets are not included in the eligibility count. ***The 2002–03 applicant number (190) and enrollee number (191) are as reported by the site. The site’s staff explained that an enroformal application. n.a.= data were not available for that year. Sources: Grant Performance Reports, supplemented by site visit databases. 
 

C-6

Exhibit C-6 

Choice Initiatives Limited to Zones Within a District 

Nevertheless Involved a Broad Array of Sending Schools 

Two large urban VPSC sites designed zonal initiatives, giving special attention to 

schools in particular zones in their district. One site eventually scaled up to cover ten 

zones and 65 schools; the other had two zones totaling 106 schools, but VPSC funds 

focused on 23 of these 106 schools. The sites chose the zones because the schools 

in them had tended to be under-enrolled and underperforming. 

 In theory, the restrictions to these zones, in lieu of districtwide coverage, would 

have classified these initiatives as having targeted selected schools. However, 

although most transferring students made within-zone transfers, at both sites students 

in the entire district were eligible to request transfers into the schools in these zones. 

Thus, the potential sending schools actually represented a broad array of schools, 

leading to the classification of these initiatives as “broad” and not “targeted.”

Exhibit C-7 

Eligible Sending Schools Were Restricted 

but Not Necessarily to Low-Performing Schools 

Five VPSC sites limited the eligibility for their choice initiatives to students 

attending specific sending schools, and not all of the schools in their districts. 

However, the designations at two of the sites were based on other factors than school 

performance: at one site that had a statewide initiative, the sending schools were 

limited to those located in the districts that had agreed to participate in the initiative, 

not all of the districts in the state; at the other site that started a charter school 

initiative, the charters were mainly designed to attract students previously attending 

specific schools in the school district, but the schools did not have to be lowperforming. 

 In contrast, the three VPSC sites that limited eligibility to low-performing schools 

did choose those schools because of their performance on state assessments.

Exhibit C-8 

Capacity-building Occurred at Receiving Schools 

That Already Were (Presumably) Higher-performing Schools 

Several of the VPSC sites enhanced academic programs at receiving schools 

even though these schools had already been designated as higher-performing 

schools. At one site, the academic changes and an infusion of computer technology 

benefited the students already enrolled at the receiving schools. At another site, a 

school previously designated as an acceptable receiving school, became “identified for 

improvement” midway through the VPSC initiative. At a third site, surveys showed 

that parents were not particularly swayed by “attractor” programs at receiving schools.
 

C-7

Exhibit C-9 

Local Sites May Need to Focus on a 

“Tapestry” of Choice Options, 

Rather Than on Any Single Initiative

 One of the two sites that had put into place the fourth type of choice 

arrangement—a “mixture” of the first three arrangements (see section 3.2)—previously 

only had one choice option serving a multidistrict region. In hindsight, the “mixture” 

may have been the site’s way of responding to the diversity of local needs by trying to 

create a tapestry of choice options rather than focusing on a single type of initiative.

Exhibit C-10 

 

“Attractor” Programs at Receiving Schools 

May Not Have Been Needed to Attract Incoming Students

 The VPSC sites frequently put new education programs into place at the receiving 

schools. One motive for such programs was that they would more readily attract 

students from other schools, either by diversifying or intensifying the curriculum to 

match the incoming students’ interests. 

 One VPSC site invested considerable effort in defining and implementing a set of 

what it called “attractor” programs at a series of receiving schools. Two years later, 

the site’s parent survey revealed that parents were attracted to the schools by their 

overall reputations, for academic excellence or even for school safety, but the parents’ 

preferences were not influenced by the presence of the attractor programs. The 

VPSC site subsequently reduced its emphasis on these programs. 

Exhibit C-11 

Explicit Efforts May Be Needed to Maintain the 

Performance of Receiving Schools

 One site’s choice initiative had predesignated sending schools that had been 

deemed low-performing according to NCLB standards. Students could transfer to 

other schools in the district, but the site also used VPSC funds to strengthen the 

academic programs at four specific receiving schools that were located near the 

sending schools. The four schools also had students with backgrounds similar to 

those at the sending schools. Two years after VPSC enrollment had started, two of 

the four receiving schools fell into the NCLB category of “identified for improvement.” 

Whether (as in the eyes of the VPSC initiative’s staff) students transferring from the 

low-performing schools exacerbated the receiving schools’ performance, explicit 

precautionary steps are needed to avert such outcomes in the implementation of 

choice initiatives. 
 

C-8
 

C-9