School Vouchers: Characteristics of Privately Funded Programs
Released: 2002-09-26T00:00:00

Letter

Background

Privately funded school voucher programs got their start in 1991 when an Indianapolis businessman founded a local program that provided tuition assistance to about 750 low-income students in grades kindergarten through 8 (K-8). After the initiation of the Indianapolis program, a number of other communities also established privately funded voucher programs based on the Indianapolis model and used funding from local donors. In 1994, a local voucher program that had been founded in San Antonio obtained a grant to establish a national clearinghouse organization—now called Children First America (CFA)—to assist new and existing local programs with funding and technical assistance. By 1997, there were 31 local programs offering over 12,000 privately funded vouchers to K-12 students in 18 states plus the District of Columbia.

In 1998, a second national organization, the Children’s Scholarship Fund (CSF), was established. CSF used the word scholarship in its name—rather than voucher—to distinguish itself from publicly funded voucher initiatives. CSF helped found several new local voucher programs and established partnerships with many existing programs. In its first year, CSF provided one-to-one matching funds for a total of 40 programs to provide vouchers for low-income students for a period of 4 years. Beginning in school year 1999-2000, CSF-funded programs provided school vouchers to about 40,000 K-8 students. CSF currently provides support to 47 affiliated programs. Nine of the programs are administered by the CSF national office, including a program that provides vouchers to students not residing in an area with a local program. Like CSF, CFA continues to be a source of funding support and technical assistance to local privately funded voucher programs. The two organizations work with each other in a number of efforts but differ somewhat in their stated missions. CSF offers tuition assistance to needy families in what it describes as a purely charitable effort, while CFA states that its mission is to promote parental choice, including both privately and publicly funded options.

In addition to programs offering vouchers for low-income students to use, with virtually no restriction as to their choice of private schools, there are other privately funded tuition support or scholarship programs. However, these programs fall outside the scope of our study because they limit student eligibility or school choice in different ways. For example, we did not review traditional scholarships offered by private schools, merit-based programs, or programs that limit choice to schools of a particular religious denomination.

According to a recent survey done by CSF, 78 privately funded school voucher programs used family income as their only eligibility criteria and permitted families to use their award at nearly any private school. Although these programs receive their funding from private individuals and groups— in contrast with publicly funded school voucher programs—they nonetheless may affect public funding in several ways. Most programs are not-for-profit organizations and, as such, are eligible for the associated federal tax benefits. Programs may also have an impact on local public school funding because much state and some federal funding to school districts is allocated on the basis of formulas incorporating the number of students attending the schools. Some states have laws in place to provide state tax credits to individuals or businesses for their contributions to tuition assistance organizations, including privately funded voucher programs.

Programs Shared Characteristics and Challenges, but Varied in Size and Voucher Amounts

The privately funded voucher programs we reviewed had a number of common characteristics, including several long-term challenges, but they varied widely in the dollar amounts of the vouchers they awarded and number of students served. Most programs had local financial support. Programs also used similar methods to determine student eligibility and many programs used a sliding scale based on family size and income. Nearly all programs required parents to contribute at least a specified minimum amount toward the cost of tuition. Many automatically accepted siblings and some ensured multiyear funding as part of an emphasis on helping families. Most programs collected only the information they needed to determine student eligibility and administer the programs, such as family income and number of siblings. However, information on other student and family characteristics was limited. Looking to the future, program officials said voucher programs face two major challenges— specifically, sustaining programs for the long term and retaining students in the programs. Some programs we contacted have begun developing ways to address these challenges.

Programs Had Many Common Characteristics but Differed Widely in Numbers of Students Served and Voucher Amounts

Although scattered across the United States, the local privately funded voucher programs we studied shared many things in common. For example, most were largely supported by local donations, and the majority of the 47 programs affiliated with CSF raised funds locally to receive matching funds from that national organization. Often programs were supported by one, or a few, “anchor” donors—people or organizations that had committed to making a substantial, and often multiyear, financial contribution to support vouchers in their community. For example, a single local donor committed to provide the matching funds for the Children’s Scholarship Fund of Seattle/Tacoma for 4 years, and in San Antonio two local donors have provided nearly all of the funding for two voucher programs serving that area.

Programs also had similar voucher recipient selection processes. Most programs selected recipients by lottery from among all eligible applicants, although a few selected them on a first-come-first-serve basis. According to officials, some programs awarded vouchers to all eligible students in the first year because they had more money for vouchers than they had qualified applicants. As more people became aware of the vouchers and demand grew, nearly every program adopted a lottery system. Many programs have established separate lotteries for private and public school students. Several program officials pointed out that awarding tuition assistance to students already in private school was not accomplishing the mission of extending school choice because those families had already exercised choice. Programs we contacted typically awarded a larger percent of their vouchers to students transferring from public schools and established a specific limit to the percent that would be awarded to students already in private school. For example, the Washington Scholarship Fund, in Washington, D.C., reported that it awards at least 75 percent of its vouchers to students coming from public schools. The program maintains two pools of applicants—one for public school students and one for private school students—and selects students by separate lottery from each pool.

Eligibility criteria were also similar among programs. Eligibility depended on both family income and size. Almost all the programs targeted the vouchers to families eligible for the federal free and reduced price lunch program—those making less than 185 percent of the federal poverty guidelines. However, many programs included families with incomes up to 270 percent of the poverty guidelines. Many programs also used a sliding scale based on family income and size to determine both eligibility and how large a voucher the family could receive. Families with higher incomes received a smaller percentage of school tuition. The CSF national organization has established sliding scale income eligibility standards for its affiliated programs. See table 1 for the CSF 2001-02 income eligibility scale. Programs also typically had a maximum voucher amount that could be awarded. Maximum award amounts for CSF programs varied from program to program but ranged from $1,000 to $2,100 for K-8 vouchers and $1,000 to $2,900 for high school vouchers. Since voucher amounts did not typically cover the full amount of tuition, the family was responsible for the balance.

Despite the size of a voucher for which a family is eligible, most programs required families to contribute a minimum amount toward their child’s tuition payment—at least about $500. However, many programs do not require families with more than one child receiving a voucher to pay the minimum amount for each child. For example, in Buffalo, New York, the program required each family to pay just $500 toward the tuition, regardless of the number of children receiving a scholarship.

For many programs, including all CSF affiliated programs, siblings of award recipients were automatically offered vouchers. In addition, the programs typically made a commitment to the families to provide the vouchers for multiple years, as long as the family remained eligible for the program. For example, in Cincinnati, all children in the family can receive a voucher, and eligible students will continue to receive a voucher for at least 4 years. Officials said accepting siblings and committing to a multiyear voucher were part of their approach to helping the children and the families. In addition, CSF plans to continue its commitment to families by extending financial support for vouchers through the year 2013 to include all younger siblings through the eighth grade.

Programs we contacted also conducted similar financial oversight activities. They typically paid voucher amounts directly to the private schools in several tuition installments throughout the school year. Prior to each payment, programs received confirmation from participating private schools that the voucher students were still enrolled. Many programs also required confirmation that the students were attending class regularly and some required parents to sign a confirmation form each time a payment was made. For example, the Houston Children’s Educational Opportunity Foundation pays voucher installments to participating schools every month during the school year. Prior to each payment, the program official sends a commitment form to each school. A school official and the parents of each voucher student are required to sign it. The CSF national organization directs its affiliated programs to have the parents of voucher students visit their schools to sign a Scholarship Verification Report. The form is also signed by the school principal and is returned to the program before payment is made. Program officials are instructed to retain the forms as an audit trail of the scholarships and proof that they were verified.

Although the programs shared many similar characteristics, they differed in the average dollar amount of their voucher awards. Award amounts are based, in part, on school tuition and the tuition at participating private schools varied considerably among program locations. For example, for K-8 voucher recipients in school year 2001-02, the average private school tuition in Buffalo, New York, was about $1,500 and the average K-8 voucher award amount for the Buffalo program was $620. The maximum Buffalo program award was $1,000, depending on family size and income. In contrast, the private schools attended by K-8 voucher students in the Atlanta CSF program charged an average tuition of about $4,300 and the average K-8 voucher amount for the Atlanta program was $1,663—with an award maximum of $2,100. The average percent of tuition covered by CSF program vouchers ranged from about 20 percent in Pittsburgh to about 65 percent in Baton Rouge. Overall, the average K-8 voucher award for the 47 CSF affiliated programs was about $1,100, according to CSF officials, and the average per-child K-8 tuition among all programs that receive funding from the CSF was about $2,550. Average high school voucher awards from individual programs also varied widely from $1,009 to $2,411. The Department of Education reports that the average Catholic or other religious private school tuition for grades K-12 in school year 2000 was about $2,800 and for private nonsectarian schools was about $8,900.

The number of vouchers awarded per program also varied considerably. For example, programs in Phoenix; St. Louis; Erie, Pennsylvania; and Midland, Texas, awarded 10 or fewer scholarships each, while the largest— the CSF local program in New York City—awarded over 3,100. Seventeen of the local programs we studied awarded more than 1,000 vouchers. In addition, the national CSF organization awarded about 3,600 vouchers, through its national program, to eligible students not living in a local area served by a voucher program.

Information on the students or families who receive privately funded vouchers is limited, as is information on the private schools the students attend. Most programs we contacted reported that they collected only the student and family data needed to determine student eligibility, and the administrative data needed to manage and oversee program activities. The eligibility data collected by programs included family income, size, and place of residence, and the administrative and oversight data included the amounts of the vouchers awarded and the tuition at the schools voucher students attended. Information on the private schools students attended was often limited to the school affiliation—for example, Catholic, nondenominational Christian, or independent. Programs reported that the majority of the schools voucher students attend are religiously affiliated. Most programs we contacted did not collect information about the public schools voucher students previously attended, and most did not gather data to evaluate program outcomes or results. While some program officials said they saw value in such analyses, several said that evaluations were costly and that they preferred to use their limited funds for student vouchers.

Some Programs Initiated Activities to Meet Future Challenges

Through our discussions with program officials, we identified two major challenges faced by privately funded voucher programs—ensuring program sustainability and maximizing student retention rates. We found that many of the programs had recognized these challenges and identified or implemented activities and initiatives to address them.

Sustaining Programs for the Long Term

Most programs we contacted were concerned with sustaining their activities over the long term. Officials generally said they expected their programs to continue operating for the foreseeable future or indefinitely. Some had determined that certain program modifications would be needed to foster sustainability. Specifically, these programs recognized that they should broaden their funding base and have more community representation on their boards of directors. Several programs had been founded and initially funded largely by a few local donors. In many cases, the donors had agreed to provide funding for 4 years. One program official said he believed there was a danger of what he called “donor fatigue” and determined it could be averted by expanding the number of donors— particularly small contributors. Some programs also saw the benefits of broadening community representation on their boards of directors. For example, the Washington Scholarship Fund told us it is currently building a broad donor base and is expanding its board of directors—which had been limited to large donors—to include members who may not be able to provide funding, but are actively involved in the community. Additionally, an official at Children First Utah told us that to help ensure stability and sustainability, the program has established a board of directors with broad community representation, including representatives from several large corporations and a public relations firm, community activists, and a representative from the Hispanic community. In addition, Children First Utah has implemented efforts to broaden its funding base, including direct mail solicitations, media public relations efforts, and several funding proposals to foundations.

The two national private voucher organizations, CFA and CSF, support local programs in both implementing and sustaining their activities. In addition to providing matching funds to some programs, they also provide technical assistance. CFA offers a detailed “how to” manual to new programs and will work on site with programs as needed at no cost. CFA officials told us they also work with existing programs to help them solve problems and enhance program stability, and they emphasized that their focus is on the sustainability of the programs. For example, CFA assists programs in broadening the community representation on their boards and among their donors. According to CSF officials, that organization provides its affiliates with extensive guidance and technical support for program administration activities such as scholarship disbursement procedures, enrollment verification, and financial planning. CSF also holds an annual conference for its affiliates, provides a detailed program management manual, and assists affiliates in the use of a sophisticated program administration database.

Maximizing Student Retention

Maximizing the retention rate among voucher recipients was another challenge faced by the programs we contacted. In many programs, about 20 percent or more of the voucher recipients leave the program each year. However, according to a CSF official, only a small percent of voucher recipients leave CSF affiliated programs because they graduate from the eighth grade. Program officials said that other reasons recipients leave might include moving or dissatisfaction with the school. Many said they believe family financial difficulties are a common cause of attrition. According to some officials, families are sometimes unprepared for all the costs of having children in private schools—for example, uniforms, books, and activity fees. Additionally, parents of voucher students may be confronted with financial emergencies that preclude their paying their required portion of the tuition and continuing in the program. Some programs we contacted attempted to determine the reasons students drop out and several had initiated activities to promote student retention. For example, the CSF program in New York City and the Washington Scholarship Fund have implemented “stay in school funds” to help parents who are experiencing financial emergencies such as unexpected medical expenses. Washington Scholarship Fund, which served nearly 900 families in school year 2001-02, reported that it has provided such funding assistance to about 26 families so far. The Horizon program in San Antonio, Texas, initiated a voucher parent group to increase the student retention rate. According to a program official, the group, “Las Comadres,” meets weekly to discuss aspects of their children’s education and learn from each other about schools and curricula. The meetings also include guest speakers on such topics as preparing children for high school and college, parental rights and duties, and children’s health issues. The Horizon program director and staff also attend the meetings to facilitate and answer questions.

Recent Research on Privately Funded Voucher Programs Has Explored Academic Achievement and Parental Perceptions of Their Children’s Schools

We identified three studies of privately funded voucher programs—in New York; Dayton, Ohio; and Washington, D.C.—that were rigorous enough to meet our criteria for inclusion. These studies all included quantitative analysis of program effects, used appropriate comparison groups, and gathered data before vouchers were awarded, as well as after, to assess the equivalence of study groups and to track program effects. In general, researchers found that African American students in New York City who used vouchers to attend private schools exhibited more substantial improvements on test scores in math and reading than African American students in the comparison group. For students from other ethnic groups, there were no sizable or significant differences in the test score gains of voucher users and comparison group students, or there were too few participants to draw any conclusions. These studies have also found that the parents of voucher users in all programs were consistently more satisfied with their children’s education, regardless of ethnic group. Although these results suggest positive achievement effects for some African American students using vouchers, they cannot be generalized beyond the specific programs, schools, and geographic areas where the studies were conducted or the low-income group of families studied.

Studies Used Same Rigorous Methodology

The studies of privately funded voucher programs in New York; Dayton, Ohio; and Washington, D.C., all used impact evaluations to study the effect of vouchers on private school attendance. Impact evaluations isolate a program effect from the effects of other factors that could influence participants’ outcomes. To isolate the program’s effect, impact evaluations divide participants into two groups: those who receive program services and a similar group who do not (the control or comparison group). Some impact evaluations assign participants randomly to one group or the other. Random assignment increases the likelihood that the two groups are roughly equivalent on all characteristics that could affect outcomes. It helps ensure, for example, roughly equal numbers of very low-income students in both groups, rather than those students being concentrated more heavily in one group or the other. The use of random assignment to create equivalent groups allows researchers to compare tests scores (or other outcomes) for the two groups and attribute any differences to the program services rather than other factors, such as differences in family income.

The studies of voucher programs in New York, Dayton, and Washington randomly assigned applicant families to the voucher offer group and the control (no voucher offer) group. Before each voucher program began, students whose families hoped to receive a voucher took the Iowa Test of Basic Skills in math and reading, and their parents completed surveys about family background characteristics and their children’s educational experiences. The researchers also screened families to determine if they were eligible to participate on the basis of their incomes. Test scores and information about family background provided researchers with a baseline for making later comparisons. Families who met the income qualifications and participated in the testing and survey sessions were eligible to enter a lottery to decide which families would receive voucher offers and which would not. Because the researchers ensured that the lotteries were completely random, the two groups—voucher offer recipients and nonrecipients—should have roughly equivalent average student test scores and family backgrounds at the outset. To collect follow-up data from both voucher offer recipients and nonrecipients in subsequent years of the studies, the researchers used the same testing and survey procedures. Only students attending a public school before the lottery were included in the follow-up testing and analysis. Table 2 provides additional details about the studies, including the numbers of applicants and vouchers awarded.

While the use of control groups helps isolate the cause of any outcome, researchers must still address circumstances that threaten the quality of their results. In the New York, Dayton, and Washington studies, the researchers addressed two such circumstances. First, not all of those offered a voucher used their voucher to attend a private school, and some students who did not receive a voucher offer ended up in a private school anyway. This meant that a comparison between the voucher offer group and control group, as designed, revealed the effect of offering a voucher to eligible students in a city, not the effect of actually using it to attend a private school. To answer the question of the effects of vouchers on those actually using them, the researchers used a data analysis technique that allowed them to compare the outcomes of those who actually used a voucher to attend private school to those of students who were most likely to have attended private school had they been offered a voucher. While both the voucher offer and voucher use analyses are valid, they answer different questions. The researchers reported primarily on the voucher use analysis. Second, substantial numbers of those who initially agreed to participate in the studies dropped out in subsequent years. When this occurs, those who remain in the study may be different from those who left, which can threaten the equivalence of the two study groups, making it difficult to assess the effect of the voucher offer. The researchers used a standard statistical procedure to minimize the possibility that the loss of some participants undermined their results.

In all three studies, the demographic characteristics of study participants surveyed at baseline reflected the programs’ targeting of vouchers to low- income families, most of whom lived in inner cities. Although these families met program eligibility requirements as low-income, they were able to pay at least a portion of the private school tuition. In both Dayton and Washington, the average income of a participant family that received a voucher and used it to attend private school was approximately $18,000. However, in New York, the average family income of participating voucher users was about $10,000, possibly reflecting the more stringent income criteria of the New York program. In all three cities, ethnic minorities made up a large majority of voucher applicants participating in the studies. According to the survey questions, study participants were asked to report their race as either Black/African American (non-Hispanic), White (non- Hispanic), Puerto Rican, Dominican, other Hispanic, American Indian or Alaskan Native, Chinese, other Asian or Pacific Islander, or other. Thus, the African Americans and Whites in the study are likely to be non-Hispanics only. (See table 3.) Among study participants who made use of the voucher offered to them, a majority attended Catholic schools in all three cities.

African American Voucher Users in New York City Had Improved Test Scores

Of the three studies, the New York study showed the greatest positive effect on voucher users’ academic achievement. In each of the 3 years of the New York study, African American students who used a voucher to attend private school performed better on the reading exams than a comparable control group. African American voucher users also performed better in math than those who did not receive a voucher and remained in public schools. This improvement in math scores was found in years 1 and 3, but not year 2, of the study. Achievement gains, however, did not extend to Hispanic voucher users. The New York study found that Hispanic students who used privately funded vouchers to switch from public to private school had math and reading scores that were statistically the same as those of the control group. The New York study did not have sufficient numbers of white students or students from other ethnic groups to estimate program effects for them. In addition, when the New York study sample is considered as a whole—pooling together African Americans, Hispanics, and white students—there is no significant difference in achievement gains between voucher users and nonusers. Although the researchers conducted additional analyses to explain why these positive effects were seen for African American students and not for others, the cause of this difference remains unclear. In Dayton, African American voucher users showed improvements in reading in the second year of the study, an effect that approached, but did not reach, statistical significance. No other effects on test scores were found for African American students or for students of any other racial or ethnic background.

In the Washington, D.C., study, the academic achievement of African American students who used a voucher to switch from public to private school was not consistently higher over the 3 years of the study than that of the control group of African American students who remained in public schools. In the first year of the study, African American voucher users scored better than control group students in math, but worse in reading. In the second year, African American voucher users scored significantly better in both math and reading than the students who remained in public schools. In the third and final year of the study, however, there was no difference between the combined math and reading test scores of African American voucher users and those of the control group. Because a large majority of participants in the Washington study were African American, there were not enough students of other ethnic groups to make a reliable estimate of program effects.

Several limitations prevent reaching more definitive conclusions from this body of research. Substantial numbers of both voucher recipients and nonrecipients who were tested at baseline did not return for follow-up testing in subsequent years (program attrition). The researchers compensated for this by weighting the results of those who did return, but in Dayton and Washington, the problem of attrition from the studies was great enough that it was unclear if the procedures used could address it sufficiently. In addition to the problem of attrition from the study, the smaller initial sample size in Dayton may have resulted in estimates that were less precise and not statistically significant. In the Washington, D.C., study, another factor limited definitive study conclusions. Specifically, only 68 percent of those who were offered a voucher and who returned for follow-up testing the first year had actually used the voucher to attend a private school. This number declined to 47 percent and 29 percent in the second and third years, respectively. The analytical procedures used in the studies are more effective when a higher percentage of students who were offered vouchers actually use them to attend a private school. The New York study had the fewest problems with attrition and voucher use, and the estimated effects from the New York study are probably the most reliable. Finally, to our knowledge, at the time this report was written, only the New York data had been examined by researchers who were not part of the original research team. Confidence in the conclusions drawn from these studies will be enhanced when other researchers reanalyze these data and examine the assumptions underlying the original research.

The findings of the studies show positive achievement gains for low- income African American voucher users in New York City who applied for vouchers while in grades 1-4; however, the findings do not provide evidence that African American students elsewhere, or any other students, would realize achievement gains if they were offered vouchers to attend private schools. Differences in how voucher programs are designed and implemented, as well as differences in the participating students and in the local public and private schools affected by such programs, make it impossible to predict the effects of larger scale programs or programs similar in scale in different cities. The authors of the studies caution against generalizing from these results to a larger scale program involving all children in a large urban school system. They point out that only a small percentage of low-income students in the three cities received vouchers, and that the outcomes of a larger program could be quite different if the applicants to the study programs differ from eligible public school students in general. Additionally, the low-income families in the studies may represent an even more distinct group: low-income as defined by program eligibility requirements, but able to pay at least the minimum amount of the private school tuition required by the program.

Parents Consistently Reported Satisfaction with Private School Instructional Programs, Teachers, and Environment

In addition to measuring student academic achievement, the studies of privately funded voucher programs in New York, Dayton, and Washington, D.C., used surveys to measure parents’ perceptions of their children’s schools. The research team found that parents who used a voucher to send their children to private schools were more likely to be satisfied with their children’s education overall and perceived their children’s schools to be better on a number of indicators, compared to parents in the control groups. These findings held true for all parents of voucher users, not only for African Americans. In all three cities in each year for which data are available, parents of voucher users were more likely than parents of control group students to give their child’s school an “A” on an A to F scale. In all three cities, in at least one study year, when asked about specific aspects of their children’s schools, the parents of voucher users were more likely than the parents of control group students to say they were “very satisfied” with school safety, teaching, and school curricula. In all three years in New York, and in the second year in Dayton, the parents of voucher users were more likely to report being very satisfied with the academic quality of their child’s school than were the parents of students who did not receive a voucher. Parents of voucher users were also more likely to be very satisfied with discipline in their child’s school than were the parents of control group students in all 3 years of the New York study and in the first year of the Dayton and Washington studies.

According to parents surveyed, the private schools attended by voucher users in New York; Dayton; and Washington, D.C., had significantly different characteristics than the public schools attended by the control group students. For example, in all three studies in every year for which data were available, the parents of voucher users reported their children’s schools had fewer students than did parents of the control group students. In at least 1 year for each of the three cities, parents of voucher users also reported that their children’s schools had smaller classes, were more likely to offer individual tutoring, and communicated with parents more frequently than did parents whose children remained in public schools. However, all three studies found that parents of voucher users reported that their children’s schools were less likely to have certain facilities. For example, based on parents’ reports, all three studies in at least 1 year found that the private schools used by voucher families were less likely than public schools attended by the control group students to have a nurse’s office. In New York and Washington, parents of voucher users were also less likely to report that their children’s schools had a cafeteria or offered programs for students with learning problems and non-English speakers than were parents of the control group students.

The studies of privately funded voucher programs in New York; Dayton; and Washington, D.C., found that the parents of voucher users reported less disruption in their children’s schools compared to the parents of control group students. Parents of voucher users were less likely than parents of control group students to report that fighting, truancy, cheating, or destruction of property were serious problems in their children’s schools. These results were true in all three studies in each year for which results were available, except for the second year of the Washington study, when only fighting showed a statistically significant difference.

As in their analysis of test scores, several issues prevented the studies from resulting in more precise estimates of the effects of the voucher programs on parental perceptions of their children’s education. In Dayton and Washington, many results that were statistically significant in the first year of the studies were no longer significant in the second year. This may have been due to the low rates of study participants that returned for follow-up in both cities, the relatively small sample size in Dayton, and the low rate of voucher use and private school attendance by those offered vouchers in Washington. Despite these limitations, many of the effects were strong enough that statistically significant differences were found between the parents of voucher users and the parents of control group students. As with the analysis of test scores, the New York study had the least problems with participants not returning for follow up and families declining voucher offers, and probably produced the most reliable estimates.

These three studies constitute an important first step toward understanding the effect of school vouchers on certain low-income (and in the case of test score effects, African American) students. However, their results cannot be taken as evidence of the effects of vouchers on other types of students in any other settings. Furthermore, to our knowledge, at the time this report was written, only the New York data had been examined by anyone other than the original research team. Further analyses of these data by other researchers and additional high quality studies in other settings and involving different types of students and schools are important next steps in informing this ongoing public debate.

Agency Comments

We received comments on a draft of this report from the Department of Education. These comments are in appendix II. Education generally agreed with the report and provided technical comments that we incorporated where appropriate. Education also agreed that more research is needed on the effects of expanded parental choice. We also received technical comments from the researchers at Harvard and Mathematica whose work we assessed and from officials at the Children’s Scholarship Fund and Children First America. These comments were also incorporated where appropriate.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 16 days from the report date. At that time, we will send copies of this report to the Secretary of Education, appropriate congressional committees, and other interested parties. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions or wish to discuss this material further, please call me at (202) 512-7215 or David Bellis at (415) 904-2272.

Description of the Privately Funded Voucher Studies Reviewed

The studies of privately funded voucher programs in New York; Dayton, Ohio; and Washington, D.C., were similar and rigorous in both design and implementation. All three were longitudinal studies, tracking participants over a 2 or 3 academic year period. In all three cities, researchers awarded vouchers randomly by lottery, thus maximizing the chances that those who received a voucher and those who did not receive a voucher (control group) were equivalent on those characteristics that could affect test score outcomes. At the outset of each study, researchers collected baseline data—information about the participants before the voucher program began—to help ensure equivalence of study groups and to compare against later outcomes. In addition to basic demographic information, the baseline data collected by the researchers included student achievement and parental satisfaction data. The researchers administered the math and reading sections of the Iowa Test of Basic Skills to applicant students who would be entering grades two and higher in the coming fall, when voucher use began. While the students were taking the tests, the accompanying parents and guardians filled out a survey that included questions about their satisfaction with a variety of aspects of their children’s education. The researchers required applicants to participate in the baseline tests and surveys as a condition of entering the random lottery to win a privately funded voucher.

In each subsequent year of the studies, researchers again administered math and reading tests to study participants and asked their parents to fill out surveys. The researchers invited both voucher and control group members back for this follow-up data collection. To ensure that as many participants as possible returned for follow-up, the researchers offered incentives for the control group members and told voucher group members that their participation was required as a condition of continuing to receive a voucher. Each year, the researchers analyzed the test scores to look for achievement differences between voucher and nonvoucher students and analyzed the survey results for differences in parental satisfaction levels between the two groups.

For all three studies, the researchers took two steps to ensure the validity of their results. First they weighted their data to adjust for participants who failed to return for follow-up data collection. The researchers used the known characteristics of study participants (e.g., family size, mother’s education, race, etc.) to estimate the probability that any given individual would return for follow-up testing. Using these calculated probabilities, they modified their data so that the responses of those who returned for follow-up testing, who were similar on these characteristics to the nonreturners, were counted more heavily in the analysis. This means that participants who did not return were represented by returning participants who were similar to them.

The second way the researchers improved the quality of their results was by using a statistical technique called an instrumental variable analysis to compensate for the fact that not all students who were offered a voucher actually used it to attend private school, and some control group students who were not offered a voucher attended private school anyway. Because of this crossover between the voucher offer and control groups, a comparison of outcomes between those offered a voucher and those not offered a voucher gave only the effect of the offer itself, not the effect of actually using a voucher to attend a private school. The instrumental variable analysis involved a statistical procedure in which the researchers used a student’s status as a voucher recipient or nonrecipient to predict whether that student attended private school. The researchers then used these results in a second statistical model, which gave the effect of actually using a voucher to attend a private school on test scores and survey results.

Tables 4, 5, and 6 summarize the results and key strengths and weaknesses of the New York City; Dayton; and Washington, D.C., studies. They also provide our interpretation of the degree to which the studies provide evidence of any effect that voucher use had on test scores and parental satisfaction.

Comments from the Department of Education

GAO Contacts and Staff Acknowledgments

GAO Contacts

David D. Bellis (415) 904-2272 (bellisd@gao.gov) Susan J. Lawless (206) 287-4792 (lawlesss@gao.gov)

Staff Acknowledgments

In addition to the individuals named above, Andrew S. Bauck, Dianne L. Whitman Miner, Shana B. Wallace, Michele C. Fejfar, Douglas M. Sloane, Gail S. MacColl, Jonathan H. Barker, Barry J. Seltser, Patrick J. Dibattista, and Stanley G. Stenersen made key contributions to this report.

Bibliography

Reports on the New York Study

Peterson, Paul E., David Myers, Josh Haimson, and William G. Howell. Initial Findings from the Evaluation of the New York School Choice Scholarships Program. Washington, D.C.: Mathematica Policy Research, November 1997.

Peterson, Paul E., David Myers, and William G. Howell. An Evaluation of the New York City School Choice Scholarships Program: The First Year. Washington, D.C.: Mathematica Policy Research, October 1998.

Myers, David, Paul Peterson, Daniel Mayer, Julia Chou, and William G. Howell. School Choice in New York City after Two Years: An Evaluation of the School Choice Scholarships Program. Washington, D.C.: Mathematica Policy Research, August 2000.

Mayer, Daniel P., Paul E. Peterson, David E. Myers, Christina Clark Tuttle, and William G. Howell. School Choice in New York City after Three Years: An Evaluation of the School Choice Scholarships Program. Washington, D.C.: Mathematica Policy Research, February 2002.

Reports on the Dayton Study

Peterson, Paul E., Jay P. Greene, William G. Howell and William McCready. Initial Findings from an Evaluation of School Choice Programs in Washington, D.C. and Dayton, Ohio. Paper presented at the annual meeting of the Association of Public Policy and Management, New York, N.Y.: October 1998.

Howell, William G. and Paul E. Peterson. School Choice in Dayton, Ohio: An Evaluation after One Year. Paper presented at the Conference on Vouchers, Charters and Public Education sponsored by the Program on Education Policy and Governance, Harvard University, Cambridge, Mass.: March 2000.

West, Martin R., Paul E. Peterson and David E. Campbell. School Choice in Dayton, Ohio After Two Years: An Evaluation of the Parents Advancing Choice in Education Scholarship Program. Cambridge, Mass.: Program on Education Policy and Governance, Harvard University, August 2001.

Reports on the Washington, D.C., Study

Peterson, Paul E., Jay P. Greene, William G. Howell and William McCready. Initial Findings from an Evaluation of School Choice Programs in Washington, D.C., and Dayton, Ohio. Paper presented at the annual meeting of the Association of Public Policy and Management, New York, N.Y.: October 1998.

Wolf, Patrick J., William G. Howell, and Paul E. Peterson. School Choice in Washington, D.C.: An Evaluation after One Year. Paper presented at the Conference on Vouchers, Charters and Public Education sponsored by the Program on Education Policy and Governance, Harvard University, Cambridge, Mass.: March 2000.

Wolf, Patrick J., Paul E. Peterson and Martin R. West. Results of a School Voucher Experiment: The Case of Washington, D.C., after Two Years. Paper presented at the annual meeting of the American Political Science Association, San Francisco, Calif.: August 2001.

Reports on the Studies in All Three Cities

Howell, William G., Patrick J. Wolf, Paul E. Peterson, and David E. Campbell. Test-Score Effects of School Vouchers in Dayton, Ohio; New York City; and Washington, D.C.: Evidence from Randomized Field Trials. Paper presented at the annual meeting of the American Political Science Association, Washington, D.C.: September 2000.

Howell, William G. and Paul E Peterson, with Patrick J. Wolf and David E. Campbell. The Education Gap: Vouchers and Urban Schools. Washington, D.C.: The Brookings Institution, 2002.

Howell, William G., Patrick J. Wolf, David E. Campbell, and Paul E. Peterson. School Vouchers and Academic Performance: Results from Three Randomized Field Trials. Journal of Policy Analysis and Management, vol. 21, no. 2 (2002): 191-217.

GAO’s Mission

The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily E-mail alert for newly released products” under the GAO Reports heading.

Order by Mail or Phone

To Report Fraud, Waste, and Abuse in Federal Programs

Public Affairs