School Vouchers: Characteristics of Privately Funded Programs
Published: 2002-09-10T00:00:00

Fast Facts:

Highlights:
section_title: What GAO Found
paragraphs: Privately funded voucher programs are a new development in the nation's experiment with school vouchers. These programs, started in the early 1990s, provide low-income families with private, nongovernmental tuition assistance at private schools for kindergarten through grade 12. Although private schools have long offered various forms of financial assistance, many of these privately funded voucher programs are different from traditional scholarship efforts in two key respects: they are open to any applicant solely on the basis of family income level, and recipients are free to decide which schools their children should attend. The 78 privately funded voucher programs GAO reviewed shared numerous characteristics and faced common challenges, but the programs varied widely in the dollar amount of the vouchers awarded and the number of students receiving them. The average voucher amount in school year 2001-2002 ranged widely among programs, from $600 to $2,000 per student, and program size ranged from as few as four students to over 3,000 students. Beyond the information needed to determine eligibility and conduct financial oversight, most programs reported collecting little data about participating students, their families, or the schools they attended. Program officials said that voucher programs face two major challenges--sustaining programs for the long term and retaining students in the programs. Some programs have begun developing ways to address these challenges. Rigorous evaluations of privately funded voucher programs in New York, Dayton, and Washington, D.C., provided some evidence that African Americans students who used vouchers to attend private schools showed greater improvements in math and reading than students in the comparison group and have also found that the parents of voucher users of all racial and ethnic groups were consistently more satisfied with their children's education than parents of comparison group students.

section_title: Letter
paragraphs: Privately funded voucher programs are a relatively new development in the  nation’s experiment with school vouchers. These programs, started in the  early 1990s, provide low-income families with private, nongovernmental  tuition assistance at private schools for kindergarten through grade 12.  While private schools have long offered various forms of financial  assistance, many of these privately funded voucher programs are different  from traditional scholarship efforts in two key respects: they are open to  any applicant solely on the basis of family income level, and recipients are  free to decide which schools their children should attend. Such programs  now serve about 46,000 of the estimated 53 million school age children  nationwide, awarding nearly $60 million in tuition assistance.
Privately funded voucher programs are becoming part of an evolving  approach to achieving greater school choice. The Congress continues to  show a strong interest in school choice issues—debating a number of  choice issues, enacting choice provisions in Title I of the No Child Left  Behind Act of 2001, and enacting the Public Charter Schools program and  the Magnet Schools Assistance program. It was within this context that you  asked us to provide information on both publicly funded and privately  funded voucher programs. We addressed publicly funded voucher  programs in a prior report. This report on privately funded voucher  programs focuses on answers to the following questions:  What are the characteristics of privately funded school voucher  programs, including such factors as amount of tuition assistance,  determination of student eligibility, and long-term challenges?
What is known about the academic performance of students  participating in these programs and the degree of parental satisfaction  with the programs?
To report on the characteristics of the programs and long-term challenges,  we focused on the 78 privately funded voucher programs that were open to  all low-income applicants and allowed recipients a wide choice in deciding  what schools their children would attend. We developed our list in  consultation with officials from two major national organizations with  which many programs are affiliated and summarized information on these  programs collected by the organizations. In addition, we searched the  literature, reviewed reports and other documents, and corroborated  information through on-site and telephone interviews with local programs  and national organization officials. To determine what is known about  student academic performance and parent satisfaction associated with  privately funded school voucher programs, we relied on existing studies.  We identified 11 studies, but for our review only included those that:  (1) analyzed student academic achievement and/or parental satisfaction,  (2) compared voucher students or their parents with an appropriate control  or comparison group of students or parents, and (3) gathered data on  student achievement or parental satisfaction both before and after the  vouchers were awarded. For our analysis, we included findings from those  studies that reached the 95 percent confidence level as statistically  significant effects. (There is a 95 percent certainty that these results would  not occur by chance alone.) See appendix I for detailed information about  these studies. Two social scientists examined each study to assess the  adequacy of the samples and measures employed, the reasonableness and  rigor of the statistical techniques used to analyze them, and the validity of  the results and conclusions that were drawn from the analyses. We  conducted our work between October 2001 and April 2002 in accordance  with generally accepted government auditing standards.
subsections: section_title: Background
paragraphs: Privately funded school voucher programs got their start in 1991 when an  Indianapolis businessman founded a local program that provided tuition  assistance to about 750 low-income students in grades kindergarten  through 8 (K-8). After the initiation of the Indianapolis program, a number  of other communities also established privately funded voucher programs  based on the Indianapolis model and used funding from local donors. In  1994, a local voucher program that had been founded in San Antonio  obtained a grant to establish a national clearinghouse organization—now  called Children First America (CFA)—to assist new and existing local  programs with funding and technical assistance. By 1997, there were  31 local programs offering over 12,000 privately funded vouchers to K-12  students in 18 states plus the District of Columbia.
In 1998, a second national organization, the Children’s Scholarship Fund  (CSF), was established. CSF used the word scholarship in its name—rather  than voucher—to distinguish itself from publicly funded voucher  initiatives.  CSF helped found several new local voucher programs and  established partnerships with many existing programs. In its first year, CSF  provided one-to-one matching funds for a total of 40 programs to provide  vouchers for low-income students for a period of 4 years. Beginning in  school year 1999-2000, CSF-funded programs provided school vouchers to  about 40,000 K-8 students. CSF currently provides support to 47 affiliated  programs.  Nine of the programs are administered by the CSF national  office, including a program that provides vouchers to students not residing  in an area with a local program. Like CSF, CFA continues to be a source of  funding support and technical assistance to local privately funded voucher  programs. The two organizations work with each other in a number of  efforts but differ somewhat in their stated missions. CSF offers tuition  assistance to needy families in what it describes as a purely charitable  effort, while CFA states that its mission is to promote parental choice,  including both privately and publicly funded options.
In addition to programs offering vouchers for low-income students to use,  with virtually no restriction as to their choice of private schools, there are  other privately funded tuition support or scholarship programs. However,  these programs fall outside the scope of our study because they limit  student eligibility or school choice in different ways. For example, we did  not review traditional scholarships offered by private schools, merit-based  programs, or programs that limit choice to schools of a particular religious  denomination.
According to a recent survey done by CSF, 78 privately funded school  voucher programs used family income as their only eligibility criteria and  permitted families to use their award at nearly any private school. Although  these programs receive their funding from private individuals and groups— in contrast with publicly funded school voucher programs—they  nonetheless may affect public funding in several ways. Most programs are  not-for-profit organizations and, as such, are eligible for the associated  federal tax benefits. Programs may also have an impact on local public  school funding because much state and some federal funding to school  districts is allocated on the basis of formulas incorporating the number of  students attending the schools. Some states have laws in place to provide  state tax credits to individuals or businesses for their contributions to  tuition assistance organizations, including privately funded voucher  programs.
section_title: Programs Shared Characteristics and Challenges, but Varied in Size and Voucher Amounts
paragraphs: The privately funded voucher programs we reviewed had a number of  common characteristics, including several long-term challenges, but they  varied widely in the dollar amounts of the vouchers they awarded and  number of students served. Most programs had local financial support.  Programs also used similar methods to determine student eligibility and  many programs used a sliding scale based on family size and income.  Nearly all programs required parents to contribute at least a specified  minimum amount toward the cost of tuition. Many automatically accepted  siblings and some ensured multiyear funding as part of an emphasis on  helping families. Most programs collected only the information they  needed to determine student eligibility and administer the programs, such  as family income and number of siblings. However, information on other  student and family characteristics was limited. Looking to the future,  program officials said voucher programs face two major challenges— specifically, sustaining programs for the long term and retaining students in  the programs. Some programs we contacted have begun developing ways  to address these challenges.
subsections: section_title: Programs Had Many Common Characteristics but Differed Widely in Numbers of Students Served and Voucher Amounts
paragraphs: Although scattered across the United States, the local privately funded  voucher programs we studied shared many things in common. For  example, most were largely supported by local donations, and the majority  of the 47 programs affiliated with CSF raised funds locally to receive  matching funds from that national organization. Often programs were  supported by one, or a few, “anchor” donors—people or organizations that  had committed to making a substantial, and often multiyear, financial  contribution to support vouchers in their community. For example, a single  local donor committed to provide the matching funds for the Children’s  Scholarship Fund of Seattle/Tacoma for 4 years, and in San Antonio two  local donors have provided nearly all of the funding for two voucher  programs serving that area.
Programs also had similar voucher recipient selection processes. Most  programs selected recipients by lottery from among all eligible applicants,  although a few selected them on a first-come-first-serve basis. According to  officials, some programs awarded vouchers to all eligible students in the  first year because they had more money for vouchers than they had  qualified applicants. As more people became aware of the vouchers and  demand grew, nearly every program adopted a lottery system.  Many  programs have established separate lotteries for private and public school  students.  Several program officials pointed out that awarding tuition  assistance to students already in private school was not accomplishing the  mission of extending school choice because those families had already  exercised choice. Programs we contacted typically awarded a larger  percent of their vouchers to students transferring from public schools and  established a specific limit to the percent that would be awarded to  students already in private school.  For example, the Washington  Scholarship Fund, in Washington, D.C., reported that it awards at least  75 percent of its vouchers to students coming from public schools. The  program maintains two pools of applicants—one for public school students  and one for private school students—and selects students by separate  lottery from each pool.
Eligibility criteria were also similar among programs. Eligibility depended  on both family income and size. Almost all the programs targeted the  vouchers to families eligible for the federal free and reduced price lunch  program—those making less than 185 percent of the federal poverty  guidelines. However, many programs included families with incomes up to  270 percent of the poverty guidelines. Many programs also used a sliding  scale based on family income and size to determine both eligibility and how  large a voucher the family could receive. Families with higher incomes  received a smaller percentage of school tuition. The CSF national  organization has established sliding scale income eligibility standards for  its affiliated programs. See table 1 for the CSF 2001-02 income eligibility  scale. Programs also typically had a maximum voucher amount that could  be awarded. Maximum award amounts for CSF programs varied from  program to program but ranged from $1,000 to $2,100 for K-8 vouchers and  $1,000 to $2,900 for high school vouchers. Since voucher amounts did not  typically cover the full amount of tuition, the family was responsible for the  balance.
Despite the size of a voucher for which a family is eligible, most programs  required families to contribute a minimum amount toward their child’s  tuition payment—at least about $500. However, many programs do not  require families with more than one child receiving a voucher to pay the  minimum amount for each child. For example, in Buffalo, New York, the  program required each family to pay just $500 toward the tuition,  regardless of the number of children receiving a scholarship.
For many programs, including all CSF affiliated programs, siblings of  award recipients were automatically offered vouchers. In addition, the  programs typically made a commitment to the families to provide the  vouchers for multiple years, as long as the family remained eligible for the  program. For example, in Cincinnati, all children in the family can receive a  voucher, and eligible students will continue to receive a voucher for at least  4 years. Officials said accepting siblings and committing to a multiyear  voucher were part of their approach to helping the children and the  families. In addition, CSF plans to continue its commitment to families by  extending financial support for vouchers through the year 2013 to include  all younger siblings through the eighth grade.
Programs we contacted also conducted similar financial oversight  activities. They typically paid voucher amounts directly to the private  schools in several tuition installments throughout the school year. Prior to  each payment, programs received confirmation from participating private  schools that the voucher students were still enrolled. Many programs also  required confirmation that the students were attending class regularly and  some required parents to sign a confirmation form each time a payment  was made. For example, the Houston Children’s Educational Opportunity  Foundation pays voucher installments to participating schools every month  during the school year. Prior to each payment, the program official sends a  commitment form to each school. A school official and the parents of each  voucher student are required to sign it. The CSF national organization  directs its affiliated programs to have the parents of voucher students visit  their schools to sign a Scholarship Verification Report. The form is also  signed by the school principal and is returned to the program before  payment is made. Program officials are instructed to retain the forms as an  audit trail of the scholarships and proof that they were verified.
Although the programs shared many similar characteristics, they differed  in the average dollar amount of their voucher awards.  Award amounts are  based, in part, on school tuition and the tuition at participating private  schools varied considerably among program locations.  For example, for  K-8 voucher recipients in school year 2001-02, the average private school  tuition in Buffalo, New York, was about $1,500 and the average K-8 voucher  award amount for the Buffalo program was $620.  The maximum Buffalo  program award was $1,000, depending on family size and income.  In  contrast, the private schools attended by K-8 voucher students in the  Atlanta CSF program charged an average tuition of about $4,300 and the  average K-8 voucher amount for the Atlanta program was $1,663—with an  award maximum of $2,100.   The average percent of tuition covered by CSF  program vouchers ranged from about 20 percent in Pittsburgh to about  65 percent in Baton Rouge.  Overall, the average K-8 voucher award for the  47 CSF affiliated programs was about $1,100, according to CSF officials,  and the average per-child K-8 tuition among all programs that receive  funding from the CSF was about $2,550.  Average high school voucher  awards from individual programs also varied widely from $1,009 to $2,411.   The Department of Education reports that the average Catholic or other  religious private school tuition for grades K-12 in school year 2000 was  about $2,800 and for private nonsectarian schools was about $8,900.
The number of vouchers awarded per program also varied considerably.  For example, programs in Phoenix; St. Louis; Erie, Pennsylvania; and  Midland, Texas, awarded 10 or fewer scholarships each, while the largest— the CSF local program in New York City—awarded over 3,100. Seventeen  of the local programs we studied awarded more than 1,000 vouchers. In  addition, the national CSF organization awarded about 3,600 vouchers,  through its national program, to eligible students not living in a local area  served by a voucher program.
Information on the students or families who receive privately funded  vouchers is limited, as is information on the private schools the students  attend. Most programs we contacted reported that they collected only the  student and family data needed to determine student eligibility, and the  administrative data needed to manage and oversee program activities. The  eligibility data collected by programs included family income, size, and  place of residence, and the administrative and oversight data included the  amounts of the vouchers awarded and the tuition at the schools voucher  students attended. Information on the private schools students attended  was often limited to the school affiliation—for example, Catholic,  nondenominational Christian, or independent. Programs reported that the  majority of the schools voucher students attend are religiously affiliated.  Most programs we contacted did not collect information about the public  schools voucher students previously attended, and most did not gather  data to evaluate program outcomes or results. While some program  officials said they saw value in such analyses, several said that evaluations  were costly and that they preferred to use their limited funds for student  vouchers.
section_title: Some Programs Initiated Activities to Meet Future Challenges
paragraphs: Through our discussions with program officials, we identified two major  challenges faced by privately funded voucher programs—ensuring program  sustainability and maximizing student retention rates. We found that many  of the programs had recognized these challenges and identified or  implemented activities and initiatives to address them.
subsections: section_title: Sustaining Programs for the Long Term
paragraphs: Most programs we contacted were concerned with sustaining their  activities over the long term. Officials generally said they expected their  programs to continue operating for the foreseeable future or indefinitely.  Some had determined that certain program modifications would be needed  to foster sustainability. Specifically, these programs recognized that they  should broaden their funding base and have more community  representation on their boards of directors. Several programs had been  founded and initially funded largely by a few local donors. In many cases,  the donors had agreed to provide funding for 4 years. One program official  said he believed there was a danger of what he called “donor fatigue” and  determined it could be averted by expanding the number of donors— particularly small contributors. Some programs also saw the benefits of  broadening community representation on their boards of directors. For  example, the Washington Scholarship Fund told us it is currently building a  broad donor base and is expanding its board of directors—which had been  limited to large donors—to include members who may not be able to  provide funding, but are actively involved in the community. Additionally,  an official at Children First Utah told us that to help ensure stability and  sustainability, the program has established a board of directors with broad  community representation, including representatives from several large  corporations and a public relations firm, community activists, and a  representative from the Hispanic community. In addition, Children First  Utah has implemented efforts to broaden its funding base, including direct  mail solicitations, media public relations efforts, and several funding  proposals to foundations.
The two national private voucher organizations, CFA and CSF, support  local programs in both implementing and sustaining their activities. In  addition to providing matching funds to some programs, they also provide  technical assistance. CFA offers a detailed “how to” manual to new  programs and will work on site with programs as needed at no cost. CFA  officials told us they also work with existing programs to help them solve  problems and enhance program stability, and they emphasized that their  focus is on the sustainability of the programs. For example, CFA assists  programs in broadening the community representation on their boards and  among their donors. According to CSF officials, that organization provides  its affiliates with extensive guidance and technical support for program  administration activities such as scholarship disbursement procedures,  enrollment verification, and financial planning. CSF also holds an annual  conference for its affiliates, provides a detailed program management  manual, and assists affiliates in the use of a sophisticated program  administration database.
section_title: Maximizing Student Retention
paragraphs: Maximizing the retention rate among voucher recipients was another  challenge faced by the programs we contacted. In many programs, about  20 percent or more of the voucher recipients leave the program each year.  However, according to a CSF official, only a small percent of voucher  recipients leave CSF affiliated programs because they graduate from the  eighth grade. Program officials said that other reasons recipients leave  might include moving or dissatisfaction with the school. Many said they  believe family financial difficulties are a common cause of attrition.  According to some officials, families are sometimes unprepared for all the  costs of having children in private schools—for example, uniforms, books,  and activity fees. Additionally, parents of voucher students may be  confronted with financial emergencies that preclude their paying their  required portion of the tuition and continuing in the program. Some  programs we contacted attempted to determine the reasons students drop  out and several had initiated activities to promote student retention. For  example, the CSF program in New York City and the Washington  Scholarship Fund have implemented “stay in school funds” to help parents  who are experiencing financial emergencies such as unexpected medical  expenses. Washington Scholarship Fund, which served nearly 900 families  in school year 2001-02, reported that it has provided such funding  assistance to about 26 families so far. The Horizon program in San Antonio,  Texas, initiated a voucher parent group to increase the student retention  rate. According to a program official, the group, “Las Comadres,” meets  weekly to discuss aspects of their children’s education and learn from each  other about schools and curricula. The meetings also include guest  speakers on such topics as preparing children for high school and college,  parental rights and duties, and children’s health issues. The Horizon  program director and staff also attend the meetings to facilitate and answer  questions.
section_title: Recent Research on Privately Funded Voucher Programs Has Explored Academic Achievement and Parental Perceptions of Their Children’s Schools
paragraphs: We identified three studies of privately funded voucher programs—in New  York; Dayton, Ohio; and Washington, D.C.—that were rigorous enough to  meet our criteria for inclusion. These studies all included quantitative  analysis of program effects, used appropriate comparison groups, and  gathered data before vouchers were awarded, as well as after, to assess the  equivalence of study groups and to track program effects. In general,  researchers found that African American students in New York City who  used vouchers to attend private schools exhibited more substantial  improvements on test scores in math and reading than African American  students in the comparison group. For students from other ethnic groups,  there were no sizable or significant differences in the test score gains of  voucher users and comparison group students, or there were too few  participants to draw any conclusions. These studies have also found that  the parents of voucher users in all programs were consistently more  satisfied with their children’s education, regardless of ethnic group.  Although these results suggest positive achievement effects for some  African American students using vouchers, they cannot be generalized  beyond the specific programs, schools, and geographic areas where the  studies were conducted or the low-income group of families studied.
subsections: section_title: Studies Used Same Rigorous Methodology
paragraphs: The studies of privately funded voucher programs in New York; Dayton,  Ohio; and Washington, D.C., all used impact evaluations to study the effect  of vouchers on private school attendance. Impact evaluations isolate a  program effect from the effects of other factors that could influence  participants’ outcomes. To isolate the program’s effect, impact evaluations  divide participants into two groups: those who receive program services  and a similar group who do not (the control or comparison group). Some  impact evaluations assign participants randomly to one group or the other.  Random assignment increases the likelihood that the two groups are  roughly equivalent on all characteristics that could affect outcomes. It  helps ensure, for example, roughly equal numbers of very low-income  students in both groups, rather than those students being concentrated  more heavily in one group or the other. The use of random assignment to  create equivalent groups allows researchers to compare tests scores (or  other outcomes) for the two groups and attribute any differences to the  program services rather than other factors, such as differences in family  income.
The studies of voucher programs in New York, Dayton, and Washington  randomly assigned applicant families to the voucher offer group and the  control (no voucher offer) group. Before each voucher program began,  students whose families hoped to receive a voucher took the Iowa Test of  Basic Skills in math and reading, and their parents completed surveys  about family background characteristics and their children’s educational  experiences. The researchers also screened families to determine if they  were eligible to participate on the basis of their incomes. Test scores and  information about family background provided researchers with a baseline  for making later comparisons. Families who met the income qualifications  and participated in the testing and survey sessions were eligible to enter a  lottery to decide which families would receive voucher offers and which  would not. Because the researchers ensured that the lotteries were  completely random, the two groups—voucher offer recipients and  nonrecipients—should have roughly equivalent average student test scores  and family backgrounds at the outset. To collect follow-up data from both  voucher offer recipients and nonrecipients in subsequent years of the  studies, the researchers used the same testing and survey procedures.  Only students attending a public school before the lottery were included in  the follow-up testing and analysis. Table 2 provides additional details about  the studies, including the numbers of applicants and vouchers awarded.
While the use of control groups helps isolate the cause of any outcome,  researchers must still address circumstances that threaten the quality of  their results. In the New York, Dayton, and Washington studies, the  researchers addressed two such circumstances. First, not all of those  offered a voucher used their voucher to attend a private school, and some  students who did not receive a voucher offer ended up in a private school  anyway. This meant that a comparison between the voucher offer group  and control group, as designed, revealed the effect of offering a voucher to  eligible students in a city, not the effect of actually using it to attend a  private school.  To answer the question of the effects of vouchers on those  actually using them, the researchers used a data analysis technique that  allowed them to compare the outcomes of those who actually used a  voucher to attend private school to those of students who were most likely  to have attended private school had they been offered a voucher.  While  both the voucher offer and voucher use analyses are valid, they answer  different questions. The researchers reported primarily on the voucher use  analysis. Second, substantial numbers of those who initially agreed to  participate in the studies dropped out in subsequent years. When this  occurs, those who remain in the study may be different from those who  left, which can threaten the equivalence of the two study groups, making it  difficult to assess the effect of the voucher offer. The researchers used a  standard statistical procedure to minimize the possibility that the loss of  some participants undermined their results.
In all three studies, the demographic characteristics of study participants  surveyed at baseline reflected the programs’ targeting of vouchers to low- income families, most of whom lived in inner cities. Although these families  met program eligibility requirements as low-income, they were able to pay  at least a portion of the private school tuition. In both Dayton and  Washington, the average income of a participant family that received a  voucher and used it to attend private school was approximately $18,000.  However, in New York, the average family income of participating voucher  users was about $10,000, possibly reflecting the more stringent income  criteria of the New York program. In all three cities, ethnic minorities made  up a large majority of voucher applicants participating in the studies.  According to the survey questions, study participants were asked to report  their race as either Black/African American (non-Hispanic), White (non- Hispanic), Puerto Rican, Dominican, other Hispanic, American Indian or  Alaskan Native, Chinese, other Asian or Pacific Islander, or other. Thus, the  African Americans and Whites in the study are likely to be non-Hispanics  only. (See table 3.) Among study participants who made use of the voucher  offered to them, a majority attended Catholic schools in all three cities.
section_title: African American Voucher Users in New York City Had Improved Test Scores
paragraphs: Of the three studies, the New York study showed the greatest positive  effect on voucher users’ academic achievement. In each of the 3 years of  the New York study, African American students who used a voucher to  attend private school performed better on the reading exams than a  comparable control group. African American voucher users also performed  better in math than those who did not receive a voucher and remained in  public schools. This improvement in math scores was found in years  1 and 3, but not year 2, of the study. Achievement gains, however, did not  extend to Hispanic voucher users. The New York study found that Hispanic  students who used privately funded vouchers to switch from public to  private school had math and reading scores that were statistically the same  as those of the control group. The New York study did not have sufficient  numbers of white students or students from other ethnic groups to  estimate program effects for them. In addition, when the New York study  sample is considered as a whole—pooling together African Americans,  Hispanics, and white students—there is no significant difference in  achievement gains between voucher users and nonusers. Although the  researchers conducted additional analyses to explain why these positive  effects were seen for African American students and not for others, the  cause of this difference remains unclear. In Dayton, African American  voucher users showed improvements in reading in the second year of the  study, an effect that approached, but did not reach, statistical  significance. No other effects on test scores were found for African  American students or for students of any other racial or ethnic background.
In the Washington, D.C., study, the academic achievement of African  American students who used a voucher to switch from public to private  school was not consistently higher over the 3 years of the study than that of  the control group of African American students who remained in public  schools. In the first year of the study, African American voucher users  scored better than control group students in math, but worse in reading. In  the second year, African American voucher users scored significantly  better in both math and reading than the students who remained in public  schools. In the third and final year of the study, however, there was no  difference between the combined math and reading test scores of African  American voucher users and those of the control group. Because a large  majority of participants in the Washington study were African American,  there were not enough students of other ethnic groups to make a reliable  estimate of program effects.
Several limitations prevent reaching more definitive conclusions from this  body of research. Substantial numbers of both voucher recipients and  nonrecipients who were tested at baseline did not return for follow-up  testing in subsequent years (program attrition). The researchers  compensated for this by weighting the results of those who did return, but  in Dayton and Washington, the problem of attrition from the studies was  great enough that it was unclear if the procedures used could address it  sufficiently. In addition to the problem of attrition from the study, the  smaller initial sample size in Dayton may have resulted in estimates that  were less precise and not statistically significant. In the Washington, D.C.,  study, another factor limited definitive study conclusions. Specifically, only  68 percent of those who were offered a voucher and who returned for  follow-up testing the first year had actually used the voucher to attend a  private school. This number declined to 47 percent and 29 percent in the  second and third years, respectively. The analytical procedures used in the  studies are more effective when a higher percentage of students who were  offered vouchers actually use them to attend a private school. The New  York study had the fewest problems with attrition and voucher use, and the  estimated effects from the New York study are probably the most reliable.  Finally, to our knowledge, at the time this report was written, only the New  York data had been examined by researchers who were not part of the  original research team. Confidence in the conclusions drawn from these  studies will be enhanced when other researchers reanalyze these data and  examine the assumptions underlying the original research.
The findings of the studies show positive achievement gains for low- income African American voucher users in New York City who applied for  vouchers while in grades 1-4; however, the findings do not provide evidence  that African American students elsewhere, or any other students, would  realize achievement gains if they were offered vouchers to attend private  schools. Differences in how voucher programs are designed and  implemented, as well as differences in the participating students and in the  local public and private schools affected by such programs, make it  impossible to predict the effects of larger scale programs or programs  similar in scale in different cities. The authors of the studies caution against  generalizing from these results to a larger scale program involving all  children in a large urban school system. They point out that only a small  percentage of low-income students in the three cities received vouchers,  and that the outcomes of a larger program could be quite different if the  applicants to the study programs differ from eligible public school students  in general. Additionally, the low-income families in the studies may  represent an even more distinct group: low-income as defined by program  eligibility requirements, but able to pay at least the minimum amount of the  private school tuition required by the program.
section_title: Parents Consistently Reported Satisfaction with Private School Instructional Programs, Teachers, and Environment
paragraphs: In addition to measuring student academic achievement, the studies of  privately funded voucher programs in New York, Dayton, and Washington,  D.C., used surveys to measure parents’ perceptions of their children’s  schools. The research team found that parents who used a voucher to send  their children to private schools were more likely to be satisfied with their  children’s education overall and perceived their children’s schools to be  better on a number of indicators, compared to parents in the control  groups. These findings held true for all parents of voucher users, not only  for African Americans. In all three cities in each year for which data are  available, parents of voucher users were more likely than parents of  control group students to give their child’s school an “A” on an A to F  scale. In all three cities, in at least one study year, when asked about  specific aspects of their children’s schools, the parents of voucher users  were more likely than the parents of control group students to say they  were “very satisfied” with school safety, teaching, and school curricula. In  all three years in New York, and in the second year in Dayton, the parents  of voucher users were more likely to report being very satisfied with the  academic quality of their child’s school than were the parents of students  who did not receive a voucher. Parents of voucher users were also more  likely to be very satisfied with discipline in their child’s school than were  the parents of control group students in all 3 years of the New York study  and in the first year of the Dayton and Washington studies.
According to parents surveyed, the private schools attended by voucher  users in New York; Dayton; and Washington, D.C., had significantly  different characteristics than the public schools attended by the control  group students. For example, in all three studies in every year for which  data were available, the parents of voucher users reported their children’s  schools had fewer students than did parents of the control group students.  In at least 1 year for each of the three cities, parents of voucher users also  reported that their children’s schools had smaller classes, were more likely  to offer individual tutoring, and communicated with parents more  frequently than did parents whose children remained in public schools.  However, all three studies found that parents of voucher users reported  that their children’s schools were less likely to have certain facilities. For  example, based on parents’ reports, all three studies in at least 1 year found  that the private schools used by voucher families were less likely than  public schools attended by the control group students to have a nurse’s  office. In New York and Washington, parents of voucher users were also  less likely to report that their children’s schools had a cafeteria or offered  programs for students with learning problems and non-English speakers  than were parents of the control group students.
The studies of privately funded voucher programs in New York; Dayton;  and Washington, D.C., found that the parents of voucher users reported less  disruption in their children’s schools compared to the parents of control  group students. Parents of voucher users were less likely than parents of  control group students to report that fighting, truancy, cheating, or  destruction of property were serious problems in their children’s schools.  These results were true in all three studies in each year for which results  were available, except for the second year of the Washington study, when  only fighting showed a statistically significant difference.
As in their analysis of test scores, several issues prevented the studies from  resulting in more precise estimates of the effects of the voucher programs  on parental perceptions of their children’s education. In Dayton and  Washington, many results that were statistically significant in the first year  of the studies were no longer significant in the second year. This may have  been due to the low rates of study participants that returned for follow-up  in both cities, the relatively small sample size in Dayton, and the low rate of  voucher use and private school attendance by those offered vouchers in  Washington. Despite these limitations, many of the effects were strong  enough that statistically significant differences were found between the  parents of voucher users and the parents of control group students. As with  the analysis of test scores, the New York study had the least problems with  participants not returning for follow up and families declining voucher  offers, and probably produced the most reliable estimates.
These three studies constitute an important first step toward  understanding the effect of school vouchers on certain low-income (and in  the case of test score effects, African American) students. However, their  results cannot be taken as evidence of the effects of vouchers on other  types of students in any other settings. Furthermore, to our knowledge, at  the time this report was written, only the New York data had been  examined by anyone other than the original research team. Further  analyses of these data by other researchers and additional high quality  studies in other settings and involving different types of students and  schools are important next steps in informing this ongoing public debate.
section_title: Agency Comments
paragraphs: We received comments on a draft of this report from the Department of  Education.  These comments are in appendix II.  Education generally  agreed with the report and provided technical comments that we  incorporated where appropriate.  Education also agreed that more  research is needed on the effects of expanded parental choice. We also  received technical comments from the researchers at Harvard and  Mathematica whose work we assessed and from officials at the Children’s  Scholarship Fund and Children First America.  These comments were also  incorporated where appropriate.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 16 days from the  report date. At that time, we will send copies of this report to the Secretary  of Education, appropriate congressional committees, and other interested  parties. In addition, the report will be available at no charge on the GAO  Web site at http://www.gao.gov.
If you or your staff have any questions or wish to discuss this material  further, please call me at (202) 512-7215 or David Bellis at (415) 904-2272.
section_title: Description of the Privately Funded Voucher Studies Reviewed
paragraphs: The studies of privately funded voucher programs in New York; Dayton,  Ohio; and Washington, D.C., were similar and rigorous in both design and  implementation. All three were longitudinal studies, tracking participants  over a 2 or 3 academic year period. In all three cities, researchers awarded  vouchers randomly by lottery, thus maximizing the chances that those who  received a voucher and those who did not receive a voucher (control  group) were equivalent on those characteristics that could affect test score  outcomes. At the outset of each study, researchers collected baseline  data—information about the participants before the voucher program  began—to help ensure equivalence of study groups and to compare against  later outcomes. In addition to basic demographic information, the baseline  data collected by the researchers included student achievement and  parental satisfaction data. The researchers administered the math and  reading sections of the Iowa Test of Basic Skills to applicant students who  would be entering grades two and higher in the coming fall, when voucher  use began. While the students were taking the tests, the accompanying  parents and guardians filled out a survey that included questions about  their satisfaction with a variety of aspects of their children’s education. The  researchers required applicants to participate in the baseline tests and  surveys as a condition of entering the random lottery to win a privately  funded voucher.
In each subsequent year of the studies, researchers again administered  math and reading tests to study participants and asked their parents to fill  out surveys. The researchers invited both voucher and control group  members back for this follow-up data collection. To ensure that as many  participants as possible returned for follow-up, the researchers offered  incentives for the control group members and told voucher group members  that their participation was required as a condition of continuing to receive  a voucher. Each year, the researchers analyzed the test scores to look for  achievement differences between voucher and nonvoucher students and  analyzed the survey results for differences in parental satisfaction levels  between the two groups.
For all three studies, the researchers took two steps to ensure the validity  of their results. First they weighted their data to adjust for participants who  failed to return for follow-up data collection. The researchers used the  known characteristics of study participants (e.g., family size, mother’s  education, race, etc.) to estimate the probability that any given individual  would return for follow-up testing. Using these calculated probabilities,  they modified their data so that the responses of those who returned for  follow-up testing, who were similar on these characteristics to the  nonreturners, were counted more heavily in the analysis. This means that  participants who did not return were represented by returning participants  who were similar to them.
The second way the researchers improved the quality of their results was  by using a statistical technique called an instrumental variable analysis to  compensate for the fact that not all students who were offered a voucher  actually used it to attend private school, and some control group students  who were not offered a voucher attended private school anyway. Because  of this crossover between the voucher offer and control groups, a  comparison of outcomes between those offered a voucher and those not  offered a voucher gave only the effect of the offer itself, not the effect of  actually using a voucher to attend a private school. The instrumental  variable analysis involved a statistical procedure in which the researchers  used a student’s status as a voucher recipient or nonrecipient to predict  whether that student attended private school. The researchers then used  these results in a second statistical model, which gave the effect of actually  using a voucher to attend a private school on test scores and survey  results.
Tables 4, 5, and 6 summarize the results and key strengths and weaknesses  of the New York City; Dayton; and Washington, D.C., studies. They also  provide our interpretation of the degree to which the studies provide  evidence of any effect that voucher use had on test scores and parental  satisfaction.
section_title: Comments from the Department of Education
section_title: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
paragraphs: David D. Bellis (415) 904-2272 (bellisd@gao.gov) Susan J. Lawless (206) 287-4792 (lawlesss@gao.gov)
section_title: Staff Acknowledgments
paragraphs: In addition to the individuals named above, Andrew S. Bauck, Dianne L.  Whitman Miner, Shana B. Wallace, Michele C. Fejfar, Douglas M. Sloane,  Gail S. MacColl, Jonathan H. Barker, Barry J. Seltser, Patrick J. Dibattista,  and Stanley G. Stenersen made key contributions to this report.
section_title: Bibliography
subsections: section_title: Reports on the New York Study
paragraphs: Peterson, Paul E., David Myers, Josh Haimson, and William G. Howell.  Initial Findings from the Evaluation of the New York School Choice  Scholarships Program. Washington, D.C.: Mathematica Policy Research,  November 1997.
Peterson, Paul E., David Myers, and William G. Howell. An Evaluation of  the New York City School Choice Scholarships Program: The First Year.  Washington, D.C.: Mathematica Policy Research, October 1998.
Myers, David, Paul Peterson, Daniel Mayer, Julia Chou, and William G.  Howell. School Choice in New York City after Two Years: An Evaluation  of the School Choice Scholarships Program. Washington, D.C.:  Mathematica Policy Research, August 2000.
Mayer, Daniel P., Paul E. Peterson, David E. Myers, Christina Clark Tuttle,  and William G. Howell. School Choice in New York City after Three Years:  An Evaluation of the School Choice Scholarships Program. Washington,  D.C.: Mathematica Policy Research, February 2002.
section_title: Reports on the Dayton Study
paragraphs: Peterson, Paul E., Jay P. Greene, William G. Howell and William McCready.  Initial Findings from an Evaluation of School Choice Programs in  Washington, D.C. and Dayton, Ohio. Paper presented at the annual  meeting of the Association of Public Policy and Management, New York,  N.Y.: October 1998.
Howell, William G. and Paul E. Peterson. School Choice in Dayton, Ohio:  An Evaluation after One Year. Paper presented at the Conference on  Vouchers, Charters and Public Education sponsored by the Program on  Education Policy and Governance, Harvard University, Cambridge, Mass.:  March 2000.
West, Martin R., Paul E. Peterson and David E. Campbell. School Choice in  Dayton, Ohio After Two Years: An Evaluation of the Parents Advancing  Choice in Education Scholarship Program. Cambridge, Mass.: Program on  Education Policy and Governance, Harvard University, August 2001.
section_title: Reports on the Washington, D.C., Study
paragraphs: Peterson, Paul E., Jay P. Greene, William G. Howell and William McCready.  Initial Findings from an Evaluation of School Choice Programs in  Washington, D.C., and Dayton, Ohio. Paper presented at the annual  meeting of the Association of Public Policy and Management, New York,  N.Y.: October 1998.
Wolf, Patrick J., William G. Howell, and Paul E. Peterson. School Choice in  Washington, D.C.: An Evaluation after One Year. Paper presented at the  Conference on Vouchers, Charters and Public Education sponsored by the  Program on Education Policy and Governance, Harvard University,  Cambridge, Mass.: March 2000.
Wolf, Patrick J., Paul E. Peterson and Martin R. West. Results of a School  Voucher Experiment: The Case of Washington, D.C., after Two Years.  Paper presented at the annual meeting of the American Political Science  Association, San Francisco, Calif.: August 2001.
section_title: Reports on the Studies in All Three Cities
paragraphs: Howell, William G., Patrick J. Wolf, Paul E. Peterson, and David E.  Campbell. Test-Score Effects of School Vouchers in Dayton, Ohio; New  York City; and Washington, D.C.: Evidence from Randomized Field  Trials. Paper presented at the annual meeting of the American Political  Science Association, Washington, D.C.: September 2000.
Howell, William G. and Paul E Peterson, with Patrick J. Wolf and David E.  Campbell. The Education Gap: Vouchers and Urban Schools. Washington,  D.C.: The Brookings Institution, 2002.
Howell, William G., Patrick J. Wolf, David E. Campbell, and Paul E.  Peterson. School Vouchers and Academic Performance: Results from  Three Randomized Field Trials. Journal of Policy Analysis and  Management, vol. 21, no. 2 (2002): 191-217.
section_title: GAO’s Mission
paragraphs: The General Accounting Office, the investigative arm of Congress, exists to  support Congress in meeting its constitutional responsibilities and to help improve  the performance and accountability of the federal government for the American  people. GAO examines the use of public funds; evaluates federal programs and  policies; and provides analyses, recommendations, and other assistance to help  Congress make informed oversight, policy, and funding decisions. GAO’s  commitment to good government is reflected in its core values of accountability,  integrity, and reliability.
section_title: Obtaining Copies of GAO Reports and Testimony
paragraphs: The fastest and easiest way to obtain copies of GAO documents at no cost is  through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older  products. The Web site features a search engine to help you locate documents  using key words and phrases. You can print these documents in their entirety,  including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and  correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site  daily. The list contains links to the full-text document files. To have GAO e-mail this  list to you every afternoon, go to www.gao.gov and select “Subscribe to daily E-mail alert for newly released products” under the GAO Reports heading.
subsections: section_title: Order by Mail or Phone
section_title: To Report Fraud, Waste, and Abuse in Federal Programs
section_title: Public Affairs