No Child Left Behind Act: Education Actions Needed to Improve Local Implementation and State Evaluation of Supplemental Educational Services
Published: 2006-08-04T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The No Child Left Behind Act of 2001 (NCLBA) requires districts with schools that have not met state performance goals for 3 consecutive years to offer their low-income students supplemental educational services (SES), such as tutoring, if these schools receive Title I funds. SES are provided outside of the regular school day by a state-approved provider, with responsibility for implementation shared by states and districts. GAO examined (1) how SES participation changed between school years 2003-2004 and 2004-2005; (2) how SES providers are working with districts to deliver SES; (3) how states are monitoring and evaluating SES; and (4) how the Department of Education (Education) monitors and supports state implementation of SES. To collect data on SES, GAO surveyed all states and a nationally representative sample of districts with schools required to offer SES. We also visited 4 school districts, interviewed 22 SES providers, reviewed SES-related research, and interviewed Education staff.
section_title: What GAO Found
paragraphs: SES participation among eligible students increased from 12 to 19 percent between school years 2003-2004 and 2004-2005, and the number of recipients also increased, due in part to a rise in the number of schools required to offer services. Districts have used some promising practices to inform parents and encourage participation, such as offering services on school campuses and at various times. However, challenges remain, including timely and effective notification of parents and attracting providers to serve certain areas and students, such as rural districts or students with disabilities. To promote improved student academic achievement, SES providers took steps to align their curriculum with district instruction and communicate with teachers and parents, though the extent of their efforts varied. A majority of the 22 providers we interviewed worked to align SES and district curriculum by hiring teachers familiar with the district curriculum as tutors. However, at least some providers did not have any contact with teachers in about 40 percent of districts. Both providers and district officials experienced challenges related to contracting and coordination of service delivery. Providers, districts, and schools reported that greater involvement of schools would improve SES delivery and coordination, as it has in some places where this is occurring. While state monitoring of district and provider efforts to implement SES has been limited in past years, more states reported conducting on-site reviews and other monitoring activities during 2005-2006. In addition, districts have taken a direct role in monitoring providers, and their monitoring efforts have similarly increased. Although states are required to withdraw approval from providers that fail to increase student academic achievement for 2 years, many states struggle to develop meaningful SES evaluations. While a few states have completed evaluations, none provides a conclusive assessment of SES providers' effect on student academic achievement. Several Education offices monitor SES activity across the country and provide SES support to states and districts through written guidance, grants, and technical assistance. However, states and districts reported needing additional SES evaluation support and technical assistance. For example, 85 percent of states reported needing assistance with methods for evaluating SES. Many also voiced interest in Education's pilot programs that increase SES flexibility, including the one that allowed certain districts identified as in need of improvement to act as providers.

section_title: Letter
paragraphs: Title I of the No Child Left Behind Act (NCLBA) supports the academic  Title I of the No Child Left Behind Act (NCLBA) supports the academic  achievement of 16.5 million low-income students nationwide by providing  achievement of 16.5 million low-income students nationwide by providing  them with additional opportunities, such as supplemental educational  them with additional opportunities, such as supplemental educational  services (SES), if their schools are low performing. SES includes tutoring  services (SES), if their schools are low performing. SES includes tutoring  and remediation that are provided outside of the regular school day by a  and remediation that are provided outside of the regular school day by a  state-approved provider, such as a for-profit company or a community- state-approved provider, such as a for-profit company or a community- based organization. When a Title I school does not meet state performance  based organization. When a Title I school does not meet state performance  goals for 2 years, including goals set under the Elementary and Secondary  goals for 2 years, including goals set under the Elementary and Secondary  Education Act (ESEA) before the enactment of NCLBA, the district must  Education Act (ESEA) before the enactment of NCLBA, the district must  offer students the choice of transferring to another school in the district  offer students the choice of transferring to another school in the district  that is not in improvement status. When a Title I school does not meet  that is not in improvement status. When a Title I school does not meet  state performance goals for 3 or more years, the district must offer SES to  state performance goals for 3 or more years, the district must offer SES to  all of the low-income students enrolled in the school. Districts with  all of the low-income students enrolled in the school. Districts with  schools required to offer school choice and SES must set aside an amount  schools required to offer school choice and SES must set aside an amount  equal to 20 percent of their Title I funds to provide choice-related  equal to 20 percent of their Title I funds to provide choice-related  transportation and SES for eligible students in these schools. In school  transportation and SES for eligible students in these schools. In school  year 2005-2006, $12.7 billion in federal Title I funding was distributed to  year 2005-2006, $12.7 billion in federal Title I funding was distributed to  nearly all school districts and approximately half of the public schools  nearly all school districts and approximately half of the public schools  nationwide in order to improve the education of low-income students.  nationwide in order to improve the education of low-income students.
While states set NCLBA performance goals and schools are judged on the  While states set NCLBA performance goals and schools are judged on the  performance of their students, responsibility for SES implementation is  performance of their students, responsibility for SES implementation is  primarily shared by states and districts under the law. States are  primarily shared by states and districts under the law. States are  responsible for approving SES providers to serve students in their states  responsible for approving SES providers to serve students in their states  and, therefore, review provider applications to assess each provider’s  and, therefore, review provider applications to assess each provider’s  record of effectiveness and program design, including its instructional  record of effectiveness and program design, including its instructional  strategies and service costs. States are also responsible for monitoring  strategies and service costs. States are also responsible for monitoring  SES providers and evaluating their services. Districts are responsible for  SES providers and evaluating their services. Districts are responsible for  notifying parents of their child’s eligibility for SES and contracting with  notifying parents of their child’s eligibility for SES and contracting with  the state-approved providers parents select for services. At the federal  the state-approved providers parents select for services. At the federal  level, the U.S. Department of Education (Education) oversees SES  level, the U.S. Department of Education (Education) oversees SES  implementation nationwide and provides guidance and technical  implementation nationwide and provides guidance and technical  assistance.  assistance.
Although some districts were first required to offer SES in 2002-2003,  Although some districts were first required to offer SES in 2002-2003,  others did not have to offer SES until 2003-2004 or the year after, and  others did not have to offer SES until 2003-2004 or the year after, and  therefore, states and districts are at different stages of implementing the  therefore, states and districts are at different stages of implementing the  SES provisions. Further, research suggests that many face challenges to  increasing student participation, working with providers to ensure  students receive quality services, and monitoring and evaluating these  services. In response to congressional interest in these and other issues  related to early implementation of SES, and in anticipation of  reauthorization of the ESEA, we agreed to answer the following questions:  (1) How has SES participation changed between school years 2003-2004  and 2004-2005, and what actions have been taken to increase  participation? (2) How are providers working with districts and schools to  provide services that increase student achievement? (3) To what extent  are states monitoring and evaluating SES implementation? (4) How does  Education monitor state SES implementation and assist state and district  efforts?
To answer our four research questions, we collected data through several  methods. First, to gather state-level information on SES, we collected  information through a Web-based survey of state SES coordinators in all  50 states, the District of Columbia, and Puerto Rico. Further, to gather  district-level information on SES, we conducted a mail survey of district  SES coordinators from a nationally representative sample of districts with  schools required to offer SES. To ensure that we obtained information  from the largest districts with schools required to offer SES, we included  all 21 districts with 100,000 or more enrolled students in our sample. In  designing our surveys, we reviewed survey questions used by Education.  The district and state surveys were both administered between January  and March 2006. Seventy-seven percent of district SES coordinators,  including all coordinators from districts with 100,000 or more enrolled  students, and all state SES coordinators responded to the surveys. All  percentage estimates presented from the district survey have margins of  error of plus or minus 8 percentage points, unless otherwise noted. While  we did not validate specific information that states and districts reported  through our surveys, we reviewed the information to determine that their  responses were complete and reasonable and found the information to be  sufficiently reliable for the purposes of this report. To gather federal-level  information on SES implementation, we interviewed Education officials  and analyzed Education’s data on SES available from state reports and  ongoing studies.
To collect additional information from states and districts, as well as  information from schools and providers, we conducted site visits to four  school districts that had experience with SES implementation (Woodburn,  Ore.; Newark, N.J.; Chicago, Ill.; and Hamilton County, Tenn.). During the  visits, we spoke with state and district officials, as well as officials from a  total of 12 schools and 15 providers. When viewed as a group, the site visit  districts provided variation across characteristics such as geographic  location, district size, student ethnicity, and the percentage of students  with limited English proficiency or disabilities. In addition to the   15 providers we spoke with during the site visits, we also interviewed   7 for-profit SES providers operating in multiple states, for a total of   22 providers. In our surveys and other data collection efforts, we asked  questions about SES implementation during specific school years, and,  therefore, all years cited refer to school years.
See appendix I for detailed information on both of our surveys as well as  our other data collection methods. We conducted our work from August  2005 through July 2006 in accordance with generally accepted government  auditing standards.
subsections: section_title: Background
paragraphs: Under NCLBA, SES primarily includes tutoring provided outside of the  regular school day that is designed to increase the academic achievement  of economically disadvantaged students in low-performing Title I schools.  These services must consist of high-quality, research-based instruction  that aligns with state educational standards and district curriculum.
subsections: section_title: Title I
paragraphs: Title I of ESEA, as amended and reauthorized by NCLBA, authorizes  federal funds to help elementary and secondary schools establish and  maintain programs that will improve the educational opportunities of  economically disadvantaged children. Title I is the largest federal program  supporting education in kindergarten through 12th grade, supplying   $12.7 billion in federal funds in fiscal year 2006. According to Education,  during the 2005-06 school year, nearly all U.S. school districts and  approximately half of public schools received some Title I funding. In  addition, the latest national data available from Education counted   16.5 million students as Title I participants in the 2002-2003 school year.
Title I funds are distributed by formula to state education agencies, which  retain a share for administration and school improvement activities before  passing most of the funds on to school districts. Districts are required to  distribute Title I funds first to schools with poverty rates over 75 percent,  with any remaining funds distributed at their discretion to schools in rank  order of poverty either districtwide or within grade spans. A school’s Title  I status can change from year to year because school enrollment numbers  and demographics may vary over time.
Enactment of NCLBA strengthened accountability by requiring states and  schools to improve the academic performance of their students so that all  students are proficient in reading and math by 2014. Under NCLBA, each  state creates its own content standards, academic achievement tests, and  proficiency levels. In 2005-2006, states were required to test all children for  reading and mathematics achievement annually in grades 3-8 and once in  high school to determine whether schools are making adequate yearly  progress (AYP).
In addition to meeting the state’s performance goals by grade, subject, and  overall student population, schools are responsible for meeting those  goals for designated groups. These groups are students who (1) are  economically disadvantaged, (2) are part of a racial or ethnic group that  represents a significant proportion of a school’s student population,   (3) have disabilities, or (4) have limited English proficiency. To make AYP,  each school must also show that each of these groups met the state  proficiency goals for both reading and math. In addition, schools must  show that at least 95 percent of students in grades required to take the test  have done so. Schools must also demonstrate that they have met state  targets for at least one other academic indicator, including graduation rate  in high schools and a state-selected measure in elementary or middle  schools.
For Title I schools that do not meet state AYP goals, NCLBA requires the  implementation of specific interventions, and these interventions must  continue until the school has met AYP for 2 consecutive years. Table 1  outlines the interventions applied after each year a Title I school misses  state performance goals. At their discretion, states may also implement  interventions for public schools that do not receive Title I funds and do  not make AYP. Although districts are not required to offer SES until a Title  I school has missed performance goals for 3 years, because some schools  had not met state goals set under ESEA before the enactment of NCLBA,  some Title I schools were first required to offer SES in 2002-2003, the first  year of NCLBA implementation.
States are also required to establish and implement AYP standards for  school districts based on the performance of all of the schools in the  district. If districts fail to meet these standards for 2 consecutive years,  states may classify districts as needing improvement. A district identified  for improvement must develop and implement an improvement plan and  remain in this status until it meets AYP standards for 2 consecutive years.  If a district remains in improvement status for 2 or more years, it may be  identified for corrective action as deemed necessary by the state.
Students are eligible for SES if they attend Title I schools that have missed  AYP for 3 consecutive years and are from low-income families. School  districts must determine family income on the same basis they use to  make allocations to schools under Title I, for which many have historically  used National School Lunch Program (NSLP) data. The NSLP is a federally  funded program that annually collects family income data from students’  parents to determine student eligibility for free and reduced-priced lunch.  A student’s state assessment scores, grades, and other academic  achievement information are generally not considered when determining  SES eligibility. However, if sufficient funds are not available to provide  SES to all eligible children, school districts must give priority to the  lowest-achieving eligible students.
SES providers may include nonprofit entities, for-profit entities, school  districts, public schools, public charter schools, private schools, public or  private institutions of higher education, educational service agencies, and  faith-based organizations. Under the Title I regulations that govern SES, a  district identified as in need of improvement or corrective action may not  be an SES provider, though its schools that are not identified as needing  improvement may. In addition, individual teachers who work in a school  or district identified as in need of improvement may be hired by any state- approved provider to serve as a tutor in its program.
A district must set aside an amount equal to 20 percent of its Title I  allocation to fund both SES and transportation for students who elect to  attend other schools under school choice. This set-aside cannot be spent  on administrative costs for these activities, and the district may reallocate  any unused set-aside funds to other Title I activities after ensuring all  eligible students have had adequate time to opt to transfer to another  school or apply for SES. Funding available for SES is, therefore, somewhat  dependent on costs for choice-related transportation, though as we found  in our 2004 report on NCLBA’s school choice provisions, few students are  participating in the school choice option. If a district does not incur any  choice-related transportation costs, it must use the full 20 percent set- aside amount to pay for SES if sufficient demand for services exists. In  addition, if the Title I set-aside is not sufficient to fund SES for interested  students, both states and districts may direct other funds for these  services at their discretion. For each student receiving SES, a district must  spend an amount equal to its Title I per-pupil allocation or the actual cost  of provider services, whichever is less.
Education oversees SES implementation by monitoring states and  providing technical assistance and support. OII leads SES policy  development and coordinates the publication of SES guidance, and OESE  oversees and monitors Title I, including SES. NCLBA and the Title I  regulations and SES guidance outline the roles and responsibilities states,  school districts, parents, and service providers have in ensuring that  eligible students receive additional academic assistance through SES   (see table 2).
During the 2005-2006 school year, Education announced the  implementation of two pilot programs intended to increase the number of  eligible students receiving SES and generate additional information about  the effectiveness of SES on students’ academic achievement. In the first,  Education permitted four districts in Virginia to offer SES instead of  school choice in schools that are in their first year of needs improvement.
In the second, Education entered into flexibility agreements with the  Boston and Chicago school districts, enabling them to act as SES  providers while in improvement status. OII and OESE coordinated  implementation of the pilots for the department. Both pilots were subject  to review at the end of the 2005-2006 school year, at which time Education  planned to evaluate their effect on student academic achievement.
section_title: SES Participation Has Increased as Districts Have Taken Steps to Ease Access, but Challenges Remain
paragraphs: SES participation increased between 2003-2004 and 2004-2005, and most  students receiving services were among the lower achieving students in  school. Districts have taken multiple actions to encourage participation,  such as offering services on or near the school campus or at various times.  Despite these efforts, challenges to increasing participation remain,  including notifying parents in a timely and effective manner, ensuring  there are providers to serve certain areas and students, and encouraging  student attendance.
subsections: section_title: Participation among Eligible Students Increased from 12 to 19 Percent between 2003-2004 and 2004-2005, and Most Participants Were Low Achieving
paragraphs: Nationally, the participation rate increased substantially from 12 percent  of eligible students receiving SES in 2003-2004 to 19 percent in 2004-2005.  In addition, the number of students receiving services almost quadrupled  between 2002-2003 and 2004-2005 from approximately 117,000 to 430,000  students nationwide, based on the best available national data (see fig. 1).  This increase may be due in part to the increase in the number of schools  required to offer SES over that time period. Specifically, between 2004- 2005 and 2005-2006 the number of schools required to offer SES increased  from an estimated 4,509 to 6,584.
Although nationally SES participation is increasing, some districts  required to offer SES have no students receiving services. Specifically, we  estimate that no students received services in about 20 percent of the  approximately 1,000 districts required to offer SES in 2004-2005. A  majority of these districts were rural or had a total enrollment of fewer  than 2,500 students. Our survey did not provide sufficient information to  explain why these districts had no students receiving services in 2004- 2005; therefore, it is unclear whether their lack of participation was related  to district SES implementation or other issues.
Nationwide, we estimate that districts required to offer SES spent   the equivalent of 5 percent of their total Title I funds for SES in 2004-2005  excluding administrative expenditures. Districts set aside an amount equal  to 20 percent of their Title I funds for SES and choice-related  transportation at the beginning of the school year, and the proportion of  the set-aside spent on SES varied by district. Specifically, in 2004-2005,  about 40 percent of districts spent 20 percent or less of the set-aside to  provide SES and almost one-fifth of districts spent over 80 percent.  Nationwide, of the total amount districts set-aside for SES, we estimate  they spent 42 percent on SES, excluding administrative expenditures.
Further, an estimated 16 percent of districts reported that the required  Title I set-aside was not sufficient to fund SES for all eligible students  whose parents requested services. For example, during our site visit to  Newark, N.J., district officials reported budgeting the entire 20 percent  Title I set-aside to fund SES in 2004-2005, but with this amount of funding,  the district was only able to fund SES for 17 percent of the students  eligible for services. In addition, according to Chicago, Ill., district  officials, the district budgeted the entire 20 percent Title I set-aside to fund  SES in 2005-2006, and because parents’ demand for services significantly  exceeded the amount of funding available, the district also allocated   $5 million in local funds to provide SES.
While approximately 1,000 of the over 14,000 districts nationwide were  required to offer SES in 2004-2005, SES recipients are concentrated in a  small group of large districts, as 56 percent of recipients attended school  in the 21 districts required to offer SES with more than 100,000 total  enrolled students (see fig. 2). Further, states ranged from having 0 districts  to 257 districts required to offer SES in 2004-2005, with most states having  fewer than 10 districts required to offer SES. State differences in the  number of districts required to offer SES may have resulted from  differences in performance or differences in state proficiency standards  and methods used to measure adequate yearly progress.
Students receiving SES in 2004-2005 shared certain characteristics, as  districts reported that most students receiving services were among the  lower achieving students in school. Specifically, an estimated 91 percent  of the districts that reviewed the academic records of students receiving  SES classified most or all of the students receiving SES as academically  low achieving. For example, Hamilton County, Tenn., school officials said  that students receiving SES are frequently behind grade level in their skills  and require special attention to increase their academic achievement.
Further, we estimate that over half of SES recipients were elementary  school students in the majority of districts and about 60 percent of schools  required to offer SES in 2004-2005 were elementary schools. Districts  varied in the percentage of students with limited English proficiency  receiving services. In about one-third of districts, less than 5 percent of  SES recipients were students with limited English proficiency; however, in  about one-fifth of districts, over half of SES recipients were students with  limited English proficiency. Students with disabilities made up less than   20 percent of students receiving services in about two-thirds of districts.  Finally, in some districts, the majority of SES recipients were African- American or Hispanic. In about 40 percent of districts, over half of SES  recipients were African-American, and in about 30 percent of districts,  over half of SES recipients were Hispanic. Because we were unable to  obtain comparable data on the characteristics of Title I students enrolled  in these districts in 2004-2005, we were unable to determine whether  certain groups of students were underserved.
We estimate that about 2,800 providers delivered services to students  nationwide in 2004-2005, and more providers were available to deliver  services in the districts with the largest student enrollments. Specifically,  about 80 percent of districts had between 1 and 5 providers delivering  services in 2004-2005. However, the number of providers delivering  services in the 21 districts with more than 100,000 total enrolled students  ranged from 4 to 45, and averaged 15 providers per district in 2004-2005.
section_title: Districts Used Several Methods to Contact Parents and Offered Services on School Campuses and at Various Times to Increase Participation
paragraphs: Districts have taken multiple actions to encourage participation, as shown  in table 3. In line with the federal statutory requirement that districts  notify parents in an understandable format of the availability of SES, over  90 percent of districts provided written information in English, held  individual meetings with parents, and encouraged school staff to talk with  parents about SES. Some districts collaborated with providers to notify  parents. For example, during our site visit, Illinois state officials described  a provider and district sharing administrative resources to increase  participation, which involved the provider printing promotional materials  and the district addressing and mailing the materials to parents. In  addition, we estimate that over 70 percent of districts lengthened the  period of time for parents to turn in SES applications, held informational  events for parents to learn about providers, and provided written  information to parents in languages other than English. During our site  visit to Woodburn, Ore., district officials reported extending the time  parents had to sign up their children for SES and hosting an event where  providers presented their programs to parents in English and Spanish.  Further, Newark, N.J., district officials told us during our site visit that the  district provided transportation for parents to attend informational events  to increase participation. Also to encourage participation, an estimated   90 percent of districts offered services at locations easily accessible to  students, such as on or near the school campus, and almost 80 percent of  districts offered services at a variety of times, such as before and after  school or on weekends. For example, Hamilton County, Tenn., worked  with providers to offer an early morning tutoring program located at the  school site in addition to providing services after school. Providers also  reported delivering SES on school campuses and at various times.  Specifically, over three-fourths of the 22 providers we interviewed  reported delivering services at the school site, although providers also  offered services off-site, such as in the home, online, or at the provider’s  facility. In addition, providers generally delivered SES after school and  some also offered SES at alternative times, such as before school, on  weekends, or during the summer. Finally, about one-third of districts  provided or arranged for transportation for participating students or  worked with a local community partner to raise awareness of the services.  For example, in Newark, N.J., the district worked with a local community  organization to inform parents and students living in public housing and  homeless shelters about SES.
States also reported taking actions to increase participation in 2005-2006,  as shown in table 4. Regarding parent notification, all states encouraged  district staff to communicate with parents about SES. In addition, almost  90 percent of states provided guidance to districts on the use of school  campuses for service delivery to encourage participation.
section_title: Remaining Challenges Include Notifying Parents in a Timely and Effective Manner and Attracting More Providers for Certain Areas and Students
paragraphs: Despite some districts’ promising approaches to encourage participation,  notifying parents in a timely manner remains a challenge for some  districts. An estimated 58 percent of districts did not notify parents that  their children may be eligible to receive SES before the beginning of the  2005-2006 school year, which may be due in part to delays in states  reporting which schools were identified for improvement. Specifically,  about half of districts that did not notify parents before the beginning of  the 2005-2006 school year did not receive notification from the state of the  schools identified for improvement by that time. Moreover, district  officials in three of the states we visited experienced delays in receiving  school improvement information, and state officials agreed that providing  timely information about whether schools have met state performance  goals has been a challenge. Almost all of the districts that did not notify  parents before the beginning of the 2005-2006 school year did so within the  first 2 months of the year.
Effectively notifying parents is also a challenge for some districts. For  example, officials in all four districts we visited reported difficulties  contacting parents to inform them about SES in part because some  families frequently move and do not always update their mailing address  with districts. In addition, some providers we interviewed indicated that  confusing parental notification letters do not effectively encourage SES  participation. For example, some of the providers we interviewed said  some districts use confusing and poorly written letters to inform parents  of SES or send letters to parents of eligible children but conduct no further  outreach to encourage participation in SES. Four of the providers we  interviewed also indicated that complicated district enrollment processes  can discourage participation. For example, one provider said certain  districts send parents multiple documents to complete in order for their  child to receive SES, such as an enrollment form to select an SES provider  and a separate contract and learning plan.
Another challenge to increasing SES participation is attracting more SES  providers for certain areas. Some rural districts surveyed indicated that no  students received services last year because of a lack of providers in the  area. We estimate that the availability of transportation for students  attending supplemental services was a moderate, great or very great  challenge for about half of rural districts. For example, one rural district  commented in our survey that there are no approved providers within   200 miles of its schools. A few other rural districts commented in our  survey that it was difficult to attract providers to their area because there  were few students to serve or providers had trouble finding staff to serve  as tutors.
In addition, ensuring there are providers to serve students with limited  English proficiency or disabilities has been a challenge for some districts.  There were not enough providers to meet the needs of students with  limited English proficiency in an estimated one-third of districts, and not  enough providers to meet the needs of students with disabilities in an  estimated one-quarter of districts. Many states also indicated that the  number of providers available to serve these groups of students was  inadequate. While over half of the providers we interviewed reported  serving students with limited English proficiency or disabilities, some  providers served these students on a limited basis and reported difficulties  meeting their needs. For example, one provider reported serving few  students with limited English proficiency and disabilities because the  amount of funding the provider receives for SES was not sufficient to pay  for specialized tutors to serve these students. Another provider said it was  difficult to find tutors to meet the needs of students with limited English  proficiency and its program was not designed for students with  disabilities. Another provider said that it was difficult to serve students  with disabilities because it required significantly modifying the tutoring  lessons to meet their needs.
Encouraging student attendance has also been a challenge, in part because  students may participate in other afterschool activities, such as sports or  work. Low parent and student demand for SES has been a challenge in  about two-thirds of districts. For example, about one-quarter of districts  reported that both competition from other afterschool programs and the  availability of services that are engaging to students were challenges to  implementing SES. In addition, providers, district and school officials in all  four districts we visited told us that SES is competing for students with  extracurricular and other activities. For example, a Chicago, Ill., high  school official indicated that student attendance at SES sessions declined  significantly as the school year progressed. To address this problem,  providers sometimes offer students incentives for participation. For  example, while 2 of the 22 providers we interviewed offered incentives for  students to sign up for services, 19 providers used incentives to encourage  student attendance, such as school supplies and gift certificates.
section_title: Providers Have Taken Steps to Deliver Quality Services, but Contracting and Coordination Remain Challenges to Working with Districts and Schools
paragraphs: To promote improved student academic achievement, providers aligned  their curriculum with district instruction primarily by hiring district  teachers and communicating with the teachers of participating students.  Providers reported communicating with teachers and parents in person as  well as mailing information and progress reports to them; however,  districts indicated the extent of provider efforts varied, as some did not  contact teachers and parents in 2004-2005. In addition, both providers and  districts experienced contracting and coordination difficulties. In part  because SES is often delivered in school facilities, providers and officials  in the districts and schools we visited reported that involvement of school  administrators and teachers can improve SES delivery and coordination.
subsections: section_title: Providers Worked to Align Curriculum and Communicate with Parents, though the Extent of Their Efforts Varied
paragraphs: In order to increase student academic achievement, providers took steps  to align their curriculum with school instruction by hiring and  communicating with teachers, though the extent of their efforts varied. A  majority of the 22 providers we interviewed hired certified teachers in the  district as tutors. Some providers said hiring district teachers promoted  curriculum alignment, in part because district teachers were apt to draw  on district curriculum during tutoring sessions. School officials in three of  our site visits also said providers’ use of district teachers as tutors helped  to align the tutoring program with what the student learned in the  classroom. In addition, some providers reported aligning curriculum by  communicating with the teachers of participating students to identify  student needs and discuss progress. The frequency of contact between  tutors and teachers ranged from mailing teachers information once prior  to the beginning of the program to contacting teachers at least weekly,  according to the providers we interviewed. A few providers also used  other methods to align their curriculum with district instruction, such as  using the same tests to evaluate student progress and allowing principals  to choose components of the tutoring curriculum for students receiving  SES in their schools. However, not all providers worked with schools to  align curriculum, as we estimate that some, most, or all providers did not  contact teachers to align curriculum with school instruction in almost 40  percent of districts in 2004-2005. For example, Woodburn, Ore., district  and school officials indicated during our site visit that instead of aligning  their services with the district curriculum, certain providers openly  questioned the district’s curriculum and teaching methods, which caused  confusion among some parents and students.
Providers reported mailing information as well as meeting with parents  over the phone and in-person to communicate about student needs and  progress; however, the frequency of communication with parents varied  by provider. A majority of the providers we interviewed communicated  with parents about student progress repeatedly, sometimes by sending  home progress reports on a monthly basis or holding parent-tutor  conferences. The frequency of contact between tutors and parents  reported by the 22 providers we interviewed ranged from meeting with  parents twice during the tutoring program to giving parents a weekly  progress report. A few providers also reported communicating with  parents by holding workshops for parents to learn about the SES program  and in some cases having the parents sign their students’ learning plans.  For example, one provider conducted workshops where parents received  reading materials to share with their children and a parent guide in English  and Spanish that explained the program and how to use the materials to  enhance student learning. Some providers also reported hiring staff  dedicated in part to maintaining communication with parents. However,  some providers faced difficulties when communicating with parents about  SES, such as language barriers or incorrect contact information. Districts  confirmed that the degree to which providers communicated with parents  varied, as we estimate that some, most, or all providers did not contact  parents to discuss student needs and progress in about 30 percent of  districts in 2004-2005.
Despite these challenges, most districts had positive relationships with  providers. Specifically, an estimated 90 percent of districts indicated that  their working relationships with providers during 2004-2005 were good,  very good, or excellent. In addition, many of the providers we interviewed  during our site visits also reported having positive working relationships  with district officials. Although other studies have found that districts  reported certain difficulties working with providers, relatively few districts  reported that their providers signed up ineligible students or billed for  services not performed in 2004-2005, as shown in figure 3.
Generally, states did not hear about these provider issues very often.  Almost half of states said the issue of providers not showing up for SES  sessions was rarely brought to their attention. Similarly, half of states  said the issue of providers billing the district for services not performed  was rarely brought to their attention. In addition, about 40 percent of  states said the issue of providers using excessive incentives was rarely  brought to their attention. Further, about 40 percent of states said the  issue of providers signing up ineligible students rarely arose. Almost      one-third of states heard about each of these issues sometimes, while few  states had these issues brought to their attention very often. For example,  during our site visits, state officials provided examples of issues that had  been brought to their attention regarding provider practices, but these  issues were often isolated incidents particular to one or a few providers in  certain districts.
section_title: Providers and Districts Experienced Contracting and Coordination Difficulties
paragraphs: While providers have taken steps to deliver quality services, both  providers and districts reported experiencing difficulties during the  contracting process. For example, some of the providers we interviewed  said certain districts imposed burdensome contract requirements, such as  requiring substantial documentation to be submitted with invoices,  limiting the marketing they could do to parents and students, or restricting  the use of school facilities to deliver services. Specifically, 7 of the   22 providers we interviewed experienced difficulties with districts  restricting provider access to school facilities, by for example, not  allowing providers to deliver services in school buildings or by charging  providers substantial fees to do so. A few providers also said contracting  with districts was a resource-intensive process, in part because contract  requirements vary by district and state. Some of the multi-state providers  we interviewed reported dedicating a team of staff to help them finalize  and manage contracts with districts. These providers commented that,  while they have the administrative capacity to manage this process,  smaller providers may not have such capacity. In addition, one provider  that delivered services exclusively online commented that contracting  with districts across the country was a challenge, particularly because  some states and districts require provider representatives to attend  meetings in-person and be fingerprinted in their states.
Contracting with providers was also a challenge for some districts.  Specifically, negotiating contracts with providers was a moderate, great, or  very great challenge in about 40 percent of districts nationwide. For  example, Woodburn, Ore., district officials described having contractual  discussions with providers around whether the district would charge fees  for the use of school facilities, the types of incentives providers could use  to encourage students to sign up, and whether the district would pay for  services when students did not attend SES sessions. While states may  review and define program design parameters as part of the provider  approval process, district officials in three of our site visits expressed  concern about their lack of authority to set parameters in provider  contracts around costs and program design, such as tutor-to-student ratios  and total hours of instruction. For example, during our site visit, a  Hamilton County, Tenn., district official expressed frustration with  providers that charged the maximum per-pupil amount but varied in the  level of services provided, such as the number of instructional hours and  tutor-to-student ratio. Chicago, Ill., district officials also expressed  concern about the variation among providers in the hours of instruction  and cost of services because the district does not have sufficient funds to  serve all eligible students and officials would like to maximize the number  of students they can serve. In part to help address district concerns, in  2005-2006, Illinois required approved providers to submit information on  the cost of providing services in each of the districts they served and made  this information available to districts and the public in order to improve  transparency and accountability. While Tennessee state officials told us  they were hesitant to set restrictions on providers and would like more  federal direction on this issue, other states have set restrictions on the  cost and design of SES programs. For example, Georgia set a maximum  tutor-to-student ratio of 1:8 for non-computer based instruction and   1:10 for computer based instruction, and New Mexico adopted a sliding fee  scale based on the educational level of tutors.
Coordination of service delivery has also been a challenge for providers,  districts, and schools. About 70 percent of states reported that the level of  coordination between providers, districts, and schools implementing SES  was a moderate to very great challenge. Sometimes these coordination  difficulties have resulted in service delays. For example, services were  delayed or withdrawn in three of the districts we visited because not  enough students signed up to meet the providers’ enrollment targets and  districts were not aware of these targets. In one district we visited,  services were delayed because school teachers hired to be tutors did not  provide evidence of their background checks and teaching certificates to  providers in a timely manner. Coordination difficulties also occurred  during the enrollment process. Though districts are responsible for  arranging SES for eligible students, in two districts we visited, both the  district and providers sent parents enrollment forms, which caused  confusion among parents as well as additional work for the district staff  processing the forms. In addition, a few providers told us they do not  know how many students they will serve until enrollment forms are  returned to district officials, which hinders planning and may delay  services since they do not know how many tutors they will need to hire  and train to deliver SES in each district.
section_title: Providers, District and School Officials Reported That a Greater Role for Schools Would Improve Local SES Implementation
paragraphs: In part because SES can be delivered in school facilities, providers and  officials in the districts and schools we visited reported that involvement  of school administrators and other staff improves SES implementation.  Although schools do not have federally-defined responsibilities for  administering SES, many officials said SES implementation is hindered  when school officials are not involved. Some providers we interviewed  said that a lack of involvement of school principals can make it difficult  for them to coordinate with schools to encourage student participation. In  addition, a few providers said certain districts contributed to this problem  by restricting communication with school officials or not defining a role  for schools in SES implementation. Officials in one of the districts we  visited also told us that encouraging participation and administering the  program was more difficult when they did not designate school staff to  assist the district with SES implementation. School officials from all four  of our site visits also said the lack of a clear role for school officials,  including principals, in administering SES has been a challenge. For  example, Illinois and Oregon school principals told us they found it  difficult to manage afterschool activities because they didn’t have  sufficient authority to oversee SES tutors operating in their buildings at  that time. Further, problems can arise when school officials are not fully  informed about the SES program. For example, Woodburn, Ore., school  officials told us that although the school was not provided SES tutoring  schedules for students, parents and students have come to school officials  for help when they were unclear about the schedule or when tutors failed  to show up.
A majority of the providers we interviewed told us that involvement of  school principals can improve participation and make it easier to deliver  services, in part because principals are familiar with the students and  manage school staff. For example, certain providers reported providing  principals with information about the tutoring program so that school staff  can assist with the enrollment process, involving principals in selecting the  curriculum used in their schools, and sending principals student progress  reports. In addition, all four districts we visited had school site  coordinators to assist with SES, such as helping with the enrollment  process and assisting with the day-to-day administration of the SES  program in the schools. For example, Woodburn, Ore., district officials  told us implementation improved when the district designated school site  coordinators to assist with parental notification and events where  providers present their programs, and meet with parents and providers to  help them complete individual student learning plans. A few providers we  interviewed also assigned a staff person at the school site to communicate  with teachers and parents. While helping to administer the SES program  adds additional administrative burden on schools, school officials in all  four of the districts we visited said they welcomed a stronger or more  clearly defined role.
section_title: States’ SES Monitoring Has Been Limited Though Their Efforts Are Increasing, and Many States Struggle to Develop Meaningful Evaluations
paragraphs: While state monitoring of SES had been limited, more states reported  taking steps to monitor both district and provider efforts to implement  SES in 2005-2006. In addition, districts have taken a direct role in  monitoring providers, and their monitoring activities similarly increased  during this time. Although states are required to withdraw approval from  providers that fail to increase student academic achievement for 2 years,  many states reported challenges evaluating SES providers. In addition, the  few states that have completed an evaluation have not yet produced  reports that provided a conclusive assessment of SES providers’ effect on  student academic achievement.
subsections: section_title: More States Reported Conducting On-Site Reviews of Districts in 2005-2006, and Many also Collected Information from Other Sources to Monitor District SES Implementation
paragraphs: State monitoring of district SES implementation, which is sometimes  performed as part of state Title I monitoring, had been limited prior to  2005-2006, though more states reported conducting on-site reviews of  districts in that year. While less than one-third of states conducted on-site  reviews of districts to monitor their implementation of SES in 2004-2005,  almost three-fourths reported conducting such reviews in 2005-2006. This  increase reflects both those states that had already begun monitoring  district SES implementation for 2005-2006 at the time of our survey and  those states planning to conduct monitoring activities before the end of  that school year. Because our data were collected during the middle of the  2005-2006 school year, we do not know whether all of the states that  planned to complete these activities before the end of the year did so. In  both years, a majority of the states that conducted on-site reviews visited  few or some of their districts. Therefore, while more states reported  conducting such reviews in 2005-2006 than in 2004-2005, the number of  districts per state receiving reviews does not appear to have increased.
In addition to on-site reviews, many states also reported reviewing  information collected from several other sources to assess district SES  implementation in 2005-2006. The most common source used by states  was district officials, as almost all states reported reviewing or planning to  review information collected from district officials to monitor their  implementation of SES in 2005-2006. Further, many states were also  collecting information from school principals, parents, and providers to  monitor districts, with between 67 and 81 percent of states reviewing or  planning to review information collected from these sources in 2005-2006.
States also reported reviewing or planning to review information related to  several aspects of district SES implementation in 2005-2006. For example,  almost all states reported reviewing district notification of parents and  SES expenditures, as shown in figure 4.
States we visited reported that some districts have had difficulties  implementing SES, in part because of district staff capacity limitations and  the complexities of administering SES at the local level. When states find  that a district is having difficulty implementing SES, most hold a meeting  with the district and provide or arrange for assistance, including  consultations or training. Half of the states also develop an action plan and  time line with the district to help improve its efforts. During our site visits,  state officials reported that notifying parents, maintaining a fair and  competitive environment for providers, ensuring providers understand  their SES roles and responsibilities, and determining an appropriate role  for schools continue to challenge some districts as they implement SES.
section_title: Though States and Districts Are Challenged to Monitor Providers, They Are Moving from Limited Monitoring of Providers to a More Active Approach
paragraphs: Although states and districts reported increasing their efforts to monitor  SES providers between 2004-2005 and 2005-2006, over two-thirds of states  reported that on-site monitoring of providers has been a challenge. In  addition, several districts commented in our survey that more provider  monitoring is needed. During all four of our site visits, state and district  officials expressed concerns about their capacity to fully administer and  oversee all required aspects of SES implementation, including provider  monitoring. Officials explained that state and district capacity to  implement SES is limited, because there is typically one staff person at  each level coordinating all of SES, and sometimes that person may also  oversee implementation of additional federal education programs. Further,  several states commented in our survey that additional training, technical  assistance, and national monitoring protocols from the federal government  would assist their efforts to monitor providers.
During our site visits, state officials noted that while they did not initially  have structured plans or procedures in place to monitor SES providers,  they took steps to develop and formalize procedures starting with the  2004-2005 and 2005-2006 school years. Nationally, in 2004-2005, states  monitored providers primarily by collecting data from district officials,  though many states reported using a more active monitoring approach in  the next year. For example, while less than one-third of states conducted  on-site reviews of providers in 2004-2005, over three-fourths had  conducted or planned to conduct such reviews in 2005-2006, as shown in  figure 5. In addition, while one-third or fewer states reviewed information  collected from school staff, parents, and students in 2004-2005, the  percentage that reported reviewing or planning to review information  collected from these sources more than doubled the next year. Similar to  2004-2005, many states continued to rely on information collected from  district officials to monitor providers in 2005-2006, with almost all states  reviewing or planning to review information collected from districts in  that year.
Federal guidance suggests states may request district assistance in  collecting data from providers to assist state monitoring activities. While  the state is ultimately responsible for monitoring providers, most states  reported that districts have taken a direct role in monitoring providers.  Similar to states, although district monitoring of providers was limited in  2004-2005, districts used a more extensive and active approach in the next  year, as shown in figure 6. For example, while we estimate that less than  half of districts collected information from on-site reviews, school staff,  parents, and students to monitor providers in 2004-2005, 70 percent or  more were collecting or planning to collect information from these  sources in 2005-2006.
Although states and districts collected information from similar sources to  monitor providers, districts collected information from more providers  than states. Specifically, while a majority of the states that conducted on- site reviews observed only some or few providers, we estimate that a  majority of districts that conducted on-site reviews observed most or all of  their providers in 2004-2005. While states and districts may both have  capacity limitations that affect their ability to conduct on-site reviews to  monitor providers, conducting such reviews is likely easier for districts  because services are often delivered on or near school campuses.
States and districts collected information on several aspects of SES  programs to monitor providers, as shown in table 5. While federal  regulations provide states flexibility to design their own SES monitoring  systems, over two-thirds or more of states and districts monitored or  planned to monitor all program elements listed, including those related to  service delivery and use of funds. For example, 94 percent of states  monitored or planned to monitor parent or student satisfaction with  providers, and 93 percent of districts monitored or planned to monitor  billing and payment for services and student attendance records.
section_title: Many States Struggle to Develop Meaningful Evaluations, and the Few State Evaluations Completed to Date Were Inconclusive
paragraphs: Many states struggle to develop evaluations to determine whether SES  providers are improving student achievement, though states are required  to evaluate and withdraw approval from providers that fail to do so after   2 years. Specifically, federal law requires states to develop standards and  techniques to evaluate the services delivered by approved providers, but it  does not require states to use specific evaluation methods or criteria for  determining provider effectiveness. Through our survey, states reported  several challenges to evaluating SES providers. Specifically, over         three-fourths of states reported that determining sufficient academic  progress of students, having the time and knowledge to analyze SES data,  and developing data systems to track SES information have been  challenges. Further, during our site visits to Illinois and New Jersey, state  officials noted they were currently in the process of improving their data  collection systems to more effectively capture and analyze data for SES  evaluations. In addition, several state officials reported that while they  have collected some information to assess provider effectiveness, they  have done little with that data. Others noted that they have not received  sufficient federal guidance on effective models for SES provider  evaluations, and because developing and implementing such evaluations  can be both time-consuming and costly, additional assistance from  Education would improve their efforts.
At the time of our survey in early 2006, only a few states had drafted or  completed an evaluation report addressing individual SES provider’s  effects on student academic achievement, and we found that no state had  produced a report that provided a conclusive assessment of this effect.  New Mexico and Tennessee were the only two states that had completed  final or draft SES evaluation reports that attempted to assess the impact of  all SES providers serving students in their states in previous years. To  measure student academic achievement, New Mexico’s report analyzed  students’ grades as well as their scores on state assessments and provider  assessments, which often differ by provider and are administered both  before SES sessions begin and at the end of SES sessions each year.
However, the report noted that these assessments produced different  results related to student academic achievement gains. While Tennessee  also planned to review students’ state assessment scores, the draft  available at the time of our analysis did not include this information. In  addition, both reports drew on information obtained through other  sources, such as teacher surveys, to assess provider effectiveness. Due to  their limitations, neither evaluation provided a conclusive assessment of  SES providers’ effect on student academic achievement.
In addition, at the time of our survey, over half of the states reported that  they were in the process of conducting an evaluation of SES providers in  order to meet the federal requirement of assessing each provider’s effect  on student academic achievement. Similar to the state evaluations  already undertaken, officials reported using different methods and criteria  to evaluate SES providers. For example, some states were collecting each  provider’s pre- and post-SES assessments of students while others were  collecting student achievement data from districts or students’ state  assessment scores. Further, while one state defined adequate student  progress as 80 percent of a provider’s students making one-grade level of  improvement after a year of SES, another state defined adequate student  progress as 50 percent or more of a provider’s students having any positive  academic achievement gain after a year of SES. While these states have yet  to produce final results from their SES provider evaluations, it is unclear  whether any of these efforts will produce a conclusive assessment of SES  providers’ effect on student academic achievement.
Likely because states are struggling to complete evaluations to assess SES  providers’ effect on student academic achievement, states did not report  that they have withdrawn approval from providers because their programs  were determined to be ineffective at achieving this goal. Rather, though  over 40 percent of states reported that they had withdrawn approval from  some providers, they most frequently reported withdrawing provider  approval because the provider was a school or district that had entered  needs improvement status, the provider asked to be removed from the  state-approved provider list, or because of provider financial impropriety.
section_title: Several Education Offices Monitor and Support SES Implementation, but States and Districts Cite the Need for Additional Assistance and Flexibility
paragraphs: Several offices within Education monitor various aspects of SES activity  across the country and provide support, but states and districts reported  needing additional assistance and flexibility with program implementation.  Education conducts SES monitoring in part through policy oversight and  compliance reviews of states and districts, and provides SES support  through guidance, grants, research, and technical assistance. However,  many states and districts reported needing additional assistance and  guidance regarding evaluation and administration of SES. Further, some  states and districts voiced interest in expansion of Education’s pilot  programs that increase SES flexibility, including the pilot that allows  certain low-achieving districts to serve as SES providers.
subsections: section_title: Education Monitors SES and Provides States and Districts with Guidance and Technical Assistance
paragraphs: OII and OESE are primarily responsible for monitoring and supporting  state and district SES implementation, and other Education offices also  contribute to these efforts (see fig. 7).
OII, which leads SES policy development and provides strategic direction,  monitors SES policy issues primarily through what it calls “desk  monitoring.” This monitoring is performed at its federal office and  includes the review of SES-related research and media reports. OII also  conducts more intensive monitoring of specific SES implementation  challenges when states, districts, and providers bring them to Education’s  attention and keeps a log documenting these issues. For example, during  2004-2005, Illinois and New Jersey officials contacted OII to clarify  guidance regarding providers affiliated with districts in need of  improvement, and OII staff provided assistance and clarification. In  addition, several providers we interviewed also mentioned that they have  contacted OII directly to discuss implementation challenges associated  with enrollment, district contracts, and provider access to school facilities.
OESE, which oversees and supports NCLBA implementation, is also  involved in monitoring SES implementation through its overall monitoring  of state compliance with Title I and NCLBA. To monitor Title I, OESE staff  visit state departments of education and selected districts within each  state to interview officials and review relevant documents. Once the  review is complete, OESE issues a report to the state outlining any  instances of Title I non-compliance, including those related to SES, and  actions needed to comply with regulations. As of June 2006, OESE had  visited and issued reports to over three-fourths of the states.
In addition to its Title I monitoring, OESE also oversees the collection of  state NCLBA data, including data on SES, through the annual Consolidated  State Performance Report (CSPR). For the CSPR, each state is required to  report the number of schools with students receiving SES, the number of  students eligible for services, and the number that received services.  However, although almost all states reported that they are collecting  information on district SES expenditures as part of their oversight,  Education does not require states to submit information on the percent or  amount of Title I funds districts spent for SES through the CSPR or other  means. Therefore, while Education tracks the extent to which states are  providing SES to eligible students, the department does not collect data on  the relative costs of serving them. Further, under NCLBA, Education is  required to present an annual summary of the CSPR data to Congress. As  of June 2006, the most recent report to Congress was for the 2002-2003  school year, though Education officials indicated they expect to submit  reports for 2003-2004 and 2004-2005 in the near future.
While OII and OESE monitoring of SES has been either limited to desk  monitoring or combined with general Title I monitoring, OIG has  conducted audits specifically focused on SES. During 2003-2004 and 2004- 2005, the office performed a series of SES implementation audits in six  states, which involved site visits to state offices and selected districts  within each state. Also during 2004-2005, OIG performed audits of five  California districts and one SES provider within each district. These  audits included an examination of district SES contracts with providers,  provider services, and provider compliance with state SES regulations.
Several Education officials reported coordinating internally to share  information, including SES monitoring results. To facilitate coordination,  OII leads an internal group comprised of staff members from other  Education offices, who meet bi-weekly to exchange information. OESE  shares its state Title I monitoring results and CSPR data with other  Education offices. In addition, OIG makes recommendations to both OII  and OESE in its state and district SES auditing reports and disseminates  the reports throughout Education and on the department’s Web site. Since  2002, OII has coordinated the publication of four versions of                   non-regulatory SES guidance, each updated to address ongoing challenges  with SES implementation. The latest and most comprehensive version of  non-regulatory SES guidance was published in June 2005. In May 2006,  Education issued a separate supplement to the guidance containing  additional information on private school participation in providing SES  and a policy letter clarifying the definition of a district-affiliated provider.
In addition to its monitoring efforts, OII also provides SES implementation  assistance, in part through presentations at conferences, and through  grants to external organizations that assist states and districts. For  example, OII staff have presented information on SES policy and  promising practices at national meetings attended by SES coordinators  and others involved in SES implementation. In addition, the office has  issued grants to the Black Alliance for Educational Options, the Hispanic  Council for Reform and Educational Options, and through the Star Schools  Program to promote SES to minority students and those in rural areas.  Further, OII funded the Supplemental Educational Services Quality Center  (SESQC), which offered SES technical assistance through tool-kits, issue  briefs, and a Web site containing SES information for state and district  officials, schools, parents, and providers. SESQC also periodically  convened representatives of organizations working on education issues to  discuss SES national coordination, challenges, and promising practices.  However, those meetings and all SESQC activities were discontinued in  December 2005 when SESQC’s grant period ended.
Other Education offices also provide SES support through various means.  For example, OESE funded the Comprehensive Centers Program through  grants that established technical assistance centers across the country to  help low-performing schools and districts close achievement gaps and  meet the goals of NCLBA. Of these, the Center on Innovation and  Improvement provides support to regional centers that assist states with  Education’s programs, including SES. In addition, Education’s Policy and  Program Studies Service, within the Office of Planning, Evaluation and  Policy Development, oversees several research studies that examine SES,  either in whole or in part. These reports, such as the National Assessment  of Title I: Interim Report and Case Studies of Supplemental Services  under the No Child Left Behind Act, provide states and districts with  information on SES implementation, challenges, and promising practices.  Further, Education’s Center for Faith-Based and Community Initiatives  offers technical assistance to faith- and community-based organizations  interested in becoming state-approved SES providers.
Given the technical assistance and support Education has already  provided to states and districts for implementation of SES and school  choice, and the department’s view that implementation of these provisions  has been uneven throughout the country, in May 2006, Education issued a  policy letter announcing the department’s plans to take significant  enforcement action. Specifically, Education plans to use the data collected  through its monitoring and evaluation efforts to take enforcement actions  such as placing conditions on state Title I grants, withholding federal  funds, or entering into compliance agreements. In the letter, the  department noted that its various monitoring activities have identified  several areas of noncompliance with SES requirements. For example, the  OIG’s audits found that each of the six states reviewed failed to adequately  monitor their districts for compliance. Consequently, nearly all of the  parental notification letters reviewed failed to include the required key  components, and several districts failed to budget sufficient funding for  services. Through our own analysis of Education’s monitoring reports, we  also found that some of the states reviewed were found to have inadequate  or incomplete processes for monitoring and evaluating SES providers.
section_title: Many States and Districts Reported Needing SES Evaluation Support and Additional Technical Assistance, and Some Suggested Increasing Flexibility through Education’s Pilot Programs
paragraphs: Despite Education’s efforts, many states and districts reported needing  more information and assistance to better comply with certain aspects of  SES implementation, including SES evaluation (see table 6). Specifically,  85 percent of states and an estimated 70 percent of districts needed  additional assistance with methods for evaluating SES, and over   60 percent also needed assistance with developing data systems. Many  districts also needed more information on provider quality and  effectiveness. Although OESE and OIG monitoring results have also  continually indicated that states and districts struggle with SES evaluation,  Education has yet to provide comprehensive assistance in this area, and  during our site visits, officials mentioned that they have been relying on  other states, organizations, or individuals for evaluation assistance. States  and districts also indicated a need for more support and technical  assistance to help them administer SES. Specifically, approximately    three-fourths of states and two-thirds of districts reported needing funding  to increase their capacity to implement SES. Many states also reported  needing tool kits and model/sample documents, as well as training from  Education, and a majority of districts needed effective parent outreach  strategies. Further, most states reported a need for conferences or  meetings where they could share lessons learned and promising practices  with other states. A few Tennessee officials mentioned that conferences  hosted by national organizations have been an effective means of allowing  SES officials to gather and share knowledge.
While three-fourths of states reported that the most recent version of  Education’s SES guidance, published in June 2005, has been very or  extremely useful, several states commented through our survey that they  needed additional or clearer guidance on certain SES provisions. This  included guidance on managing SES costs and fees, implementing SES in  rural areas, and handling provider complaints. During three of our site  visits, officials also expressed some concern about the lack of clarity in  the SES guidance with regards to student eligibility requirements and how  to craft a parental SES notification letter that is both complete and easy  for parents to understand.
Regarding parental notification letters, though both OESE and OIG found  many states and districts to be non-compliant with the federal requirement  that district SES parental notification letters include several specific  elements, Education’s SES guidance, which is coordinated by OII,  provides a sample that does not clearly specify all of the key elements  required by SES law and regulations. For example, the sample letter does  not include information on provider services, qualifications, and  effectiveness. Furthermore, a few state and district officials commented  that, when followed, the Title I regulations governing SES yield a letter  that is unreasonably long and complex, which may be difficult for parents  to understand.
Many states and districts expressed interest in the flexibility offered  through two pilot programs that Education implemented during 2005-2006.  The department designed these pilots to increase the number of eligible  students receiving SES and to generate additional information about the  effect of SES on student academic achievement. For example, several  state and district SES coordinators expressed interest in Education’s pilot  program that allowed two districts in needs improvement status to act as  SES providers in exchange for their expansion of student access to SES  providers and collection of achievement data to determine SES program  effectiveness. During three of our four site visits, state and district officials  expressed concern that districts identified for needs improvement are  excluded from delivering SES, and one state official noted that removing  districts from the state approved provider list may result in lower SES  participation. Further, in our surveys, three state SES coordinators and   17 district SES coordinators wrote in comments that permitting districts in  needs improvement status to provide services would assist their efforts.  Through both our surveys and site visits, officials suggested that allowing  districts to act as providers may ease student access to SES for rural  districts that do not have providers located nearby, allow more students to  participate in SES because district costs to provide services are sometimes  lower than other providers’ costs, and enable districts to continue their  existing tutoring programs that they feel are effective and meet the same  goals as SES. Overall, we estimate that 15 percent of districts were state- approved providers in 2004-2005. However, another national survey  recently found the percentage of urban and suburban districts that are  state-approved SES providers is declining.
The other SES pilot allowed four districts in Virginia to offer SES instead  of school choice in schools that have missed state performance goals for   2 years and are in their first year of needs improvement. During our site  visits and through our surveys, many states and districts expressed  interest in adjusting the order of the SES and school choice interventions.  Specifically, half of states and over 60 percent of districts suggested that  SES should be made available before school choice (see table 7). Further,  approximately three-quarters of both states and districts indicated that  SES should be offered either before or simultaneously with choice. As we  found through our previous work on school choice, few students are  opting to transfer schools in the first year of needs improvement, and  therefore this change would provide students with another option to  receive additional academic support in that year. Further, in a recent  national study, district and school officials noted that parents and students  are often not interested in changing schools, in part because of potential  long commutes and satisfaction with their current schools, which suggests  that parents and students would likely prefer to receive SES in their own  schools and neighborhoods rather than school choice. In line with  interest in increased flexibility with these interventions, in May 2006,  Education announced that due to the positive results in Virginia districts  under the pilot, the department plans to extend and expand this pilot in  2006-2007.
section_title: Conclusions
paragraphs: Over the last few years, almost all states and approximately 1,000 districts  have been required to implement SES across the country and, if current  trends continue, more schools will be required to offer services in the  future. Although some states and districts are beginning to gain experience  in implementing SES and use promising approaches to increase SES  participation, many students are still not receiving services, in part  because providers are sometimes not available to serve certain areas and  groups. In addition, some districts are unsure how to involve school  officials in facilitating local coordination of SES implementation and  providing effective parental notification. While Education has provided  support to states and districts through guidance and technical assistance,  many report needing additional assistance to address these challenges.  Further, the lack of clarity between policy guidance issued by OII and  criteria used by OESE in their compliance reviews of states’  implementation efforts creates additional challenges in meeting the federal  requirements for parental notification letters. Providing states with clear  guidance that has been coordinated across Education offices is especially  important now that the department has announced plans to take  significant enforcement actions to ensure states comply with federal SES  requirements.
While some districts do not have any students receiving services and,  therefore, are not spending any Title I funds for SES, other districts are  spending more than their entire set-aside and still have students on waiting  lists to receive services. Two districts have been able to participate in  Education’s pilot program waiving federal regulations that preclude  districts in need of improvement from providing SES, which may help  them provide services to more students at a lower cost. However,  extending this flexibility to more districts depends on the evaluation of the  quality of these services to determine if SES is having a positive impact on  student academic achievement. In addition, the absence of strategies that  states can use to set parameters around program design and costs further  hinders their ability to stretch available funding to serve more students.  Federal and state oversight of district efficiency in using federal funds to  provide SES services is hindered by incomplete reporting requirements  that require states to report on the number of eligible children receiving  SES, but not the data they collect on the amount of Title I funding used to  serve them. This information gap limits Education’s ability to track state  and district compliance in spending funds for SES. Further, Education’s  ability to ensure that federal dollars are effectively spent to improve  student academic achievement is limited until states increase their  capacity to monitor and evaluate provider performance. In the absence of  additional federal technical assistance and access to information about  state and district promising practices, some states may continue to  struggle with implementation and evaluation of SES.
section_title: Recommendations
paragraphs: To help states and districts implement SES more effectively, we  recommend that the Secretary of Education use the department’s Web site  and the Center on Innovation and Improvement, as well as other means of  communication, to:    Provide federal guidance on SES parental notification letters that is  clear and has been coordinated internally between OII and OESE to  provide additional assistance to states and districts to help them  comply with federal requirements and ensure that letters are easy for  parents to understand. Education might consider providing several  samples of actual district notification letters that meet these criteria.    Collect and disseminate information on promising practices used by  states to attract more providers for certain areas and groups and  promising practices used by districts to improve parental notification  of SES services and providers’ ability to serve specific groups of  students and to encourage student attendance.
Provide examples of how districts can involve schools and school  officials to facilitate local coordination with providers.
To improve states’ and districts’ ability to make the most of funding for  SES and provide services to the maximum number of students, we  recommend that the Secretary of Education:    Consider expanding the 2005-2006 pilot that allowed two districts in  need of improvement to enter into flexibility agreements to serve as  SES providers if evaluation results show that these districts can  provide quality SES services.
Clarify what states can do through the provider approval process to set  parameters around program design and costs. For example, Education  could issue guidance to states that clarifies their authority to set  parameters on issues such as minimum hours of SES per student,  minimum tutor qualifications, and cost ranges. In addition, Education  could suggest to states that they coordinate these discussions with  districts to address their concerns about program design and costs.
To improve federal and state monitoring of SES, we recommend that the  Secretary of Education:    Require states to report information necessary to determine the  amount of funds spent by districts to provide SES and the percentage  of their Title I allocations that this amount represents. Because almost  all states reported that they are planning to monitor district SES  expenditures, Education could require the states to submit these data  through the annual NCLBA Consolidated State Performance Reports.
Provide states with technical assistance and guidance on how to  evaluate the effect of SES on student academic achievement. For  example, Education might require the Center on Innovation and  Improvement to focus its SES assistance on providing states with  suggested evaluation methods, measures to assess the impact of SES  on achievement, and criteria for using this information to monitor and  withdraw state approval from providers. Further, lessons learned and  promising practices on evaluation could also be shared with states on  the Center on Innovation and Improvement’s Web site or during  national or regional meetings, trainings, or conferences.
section_title: Agency Comments
paragraphs: We provided a draft of this report to Education for review and comment.  Educations’ written comments are reprinted in appendix II, and the  department's technical comments were incorporated into the report as  appropriate. In its written comments, Education expressed appreciation  for the report’s recommendations and cited actions the department has  already initiated or plans to take in addressing them. Specifically,  Education noted several projects under development that might assist in  carrying out our recommendations to provide more assistance to states on  notifying parents, attracting providers for certain areas and groups, and  involving schools. The department also said that is currently considering  expanding the pilot allowing districts in need of improvement to apply to  become SES providers, per our recommendation. Regarding our  recommendation that Education clarify what states can do to set  parameters around program design and costs, Education said it would  consider addressing this issue further in the next set of revisions to the  SES non-regulatory guidance. In addition, Education said it would address  our recommendations to improve federal and state monitoring of SES by  proposing that districts report on their SES spending and by providing  more SES evaluation assistance to states through an updated issue brief as  well as technical assistance provided by the Comprehensive Center on  Innovation and Improvement and at a conference this fall.
We are sending copies of this report to appropriate congressional  committees, the Secretary of Education, and other interested parties.  Copies will also be made available upon request. In addition, the report  will be available at no charge on GAO’s Web site at http://www.gao.gov. If  you or your staff have any questions about the report, please contact me at  (202) 512-7215. Contact points for our Offices of Congressional Relations  and Public Affairs may be found on the last page of this report. GAO staff  who made major contributions to this report are listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: To obtain nationally representative information on supplemental  educational services (SES) participation, state and local implementation,  and federal oversight, we conducted a Web-based survey of state SES  coordinators and a mail survey of district SES coordinators from a  nationally representative sample of districts with schools required to offer  SES. We also conducted site visits during which we interviewed state,  district, and school officials representing four school districts, and we  conducted interviews with 22 SES providers both during the site visits and  separately. In addition, we spoke with staff at Education involved in SES  oversight and implementation and reviewed Education’s data on SES. We  conducted our work from August 2005 through July 2006 in accordance  with generally accepted government auditing standards.
subsections: section_title: Survey of States
paragraphs: To better understand state SES implementation, particularly how states  are monitoring and evaluating SES, we designed and administered a Web- based survey of state SES coordinators in all 50 states, the District of  Columbia, and Puerto Rico. The survey was conducted between January  and March 2006 with 100 percent of state SES coordinators responding.  The survey included questions about student participation in SES, actions  taken to increase participation, SES funding and expenditures, methods  used to monitor and evaluate implementation, implementation challenges,  and assistance received from Education.
Because this was not a sample survey, there are no sampling errors.  However, the practical difficulties of conducting any survey may introduce  nonsampling errors, such as variations in how respondents interpret  questions and their willingness to offer accurate responses. We took steps  to minimize nonsampling errors, including pre-testing draft instruments  and using a Web-based administration system. Specifically, during survey  development, we pre-tested draft instruments with officials in Oregon,  Maryland, and Washington between October and November 2005. In the  pre-tests, we were generally interested in the clarity of the questions and  the flow and layout of the survey. For example, we wanted to ensure  definitions used in the surveys were clear and known to the respondents,  categories provided in closed-ended questions were complete and  exclusive, and the ordering of survey sections and the questions within  each section was appropriate. On the basis of the pre-tests, the Web  instrument underwent some slight revisions. A second step we took to  minimize nonsampling errors was using a Web-based survey. By allowing  respondents to enter their responses directly into an electronic  instrument, this method automatically created a record for each  respondent in a data file and eliminated the need for and the errors (and  costs) associated with a manual data entry process. To further minimize  errors, programs used to analyze the survey data and make estimations  were independently verified to ensure the accuracy of this work.
While we did not fully validate specific information that states reported  through our survey, we took several steps to ensure that the information  was sufficiently reliable for the purposes of this report. For example, after  the survey was closed, we made comparisons between select items from  our survey data and other national-level data sets. We found our survey  data were reasonably consistent with the other data sets. On the basis of  our checks, we believe our survey data are sufficient for the purposes of  our work.
section_title: Survey of School Districts
paragraphs: To obtain national-level information on district implementation of SES, we  administered a mail survey to a nationally representative sample of  districts that had schools required to offer SES in school year 2004-2005.  The survey was conducted between January and March 2006. To obtain  the maximum number of responses to our survey, we sent a reminder  postcard to nonrespondents approximately 1 week after the initial mailing  of the survey instrument, a follow-up mailing with the full survey  instrument to nonrespondents approximately 3 weeks after the initial  mailing, and a second follow-up mailing with the full survey instrument  approximately 4 weeks later. The survey included questions about student  participation in SES, characteristics of students receiving SES, actions  taken to increase participation, SES funding and expenditures, methods  used to monitor and evaluate implementation, implementation challenges,  and assistance received and still needed.
subsections: section_title: Population
paragraphs: The target population of 1,095 districts consisted of public school districts  with at least one school in each of their jurisdictions required to provide  SES in the 2004-2005 school year. To define our population, we collected  school improvement information from state education agency Web sites  and the NCLBA Consolidated State Performance Reports: Part I for  Reporting on School Year 2003-2004 that states submitted to Education.  When available, we checked both sources for school improvement  information and used the source that provided the most recently updated  data, as this data is often updated by states over the course of the school  year. After constructing our population of districts, we used Education’s  Common Core of Data Local Education Agency (School District)  preliminary file and the Public Elementary/Secondary School preliminary  file for the 2003-2004 school year to further define the characteristics of  our population. On the basis of our review of these data, we determined  these sources to be adequate for the purposes of our work.
section_title: Sample Design and Errors
paragraphs: The sample design for the mail survey was a stratified random sample of  districts with one certainty stratum containing all of the districts with  100,000 students or more and one stratum containing all other districts in  the universe. We included the 21 districts with 100,000 or more students  with certainty in the sample to ensure we gathered information from the  largest districts required to offer SES.
We selected a simple random sample of districts in the non-certainty  stratum and calculated the sample size to achieve a precision of plus and  minus 7 percent at the 95 percent confidence level for an expected  proportion of 50 percent. To ensure the sample sizes were adequate, we  increased the sample size assuming we would obtain a 70 percent  response rate. The total sample size for this stratum was 237 districts. In  the sample, each district in the population had a known, nonzero  probability of being selected. Each selected district was subsequently  weighted in the analysis to account statistically for all the schools in the  population, including those that were not selected. Table 8 provides a  description of the universe and sample of districts.
Because we surveyed a sample of districts, our results are estimates of a  population of districts and thus are subject to sampling errors that are  associated with samples of this size and type. Our confidence in the  precisions of the results from this sample is expressed in 95 percent  confidence intervals, which are expected to include the actual results in   95 percent of the samples of this type. We calculated confidence intervals  for this sample based on methods that are appropriate for a stratified  random sample.
We determined that 10 of the sampled districts were out of scope because  they did not have any schools required to provide SES in the 2004-2005  school year. All estimates produced from the sample and presented in this  report are for the estimated target population of 1,034 districts with at  least one school required to provide SES in the 2004-2005 school year. All  percentage estimates included in this report have margins of error of plus  or minus 8 percentage points or less, except for those shown in table 9. All  other numerical estimates, such as the total number of schools required to  offer SES in 2004-2005, included in this report have margins of error of  plus or minus 18 percent or less.
We took steps to minimize nonsampling errors that are not accounted for  through statistical tests, like sampling errors. In developing the mail  survey, we conducted several pretests of draft instruments. We pretested  the survey instrument with district officials in Woodburn, Ore.; Tacoma,  Wash.; Baltimore, Md.; and Alexandria, Va., between October and  November 2005. These pre-tests were similar to the state Web survey pre- tests in design and content. On the basis of the pre-tests, the draft survey  instrument underwent some slight revisions.
While we did not fully validate specific information that districts reported  through our survey, we took several steps to ensure that the information  was sufficiently reliable for the purposes of this report. For example, data  from the surveys were double-keyed to ensure data entry accuracy, and  the information was analyzed using statistical software. After the survey  was closed, we also made comparisons between select items from our  survey data and other national-level data sets. We found our survey data  were reasonably consistent with the external sources. On the basis of our  checks, we believe our survey data are sufficient for the purposes of our  work.
section_title: Response Rate
paragraphs: We received survey responses from 73 percent of the 258 district Title  I/SES coordinators in our sample. The response rate, adjusted for the  known and estimated districts that were out of scope, was 77 percent.
After the survey was closed, we analyzed the survey respondents to  determine if there were any differences between the responding districts,  the nonresponding districts, and the population. We performed this  analysis for three characteristics—total number of students enrolled, total  number of special education students, and total number of English  language learner students. We determined whether sample-based  estimates of these characteristics compared favorably with the known  population values. The population value for all of the characteristics we  examined fell within the 95 percent confidence intervals for the estimates  from the survey respondents. On the basis of the 77 percent response rate  and this analysis, we chose to include the survey results in our report and  produce sample-based estimates to the population of districts required to  provide SES in the 2004-2005 school year.
section_title: Site Visits
paragraphs: To understand SES implementation at the local level, we conducted site  visits to four districts between October 17, 2005, and February 16, 2006.  The districts visited included Woodburn School District (Woodburn, Ore.),  Hamilton County Schools (Chattanooga, Tenn.), Newark Public Schools  (Newark, N.J.), and Chicago Public Schools (Chicago, Ill.). The four  districts visited were selected because they had experience implementing  SES in their schools and were recommended by stakeholders as having  promising parent outreach and/or monitoring practices. When viewed as a  group, the districts also provided variation across characteristics such as  geographic location, district size, student ethnicity, and the percentage of  students with limited English proficiency or disabilities.
During the site visits, we interviewed state officials, including the state  SES coordinator, and district officials, including the superintendent and  SES coordinator. We also interviewed officials representing 12 schools,  including principals and other school staff involved with SES. In total, we  visited several schools of each level, from elementary to high, and though  district officials selected the schools we visited, all of the schools had  experience implementing SES. Through our interviews with state, district,  and school officials, we collected information on district efforts to notify  parents and fulfill implementation responsibilities, student participation,  providers, local implementation challenges, and implementation  assistance received and needed. During the visits, we also interviewed  providers and observed tutoring sessions in order to better understand  implementation. During our visit to Woodburn, Ore., we also observed a  provider fair.
In addition to our site visits to four districts, we also visited the Rhode  Island Department of Elementary and Secondary Education in March 2006  to gather additional information on state efforts to monitor and evaluate  SES. Rhode Island invited us to attend two meetings the state held with  districts implementing SES and providers serving students in Rhode  Island, during which SES challenges, ways to improve implementation,  and state efforts to evaluate providers were discussed.
section_title: Provider Interviews
paragraphs: In total, we conducted interviews with 22 providers, including 15 providers  during the site visits and 7 providers operating in multiple states. The  Education Industry Association assisted our efforts to contact multi-state  providers, and most of the multi-state providers we interviewed were  association members. Multi-state provider interviews were conducted  between November and December 2005.
Through all of our provider interviews, we collected information on  provider efforts to increase participation in SES, align services with state  standards and district curriculum, and communicate with parents and  schools to ensure students are receiving needed services. We also  collected information on students served, tutor and program  characteristics, and provider challenges to SES implementation. While the  providers we interviewed reflect some variety in provider characteristics,  our selections were not intended to be representative. Thus, the findings  from our interviews cannot be used to make inferences about all  providers.
section_title: Education Data Analyses
paragraphs: We analyzed state data submitted to Education through the NCLBA  Consolidated State Performance Reports (CSPR) for school years 2002- 2003, 2003-2004, and 2004-2005. State reports from all 3 years included the  number of students receiving SES and the number of schools those  students attended, and state reports from 2003-2004 and 2004-2005 also  included the number of students eligible for SES. Data from the 2003-2004  CSPRs were used to assist our analysis of SES participation. To assess the  reliability of the 2003-2004 data, we performed a series of tests, which  included checking to ensure that data were consistent, that subtotals  added to overall totals and that data provided for 1 year bore a reasonable  relationship to the next year’s data and to data reported elsewhere,  including state education reports. We also spoke with Education officials  about their follow-up efforts to verify the data. At the time of our review,  Education was in the process of completing efforts to verify the 2003-2004  data.
While we compared the 2004-2005 CSPR data to data obtained through our  state and district surveys to further verify our data, we generally did not  use the 2004-2005 CSPR data for our analysis. During this comparison  analysis, where we found discrepancies or sought clarification, we  followed up with state officials. In several states, officials revised the  numbers that they had initially reported to us or to Education. On the  basis of our review of these data, we determined these sources to be  adequate for the purposes of our work.
We also considered SES-related findings from Education studies, including  the Evaluation of Title I Accountability Systems and School  Improvement Efforts: Findings From 2002-03 (2005) and the National  Assessment of Title I: Interim Report (2006). To ensure the findings from  these studies were generally reliable, we reviewed each study’s  methodology, including data sources and analyses, limitations, and  conclusions. In addition, in designing our state and district surveys, we  reviewed SES-related survey questions used by Education in these studies.
section_title: Appendix II: Comments from the Department of Education
section_title: Appendix III: GAO Contacts and Staff Acknowledgments
subsections: section_title: GAO Contacts
section_title: Acknowledgments
paragraphs: Cindy Ayers, Assistant Director, and Rachel Frisk, Analyst-in-Charge,  managed this assignment and made significant contributions to all aspects  of this report. Cathy Roark, Ted Burik, and David Perkins also made  significant contributions. Kevin Jackson, Jean McSween, Jim Ashley, and  Jerry Sandau provided methodological expertise and assistance; Rachael  Valliere assisted with message and report development; and Rasheeda  Curry made contributions during study design. In addition, Jessica  Botsford assisted in the legal analysis.
section_title: Related GAO Products
paragraphs: No Child Left Behind Act: Assistance from Education Could Help States  Better Measure Progress of Students with Limited English Proficiency.   GAO-06-815. Washington, D.C.: July 26, 2006.
No Child Left Behind Act: States Face Challenges Measuring Academic  Growth that Education’s Initiatives May Help Address. GAO-06-661.  Washington, D.C.: July 17, 2006.
No Child Left Behind Act: Most Students with Disabilities Participated  in Statewide Assessments, but Inclusion Options Could Be Improved.  GAO-05-618. Washington, D.C.: July 20, 2005.
No Child Left Behind Act: Education Needs to Provide Additional  Technical Assistance and Conduct Implementation Studies for School  Choice Provision. GAO-05-7. Washington, D.C.: December 10, 2004.
No Child Left Behind Act: Improvements Needed in Education’s Process  for Tracking States’ Implementation of Key Provisions. GAO-04-734.  Washington, D.C.: September 30, 2004.
No Child Left Behind Act: Additional Assistance and Research on  Effective Strategies Would Help Small Rural Districts. GAO-04-909.  Washington, D.C.: September 23, 2004.
Disadvantaged Students: Fiscal Oversight of Title I Could Be Improved.  GAO-03-377. Washington, D.C.: February 28, 2003.
Title I Funding: Poor Children Benefit Though Funding Per Poor Child  Differs. GAO-02-242. Washington, D.C.: January 31, 2002.