United States Government Accountability Office

GAO Report to Congressional Requesters

NO CHILD LEFT 

BEHIND ACT 

Education Actions 

Needed to Improve 

Local Implementation 

and State Evaluation 

of Supplemental 

Educational Services 

August 2006 

 

GAO-06-758 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

August 2006

NO CHILD LEFT BEHIND ACT 

Education Actions Needed to Improve 

Local Implementation and State 

Evaluation of Supplemental Educational 

Services 

Highlights of GAO-06-758, a report to 

congressional requesters 

The No Child Left Behind Act of 

2001 (NCLBA) requires districts 

with schools that have not met 

state performance goals for 

3 consecutive years to offer their 

low-income students supplemental 

educational services (SES), such as 

tutoring, if these schools receive 

Title I funds. SES are provided

outside of the regular school day by 

a state-approved provider, with

responsibility for implementation 

shared by states and districts. GAO 

examined (1) how SES 

participation changed between 

school years 2003-2004 and 2004-

2005; (2) how SES providers are 

working with districts to deliver 

SES; (3) how states are monitoring

and evaluating SES; and (4) how 

the Department of Education 

(Education) monitors and supports 

state implementation of SES. To

collect data on SES, GAO surveyed 

all states and a nationally

representative sample of districts 

with schools required to offer SES.

We also visited 4 school districts, 

interviewed 22 SES providers, 

reviewed SES-related research, and

interviewed Education staff. 

SES participation among eligible students increased from 12 to 19 percent

between school years 2003-2004 and 2004-2005, and the number of recipients 

also increased, due in part to a rise in the number of schools required to offer 

services. Districts have used some promising practices to inform parents and 

encourage participation, such as offering services on school campuses and at 

various times. However, challenges remain, including timely and effective

notification of parents and attracting providers to serve certain areas and

students, such as rural districts or students with disabilities. 

To promote improved student academic achievement, SES providers took steps 

to align their curriculum with district instruction and communicate with 

teachers and parents, though the extent of their efforts varied. A majority of the 

22 providers we interviewed worked to align SES and district curriculum by

hiring teachers familiar with the district curriculum as tutors. However, at least 

some providers did not have any contact with teachers in about 40 percent of 

districts. Both providers and district officials experienced challenges related to 

contracting and coordination of service delivery. Providers, districts, and 

schools reported that greater involvement of schools would improve SES 

delivery and coordination, as it has in some places where this is occurring. 

While state monitoring of district and provider efforts to implement SES has

been limited in past years, more states reported conducting on-site reviews and 

other monitoring activities during 2005-2006. In addition, districts have taken a 

direct role in monitoring providers, and their monitoring efforts have similarly 

increased. Although states are required to withdraw approval from providers

that fail to increase student academic achievement for 2 years, many states 

struggle to develop meaningful SES evaluations. While a few states have 

completed evaluations, none provides a conclusive assessment of SES providers’ 

effect on student academic achievement. 

Several Education offices monitor SES activity across the country and provide 

SES support to states and districts through written guidance, grants, and

technical assistance. However, states and districts reported needing additional

SES evaluation support and technical assistance. For example, 85 percent of

states reported needing assistance with methods for evaluating SES. Many also 

voiced interest in Education’s pilot programs that increase SES flexibility,

including the one that allowed certain districts identified as in need of

improvement to act as providers. 

SES Is Often Delivered after School and on School Campuses

Source: GAO.

What GAO Recommends

GAO recommends that Education 

disseminate information on

promising practices used to 

improve SES implementation, 

provide states with technical 

assistance to improve evaluation of 

SES’s effect on student

achievement, and expand program 

flexibility where appropriate. 

Education generally supported 

GAO’s recommendations. 

www.gao.gov/cgi-bin/getrpt?GAO-06-758.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Marnie Shaul at 

(202) 512-7215 or shaulm@gao.gov. 
Contents 

Letter 1

Results in Brief 3 

Background 7

SES Participation Has Increased as Districts Have Taken Steps to 

Ease Access, but Challenges Remain 13 

Providers Have Taken Steps to Deliver Quality Services, but 

Contracting and Coordination Remain Challenges to Working 

with Districts and Schools 22 

States’ SES Monitoring Has Been Limited Though Their Efforts Are 

Increasing, and Many States Struggle to Develop Meaningful 

Evaluations 28

Several Education Offices Monitor and Support SES 

Implementation, but States and Districts Cite the Need for 

Additional Assistance and Flexibility 37 

Conclusions 46

Recommendations 47

Agency Comments 48 

Appendix I Scope and Methodology 51 

Appendix II Comments from the Department of Education 60 

Appendix III GAO Contacts and Staff Acknowledgments 64 

Related GAO Products 65 

Tables 

Table 1: NCLBA Interventions for Schools Not Meeting Yearly

Performance Goals over Time 9 

Table 2: SES Stakeholder Roles and Responsibilities 12 

Table 3: District Actions Taken to Encourage SES Participation 

(2005-2006) 19

Table 4: State Actions Taken to Encourage SES Participation (2005-

2006)19 

Page i GAO-06-758 No Child Left Behind Act
Table 5: Percentage of States and Districts That Reviewed 

Specified Program Elements to Monitor Providers in 2005-

200634 

Table 6: Percentage of States and Districts in Need of Different 

Types of Information or Assistance with SES 

Implementation 43

Table 7: State and District Opinion on the Ordering of School

Choice and SES 45 

Table 8: Description of the Population and Sample of Districts 53 

Table 9: Sampling Error Calculations 55 

Table 10: SES District Survey Response Rates 56 

Figures 

Figure 1: Number of Students Receiving SES Nationwide (2002-

2003 to 2004-2005) 14 

Figure 2: School Districts Required to Offer SES in 2004-2005 16 

Figure 3: Estimated Percentage of Districts Experiencing Certain 

Issues with None, Few, Some, Most, or All of Their 

Providers in 2004-2005 24 

Figure 4: Percentage of States That Reviewed Specific Program 

Elements to Monitor District SES Implementation in 2005-

2006 29

Figure 5: Percentage of States Reviewing Information Collected 

from Different Sources to Monitor SES Providers in 2004-

2005 and 2005-2006 31 

Figure 6: Estimated Percentage of Districts Reviewing Information 

Collected from Different Sources to Monitor SES

Providers in 2004-2005 and 2005-2006 33 

Figure 7: U.S. Department of Education Offices Monitoring and 

Supporting SES 38 

Page ii GAO-06-758 No Child Left Behind Act
Abbreviations 

AYP adequate yearly progress 

CSPR Consolidated State Performance Report 

ESEA Elementary and Secondary Education Act 

NCLBA No Child Left Behind Act

NSLP National School Lunch Program 

OESE Office of Elementary and Secondary Education 

OIG Office of the Inspector General 

OII Office of Innovation and Improvement 

SES supplemental educational services 

SESQC Supplemental Educational Services Quality Center 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 

Page iii GAO-06-758 No Child Left Behind Act
United States Government Accountability Office

Washington, DC 20548 

August 4, 2006 

Congressional Requesters

Title I of the No Child Left Behind Act (NCLBA) supports the academic 

achievement of 16.5 million low-income students nationwide by providing 

them with additional opportunities, such as supplemental educational 

services (SES), if their schools are low performing. SES includes tutoring 

and remediation that are provided outside of the regular school day by a 

state-approved provider, such as a for-profit company or a communitybased organization. When a Title I school does not meet state performance 

goals for 2 years, including goals set under the Elementary and Secondary 

Education Act (ESEA) before the enactment of NCLBA, the district must 

offer students the choice of transferring to another school in the district 

that is not in improvement status. When a Title I school does not meet 

state performance goals for 3 or more years, the district must offer SES to 

all of the low-income students enrolled in the school. Districts with 

schools required to offer school choice and SES must set aside an amount 

equal to 20 percent of their Title I funds to provide choice-related 

transportation and SES for eligible students in these schools. In school 

year 2005-2006, $12.7 billion in federal Title I funding was distributed to 

nearly all school districts and approximately half of the public schools 

nationwide in order to improve the education of low-income students. 

Title I of the No Child Left Behind Act (NCLBA) supports the academic 

achievement of 16.5 million low-income students nationwide by providing 

them with additional opportunities, such as supplemental educational 

services (SES), if their schools are low performing. SES includes tutoring 

and remediation that are provided outside of the regular school day by a 

state-approved provider, such as a for-profit company or a communitybased organization. When a Title I school does not meet state performance 

goals for 2 years, including goals set under the Elementary and Secondary 

Education Act (ESEA) before the enactment of NCLBA, the district must 

offer students the choice of transferring to another school in the district 

that is not in improvement status. When a Title I school does not meet 

state performance goals for 3 or more years, the district must offer SES to 

all of the low-income students enrolled in the school. Districts with 

schools required to offer school choice and SES must set aside an amount 

equal to 20 percent of their Title I funds to provide choice-related 

transportation and SES for eligible students in these schools. In school 

year 2005-2006, $12.7 billion in federal Title I funding was distributed to 

nearly all school districts and approximately half of the public schools 

nationwide in order to improve the education of low-income students. 

While states set NCLBA performance goals and schools are judged on the 

performance of their students, responsibility for SES implementation is 

primarily shared by states and districts under the law. States are 

responsible for approving SES providers to serve students in their states 

and, therefore, review provider applications to assess each provider’s 

record of effectiveness and program design, including its instructional 

strategies and service costs. States are also responsible for monitoring 

SES providers and evaluating their services. Districts are responsible for 

notifying parents of their child’s eligibility for SES and contracting with 

the state-approved providers parents select for services. At the federal

level, the U.S. Department of Education (Education) oversees SES 

implementation nationwide and provides guidance and technical 

assistance. 

While states set NCLBA performance goals and schools are judged on the 

performance of their students, responsibility for SES implementation is 

primarily shared by states and districts under the law. States are 

responsible for approving SES providers to serve students in their states 

and, therefore, review provider applications to assess each provider’s 

record of effectiveness and program design, including its instructional 

strategies and service costs. States are also responsible for monitoring 

SES providers and evaluating their services. Districts are responsible for 

notifying parents of their child’s eligibility for SES and contracting with 

the state-approved providers parents select for services. At the federal

level, the U.S. Department of Education (Education) oversees SES 

implementation nationwide and provides guidance and technical 

assistance. 

Although some districts were first required to offer SES in 2002-2003, 

others did not have to offer SES until 2003-2004 or the year after, and 

therefore, states and districts are at different stages of implementing the 

Although some districts were first required to offer SES in 2002-2003, 

others did not have to offer SES until 2003-2004 or the year after, and 

therefore, states and districts are at different stages of implementing the 

Page 1 GAO-06-758 No Child Left Behind Act
SES provisions. Further, research suggests that many face challenges to 

increasing student participation, working with providers to ensure 

students receive quality services, and monitoring and evaluating these 

services. In response to congressional interest in these and other issues 

related to early implementation of SES, and in anticipation of

reauthorization of the ESEA, we agreed to answer the following questions: 

(1) How has SES participation changed between school years 2003-2004 

and 2004-2005, and what actions have been taken to increase 

participation? (2) How are providers working with districts and schools to 

provide services that increase student achievement? (3) To what extent 

are states monitoring and evaluating SES implementation? (4) How does 

Education monitor state SES implementation and assist state and district 

efforts? 

To answer our four research questions, we collected data through several

methods. First, to gather state-level information on SES, we collected 

information through a Web-based survey of state SES coordinators in all 

50 states, the District of Columbia, and Puerto Rico. Further, to gather 

district-level information on SES, we conducted a mail survey of district 

SES coordinators from a nationally representative sample of districts with 

schools required to offer SES. To ensure that we obtained information 

from the largest districts with schools required to offer SES, we included 

all 21 districts with 100,000 or more enrolled students in our sample. In 

designing our surveys, we reviewed survey questions used by Education. 

The district and state surveys were both administered between January 

and March 2006. Seventy-seven percent of district SES coordinators, 

including all coordinators from districts with 100,000 or more enrolled 

students, and all state SES coordinators responded to the surveys. All 

percentage estimates presented from the district survey have margins of 

error of plus or minus 8 percentage points, unless otherwise noted. While 

we did not validate specific information that states and districts reported 

through our surveys, we reviewed the information to determine that their

responses were complete and reasonable and found the information to be 

sufficiently reliable for the purposes of this report. To gather federal-level 

information on SES implementation, we interviewed Education officials 

and analyzed Education’s data on SES available from state reports and 

ongoing studies. 

To collect additional information from states and districts, as well as 

information from schools and providers, we conducted site visits to four 

school districts that had experience with SES implementation (Woodburn, 

Ore.; Newark, N.J.; Chicago, Ill.; and Hamilton County, Tenn.). During the 

visits, we spoke with state and district officials, as well as officials from a 

Page 2 GAO-06-758 No Child Left Behind Act
total of 12 schools and 15 providers. When viewed as a group, the site visit

districts provided variation across characteristics such as geographic 

location, district size, student ethnicity, and the percentage of students 

with limited English proficiency or disabilities. In addition to the 

15 providers we spoke with during the site visits, we also interviewed 

7 for-profit SES providers operating in multiple states, for a total of 

22 providers. In our surveys and other data collection efforts, we asked 

questions about SES implementation during specific school years, and, 

therefore, all years cited refer to school years. 

See appendix I for detailed information on both of our surveys as well as 

our other data collection methods. We conducted our work from August 

2005 through July 2006 in accordance with generally accepted government 

auditing standards. 

Nationally, SES participation increased from 12 percent of eligible 

students receiving services in 2003-2004 to 19 percent in 2004-2005, but 

challenges to increasing participation remain. The total number of 

students receiving services rose from approximately 117,000 in 2002-2003 

to 430,000 in 2004-2005, due in part to a rise in the number of schools 

required to offer services. An estimated 20 percent of districts required to

offer SES had no students receiving services, and the majority of those 

districts were rural or had less than 2,500 enrolled students. While these 

districts, therefore, spent no funds for SES, some other districts that were 

providing services reported needing more than the 20 percent Title I setaside to serve all eligible students that had requested services. While a 

student’s eligibility for SES is based on income and not academic 

achievement, most students receiving services were among the lower 

achieving students in school. Districts have taken multiple actions to 

encourage participation, such as using several methods to contact parents 

and offering services on school campuses and at various times. For 

example, 90 percent or more of districts provided parents with written 

information, held individual meetings with parents, encouraged school 

staff to talk with parents about SES, and offered services in locations 

easily accessible to students, such as on or near the school campus. 

Although some districts have used promising practices to increase SES 

participation, notifying parents in a timely and effective way continues to

be a challenge for districts. About half of the districts did not notify 

parents before the beginning of the current school year, due in part to 

delays in receipt of school improvement results from their state. Further, 

several providers indicated that confusing parental notification letters and

enrollment forms may discourage participation. States and districts have 

Results in Brief 

Page 3 GAO-06-758 No Child Left Behind Act
also been challenged to ensure there are providers to serve students from 

rural areas or students with limited English proficiency or disabilities. 

Encouraging student attendance has also been a challenge, in part because 

students may participate in other after-school activities, such as sports or 

work. 

Providers reported that they are working with districts and schools to 

deliver quality services, but contracting with districts and coordinating 

with schools remain challenges, and all parties reported that 

implementation would be improved with greater school involvement. To 

help improve student academic achievement, providers align their 

curriculum with district instruction primarily by hiring teachers familiar 

with the district curriculum and communicating with the teachers of 

participating students. Providers reported communicating with teachers in 

person as well as mailing information and progress reports to them, which 

are similar to the methods reported for communicating with parents. 

However, we estimate that some, most, or all providers did not contact 

teachers in about 40 percent of districts and did not contact parents in 

about 30 percent of districts during the 2004-2005 school year. In addition, 

contracting for services was sometimes a challenge for providers and 

districts. Some of the providers we interviewed said certain districts 

impose burdensome contract requirements, such as limiting provider 

marketing and use of school facilities. About 40 percent of districts also 

considered contracting to be a challenge, and officials in three of the four

districts we visited expressed concerns about their lack of authority under 

the law to set parameters around costs and program design, such as the 

number of service hours per student and the tutor-to-student ratio. While 

officials from one state we visited expressed interest in additional federal 

guidance on setting such parameters, another state helped to address 

some of its district’s concerns and improve transparency and 

accountability by requiring approved providers to submit cost information 

and sharing this information with districts. About 70 percent of the states 

also reported that coordination among providers, districts, and schools is 

a moderate to very great challenge. For example, services were delayed or

withdrawn in three of the four districts we visited because not enough 

students signed up to meet the providers’ enrollment targets and districts 

were not aware of these targets. Because SES may be delivered in school 

facilities, providers and officials in the districts and schools we visited 

reported that involvement of school administrators and teachers can ease 

SES delivery and coordination with providers. 

Prior to 2005-2006, state SES monitoring of both districts and providers 

was limited, and over the last few years, many states have struggled to 

Page 4 GAO-06-758 No Child Left Behind Act
develop meaningful SES evaluations. In regard to monitoring districts, 

more states reported conducting on-site reviews in 2005-2006, and many 

states also collected information from other sources, such as school 

principals and parents. Further, nearly all states were collecting 

information on districts’ expenditures for SES and their efforts to notify 

parents of available services. In regard to monitoring providers, though 

states were challenged by their oversight capacity, many states reported 

using a more active monitoring approach in 2005-2006. For example, 

approximately three-fourths indicated they were conducting on-site 

reviews, and many were also collecting information from districts, 

schools, and parents to monitor SES providers in that year. In addition to 

states, districts have also taken an active role in monitoring providers, and 

similarly increased their efforts in 2005-2006. While states are required to 

withdraw approval from providers that fail to increase student academic 

achievement for 2 years, many are struggling to develop meaningful SES 

evaluations that assess this outcome. Approximately three-fourths of 

states reported that they are experiencing challenges evaluating SES, 

including designing methods for determining sufficient academic progress 

of students, having the time and knowledge to analyze SES data, and 

developing data systems to track SES information. For example, during 

our site visits to Illinois and New Jersey, state officials noted that they 

were currently in the process of improving their data collection systems to 

more effectively capture and analyze data for SES evaluations. At the time 

of our survey, only a few states had drafted or completed an evaluation 

report addressing SES providers’ effect on student academic achievement, 

and no state had produced a report that provided a conclusive assessment 

of this effect. In addition, over half of the states reported that they were in 

the process of conducting an SES evaluation in order to meet the federal

requirement. 

Several Education offices monitor SES and support implementation 

through written guidance and technical assistance grants, but states and 

districts cite the need for additional assistance and flexibility. Two 

Education offices are primarily responsible for SES monitoring. The Office 

of Innovation and Improvement (OII) monitors SES implementation by 

reviewing SES-related reports and responding to state, district, and 

provider concerns brought to its attention, while the Office of Elementary 

and Secondary Education (OESE) visits states and districts as part of its 

overall Title I monitoring. The Office of the Inspector General (OIG) also 

conducts monitoring through on-site SES audits. Education shares 

monitoring results internally and has used them to help states and districts 

address implementation challenges. For example, OII has updated the SES

non-regulatory guidance several times since 2002, and OESE has 

Page 5 GAO-06-758 No Child Left Behind Act
recommended actions needed to improve state compliance with federal 

regulations. Education also issues grants that provide SES implementation 

support. For example, OESE recently funded the Center on Innovation and 

Improvement, which provides technical assistance to regional centers to 

help them work with states to improve Education’s programs, including 

SES. Despite Education’s efforts, many states and districts reported that 

they need additional information or assistance with certain aspects of SES

implementation to better comply with SES requirements. For example, 

85 percent of states and an estimated 70 percent of districts needed 

assistance with methods for evaluating SES. Further, states and districts 

both indicated needing additional capacity to implement SES. Officials 

also reported needing additional or clearer guidance from Education on 

certain SES provisions, such as crafting a notification letter that is both 

complete and easy for parents to understand. Although Education has 

cited several states for districts’ incomplete notification letters, the 

department’s current SES guidance provides a sample parental 

notification letter that does not clearly specify all of the required elements. 

Many states also reported needing a forum to share promising practices 

with peers, such as meetings and conferences. In addition, many states 

and districts expressed interest in expansion of Education’s pilot 

programs. For example, some officials suggested expanding Education’s 

pilot that enables low-achieving districts typically restricted from being 

SES providers to provide these services on the condition that they 

evaluate the effect of their services on student achievement. Officials 

noted this may ease access to SES and potentially reduce costs. 

To help states and districts implement SES more effectively, we are 

recommending that Education collect and disseminate information on 

promising practices used by districts, including examples of how districts 

have successfully involved school officials in SES implementation, and 

sample parental notification letters that meet federal requirements and are 

easy for parents to understand. Further, to improve states’ and districts’ 

ability to provide services to the maximum number of students, we are 

recommending that Education consider expanding its current SES pilot 

program allowing selected districts in need of improvement to serve as 

providers and clarify state authority to set parameters around service 

design and costs. To improve federal and state monitoring of SES, we are 

recommending that Education require states to collect and submit 

information on the amount and percent of Title I funds spent on SES by 

districts and provide states with technical assistance and additional 

guidance on how to evaluate the effect of SES on student academic 

achievement. 

Page 6 GAO-06-758 No Child Left Behind Act
In written comments on a draft of this report, Education supported our 

recommendations and cited actions the department has already initiated

or plans to take in addressing them. Education also provided technical 

comments, which were incorporated into the report as appropriate. 

Under NCLBA, SES primarily includes tutoring provided outside of the 

regular school day that is designed to increase the academic achievement 

of economically disadvantaged students in low-performing Title I schools. 

These services must consist of high-quality, research-based instruction 

that aligns with state educational standards and district curriculum. 

Title I1

 of ESEA, as amended and reauthorized by NCLBA, authorizes 

federal funds to help elementary and secondary schools establish and 

maintain programs that will improve the educational opportunities of 

economically disadvantaged children. Title I is the largest federal program 

supporting education in kindergarten through 12th grade, supplying 

$12.7 billion in federal funds in fiscal year 2006. According to Education, 

during the 2005-06 school year, nearly all U.S. school districts and 

approximately half of public schools received some Title I funding. In 

addition, the latest national data available from Education counted 

16.5 million students as Title I participants in the 2002-2003 school year. 

Title I funds are distributed by formula to state education agencies, which

retain a share for administration and school improvement activities before

passing most of the funds on to school districts. Districts are required to 

distribute Title I funds first to schools with poverty rates over 75 percent, 

with any remaining funds distributed at their discretion to schools in rank 

order of poverty either districtwide or within grade spans. A school’s Title 

I status can change from year to year because school enrollment numbers 

and demographics may vary over time. 

Enactment of NCLBA strengthened accountability by requiring states and 

schools to improve the academic performance of their students so that all 

students are proficient in reading and math by 2014. Under NCLBA, each 

state creates its own content standards, academic achievement tests, and 

Background 

Title I 

NCLBA 

 1

 In this report, we refer to Title I, Part A of ESEA as “Title I.” Other Parts of Title I (Parts B,

C, and D) are targeted at specific populations or purposes and are commonly referred to by

their program names, such as Even Start.

Page 7 GAO-06-758 No Child Left Behind Act
proficiency levels. In 2005-2006, states were required to test all children for 

reading and mathematics achievement annually in grades 3-8 and once in 

high school to determine whether schools are making adequate yearly 

progress (AYP).2

In addition to meeting the state’s performance goals by grade, subject, and

overall student population, schools are responsible for meeting those 

goals for designated groups. These groups are students who (1) are 

economically disadvantaged, (2) are part of a racial or ethnic group that 

represents a significant proportion of a school’s student population, 

(3) have disabilities, or (4) have limited English proficiency. To make AYP, 

each school must also show that each of these groups met the state 

proficiency goals for both reading and math. In addition, schools must 

show that at least 95 percent of students in grades required to take the test 

have done so. Schools must also demonstrate that they have met state 

targets for at least one other academic indicator, including graduation rate

in high schools and a state-selected measure in elementary or middle 

schools.3

For Title I schools that do not meet state AYP goals, NCLBA requires the 

implementation of specific interventions, and these interventions must 

continue until the school has met AYP for 2 consecutive years. Table 1 

outlines the interventions applied after each year a Title I school misses 

state performance goals. At their discretion, states may also implement 

interventions for public schools that do not receive Title I funds and do 

not make AYP. Although districts are not required to offer SES until a Title 

I school has missed performance goals for 3 years, because some schools 

had not met state goals set under ESEA before the enactment of NCLBA, 

some Title I schools were first required to offer SES in 2002-2003, the first 

year of NCLBA implementation. 

 2

 State testing has been phased in, whereby from 2002-2004 states were required to 

administer tests at least once in grades 3-5, 6-9, and 10-12. Beginning in 2005-2006, states 

must administer annual tests in grades 3-8, and at least once in grades 10-12. Testing for 

science proficiency will begin in 2007-2008. 

3

 Measures for elementary and middle schools may include, but are not limited to, 

attendance, grade-to-grade retention rates, or changes in the percentage of students 

completing gifted and talented programs. 

Page 8 GAO-06-758 No Child Left Behind Act
Table 1: NCLBA Interventions for Schools Not Meeting Yearly Performance Goals 

over Time 

Number of years 

school misses 

performance goals

School status in the next

year 

NCLBA interventions for Title I 

schools

First year missed N/A None 

Second year missed Needs Improvement – 

First Year 

Required to offer school choice 

Third year missed Needs Improvement – 

Second Year 

Required to offer school choice 

and SES 

Fourth year missed Corrective ActionaRequired to offer school choice 

and SES 

Fifth year missed Planning for Restructuringb Required to offer school choice 

and SES 

Sixth year missed Implementation of 

Restructuring 

Required to offer school choice 

and SES 

Source: GAO analysis of NCLBA. 

Note: N/A = not applicable. 

a

Corrective action is a significant intervention in a school that is designed to remedy the school’s 

persistent inability to make adequate progress toward all students becoming proficient in reading and

mathematics. 

b

Restructuring is a major reorganization of a school, involving fundamental reforms, such as 

significant changes in the school’s staffing and governance. For example, some schools may be 

converted to charter schools during restructuring. 

States are also required to establish and implement AYP standards for 

school districts based on the performance of all of the schools in the 

district. If districts fail to meet these standards for 2 consecutive years, 

states may classify districts as needing improvement. A district identified

for improvement must develop and implement an improvement plan and

remain in this status until it meets AYP standards for 2 consecutive years. 

If a district remains in improvement status for 2 or more years, it may be 

identified for corrective action as deemed necessary by the state. 

SES under NCLBA Students are eligible for SES if they attend Title I schools that have missed 

AYP for 3 consecutive years and are from low-income families. School 

districts must determine family income on the same basis they use to 

make allocations to schools under Title I, for which many have historically 

used National School Lunch Program (NSLP) data. The NSLP is a federally 

funded program that annually collects family income data from students’ 

Page 9 GAO-06-758 No Child Left Behind Act
parents to determine student eligibility for free and reduced-priced lunch.

4

A student’s state assessment scores, grades, and other academic 

achievement information are generally not considered when determining 

SES eligibility. However, if sufficient funds are not available to provide 

SES to all eligible children, school districts must give priority to the 

lowest-achieving eligible students. 

SES providers may include nonprofit entities, for-profit entities, school 

districts, public schools, public charter schools, private schools, public or 

private institutions of higher education, educational service agencies, and 

faith-based organizations. Under the Title I regulations that govern SES, a 

district identified as in need of improvement or corrective action may not 

be an SES provider, though its schools that are not identified as needing

improvement may. In addition, individual teachers who work in a school 

or district identified as in need of improvement may be hired by any stateapproved provider to serve as a tutor in its program. 

A district must set aside an amount equal to 20 percent of its Title I 

allocation to fund both SES and transportation for students who elect to

attend other schools under school choice. This set-aside cannot be spent 

on administrative costs for these activities, and the district may reallocate 

any unused set-aside funds to other Title I activities after ensuring all 

eligible students have had adequate time to opt to transfer to another 

school or apply for SES. Funding available for SES is, therefore, somewhat

dependent on costs for choice-related transportation, though as we found 

in our 2004 report on NCLBA’s school choice provisions, few students are 

participating in the school choice option.5

 If a district does not incur any 

choice-related transportation costs, it must use the full 20 percent set-

 4

 In addition, NSLP allows certain schools and districts with high percentages of lowincome families to certify students as eligible for free and reduced price lunches about 

once every 4 years. These alternatives to the traditional requirements for annual 

certification, known as “Provision 2” and “Provision 3,” reduce local paperwork and

administrative burden. Education has determined that, for purposes of identifying students

as “economically disadvantaged” for SES, school officials may deem all students in 

Provision 2 and Provision 3 schools as “economically disadvantaged.” See policy memo: 

U.S. Department of Agriculture, Food and Nutrition Service, “No Child Left Behind Act

(NCLB),” School Meals (Washington, D.C.: Feb. 2003),

http://www.fns.usda.gov/end/governance/policy-memos/2002-02-20.htm 

5

 Only 1 percent of students eligible for school choice elected to transfer to another school 

during the 2003-2004 school year. See GAO, No Child Left Behind Act: Education Needs to 

Provide Additional Technical Assistance and Conduct Implementation Studies for

School Choice Provision, GAO-05-07 (Washington, D.C.: Dec. 10, 2004). 

Page 10 GAO-06-758 No Child Left Behind Act
aside amount to pay for SES if sufficient demand for services exists. In 

addition, if the Title I set-aside is not sufficient to fund SES for interested 

students, both states and districts may direct other funds for these 

services at their discretion. For each student receiving SES, a district must 

spend an amount equal to its Title I per-pupil allocation or the actual cost 

of provider services, whichever is less.

6

Education oversees SES implementation by monitoring states and 

providing technical assistance and support. OII leads SES policy 

development and coordinates the publication of SES guidance, and OESE 

oversees and monitors Title I, including SES. NCLBA and the Title I 

regulations and SES guidance outline the roles and responsibilities states, 

school districts, parents, and service providers have in ensuring that 

eligible students receive additional academic assistance through SES 

(see table 2). 

 6

 A district calculates the Title I per pupil allocation by dividing its total Title I, Part A 

allocation by the number of children residing within the district aged 5-17 who are from 

families below the poverty level, as determined by the most recent Census Bureau 

estimates from the Department of Commerce. 

Page 11 GAO-06-758 No Child Left Behind Act
Table 2: SES Stakeholder Roles and Responsibilities 

Stakeholder Roles and responsibilities 

State Set criteria and standards for approving providers 

Identify, approve, and maintain public list of providers 

Ensure that the list of approved providers includes organizations that are able to serve students with disabilities and 

limited English proficiency 

Monitor and evaluate the effectiveness of provider services

Monitor district SES implementation 

Develop and use policy criteria for withdrawing providers from state-approved list, including if 

• provider fails for 2 consecutive years to increase student proficiency relative to state academic content and 

achievement standards 

• provider fails to adhere to applicable health, safety, and civil rights requirements 

School district Provide an annual notice to parents, which must identify available providers, describe the enrollment process and 

timeline, describe the services, qualifications, and demonstrated effectiveness of each provider, and be easily 

understandable 

Help parents choose a provider, if requested

Protect the privacy of students eligible for and receiving services 

Calculate and establish the SES per pupil allocation if not determined by the state 

Determine which students should receive services if more students apply for SES than can be served with available 

funds 

Enter into contracts with providers 

Ensure eligible students with disabilities and eligible students with limited English proficiency may participate in SES

At the discretion of the state, may be involved in collecting data from providers to assist state monitoring and 

evaluation activities 

Providers Provide services in accordance with district agreements 

Enable students to attain their individual achievement goals

Measure student progress and inform parents and teachers of progress made by students 

Ensure non-disclosure of student data to the public 

Provide services consistent with applicable health, safety, and civil rights laws 

Provide services that are secular, neutral, and non-ideological 

Parents Choose a provider from the state-approved list 

Are encouraged to be actively involved in their child’s SES program 

Source: GAO, per P.L.107-110, 34 C.F.R. Part 200, or the U.S. Department of Education, Supplemental Educational Services NonRegulatory Guidance, June 2005. 

During the 2005-2006 school year, Education announced the 

implementation of two pilot programs intended to increase the number of 

eligible students receiving SES and generate additional information about 

the effectiveness of SES on students’ academic achievement. In the first, 

Education permitted four districts in Virginia to offer SES instead of 

school choice in schools that are in their first year of needs improvement. 

Page 12 GAO-06-758 No Child Left Behind Act
In the second, Education entered into flexibility agreements with the 

Boston and Chicago school districts, enabling them to act as SES 

providers while in improvement status. OII and OESE coordinated 

implementation of the pilots for the department. Both pilots were subject 

to review at the end of the 2005-2006 school year, at which time Education 

planned to evaluate their effect on student academic achievement. 

SES participation increased between 2003-2004 and 2004-2005, and most 

students receiving services were among the lower achieving students in 

school. Districts have taken multiple actions to encourage participation, 

such as offering services on or near the school campus or at various times. 

Despite these efforts, challenges to increasing participation remain, 

including notifying parents in a timely and effective manner, ensuring 

there are providers to serve certain areas and students, and encouraging 

student attendance. 

Nationally, the participation rate increased substantially from 12 percent 

of eligible students receiving SES in 2003-2004 to 19 percent in 2004-2005.7

In addition, the number of students receiving services almost quadrupled 

between 2002-2003 and 2004-2005 from approximately 117,000 to 430,000 

students nationwide, based on the best available national data (see fig. 1).

This increase may be due in part to the increase in the number of schools 

required to offer SES over that time period. Specifically, between 2004-

2005 and 2005-2006 the number of schools required to offer SES increased 

from an estimated 4,509 to 6,584.8

SES Participation Has 

Increased as Districts 

Have Taken Steps to 

Ease Access, but 

Challenges Remain 

Participation among 

Eligible Students 

Increased from 12 to 19 

Percent between 2003-2004 

and 2004-2005, and Most 

Participants Were Low 

Achieving 

 7

 Certain states did not submit SES recipient information to Education through their 

NCLBA Consolidated State Performance Reports for all years. See notes to figure 1 for

details. Further, Education did not collect data on the number of students eligible for SES

in 2002-2003, and therefore an estimate of the SES participation rate is unavailable for that 

year. 

8

 In addition to our analysis, the Center on Education Policy reported that within districts

required to offer SES, the percentage of all schools required to offer SES rose from 27

percent of schools in 2002-2003 to 65 percent of schools in 2005-2006. See the Center on 

Education Policy, From the Capital to the Classroom, Year 4 of the No Child Left Behind 

Act (Washington D.C.: March 2006), for more information. 

Page 13 GAO-06-758 No Child Left Behind Act
Figure 1: Number of Students Receiving SES Nationwide (2002-2003 to 2004-2005) 

0

50,000

100,000

150,000

200,000

250,000

300,000

350,000

400,000

450,000

500,000

200

2004-2005

3-2004

2002-2003

Number of students

Source: Education’s NCLBA Consolidated State Performance Reports and GAO state survey.

School year

116,626 

245,267

430,044

Note: Certain states did not submit SES recipient information to Education through their NCLBA 

Consolidated State Performance Reports for all years. Specifically, 2002-2003 data from Kansas and 

North Dakota, 2003-2004 data from Pennsylvania, and 2004-2005 data from New Jersey are not 

included in the figure. In addition, 2002-2003 data from New York only include information from New

York City. 

Although nationally SES participation is increasing, some districts 

required to offer SES have no students receiving services. Specifically, we

estimate that no students received services in about 20 percent of the 

approximately 1,000 districts required to offer SES in 2004-2005. A 

majority of these districts were rural or had a total enrollment of fewer 

than 2,500 students. Our survey did not provide sufficient information to 

explain why these districts had no students receiving services in 2004-

2005; therefore, it is unclear whether their lack of participation was related 

to district SES implementation or other issues. 

Nationwide, we estimate that districts required to offer SES spent 

the equivalent of 5 percent of their total Title I funds for SES in 2004-2005

excluding administrative expenditures. Districts set aside an amount equal 

to 20 percent of their Title I funds for SES and choice-related 

transportation at the beginning of the school year, and the proportion of 

the set-aside spent on SES varied by district. Specifically, in 2004-2005, 

Page 14 GAO-06-758 No Child Left Behind Act
about 40 percent of districts spent 20 percent or less of the set-aside to 

provide SES and almost one-fifth of districts spent over 80 percent.9

Nationwide, of the total amount districts set-aside for SES, we estimate 

they spent 42 percent on SES, excluding administrative expenditures. 

Further, an estimated 16 percent of districts reported that the required 

Title I set-aside was not sufficient to fund SES for all eligible students 

whose parents requested services. For example, during our site visit to 

Newark, N.J., district officials reported budgeting the entire 20 percent 

Title I set-aside to fund SES in 2004-2005, but with this amount of funding, 

the district was only able to fund SES for 17 percent of the students 

eligible for services. In addition, according to Chicago, Ill., district 

officials, the district budgeted the entire 20 percent Title I set-aside to fund

SES in 2005-2006, and because parents’ demand for services significantly 

exceeded the amount of funding available, the district also allocated 

$5 million in local funds to provide SES. 

While approximately 1,000 of the over 14,000 districts nationwide were 

required to offer SES in 2004-2005, SES recipients are concentrated in a 

small group of large districts, as 56 percent of recipients attended school 

in the 21 districts required to offer SES with more than 100,000 total 

enrolled students (see fig. 2). Further, states ranged from having 0 districts 

to 257 districts required to offer SES in 2004-2005, with most states having

fewer than 10 districts required to offer SES. State differences in the 

number of districts required to offer SES may have resulted from 

differences in performance or differences in state proficiency standards 

and methods used to measure adequate yearly progress. 

 9

 These district estimates have a margin of error that exceeds plus or minus 8 percentage 

points. See table 9 in appendix I for more information. 

Page 15 GAO-06-758 No Child Left Behind Act
Figure 2: School Districts Required to Offer SES in 2004-2005 

















  





 







 













 







   







 



 





 



 





 







 



 





   

    

 



 







 





   

 









 

 



 



 











      







 









 

 











 













 

 













 

 



  





 

 



     

 



















  



 







  



 



    



   

  











  













   







 















 

 



  



 





 





















  

 





 

  







 



 

 

  





 













  











 





  



  







  

 





  



 

 



 

    













 











 



 







 











   



  



 

   



   

 

 

















 



 

 



 

 

    

  



 

   













 



  













   















  









   

   







 







   















  









 



 













   

 



 















 

 

   

 













 



 



   



















  







 





 





 

 











 







  

  







 

  







 















 

  







    

  







 

     











 























    

 



  



  



  

 

 











   









 



 







 



 













    

 



   







 

   







 

 





 

         





  

 

   

 



    

  

  

     

 



 









  



 



  

 











   

     

   







 





 

























 

  

     





      

 

  

    

    









 

 



















Other districts required to offer SES

Districts required to offer SES with over 100,000 total enrolled students

Source: GAO.

Students receiving SES in 2004-2005 shared certain characteristics, as 

districts reported that most students receiving services were among the 

lower achieving students in school. Specifically, an estimated 91 percent 

of the districts that reviewed the academic records of students receiving 

SES classified most or all of the students receiving SES as academically 

low achieving.

10 For example, Hamilton County, Tenn., school officials said

 10We did not review the academic achievement records of students receiving SES or 

independently verify this information obtained through the district survey. 

Page 16 GAO-06-758 No Child Left Behind Act
that students receiving SES are frequently behind grade level in their skills 

and require special attention to increase their academic achievement. 

Further, we estimate that over half of SES recipients were elementary 

school students in the majority of districts and about 60 percent of schools 

required to offer SES in 2004-2005 were elementary schools.11 Districts 

varied in the percentage of students with limited English proficiency 

receiving services. In about one-third of districts, less than 5 percent of 

SES recipients were students with limited English proficiency; however, in

about one-fifth of districts, over half of SES recipients were students with 

limited English proficiency. Students with disabilities made up less than 

20 percent of students receiving services in about two-thirds of districts. 

Finally, in some districts, the majority of SES recipients were AfricanAmerican or Hispanic. In about 40 percent of districts, over half of SES 

recipients were African-American, and in about 30 percent of districts, 

over half of SES recipients were Hispanic. Because we were unable to 

obtain comparable data on the characteristics of Title I students enrolled 

in these districts in 2004-2005, we were unable to determine whether 

certain groups of students were underserved. 

We estimate that about 2,800 providers delivered services to students 

nationwide in 2004-2005, and more providers were available to deliver 

services in the districts with the largest student enrollments.12 Specifically, 

about 80 percent of districts had between 1 and 5 providers delivering 

services in 2004-2005. However, the number of providers delivering 

services in the 21 districts with more than 100,000 total enrolled students 

ranged from 4 to 45, and averaged 15 providers per district in 2004-2005. 

 11 Many of the district estimates included in this paragraph have a margin of error that

exceeds plus or minus 8 percentage points. See table 9 in appendix I for more information. 

12 In addition to our analysis, the Center on Education Policy reported that that as of

August 2005, more than half of approved SES providers were private, for-profit entities. See 

the Center on Education Policy, From the Capital to the Classroom, Year 4 of the No Child 

Left Behind Act (Washington D.C.: March 2006), for more information.

Page 17 GAO-06-758 No Child Left Behind Act
Districts have taken multiple actions to encourage participation, as shown 

in table 3. In line with the federal statutory requirement that districts 

notify parents in an understandable format of the availability of SES, over 

90 percent of districts provided written information in English, held 

individual meetings with parents, and encouraged school staff to talk with 

parents about SES. Some districts collaborated with providers to notify 

parents. For example, during our site visit, Illinois state officials described 

a provider and district sharing administrative resources to increase 

participation, which involved the provider printing promotional materials 

and the district addressing and mailing the materials to parents. In 

addition, we estimate that over 70 percent of districts lengthened the

period of time for parents to turn in SES applications, held informational 

events for parents to learn about providers, and provided written 

information to parents in languages other than English. During our site 

visit to Woodburn, Ore., district officials reported extending the time 

parents had to sign up their children for SES and hosting an event where 

providers presented their programs to parents in English and Spanish. 

Further, Newark, N.J., district officials told us during our site visit that the 

district provided transportation for parents to attend informational events

to increase participation. Also to encourage participation, an estimated 

90 percent of districts offered services at locations easily accessible to 

students, such as on or near the school campus, and almost 80 percent of 

districts offered services at a variety of times, such as before and after 

school or on weekends. For example, Hamilton County, Tenn., worked 

with providers to offer an early morning tutoring program located at the 

school site in addition to providing services after school. Providers also 

reported delivering SES on school campuses and at various times. 

Specifically, over three-fourths of the 22 providers we interviewed 

reported delivering services at the school site, although providers also 

offered services off-site, such as in the home, online, or at the provider’s 

facility. In addition, providers generally delivered SES after school and 

some also offered SES at alternative times, such as before school, on 

weekends, or during the summer. Finally, about one-third of districts 

provided or arranged for transportation for participating students or 

worked with a local community partner to raise awareness of the services. 

For example, in Newark, N.J., the district worked with a local community 

organization to inform parents and students living in public housing and 

homeless shelters about SES. 

Districts Used Several 

Methods to Contact 

Parents and Offered 

Services on School

Campuses and at Various 

Times to Increase 

Participation 

Page 18 GAO-06-758 No Child Left Behind Act
Table 3: District Actions Taken to Encourage SES Participation (2005-2006) 

Action taken during the 2005-2006 school year 

Estimated percentage

of districts

Provided written information in English to parents 99

Held individual meetings and/or phone conversations with interested parents 95

Encouraged principals, teachers, or other school staff to talk with parents 93

Offered supplemental services in locations that are easily accessible to students after school 

(e.g., on or near the school campus) 

90

Offered SES at a variety of times (e.g., after school, weekends, summer break) 79

Lengthened the period of time parents have to submit applications for SES 79

Held events where parents of eligible students can learn about providers 78

Provided written information in language(s) other than English about SES to parents 72

Made public announcements (e.g., television, billboards, newspaper ads, school newsletters) 67

Worked with a local community partner to raise awareness of SES (e.g., Parent Information 

Resource Center) 

39

Provided or arranged for transportation of students receiving SES to off-site providers 33

Source: GAO.

States also reported taking actions to increase participation in 2005-2006, 

as shown in table 4. Regarding parent notification, all states encouraged 

district staff to communicate with parents about SES. In addition, almost 

90 percent of states provided guidance to districts on the use of school 

campuses for service delivery to encourage participation. 

Table 4: State Actions Taken to Encourage SES Participation (2005-2006) 

Action taken during the 2005-2006 school year Percentage of states

Encouraged district superintendents, supplemental educational services coordinators, or other district staff to 

talk with parents about supplemental educational services 

100

Increased the number of approved providers 94

Provided guidance to districts on the use of school sites to deliver supplemental educational services 87

Increased the number of approved providers that offer supplemental educational services online 73

Worked with a local community partner to raise awareness of SES (e.g., Parent Information Resource Center) 40

Made public announcements (e.g., television, billboards, newspaper ads) 23

Source: GAO.

Page 19 GAO-06-758 No Child Left Behind Act
Despite some districts’ promising approaches to encourage participation, 

notifying parents in a timely manner remains a challenge for some 

districts. An estimated 58 percent of districts did not notify parents that 

their children may be eligible to receive SES before the beginning of the 

2005-2006 school year, which may be due in part to delays in states 

reporting which schools were identified for improvement.13 Specifically, 

about half of districts that did not notify parents before the beginning of 

the 2005-2006 school year did not receive notification from the state of the 

schools identified for improvement by that time.14 Moreover, district 

officials in three of the states we visited experienced delays in receiving

school improvement information, and state officials agreed that providing 

timely information about whether schools have met state performance 

goals has been a challenge. Almost all of the districts that did not notify 

parents before the beginning of the 2005-2006 school year did so within the 

first 2 months of the year.15

Remaining Challenges 

Include Notifying Parents 

in a Timely and Effective 

Manner and Attracting 

More Providers for Certain 

Areas and Students 

Effectively notifying parents is also a challenge for some districts. For 

example, officials in all four districts we visited reported difficulties 

contacting parents to inform them about SES in part because some 

families frequently move and do not always update their mailing address 

with districts. In addition, some providers we interviewed indicated that 

confusing parental notification letters do not effectively encourage SES 

participation. For example, some of the providers we interviewed said 

some districts use confusing and poorly written letters to inform parents 

of SES or send letters to parents of eligible children but conduct no further 

outreach to encourage participation in SES. Four of the providers we 

interviewed also indicated that complicated district enrollment processes 

can discourage participation. For example, one provider said certain 

districts send parents multiple documents to complete in order for their 

child to receive SES, such as an enrollment form to select an SES provider

and a separate contract and learning plan. 

 13 GAO previously reported that some states have difficulty notifying schools of their status

in meeting proficiency goals in a timely fashion in part because of the time involved in 

identifying and correcting errors in student assessment data. See GAO, No Child Left 

Behind Act: Improvements Needed in Education’s Process for Tracking States’

Implementation of Key Provisions, GAO-04-734 (Washington D.C.: Sept. 30, 2004), for

more information. 

14 This district estimate has a margin of error that exceeds plus or minus 8 percentage 

points. See table 9 in appendix I for more information. 

15 This district estimate has a margin of error that exceeds plus or minus 8 percentage 

points. See table 9 in appendix I for more information. 

Page 20 GAO-06-758 No Child Left Behind Act
Another challenge to increasing SES participation is attracting more SES 

providers for certain areas. Some rural districts surveyed indicated that no

students received services last year because of a lack of providers in the 

area.16 We estimate that the availability of transportation for students 

attending supplemental services was a moderate, great or very great 

challenge for about half of rural districts.17 For example, one rural district 

commented in our survey that there are no approved providers within 

200 miles of its schools. A few other rural districts commented in our 

survey that it was difficult to attract providers to their area because there 

were few students to serve or providers had trouble finding staff to serve 

as tutors. 

In addition, ensuring there are providers to serve students with limited 

English proficiency or disabilities has been a challenge for some districts.

There were not enough providers to meet the needs of students with 

limited English proficiency in an estimated one-third of districts, and not 

enough providers to meet the needs of students with disabilities in an 

estimated one-quarter of districts. Many states also indicated that the 

number of providers available to serve these groups of students was 

inadequate. While over half of the providers we interviewed reported 

serving students with limited English proficiency or disabilities, some 

providers served these students on a limited basis and reported difficulties

meeting their needs. For example, one provider reported serving few 

students with limited English proficiency and disabilities because the 

amount of funding the provider receives for SES was not sufficient to pay 

for specialized tutors to serve these students. Another provider said it was 

difficult to find tutors to meet the needs of students with limited English 

proficiency and its program was not designed for students with 

disabilities. Another provider said that it was difficult to serve students 

with disabilities because it required significantly modifying the tutoring 

lessons to meet their needs. 

 16 GAO previously reported that geographic isolation created difficulties for rural districts

in implementing SES. Specifically, rural district officials stated that traveling long distances

to meet providers was not a viable option and use of online providers was challenging in 

some small rural districts where it was difficult to establish and maintain Internet service. 

See GAO, No Child Left Behind Act: Additional Assistance and Research on Effective 

Strategies Would Help Small Rural Districts, GAO-04-909 (Washington D.C.: Sept. 23, 

2004), for more details. 

17 This district estimate has a margin of error that exceeds plus or minus 8 percentage 

points. See table 9 in appendix I for more information. 

Page 21 GAO-06-758 No Child Left Behind Act
Encouraging student attendance has also been a challenge, in part because 

students may participate in other afterschool activities, such as sports or 

work. Low parent and student demand for SES has been a challenge in 

about two-thirds of districts. For example, about one-quarter of districts 

reported that both competition from other afterschool programs and the 

availability of services that are engaging to students were challenges to 

implementing SES. In addition, providers, district and school officials in all 

four districts we visited told us that SES is competing for students with 

extracurricular and other activities. For example, a Chicago, Ill., high 

school official indicated that student attendance at SES sessions declined

significantly as the school year progressed. To address this problem, 

providers sometimes offer students incentives for participation. For 

example, while 2 of the 22 providers we interviewed offered incentives for 

students to sign up for services, 19 providers used incentives to encourage

student attendance, such as school supplies and gift certificates. 

To promote improved student academic achievement, providers aligned 

their curriculum with district instruction primarily by hiring district 

teachers and communicating with the teachers of participating students. 

Providers reported communicating with teachers and parents in person as 

well as mailing information and progress reports to them; however, 

districts indicated the extent of provider efforts varied, as some did not 

contact teachers and parents in 2004-2005. In addition, both providers and 

districts experienced contracting and coordination difficulties. In part 

because SES is often delivered in school facilities, providers and officials

in the districts and schools we visited reported that involvement of school 

administrators and teachers can improve SES delivery and coordination. 

In order to increase student academic achievement, providers took steps 

to align their curriculum with school instruction by hiring and 

communicating with teachers, though the extent of their efforts varied. A 

majority of the 22 providers we interviewed hired certified teachers in the 

district as tutors. Some providers said hiring district teachers promoted 

curriculum alignment, in part because district teachers were apt to draw 

on district curriculum during tutoring sessions. School officials in three of

our site visits also said providers’ use of district teachers as tutors helped 

to align the tutoring program with what the student learned in the 

classroom. In addition, some providers reported aligning curriculum by 

communicating with the teachers of participating students to identify 

Providers Have Taken 

Steps to Deliver 

Quality Services, but 

Contracting and 

Coordination Remain

Challenges to 

Working with

Districts and Schools 

Providers Worked to Align 

Curriculum and 

Communicate with 

Parents, though the Extent 

of Their Efforts Varied 

Page 22 GAO-06-758 No Child Left Behind Act
student needs and discuss progress. The frequency of contact between 

tutors and teachers ranged from mailing teachers information once prior 

to the beginning of the program to contacting teachers at least weekly, 

according to the providers we interviewed. A few providers also used 

other methods to align their curriculum with district instruction, such as 

using the same tests to evaluate student progress and allowing principals 

to choose components of the tutoring curriculum for students receiving

SES in their schools. However, not all providers worked with schools to 

align curriculum, as we estimate that some, most, or all providers did not 

contact teachers to align curriculum with school instruction in almost 40 

percent of districts in 2004-2005. For example, Woodburn, Ore., district 

and school officials indicated during our site visit that instead of aligning

their services with the district curriculum, certain providers openly 

questioned the district’s curriculum and teaching methods, which caused 

confusion among some parents and students. 

Providers reported mailing information as well as meeting with parents 

over the phone and in-person to communicate about student needs and 

progress; however, the frequency of communication with parents varied 

by provider. A majority of the providers we interviewed communicated 

with parents about student progress repeatedly, sometimes by sending 

home progress reports on a monthly basis or holding parent-tutor 

conferences. The frequency of contact between tutors and parents 

reported by the 22 providers we interviewed ranged from meeting with 

parents twice during the tutoring program to giving parents a weekly 

progress report. A few providers also reported communicating with 

parents by holding workshops for parents to learn about the SES program 

and in some cases having the parents sign their students’ learning plans. 

For example, one provider conducted workshops where parents received 

reading materials to share with their children and a parent guide in English 

and Spanish that explained the program and how to use the materials to 

enhance student learning. Some providers also reported hiring staff 

dedicated in part to maintaining communication with parents. However, 

some providers faced difficulties when communicating with parents about 

SES, such as language barriers or incorrect contact information. Districts 

confirmed that the degree to which providers communicated with parents 

varied, as we estimate that some, most, or all providers did not contact 

parents to discuss student needs and progress in about 30 percent of 

districts in 2004-2005. 

Despite these challenges, most districts had positive relationships with 

providers. Specifically, an estimated 90 percent of districts indicated that 

their working relationships with providers during 2004-2005 were good, 

Page 23 GAO-06-758 No Child Left Behind Act
very good, or excellent. In addition, many of the providers we interviewed 

during our site visits also reported having positive working relationships 

with district officials. Although other studies have found that districts 

reported certain difficulties working with providers, relatively few districts 

reported that their providers signed up ineligible students or billed for 

services not performed in 2004-2005, as shown in figure 3. 

Figure 3: Estimated Percentage of Districts Experiencing Certain Issues with None, 

Few, Some, Most, or All of Their Providers in 2004-2005 

Issues experienced 

Percentage of districts

Source: GAO. 

0 10 20 30 40 50 60 70 80 90 100 

Not submitting attendance

data on a regular basis

Not showing up at the agreed upon

time and location to deliver services

Using incentives prohibited

by state or district policy

Billing for services not performed 

Signing up ineligible students

None/few providers

Some/most/all providers

Don’t know

89 4 7 

83 3 14 

79 11 10 

71 23 5 

90 5 5 

Generally, states did not hear about these provider issues very often. 

Almost half of states said the issue of providers not showing up for SES 

sessions was rarely brought to their attention.18 Similarly, half of states 

said the issue of providers billing the district for services not performed 

was rarely brought to their attention. In addition, about 40 percent of 

states said the issue of providers using excessive incentives was rarely 

brought to their attention. Further, about 40 percent of states said the 

 18 The other states included five that did not respond to the applicable survey question. A 

similar number of states did not respond to the other survey questions discussed in this 

paragraph. 

Page 24 GAO-06-758 No Child Left Behind Act
issue of providers signing up ineligible students rarely arose. Almost 

one-third of states heard about each of these issues sometimes, while few 

states had these issues brought to their attention very often. For example, 

during our site visits, state officials provided examples of issues that had

been brought to their attention regarding provider practices, but these 

issues were often isolated incidents particular to one or a few providers in 

certain districts. 

Providers and Districts 

Experienced Contracting 

and Coordination 

Difficulties 

While providers have taken steps to deliver quality services, both 

providers and districts reported experiencing difficulties during the 

contracting process. For example, some of the providers we interviewed 

said certain districts imposed burdensome contract requirements, such as 

requiring substantial documentation to be submitted with invoices, 

limiting the marketing they could do to parents and students, or restricting 

the use of school facilities to deliver services. Specifically, 7 of the 

22 providers we interviewed experienced difficulties with districts 

restricting provider access to school facilities, by for example, not 

allowing providers to deliver services in school buildings or by charging

providers substantial fees to do so. A few providers also said contracting 

with districts was a resource-intensive process, in part because contract 

requirements vary by district and state. Some of the multi-state providers 

we interviewed reported dedicating a team of staff to help them finalize 

and manage contracts with districts. These providers commented that, 

while they have the administrative capacity to manage this process, 

smaller providers may not have such capacity. In addition, one provider 

that delivered services exclusively online commented that contracting 

with districts across the country was a challenge, particularly because 

some states and districts require provider representatives to attend 

meetings in-person and be fingerprinted in their states. 

Contracting with providers was also a challenge for some districts. 

Specifically, negotiating contracts with providers was a moderate, great, or

very great challenge in about 40 percent of districts nationwide. For 

example, Woodburn, Ore., district officials described having contractual 

discussions with providers around whether the district would charge fees 

for the use of school facilities, the types of incentives providers could use 

to encourage students to sign up, and whether the district would pay for 

services when students did not attend SES sessions. While states may 

review and define program design parameters as part of the provider 

approval process, district officials in three of our site visits expressed 

concern about their lack of authority to set parameters in provider 

contracts around costs and program design, such as tutor-to-student ratios

Page 25 GAO-06-758 No Child Left Behind Act
and total hours of instruction. For example, during our site visit, a 

Hamilton County, Tenn., district official expressed frustration with 

providers that charged the maximum per-pupil amount but varied in the 

level of services provided, such as the number of instructional hours and 

tutor-to-student ratio. Chicago, Ill., district officials also expressed 

concern about the variation among providers in the hours of instruction 

and cost of services because the district does not have sufficient funds to 

serve all eligible students and officials would like to maximize the number 

of students they can serve. In part to help address district concerns, in 

2005-2006, Illinois required approved providers to submit information on 

the cost of providing services in each of the districts they served and made 

this information available to districts and the public in order to improve 

transparency and accountability. While Tennessee state officials told us 

they were hesitant to set restrictions on providers and would like more 

federal direction on this issue, other states have set restrictions on the 

cost and design of SES programs. For example, Georgia set a maximum 

tutor-to-student ratio of 1:8 for non-computer based instruction and 

1:10 for computer based instruction, and New Mexico adopted a sliding fee 

scale based on the educational level of tutors. 

Coordination of service delivery has also been a challenge for providers, 

districts, and schools. About 70 percent of states reported that the level of

coordination between providers, districts, and schools implementing SES 

was a moderate to very great challenge. Sometimes these coordination 

difficulties have resulted in service delays. For example, services were 

delayed or withdrawn in three of the districts we visited because not 

enough students signed up to meet the providers’ enrollment targets and 

districts were not aware of these targets.19 In one district we visited, 

services were delayed because school teachers hired to be tutors did not 

provide evidence of their background checks and teaching certificates to 

providers in a timely manner. Coordination difficulties also occurred 

during the enrollment process. Though districts are responsible for 

arranging SES for eligible students, in two districts we visited, both the 

district and providers sent parents enrollment forms, which caused 

confusion among parents as well as additional work for the district staff 

processing the forms. In addition, a few providers told us they do not 

 19 In addition to our analysis, the Center on Education Policy case studies also found that in 

some cases, approved providers that initially expressed interest in serving a certain district 

later decided not to provide services because too few students enrolled. See the Center on

Education Policy, From the Capital to the Classroom, Year 4 of the No Child Left Behind 

Act (Washington D.C.: March 2006), for more information. 

Page 26 GAO-06-758 No Child Left Behind Act
know how many students they will serve until enrollment forms are 

returned to district officials, which hinders planning and may delay 

services since they do not know how many tutors they will need to hire 

and train to deliver SES in each district. 

Providers, District and 

School Officials Reported 

That a Greater Role for 

Schools Would Improve 

Local SES Implementation 

In part because SES can be delivered in school facilities, providers and 

officials in the districts and schools we visited reported that involvement 

of school administrators and other staff improves SES implementation. 

Although schools do not have federally-defined responsibilities for 

administering SES, many officials said SES implementation is hindered 

when school officials are not involved. Some providers we interviewed 

said that a lack of involvement of school principals can make it difficult 

for them to coordinate with schools to encourage student participation. In

addition, a few providers said certain districts contributed to this problem 

by restricting communication with school officials or not defining a role 

for schools in SES implementation. Officials in one of the districts we 

visited also told us that encouraging participation and administering the 

program was more difficult when they did not designate school staff to 

assist the district with SES implementation. School officials from all four 

of our site visits also said the lack of a clear role for school officials, 

including principals, in administering SES has been a challenge. For 

example, Illinois and Oregon school principals told us they found it 

difficult to manage afterschool activities because they didn’t have 

sufficient authority to oversee SES tutors operating in their buildings at 

that time. Further, problems can arise when school officials are not fully 

informed about the SES program. For example, Woodburn, Ore., school 

officials told us that although the school was not provided SES tutoring 

schedules for students, parents and students have come to school officials 

for help when they were unclear about the schedule or when tutors failed 

to show up. 

A majority of the providers we interviewed told us that involvement of 

school principals can improve participation and make it easier to deliver 

services, in part because principals are familiar with the students and 

manage school staff. For example, certain providers reported providing 

principals with information about the tutoring program so that school staff 

can assist with the enrollment process, involving principals in selecting the 

curriculum used in their schools, and sending principals student progress 

reports. In addition, all four districts we visited had school site

coordinators to assist with SES, such as helping with the enrollment 

process and assisting with the day-to-day administration of the SES 

program in the schools. For example, Woodburn, Ore., district officials 

told us implementation improved when the district designated school site 

Page 27 GAO-06-758 No Child Left Behind Act
coordinators to assist with parental notification and events where 

providers present their programs, and meet with parents and providers to 

help them complete individual student learning plans. A few providers we 

interviewed also assigned a staff person at the school site to communicate 

with teachers and parents. While helping to administer the SES program 

adds additional administrative burden on schools, school officials in all 

four of the districts we visited said they welcomed a stronger or more 

clearly defined role. 

While state monitoring of SES had been limited, more states reported 

taking steps to monitor both district and provider efforts to implement 

SES in 2005-2006. In addition, districts have taken a direct role in 

monitoring providers, and their monitoring activities similarly increased

during this time. Although states are required to withdraw approval from 

providers that fail to increase student academic achievement for 2 years, 

many states reported challenges evaluating SES providers. In addition, the 

few states that have completed an evaluation have not yet produced 

reports that provided a conclusive assessment of SES providers’ effect on 

student academic achievement. 

State monitoring of district SES implementation, which is sometimes 

performed as part of state Title I monitoring, had been limited prior to 

2005-2006, though more states reported conducting on-site reviews of 

districts in that year. While less than one-third of states conducted on-site 

reviews of districts to monitor their implementation of SES in 2004-2005, 

almost three-fourths reported conducting such reviews in 2005-2006. This

increase reflects both those states that had already begun monitoring 

district SES implementation for 2005-2006 at the time of our survey and

those states planning to conduct monitoring activities before the end of

that school year. Because our data were collected during the middle of the 

2005-2006 school year, we do not know whether all of the states that 

planned to complete these activities before the end of the year did so. In 

both years, a majority of the states that conducted on-site reviews visited 

few or some of their districts. Therefore, while more states reported 

conducting such reviews in 2005-2006 than in 2004-2005, the number of 

districts per state receiving reviews does not appear to have increased. 

States’ SES 

Monitoring Has Been 

Limited Though Their 

Efforts Are 

Increasing, and Many 

States Struggle to 

Develop Meaningful 

Evaluations 

More States Reported 

Conducting On-Site 

Reviews of Districts in 

2005-2006, and Many also 

Collected Information 

from Other Sources to 

Monitor District SES 

Implementation 

In addition to on-site reviews, many states also reported reviewing 

information collected from several other sources to assess district SES 

Page 28 GAO-06-758 No Child Left Behind Act
implementation in 2005-2006. The most common source used by states 

was district officials, as almost all states reported reviewing or planning to

review information collected from district officials to monitor their 

implementation of SES in 2005-2006. Further, many states were also 

collecting information from school principals, parents, and providers to 

monitor districts, with between 67 and 81 percent of states reviewing or 

planning to review information collected from these sources in 2005-2006. 

States also reported reviewing or planning to review information related to

several aspects of district SES implementation in 2005-2006. For example, 

almost all states reported reviewing district notification of parents and 

SES expenditures, as shown in figure 4. 

Figure 4: Percentage of States That Reviewed Specific Program Elements to

Monitor District SES Implementation in 2005-2006 

0

20

40

60

80

100

Parent

Eligibility

District

Student

SES

District

Percentage of states

Program element

Planned to review before end of 2005-2006

Reviewed for 2005-2006 at time of survey

Source: GAO.

82

97

80

86 84

94

38

60

42 38 40 40

44

37

48 44 40

52

outreach

efforts

determination

process

contracts with

providers

participation

rates

expenditures

notification

of parents

Note: The percentage of states that did not review or plan to review these program elements to 

monitor district SES implementation in 2005-2006 and the percentage of states that did not answer 

these survey questions are not shown in this figure.

Page 29 GAO-06-758 No Child Left Behind Act
States we visited reported that some districts have had difficulties 

implementing SES, in part because of district staff capacity limitations and 

the complexities of administering SES at the local level. When states find 

that a district is having difficulty implementing SES, most hold a meeting 

with the district and provide or arrange for assistance, including 

consultations or training. Half of the states also develop an action plan and

time line with the district to help improve its efforts. During our site visits, 

state officials reported that notifying parents, maintaining a fair and 

competitive environment for providers, ensuring providers understand 

their SES roles and responsibilities, and determining an appropriate role 

for schools continue to challenge some districts as they implement SES. 

Though States and 

Districts Are Challenged to 

Monitor Providers, They 

Are Moving from Limited 

Monitoring of Providers to 

a More Active Approach 

Although states and districts reported increasing their efforts to monitor 

SES providers between 2004-2005 and 2005-2006, over two-thirds of states 

reported that on-site monitoring of providers has been a challenge. In

addition, several districts commented in our survey that more provider 

monitoring is needed. During all four of our site visits, state and district 

officials expressed concerns about their capacity to fully administer and 

oversee all required aspects of SES implementation, including provider 

monitoring. Officials explained that state and district capacity to 

implement SES is limited, because there is typically one staff person at 

each level coordinating all of SES, and sometimes that person may also 

oversee implementation of additional federal education programs. Further, 

several states commented in our survey that additional training, technical 

assistance, and national monitoring protocols from the federal government 

would assist their efforts to monitor providers. 

During our site visits, state officials noted that while they did not initially 

have structured plans or procedures in place to monitor SES providers, 

they took steps to develop and formalize procedures starting with the 

2004-2005 and 2005-2006 school years. Nationally, in 2004-2005, states 

monitored providers primarily by collecting data from district officials, 

though many states reported using a more active monitoring approach in 

the next year. For example, while less than one-third of states conducted 

on-site reviews of providers in 2004-2005, over three-fourths had 

conducted or planned to conduct such reviews in 2005-2006, as shown in 

figure 5. In addition, while one-third or fewer states reviewed information 

collected from school staff, parents, and students in 2004-2005, the 

percentage that reported reviewing or planning to review information 

collected from these sources more than doubled the next year. Similar to 

2004-2005, many states continued to rely on information collected from 

district officials to monitor providers in 2005-2006, with almost all states 

Page 30 GAO-06-758 No Child Left Behind Act
reviewing or planning to review information collected from districts in 

that year. 

Figure 5: Percentage of States Reviewing Information Collected from Different 

Sources to Monitor SES Providers in 2004-2005 and 2005-2006 

0

10

20

30

40

50

60

70

80

90

100

2005-2006

2004-2005

2005-2006

2004-2005

2004-20

2005-2006

05

2005-2006

2004-2005

2005-2006

2004-2005

Percentage of states

District officials Parents S On-site reviews School staff tudents

Source: GAO.

Source of collected information

Planned to review before end of 2005-2006

Reviewed for 2004-2005 and 2005-2006 (at time of survey)

31 31

46

50

42

23

48

27

56

13

37

67

19

33

25

77

92

71

83

50

Note: The percentage of states that did not review or plan to review information collected from these 

sources to monitor providers in each year and the percentage of states that did not answer these 

survey questions are not shown in this figure. 

Federal guidance suggests states may request district assistance in 

collecting data from providers to assist state monitoring activities. While

the state is ultimately responsible for monitoring providers, most states 

reported that districts have taken a direct role in monitoring providers. 

Similar to states, although district monitoring of providers was limited in 

2004-2005, districts used a more extensive and active approach in the next 

Page 31 GAO-06-758 No Child Left Behind Act
year, as shown in figure 6.20 For example, while we estimate that less than

half of districts collected information from on-site reviews, school staff, 

parents, and students to monitor providers in 2004-2005, 70 percent or 

more were collecting or planning to collect information from these 

sources in 2005-2006. 

 20 Similar to states, districts reported on monitoring activities through our survey in early 

2006; therefore, while some had already begun to monitor providers, others reported on

their plans to monitor providers before the end of the 2005-2006 school year. The increases

in monitoring reflect both those districts that had already begun monitoring activities and 

those planning monitoring activities for later in 2005-2006. Because our data were collected 

during the middle of the 2005-2006 school year, we do not know whether the estimated 

percentage of districts that planned to complete these activities before the end of the year 

did so. 

Page 32 GAO-06-758 No Child Left Behind Act
Figure 6: Estimated Percentage of Districts Reviewing Information Collected from

Different Sources to Monitor SES Providers in 2004-2005 and 2005-2006 

0

10

20

30

40

50

60

70

80

90

100

2005-2

2004-2005

2005-2006

006

2004-2005

2005-2006

2004-20052005-2006

2004-2005

Percentage of districts

School staff Parents S On-site reviews tudents

Source: GAO.

Source of collected information

Planned to review before end of 2005-2006

Reviewed for 2004-2005 and 2005-2006 (at time of survey)

45 45

27

47

38

40

40

46

32

48

20

50

78

86

80

70

Although states and districts collected information from similar sources to 

monitor providers, districts collected information from more providers 

than states. Specifically, while a majority of the states that conducted onsite reviews observed only some or few providers, we estimate that a 

majority of districts that conducted on-site reviews observed most or all of 

their providers in 2004-2005.21 While states and districts may both have 

capacity limitations that affect their ability to conduct on-site reviews to 

monitor providers, conducting such reviews is likely easier for districts 

because services are often delivered on or near school campuses. 

 21 The district estimate has a margin of error that exceeds plus or minus 8 percentage 

points. See table 9 in appendix I for more information. 

Page 33 GAO-06-758 No Child Left Behind Act
States and districts collected information on several aspects of SES 

programs to monitor providers, as shown in table 5. While federal 

regulations provide states flexibility to design their own SES monitoring 

systems, over two-thirds or more of states and districts monitored or 

planned to monitor all program elements listed, including those related to

service delivery and use of funds. For example, 94 percent of states 

monitored or planned to monitor parent or student satisfaction with 

providers, and 93 percent of districts monitored or planned to monitor 

billing and payment for services and student attendance records. 

Table 5: Percentage of States and Districts That Reviewed Specified Program Elements to Monitor Providers in 2005-2006 

Percentage of states Estimated percentage of districts 

Program element Monitored

Planned to

monitor

Monitored

or planned

to monitor Monitored

Planned to

monitor

Monitored

or planned

to monitor

Parent/student satisfaction with a provider 27 67 94 34 57 91

Provider communication with teachers and 

parents 37 56 92 46 43 89

Extent to which a provider’s program, as 

enacted, reflects its program design, as outlined 

in its application to your state 19 73 92 30 41 70

Evidence of meeting academic achievement 

goals as stated on student learning plan 23 65 88 28 60 88

Evidence of improved student achievement 

based on any statewide assessment 15 71 87 26 65 91

Alignment of provider curriculum with 

district/school curriculum or instruction 25 62 87 35 39 74

Student attendance records 27 56 83 67 25 93

Evidence of improved student achievement 

based on provider assessments 27 56 83 39 52 91

Protection of student privacy 33 50 83 55 28 82

Adherence to applicable health, safety, and civil 

rights laws 29 48 77 48 26 74

Provider financial stability (e.g., audits, financial 

statements) 31 42 73 N/A N/A N/A

Evidence of improved student achievement 

based on grades, promotion, and/or graduation 12 58 69 23 57 80

Billing and payment for services N/A N/A N/A 72 21 93

Source: GAO.

Note: The percentage of states that did not review or plan to review these program elements to 

monitor providers in 2005-2006 and the percentage of states that did not answer these survey

questions are not shown in this table. In addition, we did not ask states if they monitored billing and 

payment for services, and we did not ask districts if they monitored provider financial stability.

Page 34 GAO-06-758 No Child Left Behind Act
Many states struggle to develop evaluations to determine whether SES 

providers are improving student achievement, though states are required 

to evaluate and withdraw approval from providers that fail to do so after 

2 years. Specifically, federal law requires states to develop standards and

techniques to evaluate the services delivered by approved providers, but it 

does not require states to use specific evaluation methods or criteria for 

determining provider effectiveness.22 Through our survey, states reported 

several challenges to evaluating SES providers. Specifically, over 

three-fourths of states reported that determining sufficient academic 

progress of students, having the time and knowledge to analyze SES data, 

and developing data systems to track SES information have been 

challenges. Further, during our site visits to Illinois and New Jersey, state 

officials noted they were currently in the process of improving their data

collection systems to more effectively capture and analyze data for SES 

evaluations. In addition, several state officials reported that while they

have collected some information to assess provider effectiveness, they 

have done little with that data. Others noted that they have not received 

sufficient federal guidance on effective models for SES provider 

evaluations, and because developing and implementing such evaluations 

can be both time-consuming and costly, additional assistance from 

Education would improve their efforts. 

Many States Struggle to 

Develop Meaningful 

Evaluations, and the Few 

State Evaluations 

Completed to Date Were 

Inconclusive 

At the time of our survey in early 2006, only a few states had drafted or 

completed an evaluation report addressing individual SES provider’s 

effects on student academic achievement, and we found that no state had

produced a report that provided a conclusive assessment of this effect. 

New Mexico and Tennessee were the only two states that had completed

final or draft SES evaluation reports that attempted to assess the impact of

all SES providers serving students in their states in previous years.23 To 

measure student academic achievement, New Mexico’s report analyzed

students’ grades as well as their scores on state assessments and provider 

assessments, which often differ by provider and are administered both 

before SES sessions begin and at the end of SES sessions each year. 

 22 Further, the federal SES guidance suggests that student performance can be measured in 

a variety of ways, including with provider assessments or state or district standardized 

assessments. However, the guidance does not discuss the limitations of using different

approaches to conduct such evaluations. 

23 At the time of our survey, several additional states, including Louisiana and Pennsylvania,

were in the process of drafting an SES evaluation report that would assess the impact of 

SES providers serving students in their states in previous years, but the reports were not

yet available to the public. 

Page 35 GAO-06-758 No Child Left Behind Act
However, the report noted that these assessments produced different 

results related to student academic achievement gains. While Tennessee 

also planned to review students’ state assessment scores, the draft 

available at the time of our analysis did not include this information. In 

addition, both reports drew on information obtained through other 

sources, such as teacher surveys, to assess provider effectiveness. Due to 

their limitations, neither evaluation provided a conclusive assessment of 

SES providers’ effect on student academic achievement. 

In addition, at the time of our survey, over half of the states reported that 

they were in the process of conducting an evaluation of SES providers in 

order to meet the federal requirement of assessing each provider’s effect 

on student academic achievement.24 Similar to the state evaluations 

already undertaken, officials reported using different methods and criteria

to evaluate SES providers. For example, some states were collecting each

provider’s pre- and post-SES assessments of students while others were 

collecting student achievement data from districts or students’ state 

assessment scores. Further, while one state defined adequate student 

progress as 80 percent of a provider’s students making one-grade level of 

improvement after a year of SES, another state defined adequate student 

progress as 50 percent or more of a provider’s students having any positive 

academic achievement gain after a year of SES. While these states have yet 

to produce final results from their SES provider evaluations, it is unclear 

whether any of these efforts will produce a conclusive assessment of SES 

providers’ effect on student academic achievement. 

Likely because states are struggling to complete evaluations to assess SES 

providers’ effect on student academic achievement, states did not report 

that they have withdrawn approval from providers because their programs 

were determined to be ineffective at achieving this goal.25 Rather, though 

over 40 percent of states reported that they had withdrawn approval from 

some providers, they most frequently reported withdrawing provider 

approval because the provider was a school or district that had entered 

 24 In addition, at least two school districts, Chicago and Minneapolis, have taken steps to

evaluate SES, but we found that their evaluation efforts have also not yet produced a

conclusive assessment of SES’s impact on student academic achievement. 

25 Only one state reported withdrawing approval from one of its providers because that 

provider’s program was generally ineffective. However, this provider’s program was found

to be ineffective because the provider did not deliver services to all of the students it 

enrolled. This state also indicated that it had not yet completed an evaluation of SES’s 

effect on student academic achievement. 

Page 36 GAO-06-758 No Child Left Behind Act
needs improvement status, the provider asked to be removed from the 

state-approved provider list, or because of provider financial impropriety. 

Several offices within Education monitor various aspects of SES activity 

across the country and provide support, but states and districts reported 

needing additional assistance and flexibility with program implementation. 

Education conducts SES monitoring in part through policy oversight and 

compliance reviews of states and districts, and provides SES support 

through guidance, grants, research, and technical assistance. However, 

many states and districts reported needing additional assistance and

guidance regarding evaluation and administration of SES. Further, some 

states and districts voiced interest in expansion of Education’s pilot 

programs that increase SES flexibility, including the pilot that allows 

certain low-achieving districts to serve as SES providers. 

OII and OESE are primarily responsible for monitoring and supporting 

state and district SES implementation, and other Education offices also 

contribute to these efforts (see fig. 7). 

Several Education 

Offices Monitor and 

Support SES 

Implementation, but 

States and Districts 

Cite the Need for 

Additional Assistance 

and Flexibility 

Education Monitors SES 

and Provides States and 

Districts with Guidance 

and Technical Assistance 

Page 37 GAO-06-758 No Child Left Behind Act
Figure 7: U.S. Department of Education Offices Monitoring and Supporting SES 

Office of Inspector 

General (OIG) 

Office of Planning, 

Evaluation and Policy 

Development 

Policy and Program 

Studies Service

Faith-Based and 

Community Initiatives

Office of the Secretary 

Office of the Deputy Secretary 

Office of Innovation and 

Improvement (OII) 

Office of Elementary 

and Secondary 

Education (OESE) 

Monitoring

 Performed six state SES

 audits during 2003-2004 

 and 2004-2005 

 Performed audits of SES

 implementation and 

 providers in five CA 

 districts during 2005 

Support

 Issued several reports focused

 on Title I, including SES (e.g.

 National Assessment of Title I:

 Interim Report)

 Produced a nine-district case

 study report, Early 

 Implementation of SES under

 the NCLBA, with a follow-up

 report for 2003-2004

Support 

 Clarifies SES policy and 

 provides technical assistance, 

 in part through presentations

 Coordinates the publication of 

 non-regulatory SES guidance 

 Issued grant for SES

 technical assistance 

 provided by SESQC 

 Issued grants to increase 

 minority participation in SES

 Issued grants to increase 

 SES access in rural areas

Support 

 Provides technical 

 assistance to faith- and 

 community-based 

 organizations interested in 

 becoming SES providers

Support

 Clarifies SES policy and 

 provides technical assistance, 

 in part through presentations

 Issued grant to fund

 Comprehensive Center on

 Innovation and

 Improvement that provides

 technical assistance for 

 Education programs, including 

 SES

Monitoring

 Performs “desk monitoring” 

 of SES information 

 Performs ad-hoc monitoring 

 of SES issues as they arise 

Monitoring

 Performs on-site visits to 

 states and districts as part 

 of Title I monitoring 

 Oversees collection and 

 analysis of Consolidated 

 State Performance Reports, 

 including SES data

Source: GAO analysis. 

Note: This figure reflects the coordination of Education’s offices rather than the statutory reporting 

relationship. 

OII, which leads SES policy development and provides strategic direction,

monitors SES policy issues primarily through what it calls “desk 

monitoring.” This monitoring is performed at its federal office and 

includes the review of SES-related research and media reports. OII also 

conducts more intensive monitoring of specific SES implementation 

challenges when states, districts, and providers bring them to Education’s 

attention and keeps a log documenting these issues. For example, during 

2004-2005, Illinois and New Jersey officials contacted OII to clarify 

guidance regarding providers affiliated with districts in need of 

improvement, and OII staff provided assistance and clarification. In 

addition, several providers we interviewed also mentioned that they have 

Page 38 GAO-06-758 No Child Left Behind Act
contacted OII directly to discuss implementation challenges associated 

with enrollment, district contracts, and provider access to school facilities. 

OESE, which oversees and supports NCLBA implementation, is also 

involved in monitoring SES implementation through its overall monitoring 

of state compliance with Title I and NCLBA. To monitor Title I, OESE staff 

visit state departments of education and selected districts within each

state to interview officials and review relevant documents. Once the 

review is complete, OESE issues a report to the state outlining any 

instances of Title I non-compliance, including those related to SES, and 

actions needed to comply with regulations. As of June 2006, OESE had 

visited and issued reports to over three-fourths of the states.

In addition to its Title I monitoring, OESE also oversees the collection of 

state NCLBA data, including data on SES, through the annual Consolidated 

State Performance Report (CSPR). For the CSPR, each state is required to

report the number of schools with students receiving SES, the number of 

students eligible for services, and the number that received services.26

However, although almost all states reported that they are collecting 

information on district SES expenditures as part of their oversight, 

Education does not require states to submit information on the percent or 

amount of Title I funds districts spent for SES through the CSPR or other 

means. Therefore, while Education tracks the extent to which states are 

providing SES to eligible students, the department does not collect data on 

the relative costs of serving them. Further, under NCLBA, Education is 

required to present an annual summary of the CSPR data to Congress. As 

of June 2006, the most recent report to Congress was for the 2002-2003 

school year, though Education officials indicated they expect to submit 

reports for 2003-2004 and 2004-2005 in the near future. 

While OII and OESE monitoring of SES has been either limited to desk 

monitoring or combined with general Title I monitoring, OIG has 

conducted audits specifically focused on SES. During 2003-2004 and 2004-

2005, the office performed a series of SES implementation audits in six 

states, which involved site visits to state offices and selected districts 

within each state.27 Also during 2004-2005, OIG performed audits of five 

 26 States have only reported the number of students eligible for SES since the 2003-2004 

CSPR. Also, for the 2004-2005 CSPR, Education gave states the option to also report the

number of students who applied for SES. 

27 States audited were Delaware, Illinois, Indiana, Michigan, New Jersey, and Nevada. 

Page 39 GAO-06-758 No Child Left Behind Act
California districts and one SES provider within each district.28 These 

audits included an examination of district SES contracts with providers, 

provider services, and provider compliance with state SES regulations. 

Several Education officials reported coordinating internally to share 

information, including SES monitoring results. To facilitate coordination, 

OII leads an internal group comprised of staff members from other 

Education offices, who meet bi-weekly to exchange information. OESE 

shares its state Title I monitoring results and CSPR data with other 

Education offices. In addition, OIG makes recommendations to both OII 

and OESE in its state and district SES auditing reports and disseminates 

the reports throughout Education and on the department’s Web site. Since 

2002, OII has coordinated the publication of four versions of 

non-regulatory SES guidance, each updated to address ongoing challenges 

with SES implementation. The latest and most comprehensive version of 

non-regulatory SES guidance was published in June 2005. In May 2006,

Education issued a separate supplement to the guidance containing 

additional information on private school participation in providing SES 

and a policy letter clarifying the definition of a district-affiliated provider. 

In addition to its monitoring efforts, OII also provides SES implementation 

assistance, in part through presentations at conferences, and through 

grants to external organizations that assist states and districts. For 

example, OII staff have presented information on SES policy and 

promising practices at national meetings attended by SES coordinators 

and others involved in SES implementation. In addition, the office has 

issued grants to the Black Alliance for Educational Options, the Hispanic 

Council for Reform and Educational Options, and through the Star Schools 

Program to promote SES to minority students and those in rural areas. 

Further, OII funded the Supplemental Educational Services Quality Center 

(SESQC), which offered SES technical assistance through tool-kits, issue 

briefs, and a Web site containing SES information for state and district 

officials, schools, parents, and providers. SESQC also periodically 

convened representatives of organizations working on education issues to 

discuss SES national coordination, challenges, and promising practices. 

However, those meetings and all SESQC activities were discontinued in 

December 2005 when SESQC’s grant period ended. 

 28 California districts audited were Los Angeles Unified, Oakland Unified, Salinas Union

High School, San Diego City Schools, and Stockton Unified.

Page 40 GAO-06-758 No Child Left Behind Act
Other Education offices also provide SES support through various means. 

For example, OESE funded the Comprehensive Centers Program through 

grants that established technical assistance centers across the country to 

help low-performing schools and districts close achievement gaps and 

meet the goals of NCLBA. Of these, the Center on Innovation and 

Improvement provides support to regional centers that assist states with 

Education’s programs, including SES. In addition, Education’s Policy and

Program Studies Service, within the Office of Planning, Evaluation and 

Policy Development, oversees several research studies that examine SES, 

either in whole or in part. These reports, such as the National Assessment 

of Title I: Interim Report and Case Studies of Supplemental Services

under the No Child Left Behind Act, provide states and districts with 

information on SES implementation, challenges, and promising practices. 

Further, Education’s Center for Faith-Based and Community Initiatives 

offers technical assistance to faith- and community-based organizations 

interested in becoming state-approved SES providers. 

Given the technical assistance and support Education has already 

provided to states and districts for implementation of SES and school 

choice, and the department’s view that implementation of these provisions 

has been uneven throughout the country, in May 2006, Education issued a 

policy letter announcing the department’s plans to take significant 

enforcement action. Specifically, Education plans to use the data collected 

through its monitoring and evaluation efforts to take enforcement actions 

such as placing conditions on state Title I grants, withholding federal 

funds, or entering into compliance agreements. In the letter, the 

department noted that its various monitoring activities have identified 

several areas of noncompliance with SES requirements. For example, the 

OIG’s audits found that each of the six states reviewed failed to adequately 

monitor their districts for compliance. Consequently, nearly all of the 

parental notification letters reviewed failed to include the required key

components, and several districts failed to budget sufficient funding for 

services. Through our own analysis of Education’s monitoring reports, we 

also found that some of the states reviewed were found to have inadequate 

or incomplete processes for monitoring and evaluating SES providers. 

Page 41 GAO-06-758 No Child Left Behind Act
Despite Education’s efforts, many states and districts reported needing 

more information and assistance to better comply with certain aspects of 

SES implementation, including SES evaluation (see table 6). Specifically, 

85 percent of states and an estimated 70 percent of districts needed 

additional assistance with methods for evaluating SES, and over 

60 percent also needed assistance with developing data systems. Many 

districts also needed more information on provider quality and

effectiveness. Although OESE and OIG monitoring results have also 

continually indicated that states and districts struggle with SES evaluation, 

Education has yet to provide comprehensive assistance in this area, and 

during our site visits, officials mentioned that they have been relying on 

other states, organizations, or individuals for evaluation assistance. States

and districts also indicated a need for more support and technical 

assistance to help them administer SES. Specifically, approximately 

three-fourths of states and two-thirds of districts reported needing funding

to increase their capacity to implement SES. Many states also reported 

needing tool kits and model/sample documents, as well as training from 

Education, and a majority of districts needed effective parent outreach 

strategies. Further, most states reported a need for conferences or 

meetings where they could share lessons learned and promising practices 

with other states. A few Tennessee officials mentioned that conferences 

hosted by national organizations have been an effective means of allowing 

SES officials to gather and share knowledge. 

Many States and Districts 

Reported Needing SES 

Evaluation Support and 

Additional Technical 

Assistance, and Some 

Suggested Increasing 

Flexibility through 

Education’s Pilot Programs 

Page 42 GAO-06-758 No Child Left Behind Act
Table 6: Percentage of States and Districts in Need of Different Types of Information or Assistance with SES Implementation 

Percentage

of states

Estimated percentage 

of districts

Evaluation information or assistance needed

Methods for evaluating SES 85 Information on provider quality or 

effectiveness 

78

Developing data systems 71 Methods for evaluating SES 70

Developing data systems for 

monitoring and evaluation 

64

Other support or technical assistance needed 

Conferences/meetings with other 

states to share lessons learned/ 

promising practices 

90 Funding to increase district capacity to 

implement SES 

67

Tool kits and model/sample

documents 

85 Information on effective parent

outreach strategies 

61

Training 85 Assistance with contract negotiation 35

Funding to increase state capacity to 

implement SES 

77

Approving and monitoring providers 71

Allowances for SES to be offered 

before school choice provisions

58

Clarification of roles for states, 

districts, and providers in implementing 

SES 

46

Waivers for states for exclusion from 

specific SES provisions 

44

Clarification of the 20-percent setaside requirements and carryover 

allowances 

44

Source: GAO.

While three-fourths of states reported that the most recent version of 

Education’s SES guidance, published in June 2005, has been very or 

extremely useful, several states commented through our survey that they 

needed additional or clearer guidance on certain SES provisions. This 

included guidance on managing SES costs and fees, implementing SES in

rural areas, and handling provider complaints. During three of our site 

visits, officials also expressed some concern about the lack of clarity in 

the SES guidance with regards to student eligibility requirements and how 

to craft a parental SES notification letter that is both complete and easy 

for parents to understand. 

Page 43 GAO-06-758 No Child Left Behind Act
Regarding parental notification letters, though both OESE and OIG found 

many states and districts to be non-compliant with the federal requirement 

that district SES parental notification letters include several specific 

elements,29 Education’s SES guidance, which is coordinated by OII, 

provides a sample that does not clearly specify all of the key elements 

required by SES law and regulations. For example, the sample letter does

not include information on provider services, qualifications, and 

effectiveness. Furthermore, a few state and district officials commented 

that, when followed, the Title I regulations governing SES yield a letter 

that is unreasonably long and complex, which may be difficult for parents 

to understand. 

Many states and districts expressed interest in the flexibility offered 

through two pilot programs that Education implemented during 2005-2006. 

The department designed these pilots to increase the number of eligible 

students receiving SES and to generate additional information about the 

effect of SES on student academic achievement. For example, several 

state and district SES coordinators expressed interest in Education’s pilot 

program that allowed two districts in needs improvement status to act as 

SES providers in exchange for their expansion of student access to SES 

providers and collection of achievement data to determine SES program 

effectiveness. During three of our four site visits, state and district officials 

expressed concern that districts identified for needs improvement are 

excluded from delivering SES, and one state official noted that removing 

districts from the state approved provider list may result in lower SES 

participation. Further, in our surveys, three state SES coordinators and 

17 district SES coordinators wrote in comments that permitting districts in 

needs improvement status to provide services would assist their efforts. 

Through both our surveys and site visits, officials suggested that allowing 

districts to act as providers may ease student access to SES for rural 

districts that do not have providers located nearby, allow more students to 

participate in SES because district costs to provide services are sometimes 

lower than other providers’ costs, and enable districts to continue their 

existing tutoring programs that they feel are effective and meet the same 

goals as SES. Overall, we estimate that 15 percent of districts were stateapproved providers in 2004-2005. However, another national survey 

 29 As discussed earlier, OIG found all six of the states it visited to be deficient with respect 

to parent notifications. In addition, in our analysis of OESE Title I monitoring reports 

issued as of June 2006, we found that OESE cited 9 of the 40 states it had visited for SES 

non-compliance with respect to district parent notifications.

Page 44 GAO-06-758 No Child Left Behind Act
recently found the percentage of urban and suburban districts that are 

state-approved SES providers is declining.30

The other SES pilot allowed four districts in Virginia to offer SES instead 

of school choice in schools that have missed state performance goals for 

2 years and are in their first year of needs improvement. During our site 

visits and through our surveys, many states and districts expressed 

interest in adjusting the order of the SES and school choice interventions. 

Specifically, half of states and over 60 percent of districts suggested that 

SES should be made available before school choice (see table 7). Further, 

approximately three-quarters of both states and districts indicated that 

SES should be offered either before or simultaneously with choice. As we 

found through our previous work on school choice, few students are 

opting to transfer schools in the first year of needs improvement, and 

therefore this change would provide students with another option to 

receive additional academic support in that year. Further, in a recent 

national study, district and school officials noted that parents and students 

are often not interested in changing schools, in part because of potential 

long commutes and satisfaction with their current schools, which suggests 

that parents and students would likely prefer to receive SES in their own 

schools and neighborhoods rather than school choice.31 In line with 

interest in increased flexibility with these interventions, in May 2006,

Education announced that due to the positive results in Virginia districts 

under the pilot, the department plans to extend and expand this pilot in 

2006-2007. 

Table 7: State and District Opinion on the Ordering of School Choice and SES 

In percent 

Order of school choice and SES States District

SES should precede school choice 48 62

Both school choice and SES should be offered at the same time 27 15

School choice should precede SES 15 23

Source: GAO.

Note: 10 percent of states did not respond or were not sure. In addition, district percentages are 

estimates. 

 30 See the Center on Education Policy, From the Capitol to the Classroom: Year 4 of the No 

Child Left Behind Act (Washington D.C.: March 2006). 

31 Ibid. 

Page 45 GAO-06-758 No Child Left Behind Act
Over the last few years, almost all states and approximately 1,000 districts 

have been required to implement SES across the country and, if current 

trends continue, more schools will be required to offer services in the 

future. Although some states and districts are beginning to gain experience 

in implementing SES and use promising approaches to increase SES 

participation, many students are still not receiving services, in part 

because providers are sometimes not available to serve certain areas and 

groups. In addition, some districts are unsure how to involve school 

officials in facilitating local coordination of SES implementation and 

providing effective parental notification. While Education has provided 

support to states and districts through guidance and technical assistance, 

many report needing additional assistance to address these challenges. 

Further, the lack of clarity between policy guidance issued by OII and 

criteria used by OESE in their compliance reviews of states’ 

implementation efforts creates additional challenges in meeting the federal 

requirements for parental notification letters. Providing states with clear 

guidance that has been coordinated across Education offices is especially

important now that the department has announced plans to take 

significant enforcement actions to ensure states comply with federal SES

requirements. 

Conclusions 

While some districts do not have any students receiving services and, 

therefore, are not spending any Title I funds for SES, other districts are 

spending more than their entire set-aside and still have students on waiting 

lists to receive services. Two districts have been able to participate in

Education’s pilot program waiving federal regulations that preclude 

districts in need of improvement from providing SES, which may help

them provide services to more students at a lower cost. However, 

extending this flexibility to more districts depends on the evaluation of the 

quality of these services to determine if SES is having a positive impact on

student academic achievement. In addition, the absence of strategies that

states can use to set parameters around program design and costs further 

hinders their ability to stretch available funding to serve more students. 

Federal and state oversight of district efficiency in using federal funds to 

provide SES services is hindered by incomplete reporting requirements 

that require states to report on the number of eligible children receiving

SES, but not the data they collect on the amount of Title I funding used to

serve them. This information gap limits Education’s ability to track state 

and district compliance in spending funds for SES. Further, Education’s 

ability to ensure that federal dollars are effectively spent to improve 

student academic achievement is limited until states increase their 

capacity to monitor and evaluate provider performance. In the absence of 

additional federal technical assistance and access to information about 

Page 46 GAO-06-758 No Child Left Behind Act
state and district promising practices, some states may continue to 

struggle with implementation and evaluation of SES. 

To help states and districts implement SES more effectively, we 

recommend that the Secretary of Education use the department’s Web site

and the Center on Innovation and Improvement, as well as other means of 

communication, to: 

Recommendations 

• Provide federal guidance on SES parental notification letters that is 

clear and has been coordinated internally between OII and OESE to 

provide additional assistance to states and districts to help them 

comply with federal requirements and ensure that letters are easy for 

parents to understand. Education might consider providing several 

samples of actual district notification letters that meet these criteria. 

• Collect and disseminate information on promising practices used by 

states to attract more providers for certain areas and groups and 

promising practices used by districts to improve parental notification 

of SES services and providers’ ability to serve specific groups of 

students and to encourage student attendance. 

• Provide examples of how districts can involve schools and school 

officials to facilitate local coordination with providers. 

To improve states’ and districts’ ability to make the most of funding for 

SES and provide services to the maximum number of students, we 

recommend that the Secretary of Education: 

• Consider expanding the 2005-2006 pilot that allowed two districts in 

need of improvement to enter into flexibility agreements to serve as 

SES providers if evaluation results show that these districts can 

provide quality SES services. 

• Clarify what states can do through the provider approval process to set 

parameters around program design and costs. For example, Education 

could issue guidance to states that clarifies their authority to set 

parameters on issues such as minimum hours of SES per student, 

minimum tutor qualifications, and cost ranges. In addition, Education 

could suggest to states that they coordinate these discussions with 

districts to address their concerns about program design and costs. 

To improve federal and state monitoring of SES, we recommend that the 

Secretary of Education: 

• Require states to report information necessary to determine the 

amount of funds spent by districts to provide SES and the percentage 

Page 47 GAO-06-758 No Child Left Behind Act
of their Title I allocations that this amount represents. Because almost 

all states reported that they are planning to monitor district SES 

expenditures, Education could require the states to submit these data 

through the annual NCLBA Consolidated State Performance Reports. 

• Provide states with technical assistance and guidance on how to 

evaluate the effect of SES on student academic achievement. For 

example, Education might require the Center on Innovation and 

Improvement to focus its SES assistance on providing states with 

suggested evaluation methods, measures to assess the impact of SES 

on achievement, and criteria for using this information to monitor and 

withdraw state approval from providers. Further, lessons learned and 

promising practices on evaluation could also be shared with states on 

the Center on Innovation and Improvement’s Web site or during 

national or regional meetings, trainings, or conferences. 

We provided a draft of this report to Education for review and comment. 

Educations’ written comments are reprinted in appendix II, and the 

department's technical comments were incorporated into the report as 

appropriate. In its written comments, Education expressed appreciation 

for the report’s recommendations and cited actions the department has 

already initiated or plans to take in addressing them. Specifically, 

Education noted several projects under development that might assist in 

carrying out our recommendations to provide more assistance to states on

notifying parents, attracting providers for certain areas and groups, and 

involving schools. The department also said that is currently considering 

expanding the pilot allowing districts in need of improvement to apply to

become SES providers, per our recommendation. Regarding our 

recommendation that Education clarify what states can do to set 

parameters around program design and costs, Education said it would 

consider addressing this issue further in the next set of revisions to the 

SES non-regulatory guidance. In addition, Education said it would address 

our recommendations to improve federal and state monitoring of SES by 

proposing that districts report on their SES spending and by providing 

more SES evaluation assistance to states through an updated issue brief as 

well as technical assistance provided by the Comprehensive Center on 

Innovation and Improvement and at a conference this fall. 

We are sending copies of this report to appropriate congressional 

committees, the Secretary of Education, and other interested parties. 

Copies will also be made available upon request. In addition, the report 

will be available at no charge on GAO’s Web site at http://www.gao.gov. If 

you or your staff have any questions about the report, please contact me at 

(202) 512-7215. Contact points for our Offices of Congressional Relations 

Agency Comments 

Page 48 GAO-06-758 No Child Left Behind Act
and Public Affairs may be found on the last page of this report. GAO staff 

who made major contributions to this report are listed in appendix III. 

Marnie S. Shaul 

Director, Education, Workforce, and

 Income Security Issues 

Page 49 GAO-06-758 No Child Left Behind Act
List of Requesters 

The Honorable Edward M. Kennedy 

Ranking Minority Member 

Committee on Health, Education, Labor, and Pensions 

United States Senate 

The Honorable Christopher J. Dodd 

Ranking Minority Member 

Subcommittee on Education and Early Childhood Development 

Committee on Health, Education, Labor, and Pensions 

United States Senate 

The Honorable Howard “Buck” McKeon 

Chairman 

The Honorable George Miller 

Ranking Minority Member 

Committee on Education and the Workforce 

House of Representatives 

The Honorable Dale E. Kildee 

Ranking Minority Member 

Subcommittee on 21st Century Competitiveness

Committee on Education and the Workforce 

House of Representatives 

The Honorable Lynn C. Woolsey 

Ranking Minority Member 

Subcommittee on Education Reform 

Committee on Education and the Workforce 

House of Representatives 

The Honorable Hillary Rodham Clinton 

The Honorable Richard J. Durbin 

The Honorable Charles E. Schumer 

United States Senate 

The Honorable Robert E. Andrews 

The Honorable John A. Boehner 

The Honorable Rahm Emanuel 

The Honorable Betty McCollum 

House of Representatives 

Page 50 GAO-06-758 No Child Left Behind Act
Appendix I: Scope and Methodology 

Page 51 GAO-06-758

Appendix I: Scope and Methodology 

To obtain nationally representative information on supplemental 

educational services (SES) participation, state and local implementation, 

and federal oversight, we conducted a Web-based survey of state SES 

coordinators and a mail survey of district SES coordinators from a 

nationally representative sample of districts with schools required to offer 

SES. We also conducted site visits during which we interviewed state, 

district, and school officials representing four school districts, and we 

conducted interviews with 22 SES providers both during the site visits and 

separately. In addition, we spoke with staff at Education involved in SES

oversight and implementation and reviewed Education’s data on SES. We 

conducted our work from August 2005 through July 2006 in accordance 

with generally accepted government auditing standards. 

Survey of States To better understand state SES implementation, particularly how states 

are monitoring and evaluating SES, we designed and administered a Webbased survey of state SES coordinators in all 50 states, the District of 

Columbia, and Puerto Rico. The survey was conducted between January 

and March 2006 with 100 percent of state SES coordinators responding. 

The survey included questions about student participation in SES, actions

taken to increase participation, SES funding and expenditures, methods 

used to monitor and evaluate implementation, implementation challenges, 

and assistance received from Education. 

Because this was not a sample survey, there are no sampling errors. 

However, the practical difficulties of conducting any survey may introduce 

nonsampling errors, such as variations in how respondents interpret 

questions and their willingness to offer accurate responses. We took steps 

to minimize nonsampling errors, including pre-testing draft instruments 

and using a Web-based administration system. Specifically, during survey 

development, we pre-tested draft instruments with officials in Oregon, 

Maryland, and Washington between October and November 2005. In the 

pre-tests, we were generally interested in the clarity of the questions and 

the flow and layout of the survey. For example, we wanted to ensure 

definitions used in the surveys were clear and known to the respondents, 

categories provided in closed-ended questions were complete and 

exclusive, and the ordering of survey sections and the questions within 

each section was appropriate. On the basis of the pre-tests, the Web 

instrument underwent some slight revisions. A second step we took to 

minimize nonsampling errors was using a Web-based survey. By allowing 

respondents to enter their responses directly into an electronic 

instrument, this method automatically created a record for each 

respondent in a data file and eliminated the need for and the errors (and 

No Child Left Behind Act
Appendix I: Scope and Methodology 

costs) associated with a manual data entry process. To further minimize 

errors, programs used to analyze the survey data and make estimations 

were independently verified to ensure the accuracy of this work. 

While we did not fully validate specific information that states reported 

through our survey, we took several steps to ensure that the information 

was sufficiently reliable for the purposes of this report. For example, after 

the survey was closed, we made comparisons between select items from 

our survey data and other national-level data sets.1

 We found our survey 

data were reasonably consistent with the other data sets. On the basis of 

our checks, we believe our survey data are sufficient for the purposes of 

our work. 

Survey of School Districts To obtain national-level information on district implementation of SES, we 

administered a mail survey to a nationally representative sample of 

districts that had schools required to offer SES in school year 2004-2005. 

The survey was conducted between January and March 2006. To obtain 

the maximum number of responses to our survey, we sent a reminder 

postcard to nonrespondents approximately 1 week after the initial mailing 

of the survey instrument, a follow-up mailing with the full survey 

instrument to nonrespondents approximately 3 weeks after the initial 

mailing, and a second follow-up mailing with the full survey instrument 

approximately 4 weeks later. The survey included questions about student 

participation in SES, characteristics of students receiving SES, actions 

taken to increase participation, SES funding and expenditures, methods 

used to monitor and evaluate implementation, implementation challenges, 

and assistance received and still needed. 

The target population of 1,095 districts consisted of public school districts 

with at least one school in each of their jurisdictions required to provide 

SES in the 2004-2005 school year. To define our population, we collected 

school improvement information from state education agency Web sites 

and the NCLBA Consolidated State Performance Reports: Part I for 

Population 

 1

 We compared our state survey data to data on SES reported by Education in the

Evaluation of Title I Accountability Systems and School Improvement Efforts: Findings

From 2002-03 (Washington, D.C.: Nov. 2005) and the National Assessment of Title I:

Interim Report (Washington, D.C.: Feb. 2006). We also compared our survey data to data 

on SES reported by the Center on Education Policy in From the Capital to the Classroom:

Year 3 of the No Child Left Behind Act (Washington, D.C.: March 2005) and From the 

Capital to the Classroom: Year 4 of the No Child Left Behind Act (Washington, D.C.: 

March 2006). 

Page 52 GAO-06-758 No Child Left Behind Act
Appendix I: Scope and Methodology 

Reporting on School Year 2003-2004 that states submitted to Education. 

When available, we checked both sources for school improvement 

information and used the source that provided the most recently updated

data, as this data is often updated by states over the course of the school 

year. After constructing our population of districts, we used Education’s 

Common Core of Data Local Education Agency (School District) 

preliminary file and the Public Elementary/Secondary School preliminary 

file for the 2003-2004 school year to further define the characteristics of 

our population. On the basis of our review of these data, we determined 

these sources to be adequate for the purposes of our work. 

The sample design for the mail survey was a stratified random sample of 

districts with one certainty stratum containing all of the districts with 

100,000 students or more and one stratum containing all other districts in 

the universe. We included the 21 districts with 100,000 or more students 

with certainty in the sample to ensure we gathered information from the 

largest districts required to offer SES. 

Sample Design and Errors 

We selected a simple random sample of districts in the non-certainty 

stratum and calculated the sample size to achieve a precision of plus and 

minus 7 percent at the 95 percent confidence level for an expected 

proportion of 50 percent. To ensure the sample sizes were adequate, we 

increased the sample size assuming we would obtain a 70 percent 

response rate. The total sample size for this stratum was 237 districts. In 

the sample, each district in the population had a known, nonzero 

probability of being selected. Each selected district was subsequently 

weighted in the analysis to account statistically for all the schools in the 

population, including those that were not selected. Table 8 provides a 

description of the universe and sample of districts. 

Table 8: Description of the Population and Sample of Districts

Stratum Population Sample size

District enrollment >= 100,000 21 21

District enrollment < 100,000 1,074 237

Total 1,095 258

Source: GAO.

Because we surveyed a sample of districts, our results are estimates of a 

population of districts and thus are subject to sampling errors that are 

associated with samples of this size and type. Our confidence in the 

precisions of the results from this sample is expressed in 95 percent 

Page 53 GAO-06-758 No Child Left Behind Act
Appendix I: Scope and Methodology 

confidence intervals, which are expected to include the actual results in 

95 percent of the samples of this type. We calculated confidence intervals 

for this sample based on methods that are appropriate for a stratified 

random sample. 

We determined that 10 of the sampled districts were out of scope because 

they did not have any schools required to provide SES in the 2004-2005 

school year. All estimates produced from the sample and presented in this

report are for the estimated target population of 1,034 districts with at 

least one school required to provide SES in the 2004-2005 school year. All 

percentage estimates included in this report have margins of error of plus 

or minus 8 percentage points or less, except for those shown in table 9. All 

other numerical estimates, such as the total number of schools required to

offer SES in 2004-2005, included in this report have margins of error of 

plus or minus 18 percent or less. 

Page 54 GAO-06-758 No Child Left Behind Act
Appendix I: Scope and Methodology 

Table 9: Sampling Error Calculations

Page Description Estimate 

Lower

bound

Upper 

bound

15 Percentage of districts that spent 20 percent or less of the amount set aside for 

SES in 2004-2005. 

38 30 47

15 Percentage of districts that spent over 80 percent of the amount set aside for SES 

in 2004-2005. 

18 11 26

17 Percentage of districts where over half of SES recipients in 2004-2005 were

elementary school students. 

56 48 65

17 Percentage of districts where less than 5 percent of SES recipients in 2004-2005 

were students with limited English proficiency. 

33 22 43

17 Percentage of districts where over half of SES recipients in 2004-2005 were

students with limited English proficiency. 

21 13 32

17 Percentage of districts where less than 20 percent of SES recipients in 2004-2005 

were students with disabilities. 

69 59 79

17 Percentage of districts where over half of SES recipients in 2004-2005 were

African-American students. 

41 31 50

17 Percentage of districts where over half of SES recipients in 2004-2005 were

Hispanic students. 

30 21 39

20 Percentage of the districts not notifying parents before the beginning of the school 

year that did not receive final school improvement data from the state prior to 

beginning of the school year. 

51 42 61

20 Percentage of the districts not notifying parents before the beginning of the school 

year that notified parents within 2 months following the beginning of the school 

year. 

91 82 96

21 Percentage of rural districts where availability of transportation for students 

attending SES was a moderate to very great challenge. 

50 37 63

32 Percentage of the districts that observed most or all providers during on-site

reviews conducted in 2004-2005. 

83 72 91

Source: GAO.

We took steps to minimize nonsampling errors that are not accounted for 

through statistical tests, like sampling errors. In developing the mail 

survey, we conducted several pretests of draft instruments. We pretested 

the survey instrument with district officials in Woodburn, Ore.; Tacoma, 

Wash.; Baltimore, Md.; and Alexandria, Va., between October and 

November 2005. These pre-tests were similar to the state Web survey pretests in design and content. On the basis of the pre-tests, the draft survey 

instrument underwent some slight revisions. 

While we did not fully validate specific information that districts reported 

through our survey, we took several steps to ensure that the information 

was sufficiently reliable for the purposes of this report. For example, data

Page 55 GAO-06-758 No Child Left Behind Act
Appendix I: Scope and Methodology 

from the surveys were double-keyed to ensure data entry accuracy, and 

the information was analyzed using statistical software. After the survey 

was closed, we also made comparisons between select items from our 

survey data and other national-level data sets.2

 We found our survey data 

were reasonably consistent with the external sources. On the basis of our 

checks, we believe our survey data are sufficient for the purposes of our 

work. 

We received survey responses from 73 percent of the 258 district Title 

I/SES coordinators in our sample. The response rate, adjusted for the 

known and estimated districts that were out of scope, was 77 percent. 

Response Rate 

Table 10: SES District Survey Response Rates 

Stratum

Districts in

population

Districts in

sample 

In-scope

districts in

sample

Districts 

responding

District enrollment 

>= 100,000 

21 21 21 21

District enrollment 

< 100,000 

1,074 237 227 167

Total 1,095 258 248 188

Source: GAO.

After the survey was closed, we analyzed the survey respondents to 

determine if there were any differences between the responding districts, 

the nonresponding districts, and the population. We performed this 

analysis for three characteristics—total number of students enrolled, total 

number of special education students, and total number of English 

language learner students. We determined whether sample-based 

estimates of these characteristics compared favorably with the known 

population values. The population value for all of the characteristics we 

examined fell within the 95 percent confidence intervals for the estimates 

from the survey respondents. On the basis of the 77 percent response rate 

 2

 We compared our district survey data to data on SES reported by Education in the

Evaluation of Title I Accountability Systems and School Improvement Efforts: Findings

From 2002-03 (Washington, D.C.: Nov. 2005) and the National Assessment of Title I:

Interim Report (Washington, D.C.: Feb. 2006). We also compared our survey data to data 

on SES reported by the Center on Education Policy in From the Capital to the Classroom:

Year 3 of the No Child Left Behind Act (Washington, D.C.: March 2005) and From the 

Capital to the Classroom: Year 4 of the No Child Left Behind Act (Washington, D.C.: 

March 2006). 

Page 56 GAO-06-758 No Child Left Behind Act
Appendix I: Scope and Methodology 

and this analysis, we chose to include the survey results in our report and 

produce sample-based estimates to the population of districts required to 

provide SES in the 2004-2005 school year. 

To understand SES implementation at the local level, we conducted site 

visits to four districts between October 17, 2005, and February 16, 2006. 

The districts visited included Woodburn School District (Woodburn, Ore.), 

Hamilton County Schools (Chattanooga, Tenn.), Newark Public Schools 

(Newark, N.J.), and Chicago Public Schools (Chicago, Ill.). The four 

districts visited were selected because they had experience implementing 

SES in their schools and were recommended by stakeholders as having 

promising parent outreach and/or monitoring practices. When viewed as a 

group, the districts also provided variation across characteristics such as 

geographic location, district size, student ethnicity, and the percentage of 

students with limited English proficiency or disabilities. 

During the site visits, we interviewed state officials, including the state 

SES coordinator, and district officials, including the superintendent and 

SES coordinator. We also interviewed officials representing 12 schools, 

including principals and other school staff involved with SES. In total, we 

visited several schools of each level, from elementary to high, and though 

district officials selected the schools we visited, all of the schools had 

experience implementing SES. Through our interviews with state, district, 

and school officials, we collected information on district efforts to notify 

parents and fulfill implementation responsibilities, student participation, 

providers, local implementation challenges, and implementation 

assistance received and needed. During the visits, we also interviewed 

providers and observed tutoring sessions in order to better understand 

implementation. During our visit to Woodburn, Ore., we also observed a 

provider fair. 

In addition to our site visits to four districts, we also visited the Rhode 

Island Department of Elementary and Secondary Education in March 2006 

to gather additional information on state efforts to monitor and evaluate 

SES. Rhode Island invited us to attend two meetings the state held with 

districts implementing SES and providers serving students in Rhode 

Island, during which SES challenges, ways to improve implementation, 

and state efforts to evaluate providers were discussed. 

In total, we conducted interviews with 22 providers, including 15 providers 

during the site visits and 7 providers operating in multiple states. The 

Site Visits 

Provider Interviews 

Page 57 GAO-06-758 No Child Left Behind Act
Appendix I: Scope and Methodology 

Education Industry Association assisted our efforts to contact multi-state 

providers, and most of the multi-state providers we interviewed were 

association members.3

 Multi-state provider interviews were conducted 

between November and December 2005. 

Through all of our provider interviews, we collected information on 

provider efforts to increase participation in SES, align services with state 

standards and district curriculum, and communicate with parents and 

schools to ensure students are receiving needed services. We also 

collected information on students served, tutor and program 

characteristics, and provider challenges to SES implementation. While the 

providers we interviewed reflect some variety in provider characteristics, 

our selections were not intended to be representative. Thus, the findings 

from our interviews cannot be used to make inferences about all 

providers. 

Education Data Analyses We analyzed state data submitted to Education through the NCLBA 

Consolidated State Performance Reports (CSPR) for school years 2002-

2003, 2003-2004, and 2004-2005. State reports from all 3 years included the 

number of students receiving SES and the number of schools those 

students attended, and state reports from 2003-2004 and 2004-2005 also 

included the number of students eligible for SES. Data from the 2003-2004 

CSPRs were used to assist our analysis of SES participation. To assess the 

reliability of the 2003-2004 data, we performed a series of tests, which 

included checking to ensure that data were consistent, that subtotals 

added to overall totals and that data provided for 1 year bore a reasonable

relationship to the next year’s data and to data reported elsewhere, 

including state education reports. We also spoke with Education officials 

about their follow-up efforts to verify the data. At the time of our review, 

Education was in the process of completing efforts to verify the 2003-2004

data. 

While we compared the 2004-2005 CSPR data to data obtained through our 

state and district surveys to further verify our data, we generally did not 

 3

 The seven multi-state providers we spoke with were Babbage Net School, Cambridge 

Educational Services, Catapult Learning, Huntington Learning Center, Newton Learning, 

Platform Learning, and University Instructors. 

Page 58 GAO-06-758 No Child Left Behind Act
Appendix I: Scope and Methodology 

use the 2004-2005 CSPR data for our analysis.4 During this comparison 

analysis, where we found discrepancies or sought clarification, we 

followed up with state officials. In several states, officials revised the 

numbers that they had initially reported to us or to Education. On the 

basis of our review of these data, we determined these sources to be 

adequate for the purposes of our work. 

We also considered SES-related findings from Education studies, including

the Evaluation of Title I Accountability Systems and School 

Improvement Efforts: Findings From 2002-03 (2005) and the National 

Assessment of Title I: Interim Report (2006). To ensure the findings from 

these studies were generally reliable, we reviewed each study’s 

methodology, including data sources and analyses, limitations, and 

conclusions. In addition, in designing our state and district surveys, we 

reviewed SES-related survey questions used by Education in these studies. 

 4

 At the time of our survey, seven states were unable to provide 2004-2005 data on SES 

recipients and students eligible to receive SES. By the time Education required states to 

submit the 2004-2005 CSPR reports in March 2006, six of these states had provided this 

data to Education; we, therefore, used this data to supplement their responses to our 

survey. The seventh state, New Jersey, was unable to provide this data to Education by the 

time 2004-2005 CSPRs were due; therefore, New Jersey data is not included in the 2004-

2005 participation analysis included in this report. 

Page 59 GAO-06-758 No Child Left Behind Act
Appendix II: Comments from the Department Appendix II: Comments from the Department of Education 

of Education 

Page 60 GAO-06-758 No Child Left Behind Act
Appendix II: Comments from the Department 

of Education 

Page 61 GAO-06-758 No Child Left Behind Act
Appendix II: Comments from the Department 

of Education 

Page 62 GAO-06-758 No Child Left Behind Act
Appendix II: Comments from the Department 

of Education 

Page 63 GAO-06-758 No Child Left Behind Act
Appendix III: GAO

A

 Contacts and Staff 

cknowledgments 

Page 64 GAO-06-758

Appendix III: GAO Contacts and Staff 

Acknowledgments 

Marnie S. Shaul (202) 512-7215 shaulm@gao.gov

Cornelia M. Ashby (202) 512-7215 ashbyc@gao.gov

Cindy Ayers, Assistant Director, and Rachel Frisk, Analyst-in-Charge, 

managed this assignment and made significant contributions to all aspects 

of this report. Cathy Roark, Ted Burik, and David Perkins also made 

significant contributions. Kevin Jackson, Jean McSween, Jim Ashley, and 

Jerry Sandau provided methodological expertise and assistance; Rachael

Valliere assisted with message and report development; and Rasheeda 

Curry made contributions during study design. In addition, Jessica 

Botsford assisted in the legal analysis. 

GAO Contacts 

Acknowledgments 

No Child Left Behind Act
Related GAO Products 

Page 65 GAO-06-758

Related GAO Products 

No Child Left Behind Act: Assistance from Education Could Help States 

Better Measure Progress of Students with Limited English Proficiency. 

GAO-06-815. Washington, D.C.: July 26, 2006. 

No Child Left Behind Act: States Face Challenges Measuring Academic 

Growth that Education’s Initiatives May Help Address. GAO-06-661. 

Washington, D.C.: July 17, 2006. 

No Child Left Behind Act: Most Students with Disabilities Participated 

in Statewide Assessments, but Inclusion Options Could Be Improved.

GAO-05-618. Washington, D.C.: July 20, 2005. 

No Child Left Behind Act: Education Needs to Provide Additional 

Technical Assistance and Conduct Implementation Studies for School 

Choice Provision. GAO-05-7. Washington, D.C.: December 10, 2004. 

No Child Left Behind Act: Improvements Needed in Education’s Process 

for Tracking States’ Implementation of Key Provisions. GAO-04-734. 

Washington, D.C.: September 30, 2004. 

No Child Left Behind Act: Additional Assistance and Research on 

Effective Strategies Would Help Small Rural Districts. GAO-04-909. 

Washington, D.C.: September 23, 2004. 

Disadvantaged Students: Fiscal Oversight of Title I Could Be Improved.

GAO-03-377. Washington, D.C.: February 28, 2003.

Title I Funding: Poor Children Benefit Though Funding Per Poor Child 

Differs. GAO-02-242. Washington, D.C.: January 31, 2002. 

(130507) No Child Left Behind Act
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER