SCHOOL 

IMPROVEMENT 

GRANTS

Education Should 

Take Additional Steps 

to Enhance 

Accountability for 

Schools and 

Contractors

Report to Congressional Requesters

April 2012

GAO-12-373

United States Government Accountability Office

GAO
United States Government Accountability Office

Highlights of GAO-12-373, a report to 

congressional requesters

April 2012

SCHOOL IMPROVEMENT GRANTS

Education Should Take Additional Steps to Enhance 

Accountability for Schools and Contractors

Why GAO Did This Study

The School Improvement Grant 

(SIG) program funds reforms in low 

performing schools. Congress 

provided $3.5 billion for SIG in fiscal 

year 2009, and a total of about $1.6 

billion was appropriated in fiscal 

years 2010-2012. SIG requirements 

changed significantly in 2010. Many 

schools receiving SIG funds must 

now use the funding for specific 

interventions, such as turning over 

certain school operations to an 

outside organization (contractor). 

GAO examined (1) what, if any, 

aspects of SIG pose challenges for 

successful implementation; (2) how 

Education and state guidance and 

procedures for screening potential 

contractors and reviewing contractor 

performance compare with leading 

practices; and (3) to what extent 

Education’s technical assistance and 

oversight activities are effectively 

supporting SIG implementation. GAO 

surveyed SIG directors in all 50 

states and the District of Columbia; 

analyzed Education and state 

documents; and interviewed officials 

from 8 states and school districts in 

those states, SIG contractors, and 

education experts.

What GAO Recommends

GAO recommends that Education (1) 

provide additional support to states 

about making evidence-based grant 

renewal decisions and (2) ensure 

that contractor performance is 

reviewed. Education generally 

supported our first recommendation 

but disagreed with the second. We 

modified our recommendation to 

address some of Education’s 

concerns.

What GAO Found

Successful SIG implementation posed a number of challenges. 

Specifically, state and district officials were challenged to build staff 

capacity and commitment for reform, facing difficulties such as recruiting 

and retaining strong staff members. In addition, the SIG requirements to 

develop teacher evaluations and increase student learning time were 

difficult to implement quickly and effectively because they required 

extensive planning and coordination. Furthermore, states sometimes had 

limited evidence about the performance of SIG schools when making 

grant renewal decisions. For example, although Education’s guidance 

identifies meeting annual student achievement goals as a key criterion for 

making renewal decisions, some states did not receive student 

achievement data by the time decisions had to be made. States also 

made decisions through qualitative assessments of schools’ 

implementation efforts, but such determinations were not always based 

on extensive interaction with schools or systematic monitoring. Education 

did not provide written guidance to states about making evidence-based 

grant renewal decisions after they encountered these challenges.

 

Districts used a significant portion of their SIG funds to hire contractors for 

a range of services, such as managing school operations and conducting 

teacher professional development. Leading practices show that screening 

potential contractors and then reviewing their performance are important 

for ensuring accountability and quality of results. Education required 

screening of contractors before contract awards were made. However, 

Education did not require review of contractors during contract 

performance, and states varied in whether they ensured that contractors 

were reviewed during the course of contract performance.

 

Education’s assistance and oversight activities are generally supporting 

SIG implementation. In our survey, nearly all states reported they were 

satisfied with Education’s technical assistance, particularly the agency’s 

SIG guidance and conferences. In addition, many states reported that 

Education’s guidance was timely. With respect to oversight, Education 

monitored 12 states in school year (SY) 2010-2011 and found 

deficiencies in 11 of the 12 states. Education is working with states to 

correct these deficiencies. For SY 2011-2012, the agency plans to use a 

risk-based approach to conduct on-site monitoring in 14 additional states. 

To maximize its oversight resources, Education also plans to conduct 

some limited monitoring in five additional states in SY 2011-2012. 

Education officials told us that they plan to monitor the remaining states in 

SY 2012-2013 and that these states represent a small percentage of SIG 

funds. View GAO-12-373. To view an e-supplement 

with more data see GAO-12-370SP. For more 

information, contact George Scott at (202) 

512-7215 or scottg@gao.gov. 
Page i GAO-12-373 School Improvement Grants

Letter 1 

Background 3

Successful SIG Implementation Has Posed a Number of Challenges 7

States and Districts Screen Contractors Before Selecting Them to 

Receive SIG Funding, but Inconsistent Approach to Reviewing 

Performance Reduces Contractor Accountability 17

Education’s Assistance and Oversight Are Supporting SIG 

Implementation 21

Conclusion 25

Recommendations 26

Agency Comments and Our Evaluation 26

Appendix I Objectives, Scope, and Methodology 29

Appendix II Comments from the Department of Education 32

Appendix III GAO Contact and Staff Acknowledgments 38

Figures

Figure 1: Percentage of SY 2010-2011 and 2011-2012 School 

Grantees Using Each Reform Model 7

Figure 2: State Responses on Whether Inadequate Action by SIG 

Schools or Districts Was a Reason Schools Were Not

Operating Very Differently 11

Figure 3: Survey Results Regarding the Helpfulness of Education’s 

Technical Assistance Offerings 22

Contents
Page ii GAO-12-373 School Improvement Grants

Abbreviations

Education U.S. Department of Education

SIG School Improvement Grant Program

SY school year

View GAO-12-373 key component

School Improvement Grants: Survey of State Educational Agencies 

Regarding Grant Implementation (GAO-12-270SP), an e-supplement to 

GAO-12-373.

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-12-373 School Improvement Grants

United States Government Accountability Office

Washington, DC 20548

April 11, 2012

The Honorable Thad Cochran

Vice Chairman

Committee on Appropriations

United States Senate

The Honorable Tom Harkin

Chairman

The Honorable Richard C. Shelby

Ranking Member

Subcommittee on Labor, Health and Human Services,

 Education, and Related Agencies

Committee on Appropriations

United States Senate

The School Improvement Grant (SIG) program is designed to fund 

significant reforms in low performing schools. Schools that receive SIG 

funding (SIG schools) receive up to $2 million annually for 3 years to 

improve student outcomes, such as standardized test scores and 

graduation rates. While the program—which provides funds to states on a 

formula basis—was first authorized in fiscal year 2002, it changed 

significantly in fiscal year 2009. Specifically, Congress greatly increased 

funding for the program from $125 million in fiscal year 2007 to $3.5 

billion in fiscal year 2009. From this amount, $3 billion was provided 

under the American Recovery and Reinvestment Act of 2009, and 

schools began spending these funds during school year (SY) 2010-2011.1

In addition to funding increases, the U.S. Department of Education 

(Education) made major changes to SIG program requirements in 2010.

Additionally, a total of approximately $1.6 billion was appropriated in fiscal 

years 2010-2012.

2

 

1

Pub. L. No. 111-5, 123 Stat. 115, 181. 

As a result of these changes, many schools receiving SIG funding must 

now use the funding to implement one of four intervention models, each 

with specific requirements, such as replacing principals or turning over 

school management or certain school operations to an outside 

2

75 Fed. Reg. 66,363 (Oct. 28, 2010).

 
 

Page 2 GAO-12-373 School Improvement Grants

organization (contractor). Also, states are now required to award SIG 

funds to school districts competitively, rather than by formula. States 

evaluate grant applications using several criteria, including the proposed 

intervention model for each school and the district’s budget and reform 

implementation plan, as well as their capacity and commitment to 

effectively implement the reforms. After grant awards are made, states 

are also responsible for deciding whether to renew schools’ SIG funding 

for additional years. State renewal decisions are to be based on criteria 

that can include whether school assessment results meet annual student 

achievement goals, and whether the school has implemented its chosen 

intervention model. Education oversees the program by reviewing state 

grant applications, monitoring implementation and providing technical 

assistance.

The recent changes to the SIG program have raised questions about how 

grants are being implemented. The Senate Appropriations Committee 

and its Subcommittee on Labor, Health and Human Services, Education, 

and Related Agencies requested that we conduct a broad review of the 

SIG program, and in July 2011 we reported on early implementation 

issues in six states during the first year of the expanded SIG program.3

To identify aspects of SIG that pose challenges to successful school 

turnaround, we administered a survey to the 50 states and the District of 

That report found that the program’s implementation time frames in some 

cases did not allow schools sufficient time to plan and fully enact reforms, 

and we recommended that the Secretary of Education consider options to 

award SIG grants to school districts earlier in the school year. In this 

review, we addressed the following questions: (1) What, if any, aspects of 

SIG pose challenges for successful implementation? (2) How do 

Education and state guidance and procedures for screening potential 

contractors and reviewing contractor performance compare with leading 

practices? (3) To what extent are Education’s technical assistance and 

oversight activities effectively supporting SIG implementation?

 

3

For more information, see GAO, School Improvement Grants: Early Implementation 

Under Way, but Reforms Affected by Short Time Frames, GAO-11-741 (Washington, 

D.C., July 25, 2011). The six states were: Delaware, Nebraska, Nevada, Rhode Island, 

Ohio, and Virginia. 
 

Page 3 GAO-12-373 School Improvement Grants

Columbia, and received a 100 percent response rate.4

We conducted this performance audit from January 2011 to April 2012 in 

accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

We also reviewed 

Education documents and interviewed Education officials. We gathered 

in-depth information from eight states that were selected to represent a 

range of size, geographic diversity, and the intervention models being 

used. These states were California, Delaware, Nebraska, Nevada, Ohio, 

Rhode Island, Texas, and Virginia. In each of the states, we also 

interviewed school district and school officials from one to three districts 

that had SIG schools. To collect information about policies and 

procedures for screening potential contractors and reviewing contractor 

performance, we examined Education guidance and state monitoring 

policies. We also interviewed Education officials and state and local 

officials from our site visit states. We identified leading practices for 

screening and reviewing contractors based on our past work. To evaluate 

Education’s technical assistance and oversight efforts, we reviewed 

relevant documents, such as SIG monitoring protocols, and interviewed 

relevant Education officials. We also gathered information from states 

through our survey and state site visits. We interviewed several 

Education-funded technical assistance providers that served our site visit 

states and various stakeholders, such as teachers’ union officials. For 

more information about our scope and methodology, see appendix I.

Education’s changes to SIG requirements in 2010 have led to new 

responsibilities for the agency, states, and school districts. These entities 

all play key roles in the SIG award and implementation process, with 

Education supporting and overseeing state SIG efforts. Before awarding 

formula grants to states, Education reviews each state’s application and 

approves the state’s proposed process for competitively awarding SIG 

 

4

This report does not contain all the results from the survey. The survey and a more 

complete tabulation of the results are included in GAO-12-370SP. When discussing 

survey results in this report, the term “states” also includes the District of Columbia. See 

appendix I for more information about our survey design and implementation.

Background 
 

Page 4 GAO-12-373 School Improvement Grants

grants and monitoring implementation. As part of the state application 

process, states identify and prioritize eligible schools into three tiers:

• Tier I schools. Receive priority for SIG funding and are the state’s 

lowest-achieving 5 percent of Title I schools (or 5 lowest-achieving 

schools, whichever number is greater) in improvement status.5

• Tier II schools. Secondary schools eligible for, but not receiving, Title I 

funds with equivalently poor performance as Tier I schools.

• Tier III schools. Title I schools in improvement status that are not Tier 

I or Tier II schools.

After states receive SIG funding, school districts submit applications to 

states describing their SIG reform plans for eligible schools. Education 

has required that districts base their plans on an analysis of each school’s 

needs, called a needs assessment. After reviewing district applications, 

states distribute their SIG dollars using their approved competitive grant 

award process, giving priority to districts seeking funding for Tier I and 

Tier II schools. Education’s regulations and guidance require districts 

receiving SIG awards for Tier I or Tier II schools to implement one of four 

intervention models in each school. Select aspects of each model are as 

follows:

• Transformation. Transformation schools must replace the principal, 

implement a transparent and equitable teacher and principal

evaluation system that incorporates student academic growth, identify 

and reward staff who are increasing student outcomes, and provide 

increased student learning time, among other requirements.

• Turnaround. In addition to implementing many requirements of the 

transformation model, turnaround schools must use locally adopted 

 

5

Title I, Part A of the Elementary and Secondary Education Act of 1965, as amended, 

provides funding to school districts and schools with high concentrations of students from 

low-income families. Under Title I, states set academic targets and measure schools’ 

progress in meeting them. Schools in improvement status have missed academic targets 

for at least two consecutive years. The definitions of Tier I and Tier II schools also include 

high schools that have a graduation rate of less than 60 percent over a number of years. 
 

Page 5 GAO-12-373 School Improvement Grants

competencies to screen existing staff and rehire no more than 50 

percent of the existing staff.

• Restart. The district must reopen the school under the management of 

a contractor, such as a charter school operator, charter management 

organization, or education management organization.

• Closure. The district must close the school and enroll its students in a 

higher achieving school within a reasonable proximity.

Districts may choose to use contractors to implement aspects of their 

reform plans. Schools enacting a restart model are required to contract 

with an organization that will assume many of the decision-making and 

leadership functions in that school. Districts employing other models may 

also contract with external organizations for services that could include 

data analysis, teacher professional development, and efforts to create 

safe school environments. Our work notes the importance of screening 

potential contractors before awarding contracts, as well as regular 

evaluation in order to ensure contractors are providing timely and quality 

services with government funds.6

In addition to reviewing district applications, states are also responsible 

for monitoring grant implementation. States make decisions about 

whether to renew funding for each SIG school for an additional year, 

based on factors such as whether schools meet annual student 

achievement goals that districts set for the schools. Pursuant to 

Education’s guidance, if a school meets its annual goals, the state must 

renew the school’s SIG grant. If a school does not meet one or more 

annual goals, Education’s guidance gives states the flexibility to consider 

other factors such as the “fidelity with which the school is implementing”

its chosen intervention model.7

Education provides states with technical assistance and oversight 

regarding SIG implementation. For example, Education funds 21 

Comprehensive Centers that help build states’ capacity to assist school 

 

6

GAO, Contract Management: Opportunities to Improve Surveillance on Department of 

Defense Service Contracts, GAO-05-274 (Washington, D.C.: Mar. 17, 2005).

7

We refer to this as fidelity of implementation. Education’s guidance does not further 

define the process or factors states should use when making this determination.
 

Page 6 GAO-12-373 School Improvement Grants

districts and schools. Sixteen of these organizations serve states in 

designated regions, and 5 provide technical assistance on specific issues, 

such as teacher quality. In addition, Education funds Regional 

Educational Laboratories, a network of 10 laboratories that serve 

designated regions by providing access to applied education research 

and development projects, studies, and other related technical assistance 

activities. Education also monitors states’ implementation of SIG. This 

monitoring process consists of visits to selected states and several SIG 

districts and schools within the monitored states, followed by reports 

documenting any findings. States have an opportunity to respond to any 

findings before the release of Education’s monitoring reports.

States have awarded funding to two cohorts of schools since the program 

was modified and expanded in 2010. In the first cohort, 867 schools 

received SIG funding to implement one of the four intervention models in 

SY 2010-2011, and in the second cohort 488 schools received funding to 

implement one of the intervention models in SY 2011-2012. Seven 

states8

 

8

The seven states are Alabama, California, Hawaii, Missouri, Rhode Island, Tennessee, 

and Vermont. In addition, the Bureau of Indian Affairs was granted a waiver.

have received waivers from Education to delay awarding funding 

to their second cohort of schools until SY 2012-2013 because of various 

issues, such as turnover of key staff in state educational agencies. The 

proportion of schools choosing each model was similar in both cohorts 

and, as shown in figure 1, most schools chose to implement the 

transformation model.
 

Page 7 GAO-12-373 School Improvement Grants

Figure 1: Percentage of SY 2010-2011 and 2011-2012 School Grantees Using Each 

Reform Model

Although most states have increased the amount of staff time devoted to 

SIG since the program was expanded, some states have struggled to 

develop the necessary staff capacity to successfully support and oversee 

SIG implementation because of budget constraints. In our survey, 29 

states told us that they have increased the staff time devoted to SIG since 

they first applied for SIG funds in the expanded SIG program. However, 

officials from four of the eight states we visited—California, Nebraska, 

Rhode Island, and Texas—told us that because of budgetary constraints, 

Successful SIG 

Implementation Has 

Posed a Number of 

Challenges 

Staff Capacity and 

Commitment to Reform 

Posed Implementation 

Challenges
 

Page 8 GAO-12-373 School Improvement Grants

the time staff could devote to administering the SIG program and 

monitoring district implementation was significantly limited. For example, 

officials in California said that as a result of the state budget crisis, the 

state legislature reduced the amount of SIG funds available for state 

administration from the allowable 5 percent9 to 0.5 percent, limiting the 

number of staff available to administer the program and monitor districts. 

Several state officials we spoke with also reported that their existing 

workloads made it difficult to focus on SIG. For example, in several states 

the program was administered by officials who also had responsibilities 

for other major education programs, such as Race to the Top.10

In addition, state officials sometimes did not have expertise in supporting 

school turnaround efforts. Officials from Education and several states and 

research groups told us that SIG required states to support local reform 

efforts to a much greater extent than they had in the past, and staff in 

some states had not yet developed the knowledge base to fulfill these 

responsibilities. Even when states were able to develop expertise in 

school reform and hire necessary staff, officials from Education told us 

that personnel turnover in many states made it difficult to retain such 

knowledge. For example, Rhode Island officials said they encountered 

difficulties filling vacated positions because many nearby states were also 

recruiting from the same small pool of qualified applicants. Several states 

increased their capacity through actions such as contracting with 

nationally recognized experts to help them run their grant competitions, 

establishing school turnaround offices, or hiring turnaround specialists 

that regularly handled an individualized caseload of SIG schools. For 

example, 18 states created new turnaround offices to help districts 

implement SIG, according to our survey.

In addition to these state level issues, many districts also struggled to 

develop the necessary staff capacity to implement successful school 

reforms. It was particularly difficult for schools to recruit and retain 

qualified staff members, according to many stakeholders, including 

officials from several states and districts we visited. They told us that SIG 

 

9

Under federal law, states are allowed to reserve not more than 5 percent of SIG grant 

funds for administrative costs. 20 U.S.C. § 6303(g)(8).

10Race to the Top awards education reform grants to select states through a competitive 

grant process. Funding for the program was first authorized through the American 

Recovery and Reinvestment Act of 2009.
 

Page 9 GAO-12-373 School Improvement Grants

schools were sometimes in rural areas or needed staff to have expertise 

that was in short supply, such as experience with reform or specialized 

academic subjects. Among the 12 states that Education monitored during 

SY 2010-2011, they found that 4 states did not ensure that turnaround 

schools met requirements to remove at least half of the schools’ staff and 

hire new staff for those positions based on staff effectiveness.11

Moreover, some districts did not have staff with expertise in using 

performance and evaluation data—such as data on student 

performance—to inform plans for reforming schools and ongoing 

instructional improvements. Education officials said that, in many cases, 

school district staff were able to collect data, but did not have experience 

linking data to needed interventions. In addition, our review of the needs 

assessments districts were required to develop when planning SIG 

interventions showed that some were more extensive than others. Also, in 

one district we visited, the new teacher evaluation process did not include 

state assessment data on student achievement as one of the evaluation 

criteria, as required by Education. Several states, districts, and 

researchers identified promising practices for recruiting and retaining staff 

or improving data usage, such as developing “grow-your-own” leadership 

programs, conducting priority hiring for SIG schools, or hiring data 

coaches to help teachers collect and analyze student data.

For 

example, Education found that one monitored district in Minnesota did not 

base hiring decisions on prospective teachers’ instructional effectiveness. 

In addition, Education found that three of the monitored states did not 

ensure appropriate replacement of the principal in turnaround or 

transformation schools.

Districts also varied in their commitment to use SIG funds to enact major 

reforms. According to our survey, 35 of 51 states awarded grants to all or 

most Tier I applicants who applied for grants starting in SY 2010-2011, 

but several officials from states we visited and research organizations 

reported that some districts receiving SIG grants were not prepared to 

make significant reforms. For example, officials in one large school district 

we visited told us they followed turnaround model requirements to rehire 

no more than 50 percent of teachers at a SIG school. However, the 

district officials said they relocated the released teachers to other SIG 

 

11Using locally adopted competencies to measure the effectiveness of staff, schools 

implementing the turnaround model must screen all existing staff and rehire no more than 

50 percent, and select new staff. 
 

Page 10 GAO-12-373 School Improvement Grants

schools in their district because those schools had almost all of the 

vacancies. Similarly, in two states we visited, district officials moved a 

school’s previous principal into another leadership position on site so that 

person could continue to work in the school even after a new principal 

was assigned. State and district officials also cited instances where 

districts chose their SIG model for reasons other than its likelihood of 

improving student success. For example, the superintendent in one 

district told us they chose the restart model because they considered it 

less restrictive than other models. Although many states responding to 

our survey told us that all or most of their transformation model schools 

were operating very differently after the first year of SIG, 33 states said 

that at least some of these schools choosing the transformation model 

were not.12

 

12For each model, our survey asked whether all, most, around half, few, or none of the 

schools in a state were operating very differently after the first year of SIG implementation. 

Thirty-three states told us that some (most, around half, or few) transformation schools 

were operating very differently after the first year, while 13 states told us that all 

transformation schools were operating very differently and 5 did not provide a response. 

For more information, about the extent to which schools were operating very differently 

after the first year of SIG and reasons cited by state officials, see our e-supplement 

Figure 2 shows responses from these 33 states about 

whether inadequate action by SIG schools or districts was a reason the 

schools were not operating very differently.

GAO12-370SP. 
 

Page 11 GAO-12-373 School Improvement Grants

Figure 2: State Responses on Whether Inadequate Action by SIG Schools or 

Districts Was a Reason Schools Were Not Operating Very Differently

SIG requirements to increase student learning time and develop new 

teacher evaluation systems were particularly challenging for some 

districts to implement fully, especially given the significant time pressure 

that we identified as a concern in our previous report on SIG.13

 

13

Indeed, 

Education found districts that were not appropriately implementing 

GAO-11-741. Both the turnaround model and the transformation model require school 

districts to provide increased learning time. Increased learning time is defined by 

Education’s guidance as significantly increasing the total number of school hours to 

include: (1) instruction in core academic subjects; (2) instruction in other subjects and 

enrichment activities that contribute to a well-rounded education; and (3) teachers to 

collaborate, plan, and engage in professional development. The transformation model also 

requires school districts to use “rigorous, transparent, and equitable evaluation systems 

for teachers and principals that take into account data on student growth as a significant 

factor.”

Key SIG Requirements Are 

Difficult to Implement, 

Especially Under Short 

Time Frames
 

Page 12 GAO-12-373 School Improvement Grants

increased learning time requirements in about half of the states that it 

monitored during SY 2010-2011 and in both states for which it had 

completed SY 2011-2012 monitoring reports by February 2012..

According 

to district officials, at least half the districts we spoke with will not have 

fully implemented new teacher evaluation systems by the end of their 

second year of SIG. In addition, during its SY 2011-2012 monitoring visit 

to Iowa, Education found that student growth was not always incorporated 

in new teacher evaluation systems, as required. Our analysis showed that 

increased learning time and teacher evaluation requirements were 

challenging because the planning needed to implement them was 

complex and time-consuming, and stakeholders, such as unions and 

parents, were sometimes reluctant to embrace the changes. Some 

districts struggled to develop increased learning time initiatives that would 

be sustainable after their 3-year SIG grant ended. More specifically:

• Interventions required extensive planning. Effectively implementing 

increased learning time and teacher evaluations required extensive 

planning. Several stakeholders stressed the importance of carefully 

designing increased learning time schedules because, for the 

intervention to be successful, it must provide quality instruction rather 

than simply increasing the amount of poor instruction. Officials from 

several districts said they were unable to fully implement their plans 

for increased learning time at the beginning of the first year of 

increased SIG funding because, for example, they first needed to fully 

analyze their existing schedules and curricula and adapt them to meet 

SIG requirements. Officials from states and districts we visited often 

stressed that developing a teacher evaluation system is timeconsuming because it requires districts to accurately and 

comprehensively identify, collect, and analyze information about 

teachers’ performance and students’ academic growth. In response to 

the challenges involved in planning and implementing teacher 

evaluation systems, Education allowed states to apply for a waiver to 

extend the planning period for this requirement, and 27 states applied 

for and received the waiver as of February 2012. In its final SIG 

requirements, Education required schools implementing the 

transformation model to implement new teacher evaluation systems 

within the first year of the grant. Districts in states receiving these 

waivers must develop their evaluation systems during SY 2011-2012;

pilot or fully implement them by SY 2012-2013; and use them to make 

decisions about retention, promotion, and compensation by SY 2013-

2014. The timeline is the same regardless of whether the SIG schools 

in the district are from the first or second SIG cohort.
 

Page 13 GAO-12-373 School Improvement Grants

• Stakeholders sometimes reluctant to embrace required changes. 

Implementation was also delayed or otherwise challenged by 

concerns from various stakeholder groups. Teachers and teachers’ 

unions were sometimes concerned about increasing student learning 

time or implementing new teacher evaluations in SIG schools, 

according to Education, state, and district officials. For example, these 

officials said unions were concerned about whether teacher 

evaluation systems that incorporated student academic growth could 

do so in a manner that would not penalize teachers working with the 

most challenging students. Such concerns sometimes led to delays in 

finalizing evaluation systems. In a few cases, officials told us that 

other stakeholder groups such as parents and school board members 

were also resistant to SIG requirements. For example, an official in 

Virginia said that some schools trying to increase learning time had 

met resistance from parents because students often had jobs or 

responsibilities at home once the traditional school day was over.

• Difficulty designing sustainable approaches for increasing learning 

time. State and district officials also questioned whether increased 

learning time initiatives would be sustainable after SIG funds were 

exhausted. For example, survey respondents from 26 states said the 

costs of increased learning time were unlikely or very unlikely to be 

sustainable after the SIG grant ends, compared with 10 states that 

reported it was likely or very likely to be sustainable.14

While many officials stressed the complexity of effectively implementing 

these requirements, some states and districts that we visited found ways 

to address the challenges they posed. This was particularly true in 

districts that had started to plan for and implement similar school reforms 

prior to applying for SIG funds. Many officials from Education, state, and 

districts stressed the importance of stakeholder involvement while 

Rhode Island 

officials noted that increased learning time benefits students enrolled 

in SIG schools during the grant cycle, but state and local financial 

constraints will make it difficult to sustain the increased learning time 

for future students. Due in part to these concerns, one of the two 

districts in the state with SIG schools limited the amount of learning 

time it added in order to avoid significant cuts in this time after grant 

funding ends.

 

14The remaining 15 respondents said that either it was as likely as unlikely that these 

costs would be sustainable; they did not know; the question was not applicable, or it was 

too early to tell.
 

Page 14 GAO-12-373 School Improvement Grants

designing and implementing SIG reforms in order to enhance buy-in and 

strengthen reform initiatives. In order to increase students’ learning time 

without increasing teacher workloads and salaries, Education officials and 

researchers told us that a few districts were working with community 

partners to fund or staff additional learning time or were staggering 

teachers’ schedules so that students would be in class longer but 

teachers would not. In addition, a few states developed sample teacher 

evaluation system that met SIG requirements so that districts could use it 

as a framework for developing their own systems.

States often had limited evidence for making decisions about whether to 

renew schools’ SIG funding. For example, in our survey, officials from 10 

states told us that they did not use schools’ achievement of annual goals 

to make grant renewal decisions after SY 2010-2011. According to state 

officials, at least half the states we interviewed did not have the annual 

student achievement data available at the time they had to make renewal 

decisions because assessment results only became available at the end 

of the summer. Officials from two of these states told us that timely 

access to annual achievement data will continue to be a problem in future 

years. In addition, even when these data were available, states frequently 

chose not to base their decisions on schools’ achievement of annual 

goals. Twenty-three of 44 states responding to our survey question said 

that, among schools that had their funding renewed, all or most did not 

meet their annual goals. Several officials from our site visits questioned 

the usefulness of annual goal data in determining whether progress was 

made, particularly because districts set their own performance targets. 

For example, California officials said they did not find annual goals data 

useful because districts often included generic annual goals in their 

applications for SIG funding instead of proposing goals based on schools’ 

unique circumstances.

Regardless of whether annual goals information was available, states 

almost always considered “fidelity of implementation”—the extent to 

which the school is implementing the requirements of its intervention 

model—when making grant renewal decisions. However, states did not 

always base decisions about this criterion on extensive information. In our 

survey, 48 of 51 states identified fidelity of implementation as an 

important factor in their decision-making process, more than any other 

factor. Several states we spoke with said that qualitative information 

about implementation was important for assessing grant progress 

because the first steps of school reform, such as efforts to change school 

culture, do not always result in measurable student achievement gains. 

Some States Had Limited 

Evidence about SIG Grant 

Performance When Making 

Grant Renewal Decisions
 

Page 15 GAO-12-373 School Improvement Grants

However, making this assessment can involve a high degree of 

subjectivity and states’ determinations were not always developed based 

on extensive interaction with schools or systematic monitoring of their 

implementation efforts. For example, officials in California told us they 

used fidelity of implementation as their key criterion for making grant 

renewal decisions, and that the primary method for evaluating this 

criterion was one telephone conversation with each district at the end of 

the year. Prior to those conversations, the state had limited interaction 

with most districts for the purpose of assessing their implementation and 

was unable to conduct SIG monitoring visits for budgetary reasons. In 

addition, a Virginia official told us the state used fidelity of implementation 

for making renewal decisions but would benefit from guidance on how to 

define and measure it.

States were in some cases reluctant to discontinue SIG funding even 

when information they collected showed that schools were not 

implementing key requirements with fidelity. Several officials from states 

we visited said they renewed all schools’ SIG funding even if the schools 

were struggling to fulfill key SIG requirements because tight 

implementation timeframes made the officials reluctant to eliminate 

funding after the first year of the grant. In our survey, 21 states reported 

that half or fewer of their Tier I and Tier II schools were able to implement 

major aspects of their plan by the beginning of SY 2010-2011, such as 

extending the school day or having new staff in place. In the 19 cases 

where these states had made renewal decisions, the state renewed all or 

most grants. Furthermore, officials in several states we visited identified 

instances where they chose to renew schools’ funding despite significant 

problems at the district or school level, such as having administrators who 

were not committed to enacting major reforms or were not ensuring that 

planned reforms were fully implemented. For example, officials from 

Nevada said they renewed such grants after the first year because they 

did not want to negatively impact students and teachers when significant 

district-level problems were outside their control.

Although Education reviewed states’ proposed grant renewal procedures 

through the state SIG application process, the agency did not provide 

written guidance after grant renewal challenges arose. Education required 

states to submit their renewal processes for review as part of their SIG 

applications. Nonetheless, in several state applications we reviewed, 

descriptions of renewal processes and criteria did not align with the 

practices the state actually implemented. For example, states that told us 

they were unable to use annual goals data to make renewal decisions 

had originally identified these goals as a key renewal criterion in their 
 

Page 16 GAO-12-373 School Improvement Grants

applications to Education. In its work with states, Education officials told 

us they found some had difficulty using annual goals data or fidelity of 

implementation and that the agency provided technical assistance to 

several states that asked for help. However, agency officials were not 

aware of how states ultimately addressed these issues, and said the 

agency has not provided any additional technical assistance on grant 

renewal.15

States renewed almost all SIG grants at the end of SY 2010-2011, and in 

some cases imposed conditions on schools for renewal. According to 

Education, 39 states chose to renew funding to every SIG school in their 

state. Eleven states and the District of Columbia chose not to renew 

funding to one or more schools, for a total of 16 nonrenewed schools 

overall.16 Of these 16 schools, about two thirds were not renewed due to 

problems with fidelity of implementation. Several states we spoke with 

chose to renew grants with conditions or required changes. For example, 

officials in Ohio told us that struggling schools were required to take 

corrective actions in the second year of the 3-year grant and that their 

level of success in taking such actions will be a key criterion in future 

renewal decisions. In addition, New York officials renewed all grants after 

SY 2010-2011 under the condition that transformation and restart schools 

would implement state and federal SIG teacher evaluation requirements 

by December 30, 2011. Once that deadline passed, state officials 

determined that no districts had met the requirements and suspended all 

SIG funding until they were met. In February 2012, the state 

commissioner reinstated funds to half of the SIG districts after 

determining that the districts had made the necessary changes. In our 

survey, 23 states reported that at least a few of their SIG schools were 

required to make major changes to their SIG plans as a condition of 

having funding renewed.

 

15Education officials told us they will review state renewal processes through monitoring 

starting in SY 2011-2012, but that their SY 2010-2011 monitoring took place prior to states 

making renewal decisions. 

16The Bureau of Indian Affairs also chose not to renew two schools. 
 

Page 17 GAO-12-373 School Improvement Grants

Contractors provide a wide range of services with SIG funds, and school 

districts have often given contractors major roles in schools using the 

restart, turnaround, and transformation models. Education’s guidance 

identifies a clear role for contractors in schools using the restart model. 

Specifically, districts must hire a contractor to take over school 

operations. For example, in the Los Angeles Unified School District, the 

Partnership for Los Angeles Schools has been given full management 

authority over five restart schools. In contrast, Education allows districts 

with schools using the turnaround and transformation models—which 

include more than 90 percent of schools receiving SIG funds—to use 

contractors, but does not identify a specific role for them. Most turnaround 

and transformation schools we visited were working with contractors. 

Although in some cases turnaround and transformation schools used 

these contractors for minor tasks, in other cases the contractors played a 

major role in school operations. For example, in Virginia, the state 

required schools implementing the turnaround and transformation 

intervention models to use a contractor for a range of services that could 

include improving teacher performance, principal and management 

leadership, or changing school culture.

Among the school districts we visited, several planned to spend 

significant amounts of their SIG grants on hiring contractors. These 

included districts using the restart, turnaround, and transformation 

models. For example, a district with one SIG school using the 

transformation model planned to spend about $450,000 for contractors in 

States and Districts 

Screen Contractors 

Before Selecting 

Them to Receive SIG 

Funding, but 

Inconsistent 

Approach to 

Reviewing 

Performance Reduces 

Contractor 

Accountability 

Contractors Provide a 

Wide Range of Services, 

and Many Receive 

Significant Amounts of SIG 

Funding 
 

Page 18 GAO-12-373 School Improvement Grants

one school year. In addition, a district that we visited with three SIG 

schools planned to spend approximately $1.5 million on contractors over 

the 3-year period for services that included data analysis and curriculum 

planning.

Our prior work and reports regarding services acquisition have shown the 

importance of building safeguards into acquisition processes to ensure 

accountability. These leading practices include screening potential 

contractors prior to award using a thorough selection process that 

evaluates their ability to achieve results and the contractors’ past 

performance.17 Once a contractor has been selected, officials should 

routinely review contractors’ work to help ensure they are providing timely 

and quality services and to help mitigate any contractor performance 

problems.18

Education required and states reported requiring that potential contractors 

be selected after a thorough screening process. Education required that 

either states or districts screen contractors prior to contract award to 

ensure their quality.19

In addition, states varied in how they approached contractor screening at 

the state level, either taking an active role in the process or delegating 

screening responsibilities to districts. According to our survey, 17 of the 

Although Education’s guidance does not provide 

specific criteria for approval, Education requires each state to describe in 

its state application how it will ensure that school districts screen 

contractors. Each of the eight states we reviewed required districts to 

describe their plans to screen contractors in their applications for SIG 

funding.

 

17GAO, Federal Contractors: Better Performance Information Needed to Support Agency 

Contract Award Decisions. GAO-09-374, (Washington, D.C.: Apr. 23, 2009). 

18For the purposes of this report, we define review during the course of contract 

performance as monitoring or quality assurance surveillance, as described in GAO-05-

274. 

19Education’s guidance requires that contractors implementing the restart model must be 

selected through a “rigorous review process,” which permits a district to examine a 

prospective contractor’s plans and strategies, and to ensure that the contractor has the 

capacity to implement its strategies. The guidance also requires districts to screen all 

other contractors for quality, stating that this screening serves a similar purpose as the 

rigorous review process used for restart contractors.

Education and States 

Required Contractors to 

Be Screened, but 

Screening Procedures 

Varied
 

Page 19 GAO-12-373 School Improvement Grants

51 developed approved lists of contractors from which districts could 

choose. For example, Virginia officials told us they enacted statewide 

contracts with four organizations, and strongly encouraged districts to 

choose one of those four organizations. Ohio officials said they developed 

a list of approximately 100 state-screened organizations from which 

districts could choose, but districts were free to use other contractors, 

provided that they screened those organizations. States that we visited 

that did not develop a list of approved contractors reported requiring 

districts to screen contractors. For example, Texas officials told us they 

required all districts to use a formal competitive process in selecting 

contractors, which included a process to evaluate contractor proposals, in 

order to be approved by the state.

Education’s monitoring protocols for the SIG program require the review 

of contractors in schools using the restart model, but they do not require 

review of contractors during contract performance for the other school 

improvement models. Education’s protocol for monitoring states’ SIG 

implementation asks whether districts have included accountability 

measures in the contract for restart schools and also asks for the district’s 

current assessment of the contractor. The protocol does not include a 

similar question for turnaround and transformation schools.20

States varied in their approaches to the review of contractors, and in 

some cases reported that they did not require that districts review 

contractors during contract performance. Among the eight states we 

spoke with, none assessed districts’ plans to review contractors in their 

SIG applications. In addition, several states reported not having any 

state-level review requirements. For example, Nebraska state officials 

said their districts conduct informal reviews of the contractors, but the 

state does not require reviews or provide districts with a formal process or 

metrics to assess performance. Similarly, in follow up calls for our state 

survey, state officials in several states said they do not require districts to 

review contractor performance and were unaware of whether districts 

 

20Education officials said they included this requirement for restart schools because their 

regulation requires districts to contract with external management organizations and that 

the contractors are selected through a rigorous review process. Education officials also 

said that they include review requirements for restart contractors because those 

contractors are an essential part of the restart model, and that they have more flexibility 

than the contractors used in a school implementing a different model. 

Education Does Not 

Require that All 

Contractors’ Performance 

Be Reviewed, and State 

Review Approaches Vary
 

Page 20 GAO-12-373 School Improvement Grants

conducted any reviews. In contrast, Nevada officials told us they require 

districts to add accountability steps for contractors in each phase of work.

Inconsistent review of contractors during contract performance reduces 

states’ and districts’ ability to ensure that they are receiving the services 

they have paid for. In our work, one stakeholder told us that in the 

absence of stronger guidance or oversight, the extent to which contracts 

include accountability measures is largely dependent on the knowledge 

and experience of the individual contract manager. Although some district 

officials in our site visits described efforts to include accountability 

measures or regular review in the contracts, others indicated that 

contractors are reviewed informally, if at all.
 

Page 21 GAO-12-373 School Improvement Grants

Education’s guidance and technical assistance on SIG implementation 

was well received by nearly all states. In our survey, nearly all states 

responded favorably about Education’s guidance and various technical 

assistance offerings for SY 2011-2012. Most states reported that 

Education’s guidance and technical assistance were helpful and many 

reported they were very helpful (see fig. 3). In our survey, we also 

inquired about the amount and timeliness of guidance provided by 

Education. Forty-one states reported that Education provided about the 

right amount of guidance for the second year of SIG. In addition, 33 

states responded that in SY 2011-2012, Education’s guidance was timely, 

allowing the state to meet its needs, while 14 states commented that the 

guidance was not timely.

Education’s 

Assistance and 

Oversight Are 

Supporting SIG 

Implementation 

Education’s Technical 

Assistance Was Well 

Received by States
 

Page 22 GAO-12-373 School Improvement Grants

Figure 3: Survey Results Regarding the Helpfulness of Education’s Technical 

Assistance Offerings

Note: The data for this graphic is from our survey, in which we asked states about the helpfulness of 

specific technical assistance offerings. In addition to the answers presented, we also included “do not 

know” as a potential response. We also asked states about the usefulness of Education’s 

communities of practice, the implementation initiative, and the on-site monitoring visit. For these three 

items, the number of states responding “do not know” was 23, 30, and 30, respectively. Because the 

“do not know” responses were numerous, we did not include them in our graphic.

Although most states told us that Education’s guidance was helpful, some 

identified additional technical assistance that would assist with SIG 

implementation. In an open-ended question on our survey that asked 

about the types of additional guidance that Education could provide, 15 

states indicated they wanted additional information about other states’ 

SIG implementation efforts that are working well. Several states that we 

met with also mentioned wanting more information on successful and 
 

Page 23 GAO-12-373 School Improvement Grants

sustainable implementation strategies, proven contractors, increased 

learning time strategies, and teacher/principal evaluation systems.

To provide additional support and enhance information sharing among 

the states, Education has recently begun three new assistance efforts. 

First, Education selected nine states to participate in the SIG 

“implementation support initiative” as an optional technical assistance 

resource.21 Under this initiative, each participating state receives a visit 

from an Education representative as well as officials from the eight other 

participating states. These site visits have two purposes—first, to provide 

technical assistance to the states, and second, to enable states to 

engage in peer-to-peer information sharing. Education reported that they 

have used information from these site visits to produce targeted technical 

assistance reports. Second, in December 2011, Education began 

conducting monthly check-in calls with state officials to better manage 

SIG implementation. Each state was assigned an Education program 

officer responsible for providing oversight and technical assistance 

support, including outreach and monthly check-in calls. Lastly, Education 

launched the School Turnaround Learning Community—an on-line forum 

to provide states and districts with access to resources and to facilitate 

networking. According to Education, this initiative offers research-based 

practices and practical examples from states, districts and schools for 

developing and implementing SIG.

Education’s oversight strategy is to monitor all states during the 3-year 

period22—starting with SY 2010-2011—in which the first cohort of schools 

will receive SIG funding. In selecting states for on-site monitoring for SY 

2010-2011, Education did not use a SIG-specific risk-based approach 

and instead used the existing Title I monitoring schedule.23

 

21According to Education officials, 17 states volunteered for this initiative, and the 9 states 

selected to participate were Alabama, Arizona, Arkansas, Connecticut, Louisiana, 

Michigan, Oklahoma, South Carolina, and Virginia. States that had received an Education 

SIG monitoring visit or were awarded a Race to the Top grant were not selected to 

participate. 

However, due 

to resource constraints, Education suspended its Title I monitoring and 

22Education officials also told us that nine of these states received technical assistance 

visits through the implementation initiative.

23Education monitors how states receiving Title I grants implement and administer the 

Title I program to determine whether states comply with program requirements.

Education is Now Using a 

Risk-Based Approach to 

Monitor All States within 3 

Years
 

Page 24 GAO-12-373 School Improvement Grants

instead focused exclusively on SIG monitoring. Education also delayed 

SY 2010-2011 monitoring to allow states and districts time to implement 

SIG before beginning monitoring in February 2011. For SY 2010-2011, 

Education conducted on-site monitoring in 12 states24 uncovering 28 

deficiencies. At least one deficiency was identified in 11 of the 12 

monitored states, with California and Pennsylvania having the most 

deficiencies with seven and five, respectively. Half of the monitored states 

had deficiencies in ensuring appropriate district implementation of the 

increased learning time requirement. In addition, two states did not 

ensure that all SIG funds were used consistent with the SIG 

requirements.25

In SY 2011-2012, Education selected states with a risk based approach 

tailored for SIG based on factors such as the size of a state’s SIG grant. 

For SY 2011-2012, Education officials initially selected 12 states to 

conduct on-site monitoring.26

 

24In SY 2010-2011, Education conducted monitoring site-visits to the following states: 

California, Indiana, Maine, Michigan, Minnesota, Mississippi, Montana, Nebraska, 

Nevada, Pennsylvania, South Dakota, and Tennessee. Education officials told us that they 

also conducted pilot monitoring site-visits in Maryland and New Mexico, but these visits 

were limited and used to test monitoring questions 

As of February 2012, Education had issued 

SY 2011-2012 monitoring reports for Iowa and Florida, containing seven 

and two deficiencies respectively. For example, in Iowa, Education found 

that funds were not used consistently with SIG grant requirements nor 

was the state monitoring SIG as written in its approved SIG application. 

Education also set aside a portion of its oversight resources so that 

additional states could be selected for monitoring as more information 

became available. As of February 2012, Colorado and South Carolina 

were also selected to receive an on-site review. To maximize its oversight 

resources, Education plans to conduct some limited “desk monitoring” in 

25Education required states to take corrective actions to address identified deficiencies. 

Education officials commented that states must submit evidence to prove that the 

identified deficiencies have been addressed. Education would then evaluate the strength 

of the evidence to determine if it adequately addressed the weakness. Education began 

issuing its monitoring reports in the spring of 2011 and as of March 2012, seven states—

Indiana, Maine, Michigan, Mississippi, Nevada, Pennsylvania, and South Dakota—had 

adequately addressed all of the weaknesses identified from their monitoring report. In 

addition, two states, Minnesota and Nebraska, have addressed some of the identified 

weaknesses, but still have other weaknesses to resolve. 

26For SY 2011-2012, Education initially planned to conduct monitoring site visits to the 

following 12 states: Florida, Georgia, Hawaii, Illinois, Iowa, Missouri, New Mexico, New 

York, Oregon, Rhode Island, Texas, and Wisconsin. 
 

Page 25 GAO-12-373 School Improvement Grants

five additional states in SY 2011-2012.27 According to Education, the desk 

monitoring protocol is similar to the on-site visit protocol, but—unlike the 

on-site monitoring—does not include interviews with school officials. 

Finally, Education officials told us that they plan to monitor the remaining 

states in SY 2012-2013, and that these states represent a small 

percentage of SIG funds.

Dramatic funding increases in a short period of time—such as those 

made to SIG—can subject federal programs to considerable financial risk. 

While states and school districts carry a large share of the responsibility 

for planning and implementing successful SIG reforms, Education also 

plays a critical role in supporting these efforts and mitigating risk through 

strong oversight and accountability. For example, it is important that 

Education have rigorous processes for reviewing state SIG applications, 

conducting oversight, and providing technical assistance when needed. 

The ability to successfully carry out these functions is vital to ensuring the 

long-term success of the SIG program and protecting taxpayer funds from 

waste and abuse. Although SIG has been challenging to implement, in 

part due to the short implementation timeframes we highlighted in our 

July 2011 report, Education has reviewed state SIG applications, 

distributed funds to states, begun its monitoring activities, and provided 

technical assistance. However, the agency’s guidance in some cases has 

not been sufficient to ensure that schools and contractors are fully 

accountable. For example, given the implementation issues we and 

Education’s monitoring have found, it is critical that states have rigorous 

SIG grant renewal procedures in place to identify schools that are not 

making progress. Education has provided limited guidance to states 

about how to make renewal decisions. Some states are using highly 

subjective review processes to renew nearly all grants, often without key 

information on SIG schools’ performance. Until Education provides 

additional support about how states should make evidence-based 

renewal decisions when, for example, state assessment results are 

received too late to be factored into these decisions, schools that are not 

making progress may continue receiving SIG funds. In addition, although 

contractors are receiving large amounts of many schools’ SIG funds, 

Education has not ensured that states or districts review contractor 

 

27As of February 2012, Education had identified Idaho and Wyoming as states that would 

receive desk monitoring. Education has completed the desk monitoring for Idaho. 

Conclusion 
 

Page 26 GAO-12-373 School Improvement Grants

performance during the terms of their contracts. Unless Education takes 

action to ensure that states or districts review contractor performance, 

districts may not receive an appropriate level of contractor services for 

their SIG funds and funds may not be well spent.

To ensure that SIG grant renewal decisions serve to hold districts and 

schools accountable, we recommend that the Secretary of Education 

provide additional support to states about how to make evidence-based 

grant renewal decisions, particularly when states do not have annual 

student achievement goal information available at the time renewal 

decisions are made.

To ensure that contractors hired with SIG funding are accountable for 

their performance, we recommend that the Secretary of Education take 

steps to ensure that the performance of SIG funded contractors, including 

those in turnaround and transformation schools, is reviewed during 

contract performance. In developing such requirements and to ensure 

that those reviews are targeted to contractors receiving large amounts of 

SIG funding, Education could consider setting a dollar threshold amount 

for contracts, above which contractor performance should be reviewed.

We provided a draft copy of this report to the Department of Education for 

review and comment. Education’s comments are reproduced in appendix 

II. Education generally supported our recommendation about SIG grant 

renewal and outlined how the agency is planning to address this 

recommendation.

Education did not agree with our draft recommendation that it should 

require states to ensure that the performance of all SIG-funded 

contractors be reviewed, including contractors in turnaround and 

transformation schools. In its comments, Education said that it believed 

that existing provisions and requirements address this issue 

appropriately. For example, Education cited a federal regulation that 

requires districts to follow their existing procurement procedures, and 

noted that districts and states have their own requirements for evaluating 

contractors to ensure accountability. Education also said that the type of 

evaluation process needed for a contractor should depend on the 

contractor’s role, and that contractors used by schools implementing the 

turnaround or transformation models may be working on small, discrete 

projects and may require less provider-specific reviews than contractors 

in schools implementing the restart model.

Recommendations for 

Executive Action

Agency Comments 

and Our Evaluation
 

Page 27 GAO-12-373 School Improvement Grants

We agree with Education that the need for performance reviews should 

be dependent on the specific role of the contractor, and we modified our 

recommendation to address some of Education’s concerns. Specifically, 

Education may wish to create a dollar threshold above which 

performance reviews are required. We continue to believe, however, that 

the current monitoring framework is inadequate. As noted in our report, 

schools implementing the turnaround and transformation models account 

for the overwhelming majority of SIG schools, and contractors operating 

in these schools are performing a range of functions, including some that 

are large or complex. In our view, there is a need for additional steps to 

ensure adequate review of contractor performance. Furthermore, our 

work shows that as a practical matter, states varied in their approaches to 

contractor review, with some imposing no requirements on districts. 

Education says that it will clarify in existing guidance the requirement for 

SIG recipients to follow state and local procurement procedures. 

Education could use this opportunity to implement our recommendation 

through additional guidance on contractor performance reviews.

In addition, Education implied that our report was based only on the first 

year of SIG implementation. This is inaccurate. We also conducted 

interviews with all eight states, reviewed SIG documents, received 

finalized survey responses, and interviewed Education officials several 

times during the second year of implementation, thereby enabling us to 

reflect activities beyond the first year. Based on the number and 

significance of deficiencies identified in Education’s SIG monitoring 

reports—including some completed during the SY 2011-2012—as well as 

our own findings, we continue to believe that Education should take 

additional steps to increase program accountability.

Education also provided technical comments that we have incorporated 

into the report as appropriate.
 

Page 28 GAO-12-373 School Improvement Grants

We are sending copies of this report to relevant congressional 

committees, the Secretary of Education, and other interested parties. In 

addition, this report will be available at no charge on GAO’s website at 

http://www.gao.gov. 

If you or your staff have any questions about this report, please contact 

me at (202) 512-7215 or scottg@gao.gov. Contact points for our Offices 

of Congressional Relations and Public Affairs may be found on the last 

page of this report. GAO staff who made key contributions to this report 

are listed in appendix III.

George A. Scott, Director

Education, Workforce, and Income Security Issues
Appendix I: Objectives, Scope, and 

Methodology

Page 29 GAO-12-373 School Improvement Grants

This study’s objectives were to answer the following questions: (1) What, 

if any, aspects of the School Improvement Grant (SIG) program pose 

challenges to successful implementation? (2) How do U.S. Department of 

Education (Education) and state guidance and procedures for screening 

potential contractors and reviewing contractor performance compare with 

leading practices? (3) To what extent are Education’s oversight and 

technical assistance activities effectively supporting SIG implementation? 

To meet these objectives, we used a variety of methods, including 

document reviews of Education and state documents; a web-based 

survey of the 50 states and the District of Columbia; interviews with 

Education officials and stakeholders; site visits to and teleconferences 

with 8 states; and a review of the relevant federal laws, regulations, and 

guidance. The survey we used was reviewed by Education and several 

external reviewers, and we incorporated their comments as appropriate.

We conducted this performance audit from January 2011 through April 

2012 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.

To identify aspects of the SIG program that pose challenges for 

successful SIG implementation, we analyzed responses to our survey of 

state educational agency officials with responsibility for SIG in the 50 

states and the District of Columbia. The wed-based survey was in the 

field from August to October 2011. In the survey, we asked states to 

provide information on challenges they faced in implementing the SIG 

program and on other aspects of the program, such as SIG grant 

renewal. We received responses from all 50 states and the District of 

Columbia, for a 100 percent response rate. We reviewed state responses 

and followed up by telephone and e-mail with select states for additional 

clarification and context.

Nonsampling error could affect data quality. Nonsampling error includes 

variations in how respondents interpret questions, respondents’ 

willingness to offer accurate responses, and data collection and 

processing errors. We included steps in developing the survey, and 

collecting and analyzing survey data to minimize such nonsampling error. 

In developing the web survey, we pretested draft versions of the 

instrument with state officials in various states to check the clarity of the 

Appendix I: Objectives, Scope, and 

Methodology 

Providing Information 

on Challenges to 

Successful SIG 

Implementation 
Appendix I: Objectives, Scope, and 

Methodology

Page 30 GAO-12-373 School Improvement Grants

questions and the flow and layout of the survey. Education officials also 

reviewed the draft survey and provided comments. On the basis of the 

pretests and reviews, we made minor revisions of the survey. Using a 

web-based survey also helped remove error in our data collection effort. 

By allowing state SIG directors to enter their responses directly into an 

electronic instrument, this method automatically created a record for each 

SIG director in a data file and eliminated the errors (and costs) associated 

with a manual data entry process. In addition, the program used to 

analyze the survey data was independently verified to ensure the 

accuracy of this work. Detailed survey results are available at 

GAO-12-370SP. 

We also conducted site visits to and teleconferences with eight states—

California, Delaware, Nebraska, Nevada, Ohio, Rhode Island, Texas, and 

Virginia—that reflect a range of population size, number of SIG schools, 

and use of the four SIG intervention models. In each state, we 

interviewed state officials, as well as district or school officials from one to 

three districts that had Tier I or Tier II SIG schools. Districts were selected 

in consultation with state officials to cover heavily and sparsely populated 

areas, and a variety of SIG intervention models. We also reviewed 

documents, such as state and district applications for SIG funds, and the 

relevant federal laws, regulations and guidance.1 We interviewed 

Education officials and stakeholders, such as teachers’ union officials 

from the national and local levels.

To gather information about state policies and procedures for selecting 

and overseeing contractors, we analyzed state survey results. Our survey 

questions included whether states had developed a list of approved 

contractors, the SIG turnaround models for which they required that 

districts work with contractors, and whether states reviewed contractor 

performance. We reviewed Education documents, including SIG 

guidance, the state application template, and monitoring protocols, and 

interviewed Education officials responsible for reviewing state 

applications and providing oversight of states. Further, we reviewed state 

and district SIG applications from the eight states to identify their 

selection and review processes for contractors, and proposed contract 

 

1

This includes Department of Education, Office of Inspector General, School Improvement 

Grants: Selected States Generally Awarded Funds Only to Eligible Schools, EDOIG/A05L002 (Washington D.C., March 2012).

Identifying How 

Education and State 

Guidance on 

Contractors Compare 

with Leading 

Practices 
Appendix I: Objectives, Scope, and 

Methodology

Page 31 GAO-12-373 School Improvement Grants

expenditures. We also spoke with state and local officials about their 

procedures for selecting and overseeing contractors, as well as with 

several contractors working with districts we visited. We compared 

Education and state requirements for selecting and overseeing SIG 

contractors to leading contracting practices that were identified through 

collaboration with our contracting experts and review of GAO-09-3742

GAO-05-274

and 

.

3

To address the extent of Education’s support and oversight of SIG 

implementation, we reviewed Education guidance, summaries of 

Education assistance, monitoring time frames, monitoring protocols, and 

monitoring reports from SY 2010-2011. In addition, we analyzed survey 

results. We asked states to provide information on the federal role in SIG, 

including their perspectives on technical assistance offered by Education 

and Education’s monitoring process. We also talked with officials from 

Comprehensive Centers and Regional Educational Laboratories serving 

several of the eight states we worked with. The technical assistance 

providers were selected to include those working with large, medium, and 

small rural states. In addition, we interviewed Education officials in charge 

of the Comprehensive Centers Program and in charge of SIG monitoring 

efforts.

 

2

GAO-09-374. 

3

GAO-05-274. 

Describing the Extent 

to Which Education 

Has Supported SIG 

Implementation 
Appendix II: Comments from the Department 

of Education

Page 32 GAO-12-373 School Improvement Grants

Appendix II: Comments from the Department 

of Education
Appendix II: Comments from the Department 

of Education

Page 33 GAO-12-373 School Improvement Grants
Appendix II: Comments from the Department 

of Education

Page 34 GAO-12-373 School Improvement Grants
Appendix II: Comments from the Department 

of Education

Page 35 GAO-12-373 School Improvement Grants
Appendix II: Comments from the Department 

of Education

Page 36 GAO-12-373 School Improvement Grants
Appendix II: Comments from the Department 

of Education

Page 37 GAO-12-373 School Improvement Grants
Appendix III: GAO Contact and Staff 

Acknowledgments

Page 38 GAO-12-373 School Improvement Grants

George Scott at (202) 512-7215 or scottg@gao.gov

In addition to the contact named above, the following staff members 

made important contributions to this report: Elizabeth Sirois, Assistant 

Director; Scott Spicer, Analyst-in-Charge; Jacques Arsenault; Melissa 

King; Salvatore Sorbello; and Barbara Steel-Lowney. In addition, Jean 

McSween, James Rebbe, Tom James, William Woods, and Kathleen Van 

Gelder provided guidance on the study.

Appendix III: GAO Contact and Staff 

Acknowledgments 

GAO Contact

Staff 

Acknowledgments 

(131117)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (www.gao.gov). Each weekday afternoon, 

GAO posts on its website newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations 

Public Affairs

Please Print on Recycled Paper.