GAO

United States General Accounting Office

Report to Congressional Requesters

June 2000 TITLE I PROGRAM

Stronger

Accountability Needed

for Performance of

Disadvantaged

Students

GAO/HEHS-00-89

Page 1 GAO/HEHS-00-89 Title I Schoolwide Programs

Contents

Letter 5

Executive Summary 6

Chapter 1

Introduction

14

Title I Program Targeted Primarily—but Not Exclusively—to

High-Poverty Schools 14

High-Poverty Schools May Choose the Schoolwide Option, Which

Focuses on All Students in the School 17

States Provide Program Accountability and Program Oversight 19

Education Provides States With Assistance and Oversight 21

Objectives, Scope, and Methodology 22

Chapter 2

Educators Believe

Schoolwide Approach

Can Provide Additional

Opportunities in

High-Poverty Schools

27

Schoolwide and Targeted Assistance Schools Generally Offered

Similar Services but Varied in Service Delivery Methods 27

Schoolwide Approach Viewed as Benefiting More Students 29

Chapter 3

Most States Not

Positioned to Hold

Schools and Districts

Accountable for

Outcomes of

Disadvantaged

Students

32

States Are Responsible for Ensuring Local Compliance; Education

Provides Oversight and Technical Assistance 32

Focus and Frequency of Program Oversight Efforts Varied

Across States 33

States Have Made Uneven Progress in Collecting and Reporting

Assessment Data Essential to Accountability 35

States’ Annual Progress Criteria Include Only Assessment Results

for Overall Student Population, Limiting Accountability for

Disadvantaged Students 44
Contents

Page 2 GAO/HEHS-00-89 Title I Schoolwide Programs

Chapter 4

Evaluations of Title I

and Schoolwide

Programs Have Been

Limited

46

Research Efforts Have Provided Limited Information on Program

Effectiveness 46

Data Limitations Present Challenges in Evaluating Program

Effectiveness 48

Chapter 5

Conclusions,

Recommendations,

Matter for

Congressional

Consideration, Agency

Comments, and Our

Evaluation

50

Conclusions 50

Recommendations 51

Matter for Congressional Consideration 51

Agency Comments and Our Evaluation 51

Appendixes Appendix I: Comments From the Department of Education 54

Appendix II: Major Contributors to This Report 57

Tables Table 1: Examples of Key Title I Requirements 20

Table 2: Characteristics of Schools Visited 24

Table 3: Principals Reporting Use of Extended-Time Learning

Opportunities 28

Table 4: Assessment Design Issues 37

Figures Figure 1: Distribution of Schools by Poverty Level and Title I Status 15

Figure 2: Example of District-School Title I Allocation Process 17

Figure 3: Frequency of States’ On-Site Monitoring 35

Figure 4: Other Outcome Measures Gathered by States 40

Figure 5: Number of States That Disaggregate Assessment and

Other Outcome Data, as of Fall 1999 41

Figure 6: Number of States Reporting Disaggregated Assessment

Data Through the Internet 42
Contents

Page 3 GAO/HEHS-00-89 Title I Schoolwide Programs

Abbreviations

CCD Common Core of Data

SERFF Study of Education Resources and Federal Funding
Page 4 GAO/HEHS-00-89 Title I Schoolwide Programs
United States General Accounting Office

Washington, D.C. 20548

Page 5 GAO/HEHS-00-89 Title I Schoolwide Programs

Health, Education, and

Human Services Division

B-282653 Letr

June 1, 2000

The Honorable James M. Jeffords

Chairman

The Honorable Edward M. Kennedy

Ranking Minority Member

Committee on Health, Education, Labor, and Pensions

United States Senate

The Honorable Christopher J. Dodd

Ranking Minority Member

Subcommittee on Children and Families

Committee on Health, Education, Labor, and Pensions

United States Senate

The Title I program was established in 1965 to help schools meet the needs of economically and

educationally disadvantaged students. Title I has traditionally directed its funds to those students who

are lowest-achieving or at highest risk for school failure. In 1994, the reauthorization of Title I

established new provisions encouraging the use of the schoolwide option and increasing

accountability for the educational outcomes of all children. You requested us to provide information

about Title I services at schoolwide and targeted assistance schools, state efforts to hold schools and

districts accountable for student achievement, and research and evaluations of Title I and schoolwide

programs.

This report was prepared under the direction of Harriet C. Ganson, Assistant Director, Education,

Workforce, and Income Security Issues, who may be reached at (202) 512-9045 if you or your staff

have any questions. Major contributors to this report are listed in app. II.

Marnie S. Shaul, Associate Director

Education, Workforce, and

Income Security Issues
Page 6 GAO/HEHS-00-89 Title I Schoolwide Programs

Executive Summary

Purpose Title I, the largest federal elementary and secondary education program,

was established in 1965 to help schools meet the needs of economically and

educationally disadvantaged students. Title I has traditionally directed its

funds—approximately $7.9 billion in fiscal year 1999—to those students

who are lowest-achieving or at highest risk for school failure (targeted

assistance). The 1994 reauthorization1 of Title I expanded the focus of the

program by increasing the number of schools eligible to use their Title I

funds to improve the school as a whole (a schoolwide program). In 1994,

the Congress also established new provisions aimed at creating greater

accountability for educational outcomes—not just for disadvantaged

children, but for all children. The addition of this broader focus to Title I—

particularly the growth in schoolwide programs—caused concern for some

educators and policymakers, who feared that some disadvantaged students

might lose services they would otherwise have received.

In light of the current discussions regarding reauthorization of Title I, the

Chairman and Ranking Minority Member of the Senate Committee on

Health, Education, Labor and Pensions and the Ranking Minority Member

of that Committee’s Subcommittee on Children and Families asked GAO to

examine how the changes to Title I, particularly the growth in schoolwide

programs, have affected Title I’s focus on disadvantaged children. GAO

reviewed (1) the services provided under Title I schoolwide and targeted

assistance programs, (2) states’ efforts to ensure compliance with key Title

I requirements and hold districts and schools accountable for educational

outcomes, and (3) research on and evaluation of Title I overall and

schoolwide programs in particular.

Background Title I allocations vary considerably across schools, depending on, among

other factors, the amount of money the district receives, the number of

children in poverty, and how the school district chooses—with certain

restrictions—to allocate its Title I dollars to individual schools. Although

high-poverty schools are more likely to receive Title I funding across the

nation, some high-poverty schools receive no Title I funds and many

schools with below-average poverty rates do receive Title I dollars.

According to 1999 Department of Education (Education) data, an

estimated 54 percent of the approximately 91,000 public elementary and

secondary schools nationwide received Title I funds, and about one-third of

1

The Improving America’s Schools Act of 1994 (P.L. 103-382).
Executive Summary

Page 7 GAO/HEHS-00-89 Title I Schoolwide Programs

the schools that received Title I funds have poverty rates near or below the

national average.

Prior to 1994, only schools with poverty rates of 75 percent or higher were

eligible to choose the schoolwide option, but the 1994 reauthorization

lowered the eligibility threshold to 50 percent. Currently, about half of Title

I schools are eligible to choose the schoolwide option, and a large

majority—an estimated 82 percent—of eligible schools have chosen this

approach. The remaining Title I schools—those that are not eligible or

choose not to adopt the schoolwide approach—are called targeted

assistance schools and are required to target their Title I dollars to lowachieving students within the school.

In addition to expanding the availability of the schoolwide option, the 1994

Title I reauthorization broadened the accountability provisions. States are

now expected to hold districts and schools accountable for educational

outcomes. States are to collect and publicly report assessment data,

develop criteria to determine whether schools and districts are performing

satisfactorily,2 and take actions to improve the performance of lowperforming schools and districts. Then, to ensure that all types of students

are making progress, states are to collect and report assessment results by

six specified student categories—gender, racial and ethnic group, English

proficiency status, disability status, migrant status, and economic status.

States had until the 1997–1998 school year to develop content and

performance standards. Education, as authorized by statute, extended the

deadline for performance standards for many states to coincide with the

deadline for assessments, which must be completely finalized by the 2000−

2001 school year.

Education has responsibility for general oversight of Title I. As part of its

oversight, Education conducts on-site reviews of each state’s program

every 3 to 4 years. This is done to assess whether states are adequately

monitoring how schools implement Title I requirements—those that are

financial and programmatic and those that are specifically related to

outcomes. In addition, Education provides technical assistance regarding

the interpretation of Title I—requirements and issues related to overall

2

According to the 1994 Title I legislation, states must develop criteria to ascertain whether

schools and/or districts are making adequate yearly progress. States may choose different

ways to define these criteria, such as using a fixed target score for all schools or requiring

schools to show improvement over time compared with previous results.
Executive Summary

Page 8 GAO/HEHS-00-89 Title I Schoolwide Programs

educational quality. This assistance takes a variety of forms such as

conferences, forums, and ongoing staff contacts. Finally, Education

conducts research on the effects of services provided under Title I.

Results in Brief Both schoolwide and targeted assistance schools generally offered similar

services, such as tutoring, and targeted additional services to students

needing extra help. However, schoolwide schools were generally more

likely than targeted assistance schools to provide services such as

extended day programs and often chose different methods of service

delivery, such as moving students in and out of flexible groups as their

achievement levels changed. Educators at the high-poverty schools GAO

visited generally preferred the schoolwide approach because they believed

that it allowed them to serve more students, facilitated faculty

collaboration, and allowed them to deliver services more efficiently and

effectively. However, some principals and teachers cautioned that schools

adopting the schoolwide approach need to be careful that low-achieving

students still receive the extra help they may need to improve their

academic performance.

Many states have yet to take all the steps necessary to oversee program

operations and hold districts and schools accountable for results. States

varied considerably in the frequency and focus of their efforts to monitor

compliance with Title I requirements and to oversee program quality. In

addition, some states had collected extensive and detailed information on

educational outcomes, but most states had substantially less information

on educational outcomes and on disadvantaged students in general. The

majority of states had established criteria to determine whether schools

and/or districts were performing satisfactorily. However, these criteria

were sometimes confusing or vague and (with only one state as an

exception) were based solely on the performance of the student population

as a whole, without reference to the achievement of specific subgroups of

children, such as students from low-income families or students with

limited English proficiency. Consequently, states are not yet in a position to

ensure accountability for the educational outcomes of disadvantaged

students, the children that remain central to the mission of the Title I

program.

Limited data and methodological problems have made it difficult to draw

firm conclusions about whether Title I in general—and schoolwide

programs in particular—are effective in improving educational outcomes.

Because schools and districts have considerable discretion in spending
Executive Summary

Page 9 GAO/HEHS-00-89 Title I Schoolwide Programs

their Title I dollars and are not required to report the specific services

provided, it has been difficult for researchers to isolate the effect of

specific services. Education plans to expand its existing data collection to

include information specific to Title I and schoolwide programs. These

changes, combined with other actions to improve the completeness and

quality of existing data, could facilitate research improvements.

Principal Findings

Educators Believe

Schoolwide Approach Can

Provide Additional

Opportunities in HighPoverty Schools

Title I schoolwide and targeted assistance schools generally offered similar

educational services. In the schools GAO visited, both schoolwide and

targeted assistance schools directed specific services (such as tutoring) at

students they had identified as needing additional assistance. However,

according to recent surveys conducted by Education, schools using the

schoolwide approach were more likely than targeted assistance schools to

offer programs with extended instructional time, such as after school,

weekend, or summer school programs.

Principals and teachers at the schools GAO visited said that the targeted

assistance approach could lead schools to choose convenient, but less

preferable, service delivery methods compared with the schoolwide

approach. One Education survey found that 80 percent of targeted

assistance elementary schools used pull-out programs, which take students

out of their regular classrooms to provide additional tutoring or assistance;

only 53 percent of the schoolwide schools used this approach. The pull-out

option provides a convenient method for tracking students who received

services and demonstrating that funds were used for students formally

identified as eligible under Title I. While the schoolwide schools GAO

visited sometimes used the pull-out strategy, they often preferred other

approaches such as moving students in and out of flexible groups as their

achievement levels changed.

Many district and school officials also said that the schoolwide approach

allowed them to serve more students, fostered increased collaboration, and

offered greater flexibility. In the high-poverty schools GAO visited,

principals and teachers said that because most or all of their students

suffered from educational disadvantages, the schoolwide approach

allowed them to address learning deficiencies for a greater number of

students, as well as to prevent such deficiencies. However, some school
Executive Summary

Page 10 GAO/HEHS-00-89 Title I Schoolwide Programs

and district officials cautioned that schools adopting the schoolwide

approach need to be careful that low-achieving students receive the extra

help they need to improve their academic performance.

District and school staff said that they believed the schoolwide approach

helped principals and teachers develop unified goals, engage in shared

decision-making, and assume responsibility for school improvement. State,

district, and school officials also stated that the schoolwide approach

allowed schools to use Title I funds without the need to direct these funds

to specific students. Consequently, schools could use their federal dollars

to employ a wider variety of instructional strategies. While those schools

that have chosen the schoolwide option see a number of advantages in

doing so, evidence regarding the overall effectiveness of the schoolwide

approach is inconclusive, as discussed later.

Most States are Not

Positioned to Hold Schools

and Districts Accountable

for Educational Outcomes

of Disadvantaged Students

State efforts to monitor compliance and oversee program quality varied

dramatically. Forty-seven states reported that they include on-site visits as

part of their monitoring procedures, and many of these states reported that

they rely solely on on-site visits to monitor certain key program

requirements. The average time between visits ranged from 2 years or less

(for 6 states) to more than 7 years (for 17 states). Three states reported that

they made no on-site visits, while three states reported that they visited all

their school districts each year.

Title I gives states considerable flexibility in deciding how to develop and

implement assessments of students’ performance. Consequently, there are

major differences in states’ assessments, including the format, content, and

difficulty of the assessments, as well as whether all students participated in

state-sponsored assessments. Moreover, while some states provided

extensive, detailed, and timely assessment data that school and district

officials found useful in improving instruction, other states collected or

reported less data on student achievement. Only about one-third of the

states have collected disaggregated data by all six required categories.

States varied in how they set criteria to judge whether schools and districts

have been performing satisfactorily. While some states had developed clear

and specific criteria for assessing adequate yearly progress for schools and

districts, other states had outlined only vague objectives. The law does not

require states to include criteria for adequate yearly progress that are based

on disaggregated data, and only one state has chosen to establish criteria

that include the performance of disadvantaged students. Without these
Executive Summary

Page 11 GAO/HEHS-00-89 Title I Schoolwide Programs

data, it will be more difficult for states to hold districts and schools

accountable for the achievement of disadvantaged students, and to discern

whether achievement gaps between disadvantaged and nondisadvantaged

students are closing.

Evaluations of Title I and

Schoolwide Programs Have

Been Limited

Research efforts to determine the effect of Title I on student achievement

have provided only limited information. Recent evaluations suggest that

Title I may have had a limited positive effect on overall student

achievement in high-poverty schools; less information is available

specifically on the effect of schoolwide programs. Education plans to

expand its data collection efforts in ways that may facilitate future research

on program effectiveness for both Title I overall and schoolwide programs.

These data could later be combined with other information about student

achievement or school characteristics to allow for more comprehensive

analyses of the effectiveness of Title I and schoolwide programs. However,

while linking key databases may facilitate more comprehensive analyses,

additional work is needed to improve data quality and consistency,

especially for key variables, such as school poverty rate.

Recommendations and

Matter for

Congressional

Consideration

GAO makes two recommendations to the Secretary of Education. First,

Education should conduct additional activities to facilitate the exchange of

information and best practices among states so they can identify ways to

improve the timeliness and specificity of their assessment data, the

collection and reporting of disaggregated assessment data, and the clarity

of their criteria for adequate yearly progress. Second, Education should

implement additional measures to improve research on the effectiveness of

specific services in both schoolwide programs and targeted assistance

schools. Such measures could include expanding and improving current

data collection efforts so that comprehensive analyses could be conducted

linking program characteristics to services and student outcomes and/or

developing an evaluation design for a study or set of studies of educational

services that would include national representation of both schoolwide and

targeted assistance schools.

To hold schools and districts accountable for improving the performance of

disadvantaged students and to help educators, parents, and others discern

whether achievement gaps between disadvantaged and nondisadvantaged

students are closing, the Congress should consider requiring that states’

criteria for progress, as expressed in their definitions of adequate yearly
Executive Summary

Page 12 GAO/HEHS-00-89 Title I Schoolwide Programs

progress, apply specifically to disadvantaged children as well as to the

overall student population.

Agency Comments GAO obtained comments on a draft of this report from the Department of

Education. Education agreed with most of the report’s findings and

conclusions but did not cite specific actions it would take in response to

our recommendations.
Page 13 GAO/HEHS-00-89 Title I Schoolwide Programs
Page 14 GAO/HEHS-00-89 Title I Schoolwide Programs

Chapter 1

Introduction Chapte1r

Title I of the Elementary and Secondary Education Act (Title I), the largest

federal program for elementary and secondary education, targets its $7.9

billion budget primarily to schools with a relatively high percentage of

students from low-income families. Nearly half of Title I schools operate on

a schoolwide basis—that is, the school can use its Title I dollars in

combination with other funds to improve the school as a whole. The

remaining schools that receive Title I funds are expected to target these

funds to lower-achieving students within the school. In 1994, the Congress

established new requirements whereby states (in addition to their general

oversight responsibilities) were to collect and report information on

educational outcomes and hold schools and districts accountable for

results.

Title I Program

Targeted Primarily—

but Not Exclusively—

to High-Poverty

Schools

Title I was established in 1965 to help schools—especially high-poverty

schools—meet the needs of disadvantaged students. Children from lowincome families often face obstacles that can reduce their chances for

success in learning. For example, children from low-income families are

more likely to be exposed to drug abuse, violence, and unhealthy living

conditions. In addition, parents in economically disadvantaged families

sometimes have limited education and involvement in their children’s

learning. However, the effects of poverty on student achievement are not

confined to students who happen to be poor themselves. Research shows

that students in schools with high poverty rates are more likely to be low

performers, independent of their own family background. These effects

may be related to lower expectations on the part of teachers in highpoverty schools, lack of highly qualified teachers or sufficient resources in

these schools, or other factors.

In recognition of these special challenges, the Title I program targets

primarily high-poverty schools. Although high-poverty schools are more

likely to receive Title I funding, some high-poverty schools receive no Title I

funds and many schools with below-average poverty rates do receive Title I

dollars. According to 1997–98 Education data, an estimated 54 percent of

all schools have poverty rates near or below the national average,

compared to about one-third of Title I schools (see fig. 1).
Chapter 1

Introduction

Page 15 GAO/HEHS-00-89 Title I Schoolwide Programs

Figure 1: Distribution of Schools by Poverty Level and Title I Status

Note: Under Title I, school districts have some discretion in selecting the measure used for schools’

poverty rates. The most commonly used measure has been students’ eligibility for free and reducedprice lunches.

Source: Jay Chambers, Joanne Lieberman, Tom Parrish, and others, Study of Educational Resources

and Federal Funding (Washington, D.C.: U.S. Department of Education, Planning and Evaluation

Service, 1999).

0

10

20

30

40

50

60

Title I Schools

All Schools

Less Than 35% 35-49% 50-74% 75% and Higher

54

33

13

15

17

25

16

27

School Poverty Rate

Percentage
Chapter 1

Introduction

Page 16 GAO/HEHS-00-89 Title I Schoolwide Programs

Title I dollars are distributed to local school districts through state

education agencies. The amount each school district receives is

determined by a complex formula that incorporates, among other factors,

the average per-pupil expenditure in the state, the number of children in

poverty, and previous allocations to the state and the district. Once Title I

dollars reach the local school district, district officials distribute the funds

to the schools. A school is Title I eligible if its school attendance area has a

poverty rate that is at least equal to the district average rate or 35 percent

(whichever is less). District officials are required to follow certain rules in

allocating Title I funds to schools based on the number of low-income

students. First, districts must provide Title I funding to all schools with a

poverty rate of 75 percent or higher before providing Title I funding to any

school with a poverty rate below 75 percent. In addition, district officials

must provide funding to schools with the same grade span in rank order of

poverty rate.1 Even within these requirements, however, district officials

have some discretion as to which schools get Title I dollars and how much

each school receives. Districts may choose to concentrate their Title I

funds on their highest-poverty schools and limit school eligibility to a

poverty level that is higher than the districtwide average. Districts may give

schools different amounts per poor child so long as schools with higher

poverty rates receive higher allocations per poor child than schools with

lower poverty rates.2 Fig. 2 provides an example of how this allocation

process works.

1

If a district chooses to fund any school with a poverty rate below 35 percent, each school

selected for Title I funding must receive a minimum amount per low-income student under

the “125 percent rule.” This minimum amount is equal to the total district allocation under

Title I divided by the total number of low-income students in the entire district, multiplied

by 125 percent.

2

District and school officials may have additional flexibility in determining allocations for

two reasons: (1) they can choose from different measures of poverty in determining school

poverty rates; and (2) they may be able to affect the measured poverty rate in specific

schools. Under Title I rules, as long as the same measure of poverty is used across schools,

districts may use poverty rates derived from a number of data sources, including children

eligible for free or reduced-price lunches (the most common measure), children eligible to

receive medical assistance under the Medicaid program, or other measures. Some districts

and schools can also take steps to increase the accuracy of the measured poverty rate. In

one district we visited, the district sponsored billboards in low-income neighborhoods

urging parents to sign their children up for free or reduced-price lunches. Similarly, at a

school in another district, the principal told us that she made special efforts to get students

to return their applications for the lunch program so that her school would exceed the 75

percent threshold and receive Title I funds.
Chapter 1

Introduction

Page 17 GAO/HEHS-00-89 Title I Schoolwide Programs

Figure 2: Example of District-School Title I Allocation Process

a

Dollar amounts shown are allocations per low-income child.

High-Poverty Schools

May Choose the

Schoolwide Option,

Which Focuses on All

Students in the School

Once funding has been distributed to the school level, Title I has

traditionally expected schools to direct the funds to students who are lowachieving or at highest risk for school failure. However, a provision known

as the schoolwide program allows a school to spend its Title I funds to

improve the school as a whole, rather than targeting Title I funds to lowachieving students. The 1994 reauthorization of Title I increased the

number of schools eligible to use Title I funds on a schoolwide basis.

Before 1994, only schools with poverty rates above 75 percent were eligible

to choose the schoolwide option, but the 1994 reauthorization lowered the
Chapter 1

Introduction

Page 18 GAO/HEHS-00-89 Title I Schoolwide Programs

eligibility threshold to 50 percent. In addition, under a schoolwide program,

the school is encouraged to take an integrated approach, combining federal

resources with other funds to implement a school plan, instead of viewing

each program in isolation.

The 1994 reauthorization encouraged the use of the schoolwide option and

provided for waivers so that some otherwise ineligible schools could also

adopt a schoolwide program. Consequently, the number of schools

operating schoolwide programs has increased dramatically—from 3,903 in

the 1993–94 school year to 19,701 in 1999. As of 1999, about half of Title I

schools were eligible to choose the schoolwide option, and a large

majority—an estimated 82 percent—of eligible schools have chosen the

schoolwide approach. The remaining Title I schools—those that are not

eligible or choose not to adopt the schoolwide approach—are called

targeted assistance schools. These schools are required to target their Title

I dollars to low-achieving students within the school.

Schoolwide schools had many of the same characteristics as high-poverty

schools in general, because eligibility for schoolwide status is determined

by poverty rate. In general, schools with poverty rates of 50 percent or

greater are more likely to be elementary schools, to be located in urban

areas, and to have a higher percentage of nonwhite students. Schools that

adopted the schoolwide approach shared these characteristics.

States varied a great deal in the proportion of schools choosing the

schoolwide option. Some states had a greater proportion of schoolwide

programs than would be expected on the basis of the number of highpoverty schools in that state, while other states had a smaller proportion of

schoolwide programs. For example, while 18 percent of New Jersey

schools had poverty rates of 50 percent or greater, only 4 percent of all

NewJersey schools (and 11 percent of Title I schools in New Jersey)

operated schoolwide programs. In contrast, 43 percent of schools in South

Carolina had poverty rates of 50 percent or greater, and many of these

schools operated schoolwide programs—39 percent of schools in the state

and 90 percent of Title I schools.3

Factors such as funding allocations, state and district support for

schoolwide programs, and local circumstances may have contributed to

3

School poverty rates were taken from Education’s Common Core of Data and schoolwide

data were obtained from GAO’s 50-state survey.
Chapter 1

Introduction

Page 19 GAO/HEHS-00-89 Title I Schoolwide Programs

the variation across states in the proportion of schools choosing

schoolwide programs. For example, the proportion of all public schools in

a state that received Title I funds varied considerably across states—from

20 percent in Nevada to 77 percent in Mississippi. A state where Title I

funds were allocated to a greater proportion of schools, particularly

schools with lower poverty rates, would likely have fewer schoolwide

programs and more targeted assistance schools compared with a state

where Title I dollars were concentrated in fewer schools. In addition,

adoption of schoolwide programs may have varied because of state and

district policies encouraging the schoolwide approach. According to school

and district officials, other local or regional circumstances may also have

played a role in the decision to choose a schoolwide program.

States Provide

Program

Accountability and

Program Oversight

The 1994 legislation gave the states additional responsibilities designed to

increase Title I’s focus on educational outcomes for all students. Under the

legislation, states were to take the major responsibility for holding schools

and districts accountable for student achievement. Specifically, states were

required to develop content standards to describe what students need to

know and performance standards to describe expected proficiency in at

least the core subject areas of reading and mathematics. States were then

to establish assessments to measure how students are doing in relation to

the content and performance standards.

Each state must report the results of assessments, as well as its other

outcome data, to schools, districts, and the public. In addition to the overall

results, states must collect and publicly report assessment results by six

specified student categories: gender, racial and ethnic group, English

proficiency status, disability status, migrant status, and economic status.

States may also select other outcome measures (such as graduation rates)

to capture important facets of school performance that cannot be

measured through assessments. States must also develop criteria for

determining whether schools and districts are performing satisfactorily.

These criteria, collectively called adequate yearly progress, are designed to

help states identify low-performing schools and districts. States may

provide additional assistance, or invoke penalties or sanctions, to help and

motivate low-performing schools to improve.

The deadline for implementing content and performance standards was the

1997–98 school year. Education, as authorized by statute, extended the

deadline for performance standards for many states to coincide with the

deadline for assessments, which must be completed by the 2000–2001
Chapter 1

Introduction

Page 20 GAO/HEHS-00-89 Title I Schoolwide Programs

school year. During the transition period, while states are developing

standards, assessments, and adequate yearly progress benchmarks, they

can use an interim process for assessing schools and districts.

The new requirements for assessment, data collection and reporting, and

accountability augment rather than replace Title I’s long-standing financial

and programmatic requirements. Under Title I, states have a significant role

in ensuring that schools and districts comply with all these requirements.

Title I also contains a number of requirements that address a variety of

important program objectives, many of them directly related to protecting

the interests of disadvantaged children. For example, requirements are

directed not only at ensuring that Title I funds are targeted to high-poverty

schools in accordance with the law, but also at promoting parental

involvement in education—a factor that educators agree is particularly

important to the success of poor and low-achieving students. Examples of

some of Title I’s key requirements are given in table 1.

Table 1: Examples of Key Title I Requirements

The Congress and Education anticipated that the new accountability

requirements, combined with Title I’s financial and programmatic

Objective Requirement

Getting parents more involved in their

children’s education

Districts that get more than $500,000 in Title I funds must spend 1 percent of their Title I

allocations for parental involvement activities.

Title I schools are required to have a school-parent compact, which describes the school’s

responsibility and the ways in which parents will be responsible for supporting their children’s

learning, such as monitoring attendance, homework completion, and television watching.

Ensuring quality schoolwide planning

that addresses the needs of

disadvantaged students

Title I schools that choose a schoolwide program must produce a schoolwide plan that includes

eight components required by law. Among these components are strategies and activities to

address the specific needs of disadvantaged children.

Title I schools are expected to implement their schoolwide plans, using any approach they

choose, as long as the plan contains the required components.

Assuring continuation of state and

local funding levels

Districts must meet maintenance-of-effort requirements—that is, aggregate state and local

education expenditures for the preceding year generally may not be less than 90 percent of the

expenditures for the year before.

Schools are expected to use federal funds to supplement, not supplant, funds that would be

available in the absence of federal funds for the education of students in Title I schools.

State and local funds must be used to provide services in Title I schools that are comparable to

services those funds are providing in other schools.

Ensuring that Title I funds are directed

to high-poverty schools and

disadvantaged students in accordance

with the law

Districts must allocate federal funds in accordance with Title I regulations that target highpoverty schools.

Targeted assistance schools must direct Title I funds primarily to services for students identified

as lowest achieving or at risk of school failure.
Chapter 1

Introduction

Page 21 GAO/HEHS-00-89 Title I Schoolwide Programs

requirements, would help all schools improve student achievement. Title I’s

programmatic and financial requirements would help ensure that important

building blocks—such as strong parental involvement and additional

funding for high-poverty schools—were in place. The availability of

assessment and other outcome data at the local level would provide useful

information to help schools identify and work on areas of weakness. The

public distribution of these outcome data would provide a powerful

incentive for improvement, and using these data to identify low-performing

schools and districts would create added accountability. The success of

this process, however, depends on states’ ability to conduct careful

oversight; collect and report detailed, specific, and valid outcome data; and

implement clear progress criteria for schools and districts. In addition,

states must collect and use the disaggregated assessment data—in

combination with their oversight of key program requirements—to protect

the interests not only of the general student population, but also of the

disadvantaged students targeted under the Title I program.

Education Provides

States With Assistance

and Oversight

Although the states have the major responsibility for ensuring that local

schools and districts comply with key Title I financial and programmatic

requirements, Education oversees how states implement Title I

requirements related to outcomes, and whether states sufficiently monitor

how schools and districts implement Title I’s financial and programmatic

requirements. Education is also responsible for providing technical

assistance to the states, not only on the interpretation of Title I

requirements, but also on other broader issues related to overall

educational quality.

As a major component of its oversight process, Education conducts on-site

reviews for each state.4 Site visits are generally conducted on a rotating

cycle every 3 to 4 years, with review teams spending about 5 business days

in each state they visit.5 Education requires states to submit a selfassessment prior to the on-site visit to help focus the review. In addition to

meeting with state education department officials, review teams generally

visit a few local districts and schools.

4

Education’s site visits focus on how the states are fulfilling their responsibilities. States also

make site visits to districts and schools to monitor compliance with key Title I requirements.

5

The four states receiving the most money are reviewed more frequently.
Chapter 1

Introduction

Page 22 GAO/HEHS-00-89 Title I Schoolwide Programs

In keeping with Education’s technical assistance role, Education staff told

us that they review state programs from a problem-solving rather than

strictly a compliance perspective. The agency has also shifted from

conducting separate reviews that focused on only a single federal program

to broader, integrated reviews—including Title I reviews—looking at

themes that cut across several federal programs. For example, themes have

included parental involvement and professional development, which are

important components of several federal programs. Although the theme is

not the sole focus of the monitoring visit, Education staff said that areas

related to the theme may be emphasized more. Education officials told us

that the process for conducting on-site reviews is evolving and they

expected to make additional changes in the future. While Education staff

expressed support for the broader, cross-cutting review process, several

individuals were concerned that this broader process may result in Title I

compliance issues being monitored less thoroughly. Education officials

said that they plan to supplement the integrated review process with

program-specific reviews to address this concern.

In its leadership role, Education can encourage states to comply with Title I

requirements. Education sponsors conferences and forums to allow states

to share information, provides written guidance to states, and discusses

issues with state officials during on-site monitoring reviews, as well as

other means of providing technical assistance for program improvement.

Education, for example, sponsored three regional meetings on assessment

systems and has provided the states with some written guidance on

progress criteria. Education is currently providing technical assistance to

states in developing their progress criteria and final assessment systems.

Objectives, Scope, and

Methodology

The Chairman and the Ranking Member of the Senate Committee on

Health, Education, Labor and Pensions and the Ranking Minority Member

of that Committee’s Subcommittee on Children and Families asked us to

examine the implications of the 1994 changes in the Title I program,

particularly the growth of schoolwide programs. Specifically, the objectives

of this study were to describe

• the services offered under the schoolwide and targeted assistance

approaches and the extent to which schoolwide and targeted assistance

schools provide special assistance to low-achieving or disadvantaged

children within the school;

• how states and the federal government exercise general oversight for

the Title I program; and how they measure, report, and create
Chapter 1

Introduction

Page 23 GAO/HEHS-00-89 Title I Schoolwide Programs

accountability for educational outcomes, both for the student

population as a whole and for low-achieving and disadvantaged students

in particular; and

• the evaluations that have been conducted to examine whether Title I

and schoolwide programs are contributing to improved student

achievement.

To address these objectives, we obtained and analyzed information from a

variety of sources at the federal, state, district, and school levels. To obtain

data on characteristics of and services provided by schoolwide and

targeted assistance schools, we reviewed and analyzed three national

databases collected by Education:

• the Common Core of Data (CCD), which contains descriptive

information on the approximately 91,000 public elementary and

secondary schools in the United States, as reported by the 50 states;

• the Study of Education Resources and Federal Funding (SERFF), which

contains information on school characteristics and services from a

nationally representative sample of over 700 schools and 180 school

districts in the 1998 school year; and

• the School-Level Implementation of Standards-Based Reform: Findings

From the Follow-up Public School Survey on Education Reform,

administered in 1998 to a nationally representative sample of about

1,600 school principals.

To obtain information on services provided under both targeted assistance

and schoolwide programs and on the implementation of accountability

requirements, we conducted on-site visits and interviewed state officials,

district staff, principals, and teachers in five states: North Carolina,

Pennsylvania, Rhode Island, Tennessee, and Texas. We chose these states

to illustrate a variety of characteristics, including the number and

percentage of schoolwide schools, the status of state assessments, the

growth in the school-age population in the past 5 years, the state share of

education expenditures, and Title I allocations per student. We visited 3

districts each in North Carolina, Pennsylvania, Tennessee, and Texas, and 1

in Rhode Island, for a total of 13 districts. In each district we interviewed

the Title I director and other officials as appropriate, including

superintendents, school support staff, and district staff involved in

assessment. We selected these districts on the basis of several criteria,

including their size and location. The districts we visited ranged in size

from just under 1,000 to nearly 70,000 students and from 49 to 89 percent of

students in poverty.
Chapter 1

Introduction

Page 24 GAO/HEHS-00-89 Title I Schoolwide Programs

Similarly, we visited between 1 and 3 Title I schools within each of these

districts, for a total of 21 schools (15 elementary schools, 5 middle schools,

and 1 high school). At each school we interviewed the principal and toured

the facility; at the majority of schools, we also spoke with teachers and

other school staff. We selected these schools on the basis of their poverty

rates, grade levels, size, status as schoolwide or targeted assistance,

diversity of student population, and other characteristics (see table 2).

Table 2: Characteristics of Schools Visited

School State

Grade

span

Number of

students

Student racial

breakdowna

(%)

Poverty rate

(percent) Schoolwide

Targeted

assistance

1 N.C. K-5 318 90 black

7 white

3 Hispanic

95 X

2 N.C. K-5 903 41 white

38 black

11 Hispanic

10 Asian

48 X

3 N.C. K-5 199 95 black

5 white

79 X

4 N.C. PK-5 247 59 black

39 white

69 X

5 Penn. K-5 663 47 black

31 white

20 Hispanic

81 X

6 Penn. K-5 738 39 black

38 white

23 Hispanic

74 X

7 Penn. K-6 180 100 white 69 X

8 Penn. 6-8 663 47 black

47 white

5 Asian

47 X

9 Penn. K-5 355 99 black 77 X

10 R.I. K-5 447 35 Hispanic

26 black

26 white

13 Asian

85 X

11 R.I. 6-8 891 55 Hispanic

25 black

13 white

7 Asian

85 X
Chapter 1

Introduction

Page 25 GAO/HEHS-00-89 Title I Schoolwide Programs

a

The student racial breakdown for each school may not add up to 100 percent. In some cases, schools

had racial groups represented by only a few students and these groups were not included. In other

cases, the total did not equal 100 percent due to rounding.

We also surveyed the Title I directors in all 50 states in December 1999 and

January 2000 to obtain information about how the states measured

educational outcomes (including not only their state assessment systems

but other outcome measures), the level of detail at which they publicized

this information, their definitions of “adequate yearly progress” for schools

and districts under Title I, and how they monitored the Title I program at

the district and school levels. To obtain additional information on the data

states make available to the public, we used a standardized data collection

instrument to review the content of the 50 state education agencies’

Internet web sites for information on assessment, other outcome measures,

and other information at the school and district levels. To obtain

information on state outcome data, accountability systems, and Title I

program monitoring, we interviewed officials from five states and reviewed

documents they provided. We also reviewed the general literature on state

accountability systems, assessments, and educational outcomes.

12 Tenn. K-4 814 53 white

42 black

49 X

13 Tenn. K, 5-6 366 52 white

42 black

93 X

14 Tenn. K-8 164 100 white 63 X

15 Tenn. K-5 535 50 white

49 black

67 X

16 Tex. PK-5 884 96 Hispanic

3 black

97 X

17 Tex. 6-8 1024 96 Hispanic

3 black

88 X

18 Tex. 9-12 1664 88 Hispanic

9 black

82 X

19 Tex. PK-2 284 43 black

40 white

17 Hispanic

68 X

20 Tex. 6-8 572 36 white

33 black

29 Hispanic

71 X

21 Tex. PK-4 417 61 Hispanic

36 white

79 X

(Continued From Previous Page)

School State

Grade

span

Number of

students

Student racial

breakdowna

(%)

Poverty rate

(percent) Schoolwide

Targeted

assistance
Chapter 1

Introduction

Page 26 GAO/HEHS-00-89 Title I Schoolwide Programs

To obtain information on current research on Title I, Title I services, and

schoolwide programs, we conducted a comprehensive literature search

and review. We interviewed Education researchers to obtain information

on their current studies and future evaluation plans.

We also interviewed staff in the Office of Compensatory Education (which

administers Title I), as well as federal officials representing federal

comprehensive technical assistance centers, regional education

laboratories, and equity assistance centers. In addition, we interviewed

representatives of major education associations and other experts. Finally,

in addition to reviewing Title I law and regulations, we also reviewed and

analyzed other Education documents, including program guidance,

monitoring manuals, training materials, and reports from completed

monitoring visits.

Our work was done between April 1999 and February 2000 in accordance

with generally accepted government auditing standards.
Page 27 GAO/HEHS-00-89 Title I Schoolwide Programs

Chapter 2

Educators Believe Schoolwide Approach Can

Provide Additional Opportunities in HighPoverty Schools Chapte2r

Schoolwide and targeted assistance schools often provided similar

educational and support services. However, many of the educators we

interviewed told us they preferred the schoolwide approach because they

believed it provided disadvantaged students additional opportunities to

improve academic achievement, especially in high-poverty schools. Both

schoolwide and targeted assistance schools aimed special services at

children who needed extra help. However, according to an education

survey, schoolwide schools were more likely to use methods that increased

instructional time (such as before-school, after-school, and summer school

programs). Principals and teachers in high-poverty schools generally

supported the schoolwide approach, primarily because they believed it

helped them serve a greater number of students. At the same time, some

educators expressed concern that schoolwide schools need to take care

that broader efforts to improve student performance do not result in

decreased services to those low-performing students who need special

assistance. Moreover, evidence regarding the overall effectiveness of the

schoolwide approach is inconclusive.

Schoolwide and

Targeted Assistance

Schools Generally

Offered Similar

Services but Varied in

Service Delivery

Methods

Title I schoolwide and targeted assistance schools often provided similar

educational services for disadvantaged students and their families. In the

schools we visited, both schoolwide and targeted assistance schools

directed specific services (such as tutoring) to students identified as

needing additional assistance. After-school programs were the most

common type of extended instruction, and 7 of the 21 schools we visited

used Title I funds to support after-school programs. Principals told us that

the after-school programs consisted primarily of tutoring for low-achieving

students and were generally available 1 or 2 days per week. Three of the

schools that operated after-school programs also provided the students

with other services, such as snacks and transportation home.

Most of the schools we visited—both schoolwide and targeted assistance—

used Title I funds for activities designed to more closely involve parents in

their children's education. These activities included parent-teacher

meetings, workshops, parent advisory councils, and home-school liaisons.

For example, one targeted assistance elementary school we visited used

Title I funds to support workshops that provided parents with information

about Title I services; ideas for helping their children with school work at

home; and summer learning activities for their children, including a

recommended reading list. A schoolwide school we visited used Title I

funds to provide tables and chairs so that parents can have lunch with their

children in a private area set aside near the school cafeteria. Some
Chapter 2

Educators Believe Schoolwide Approach Can

Provide Additional Opportunities in HighPoverty Schools

Page 28 GAO/HEHS-00-89 Title I Schoolwide Programs

schoolwide and targeted assistance schools also provided family literacy

programs for parents and students. For example, one school district we

visited operated a family literacy center in collaboration with a local

community college. Throughout the year, the center offered basic skills and

parenting workshops to any parent with children attending the local public

schools.

In several schools, Title I funds were also used to pay the salaries of homeschool liaisons who linked schools with parents, students, and the

community. In one targeted assistance elementary school, for example, the

principal told us that the home-school liaison made about 1,100 home visits

during the past school year to discuss with parents issues concerning their

children such as attendance, homework, and other classroom or behavioral

matters. At another schoolwide school we visited, the home-school liaison

not only made home visits but also held monthly meetings for parents at

school, regularly contacted parents to increase their involvement in school

activities, coordinated the activities of parent volunteers at the school, and

published a school newsletter. The liaison also established relationships

with community members, recruiting a local business owner who then

donated both time and money to the school.

While schoolwide and targeted assistance schools generally provided

similar services for their students, schoolwide schools tended to offer a

wider array of programs. In responding to Education's recent surveys,

school officials reported that schools using the schoolwide approach were

more likely than targeted assistance schools to offer programs with

extended-time learning opportunities (see table 3).

Table 3: Principals Reporting Use of Extended-Time Learning Opportunities

a

Statistically significant.

Source: Camilla Heid and Ann Webber, School-Level Implementation of Standards-Based Reform:

Findings from the Follow-Up Public School Survey on Education Reform (Washington, D.C.: U.S.

Department of Education, Planning and Evaluation Service, 1999).

Extended-time programs

Schoolwide

schools

(percentage)

Targeted assistance

schools (percentage)

Before school 18 14

After school 53a 36a

Weekend 7a 3a

Summer school 39 36
Chapter 2

Educators Believe Schoolwide Approach Can

Provide Additional Opportunities in HighPoverty Schools

Page 29 GAO/HEHS-00-89 Title I Schoolwide Programs

State, district, and school officials also told us that the schoolwide

approach allowed schools to adopt instructional methods they considered

more appropriate for all of their students. For example, one schoolwide

elementary school we visited used interactive computer software designed

to help teachers improve students' reading abilities. The computer program

helped teachers identify students' strengths and weaknesses so that

students were able to receive individualized assistance in areas of

identified need. Under the schoolwide approach, this school could use Title

I funds to purchase and implement the computer software for the whole

class, without targeting it specifically to the lower- achieving students.

Similarly, staff at one elementary school told us they preferred to keep all

their students together in the same classroom and move them in and out of

flexible groups as their achievement levels changed. These fluid groups

allowed students to obtain long- or short-term remedial help as needed

within the classroom, without being formally identified as Title I eligible.

School officials also described classroom settings in which students were

placed in small groups, combining strong and weak performers, and

encouraged to help each other with reading.

In contrast, targeted assistance schools were more likely to rely on “pullout” programs, using Title I funds to pay for teachers who provide remedial

instruction targeted to specific students. The pull-out option can provide a

convenient method for tracking who received Title I services and how the

funds were expended, allowing targeted assistance schools to more easily

demonstrate that funds were used for those students formally identified as

eligible under Title I. However, while the schoolwide schools we visited

sometimes used the pull-out strategy, they preferred other approaches.

Schoolwide Approach

Viewed as Benefiting

More Students

Under the schoolwide approach, schools can provide Title I services to

more students, some of whom would not be eligible under targeted

assistance. In the high-poverty schools we visited, many educators viewed

this broader focus as a primary advantage of the schoolwide approach.

They explained that most or all of their students suffered from educational

disadvantages and that the schoolwide approach allowed them to prevent

as well as address learning deficiencies. For example, one high-poverty,

urban elementary school we visited used its Title I funds to help implement

broad changes in the school curriculum. To help students improve their

critical thinking skills, the school integrated a science theme throughout

the curriculum, using new instructional methods such as hands-on

activities and computers with science-related software programs. The
Chapter 2

Educators Believe Schoolwide Approach Can

Provide Additional Opportunities in HighPoverty Schools

Page 30 GAO/HEHS-00-89 Title I Schoolwide Programs

principal told us the schoolwide approach provided more opportunities to

develop strategies that could benefit a greater number of students.

In contrast, targeted assistance schools more often relied on pull-out

programs to provide remedial assistance to low-achieving students. For

example, one Education survey found that 80 percent of targeted

assistance schools used pull-out programs, while 53 percent of schoolwide

schools used this approach.1 Although pull-out programs provide intensive

tutoring to low-performing students, they serve only a limited number of

students each school year and may exclude other low or marginally

performing students in need of additional instruction. One elementary

school we visited, for example, used a well-known pull-out program to

provide individualized reading instruction to low achievers in the early

grades. In this school, one reading specialist provided one-on-one tutoring

to about 10 children throughout the school year. While the number of

students who receive these services will vary according to the number of

reading specialists available, some district and school officials told us they

did not adopt this specific program because it is expensive and can serve

only a small number of students.

Many district and school officials also told us they favored schoolwide

programs because this approach offered other advantages, such as

fostering increased collaboration and greater flexibility. Before adopting a

schoolwide approach, schools are required to develop school plans that

describe strategies to improve academic achievement, both for lowachieving students and for the rest of the school. School officials told us

that school staff participating in these planning activities worked together

in new ways, creating better coordination and communication between

instructional programs. For example, staff told us that during the planning

process, classroom strategies such as team teaching were developed

because of the teachers' willingness to try different approaches that may

help improve all students' academic performance.

In addition to serving more students, the schoolwide approach can give

schools greater flexibility in choosing educational strategies to improve

student achievement. For example, one rural elementary school we visited

implemented an approach that relied on reducing class size to give students

1

Heid and Webber.
Chapter 2

Educators Believe Schoolwide Approach Can

Provide Additional Opportunities in HighPoverty Schools

Page 31 GAO/HEHS-00-89 Title I Schoolwide Programs

more individualized attention, while another school invested in

professional development to help teachers implement state standards.

Other schools we visited purchased computers and software that allowed

students to progress at their own pace. In all of these schools, principals

told us that the flexibility of the schoolwide program was an important

factor in encouraging them to adopt the changes.

Some district and school officials cautioned that schools that adopt the

schoolwide approach need to be careful to make sure that low-performing

students receive the extra help they need to improve their academic

performance and not be lost in the overall program. For several district and

state officials, this concern was especially important for schools with

relatively lower poverty rates and fewer disadvantaged children.

Nearly all of the schools GAO visited had high poverty rates and used the

schoolwide approach. Most schools chose the schoolwide approach

because school officials believed there were a number of advantages in

doing so. However, research on the overall effectiveness of the schoolwide

approach, as discussed in chapter 4, is inconclusive.
Page 32 GAO/HEHS-00-89 Title I Schoolwide Programs

Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students Chapte3r

Although Education is responsible for overseeing states’ implementation of

Title I, states continue to play a central role in program oversight and

accountability at the school and district levels. Historically, states have

been responsible for ensuring that districts and schools comply with Title I

programmatic and financial requirements, such as protecting the integrity

of federal funds and ensuring quality schoolwide planning that addresses

the needs of disadvantaged students. Under the oversight of Education,

states are expected to collect and report data on assessment outcomes and

to hold schools and districts accountable for these outcomes.

States varied dramatically in their efforts to oversee schools’ and districts’

compliance with Title I’s programmatic and financial requirements. States

focused their reviews on different requirements and adopted different

oversight methods. In addition, while some states made annual oversight

visits to schools and districts, others never made visits. States also varied in

how they reviewed schools’ and districts’ compliance with Title I

requirements and the degree to which they implemented Title I’s data

collection and reporting requirements. Some states provided extensive,

detailed, and timely assessment data that school and district officials found

useful in improving instruction. Other states collected and reported less

information on assessments, including results for economically

disadvantaged students—information that is required beginning in the

2000–2001 school year. Furthermore, states’ criteria for determining

whether schools and districts have performed satisfactorily have, with one

exception, been based solely on the performance of the student population

as a whole, without reference to the performance of specific subgroups of

children. Although the law does not require disaggregated data for yearly

progress assessments, lack of these data makes it difficult for states to hold

districts and schools accountable for the achievement of disadvantaged

students, and to discern whether achievement gaps between disadvantaged

and nondisadvantaged students are closing.

States Are Responsible

for Ensuring Local

Compliance;Education

Provides Oversight and

Technical Assistance

Under the oversight of Education, states play two distinct roles in ensuring

accountability for the Title I program. First, the states must ensure that

local schools and districts comply with the programmatic and financial

requirements that apply to them. Second, the states are responsible for

collecting and reporting data on assessment outcomes and devising criteria

to hold schools and districts accountable for these outcomes.

In both these roles, states have considerable discretion in how they carry

out their responsibilities and are generally free to develop their own
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 33 GAO/HEHS-00-89 Title I Schoolwide Programs

oversight methods and procedures. States also have discretion in designing

and gathering outcome data. The states set standards that determine the

key content children are expected to know. States can choose from a

variety of assessments—from multiple-choice standardized exams to

collected portfolios of student work—to measure student achievement as

related to these standards. States can also collect and report other

educational outcome measures. Finally, states have broad latitude in how

they establish criteria for determining whether schools and districts are

performing satisfactorily—for example, whether schools and districts will

be evaluated solely on the basis of assessment results or whether criteria

will include other measures of educational outcomes, such as graduation

and dropout rates.

Education is responsible for assessing whether states implement Title I

requirements related to assessments and sufficiently monitor schools’ and

districts’ implementation of Title I’s financial and programmatic

requirements. Education also provides technical assistance to the states on

Title I regulations and on program quality issues.

Focus and Frequency

of Program Oversight

Efforts Varied Across

States

States varied in how they focused their program oversight efforts. Like the

federal Department of Education, some states have approached their

oversight process from a technical assistance rather than strictly a

compliance perspective. For example, one state we visited used peer

reviewers to help schools and districts interpret data on teaching practices

and other key areas and placed less emphasis on compliance with financial

requirements. In contrast, another state focused more on district-level

program administration. Similarly, while the majority of states (34)

conducted reviews that covered multiple federal programs, some states

focused their review process on Title I alone.

States differed not only in their general approach to monitoring, but also in

the methods they used in their Title I reviews. These methods included

visits to districts and schools, reviews of state-required annual reports and

self-assessment documents, and reviews of districts’ financial audits.1 For

the most part, states depend on districts’ data gathering and reporting, even

though they may be incomplete in some areas. Consequently, on-site visits

1

Under the Single Audit Act, recipients of federal funds may be required to have an annual

single audit of the funds received from all federal programs to ensure that federal dollars are

spent in compliance with applicable requirements.
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 34 GAO/HEHS-00-89 Title I Schoolwide Programs

have generally been viewed as important because they provided state

officials with the opportunity to look directly at program implementation.

Forty-seven states reported that they included visits to school districts as

part of their monitoring procedures, and many of these states reported that

they relied solely on on-site visits to monitor certain key requirements. For

example, 26 states reported reviewing the components of schoolwide plans

only during on-site visits.

States varied dramatically in how frequently they conducted on-site

reviews. For example, while most states made on-site visits to schools and

districts, the average time between visits to districts ranged from 2 years or

less (for 6 states) to more than 7 years (for 17 states), as fig. 3 shows. Three

states reported that they made no on-site visits at all, while three states

reported that they visited all their school districts each year. The fact that

some states relied solely on site visits for oversight of key financial and

programmatic requirements, and made only very infrequent visits, raises

questions about whether these states conduct sufficient oversight.
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 35 GAO/HEHS-00-89 Title I Schoolwide Programs

Figure 3: Frequency of States’ On-Site Monitoring

Source: GAO survey of the 50 state education agencies.

States Have Made

Uneven Progress in

Collecting and

Reporting Assessment

Data Essential to

Accountability

States determine the content covered in state standards and how student

performance is measured. States varied in the types of assessments used

(for example, multiple-choice tests, questions requiring written responses,

or alternative assessments such as portfolios of student work), the test

length, the frequency with which assessments are done, and the analytic

techniques used. States, districts, and schools also have some discretion

over whether certain students (such as those with limited English

proficiencies) are included in state-sponsored assessments. Nearly all

states have collected some type of assessment or outcome data; however,

few have obtained data specifically on certain groups of disadvantaged

0

5

10

15

20

Make Visits,

but

No Data

Available

Do Not

Make Visits

2 Years or Less 2 to 7 Years Over 7 Years

6

18

13

17

3

6

Average Cycle Time for On-Site Visits

Number of States
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 36 GAO/HEHS-00-89 Title I Schoolwide Programs

students. Furthermore, although most states publicly reported assessment

data for the overall student population of the district or school, many have

not reported similar data on different groups of disadvantaged students.

States Face Challenges in

Designing Valid

Assessments and Ensuring

Full Participation

States are responsible for developing content and student performance

standards as well as assessments to measure students’ proficiencies. Every

state except one has adopted content standards and established

assessments; some states have established assessments in only reading and

mathematics, while others also require assessment in areas such as science

and social studies. In two states, the state allowed local districts to choose

their own assessments; however, if districts choose different assessments,

results may not be comparable.2 In the remaining 48 states, the state

education agency has required the same assessment on a statewide basis so

results can be compared across all districts and schools.

Because so much of schools’ and districts’ accountability hinges on state

assessments, it is important to ensure that these tests serve as valid

measures of student achievement. States have made choices about the

type, format, length, and difficulty level of the tests, and each of these

choices can have implications for the usefulness of the assessment. For

example, an assessment with a large number of multiple-choice items that

focus on basic skills can provide extensive information to help schools

improve the performance of low-achieving students, but might be less

useful in promoting continuous improvement in schools with higher levels

of academic achievement. An assessment that requires written responses

to a smaller number of detailed and more difficult questions might better

measure critical thinking skills but cover a narrower range of topics and

provide less information on what material low-achieving students have

mastered. Table 4 shows several important design issues, their implications

for the usefulness of an assessment in accomplishing various purposes, and

how states have responded to these issues.

2

In one of these states, a large majority of districts have chosen to administer the same test.

For more information about these states’ systems, see Craig D. Jerald, “The State of the

States,” Education Week: Quality Counts 2000 (January 13, 2000), p. 62.
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 37 GAO/HEHS-00-89 Title I Schoolwide Programs

Table 4: Assessment Design Issues

a

As noted previously, two states leave the format of the assessment up to local school districts. For

more detailed information on the use of open-response format questions on state assessments, see

Jerald, p. 62.

Both federal legislation and Education’s guidance have strongly

emphasized that, to the extent practicable, all students should participate

in state assessments. If certain students do not take the test, it impairs the

state’s ability to hold schools and districts accountable for those students’

achievement. However, in some circumstances it may be considered

Issue Options Implications States’ responses

Type of test States must choose either a normreferenced assessment, which may

show a student’s or school’s results

compared with a national average, or

a criterion-referenced assessment,

which measures knowledge in

relationship to specific criteria (such

as may be incorporated in the state

standards).

Norm-referenced tests can provide national

comparison information and may be easier to use

because they are commercially available. However, a

criterion-referenced test may do a better job of

measuring students’ knowledge compared to specific

elements of the state standards.

27 states use a normreferenced test in at least

one subject; 38 states

use a criterionreferenced test in at least

one subject.

Test length States must choose the number of

items they include in their

assessments.

States with very detailed and specific standards may

find it difficult to cover the wide range of topics and

also include enough items to provide a valid measure

of knowledge in any one area. If state tests have too

few items, they may not adequately test the students’

knowledge of the range of topics. However, tests with

an excessive number of items may result in students

spending unnecessary time taking tests rather than

being engaged in classroom instruction.

Although we did not

specifically examine

variations in test length

as part of our study, it

seems reasonable to

expect that length of

assessments varies

across states.

Test format States must choose from more

traditional objective-style questions

(like multiple-choice), questions

requiring written responses, and

alternative assessments (such as

portfolios of student work).

Traditional, multiple-choice questions can be

machine-scored, and the results of the scoring do not

depend on the individual grading the test. Some

experts prefer formats that require students to

generate their own response. Assessments with such

questions, however, can be more costly and difficult

to score. Similarly, portfolios may be preferred

because they reflect the student’s work over time

rather than on one test day; however, obtaining

consistent, objective scoring of portfolios can also be

time-consuming and expensive.

Only 4 states rely

exclusively on traditional

multiple-choice

assessments in their

core subjects, 44

included open-response

formats in their

assessments, and 2

required the use of

portfolios statewide.a

Difficulty level States must choose the overall

difficulty level of the assessment.

A more difficult assessment that sets a high standard

may motivate schools and districts to concentrate on

challenging material but provide less information to

use to help low-achieving students improve. If the

lowest-achieving students can answer only a very

few questions correctly, the test cannot help teachers

distinguish between material these students have

mastered and the material they have not.

Only limited information

is available, but one

study, which reviewed

tests used in several

states, found wide

variation in difficulty level

among 8th grade

mathematics tests.
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 38 GAO/HEHS-00-89 Title I Schoolwide Programs

appropriate to exempt an individual student or group of students from

state-sponsored assessments. Students with severe disabilities, for

example, may be working toward different educational goals than the

general population, and thus it may not be considered appropriate to

measure their progress against state standards.3

States varied in how they attempted to strike the balance between

including as many students as possible and making sure that students were

not subjected to inappropriate assessments. In our survey, 45 states

reported that some students were exempted; however, over half (24) were

unable to provide any information on precisely how many students were

exempted.4 Similarly, of the 41 states that reported district- or school-level

assessment data through their Internet site, only 20 provided any additional

information about the extent to which students participated in or were

excluded from assessments, and even fewer states (4) provided

information that specified the reasons for student exclusions. Without such

information, state officials and outside observers cannot determine

whether state assessment results reflect achievement for all students or

only for a selected group. Furthermore, if different numbers or types of

students are excluded in different schools, school-to-school comparisons

may become less meaningful.5

3

Federal law expressly allows some students to be excluded from assessment. The decision

to exclude a student with a disability is left to the student’s Individualized Education

Program team, which comprises the student’s parents, teachers, district officials, and

others. Students with disabilities may also (at the discretion of the team) receive

accommodations—such as being allowed extra time or having part of the assessment

presented orally—to help them participate in the assessment.

4

Forty states excluded some students with limited English proficiency, and 40 states

excluded some students with disabilities. In addition, 18 states sponsored alternative

assessments for students with limited English proficiency (such as a Spanish-language

version), and 18 states sponsored alternative assessments for students with disabilities.

5

Although we did not collect information about the extent of accommodations for students

with disabilities, a recent research study confirms that similar issues arise when states,

schools, and districts differ in their practices in granting accommodations. For more

information about this issue, see Anne Lewis, 1998 CRESST Conference Proceedings,

Comprehensive Systems for Educational Accounting and Improvement: R&D Results,

National Center for Research on Evaluation, Standards, and Student Testing (CRESST), CSE

Technical Report 504 (Los Angeles: University of California, June 1999).
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 39 GAO/HEHS-00-89 Title I Schoolwide Programs

Some States Collected

Assessment Data

Specifically on

Disadvantaged Students;

Few States Collected Other

Outcome Data for These

Students

States varied in the type and amount of data they collected from schools

and districts on assessment results and other outcome measures for

disadvantaged students. Some states collected information on assessment

results specifically of disadvantaged students. Fewer states indicated that

they collected information on other educational outcomes, such as dropout

or attendance rates of these students.

Title I requires states to collect assessment results separately for specific

subgroups of students when final assessments are in place; however, many

states have not yet collected all these data. As of fall 1999, only 17 states—

less than half—had disaggregated their assessment results by all six

required categories (gender, racial and ethnic group, migrant status,

disability status, English proficiency status, and economic status). The

most frequently omitted category was migrant status, perhaps reflecting

the limited outcome information generally available on migrant students.6

Many states also did not disaggregate assessment data by economic status.

In addition, 38 states reported that they gathered other outcome data not

required by Title I legislation. The most commonly used measures were

student attendance, dropout, and graduation rates, as shown in fig. 4. Only

4 states indicated that they disaggregated these data by the six categories

required for assessment data; 14 states did not disaggregate their other

outcome data by any of these categories (see fig. 5).

6

For more information on these issues, see Migrant Children: Education and HHS Need to

Improve the Exchange of Participant Information (GAO/HEHS-00-4, Oct. 15, 1999).
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 40 GAO/HEHS-00-89 Title I Schoolwide Programs

Figure 4: Other Outcome Measures Gathered by States

Source: GAO survey of the 50 state education agencies.

 

 

 

 

 

Other Outcomes Measured

 umber o States

0

5

 0

 5

 0

 5

 0

 5

 eacher Other

Attendance

 ro otion 

 etention

Student

 o i ity

 u er

 a ing

A 

 ourses

SA A 

 esu ts

 isci inary

Actions

Student Graduation

Attendance

 ro out

Schoo 

 istrict

 

 

 

 

 

 

 4

 4

 

 4 4

4 4
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 41 GAO/HEHS-00-89 Title I Schoolwide Programs

Figure 5: Number of States That Disaggregate Assessment and Other Outcome Data, as of Fall 1999

Source: GAO survey of the 50 state education agencies.

Most States Publicly Report

Outcome Data but Fewer

States Report Data for

Various Subgroups of

Disadvantaged Students

While most states report assessment results to the public, many states do

not separately report data for various subgroups of disadvantaged

students. In the 1994 Title I reauthorization legislation, the Congress

recognized that publicity could be an important tool for holding schools

and districts accountable for student performance. By making test results

available to the public, states give parents, educators, and the community

information on how their schools are doing and provide an incentive for

schools and districts to improve their results. Nearly all states (48) said

they made their assessment data available to the public; fewer (34)

reported their other outcome data. State survey responses indicated that

the majority of states report assessment information to the public through

23

40

21

38

11

Disaggregated Data

Number of States

0

10

20

30

40

50

None of

the Groups

Migrant All Groups

Status

Racial/Ethnic LEP Poverty

Group

Gender Disability

Other Outcome Data

Assessment Data

8

26

7

23

4 3

44

13

17

14

39
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 42 GAO/HEHS-00-89 Title I Schoolwide Programs

press releases and by making information available on states’ Internet web

sites.

While most states publicly reported assessment and other outcome data for

the overall population of the district or school, some states did not report

such data by specific groups of disadvantaged students. For example, one

state official told us that although the state keeps disaggregated

assessment data, his office does not use them. States identified the Internet

as one of the most widely used means of publicizing assessment results,

and it is a natural choice for distributing the more voluminous information

from disaggregated assessments. However, very few states used the

Internet to report disaggregated data, as fig. 6 illustrates.

Figure 6: Number of States Reporting Disaggregated Assessment Data Through the

Internet

Source: GAO review of the Internet web sites of the 50 state education agencies.

Assessment and other outcome data clearly play an important role in

holding schools and districts accountable for student performance in

general and for disadvantaged students in particular. However, staff at

many of the districts and schools we visited also emphasized how these

data helped them improve instruction. District officials and principals told

None of the 6 Categories

43

All of the 6 Categories

1

Some of the 6 Categories

6
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 43 GAO/HEHS-00-89 Title I Schoolwide Programs

us that they used assessment and other outcome measures to focus

attention and effort on improving student achievement. For example,

several schools we visited displayed signs, banners, or bulletin boards

showing their progress in improving assessment results. District and

school staff also reported using assessment information to identify and

address weaknesses in instruction. One school we visited used a detailed

analysis of the state assessment to identify key mathematics concepts

students needed to work on, and set aside a 15-minute period each day for

students to play special math games to develop and reinforce these specific

skills.

School and district officials emphasized that data must be detailed and

timely to be useful in improving instruction. However, the information

reported by states was sometimes limited or late. School and district staff

found assessment and other outcome data especially useful when broken

down by teacher or objective, but only 12 states provided assessment data

by teacher, 23 by objective, and 11 by individual test item. In addition, in

two of the five states we visited, district and school officials told us that the

results of the state assessments were not provided to them in a timely

manner—making it difficult for them to make significant instructional

changes to address students’ needs.7

7

In a number of districts we visited, district officials had implemented assessment programs

of their own to better meet their needs for detailed and timely achievement data. Because

these district-administered assessments were scored quickly and analyzed in detail, schools

could identify problems and tailor solutions to the individual child, classroom, or school.

Nine states made at least some questions from previous assessments available through their

Internet site.
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 44 GAO/HEHS-00-89 Title I Schoolwide Programs

States’ Annual

Progress Criteria

Include Only

Assessment Results for

Overall Student

Population, Limiting

Accountability for

Disadvantaged

Students

In addition to requiring that states collect and report data on educational

outcomes, Title I requires states to establish criteria to evaluate whether

schools and districts are performing satisfactorily—that is, making

“adequate yearly progress” toward improving student achievement. States

have chosen different ways to define criteria for determining adequate

yearly progress. Some states defined their criteria on the basis of a fixedtarget assessment score for all schools, while others required schools to

show improvement compared with previous results. For example, one state

required that 75 percent of all students in grades 4 and 8, and 85 percent of

all students in grade 11, meet proficiency levels on the state assessment. In

another state, for a school to meet the criteria, at least 50 percent of the

students had to score at a satisfactory level in mathematics and reading for

all grades tested for 2 consecutive years, or the number of students scoring

at a satisfactory level had to increase by at least 5 percent from one year to

the next. Although states are not required to establish their final adequate

yearly progress criteria until the 2000–2001 school year, a majority of states

reported these criteria in response to our survey of the 50 states.

Several states have very lengthy and complex adequate yearly progress

criteria that can be difficult for school and district staff to understand and

interpret. Officials in two districts we visited, in two different states, told us

that they had encountered difficulties interpreting their assessment results

in light of the states’ expectations of performance. Other criteria we

reviewed also appeared confusing or vague. If the school and district staff

do not fully understand the criteria on which their performance will be

judged, it is likely that they will have difficulty developing strategies for

improvement. In addition, states have—with only two exceptions—defined

adequate yearly progress solely in terms of assessment results, without

including other educational outcome measures such as graduation,

attendance, or dropout rates. These other outcome measures are also

important in considering schools’ and districts’ performance, yet are

generally not included in the yearly progress criteria. Because adequate

yearly progress is a new concept, some states have had difficulty

developing their criteria.

Annual progress criteria are not required by law to be disaggregated by

subgroups of students, and only one state has chosen this approach. As a

result, schools and districts may not be held accountable for closing

achievement gaps over time among subgroups of students but only for

improving the average overall student achievement. Because subgroups of

students can have poor results on assessments even if the overall student
Chapter 3

Most States Not Positioned to Hold Schools

and Districts Accountable for Outcomes of

Disadvantaged Students

Page 45 GAO/HEHS-00-89 Title I Schoolwide Programs

body is performing at higher levels, disadvantaged children may fall further

behind even in schools making satisfactory progress according to states’

criteria. Basing progress criteria on each student subgroup may

unnecessarily complicate states’ evaluation of school and district

performance. However, in Texas, the state has included two specific

subgroups—economically disadvantaged and race/ethnicity—in its

definition of adequate yearly progress to improve accountability for the

achievement of disadvantaged students.
Page 46 GAO/HEHS-00-89 Title I Schoolwide Programs

Chapter 4

Evaluations of Title I and Schoolwide

Programs Have Been Limited Chapte4r

Nationally representative evaluations of the effectiveness of Title I are

needed because state assessment data on the progress of disadvantaged

students are currently limited and state data on overall student

achievement vary in quality and type. However, research on the

effectiveness of the Title I program is also currently limited, particularly for

the more recent schoolwide approach. Previous research suggests that

Title I may have a slightly positive effect on student achievement in highpoverty schools. Research on the general effectiveness of the schoolwide

approach for improving student achievement in high-poverty schools is

inconclusive.

Conducting research on the effectiveness of Title I presents challenges,

including the difficulty of discerning the effect of Title I on the regular

school program. Education’s planned improvements in data collection

show some promise for facilitating better research, but these efforts will

need to be expanded to ensure that specific educational programs and

services can be linked to achievement—both for disadvantaged students

and all students.

Research Efforts Have

Provided Limited

Information on

Program Effectiveness

Title I may have a limited positive impact on overall student achievement in

high-poverty schools, according to the two most recent congressionally

mandated studies completed by Education. Education’s most recent

national assessment generally concluded that, since 1992, national

performance in reading and mathematics has improved for 9-year-olds in

the highest-poverty public schools—those schools most likely to receive

Title I funds.1 However, the report also noted that a substantial

achievement gap remains between students in the highest- and lowestpoverty schools. Education concluded in its earlier mandated study that

Title I may have helped educational achievement but did not bring the

participating students up to the level of their classmates.2 Other research

findings are based on data that are not nationally representative; these

studies provide either weak positive evidence or mixed evidence about the

effectiveness of Title I.

1

U.S. Department of Education, Office of the Under Secretary, Planning and Evaluation

Service, Promising Results, Continuing Challenges: The Final Report of the National

Assessment of Title I (Washington, D.C.: 1999).

2

Abt Associates, Inc., Prospects: Final Report on Student Outcomes, prepared under

contract to the Planning and Evaluation Service, U.S. Department of Education

(Washington, D.C.: U.S. Department of Education, Apr. 1997).
Chapter 4

Evaluations of Title I and Schoolwide

Programs Have Been Limited

Page 47 GAO/HEHS-00-89 Title I Schoolwide Programs

Because of data and methodological limitations, neither the national

evaluations nor smaller studies have been able to support any conclusions

on whether schoolwide programs are effective. The earlier national

evaluation did identify five high-performing, high-poverty schools, three of

which had adopted schoolwide programs; however, because there were so

few schools, researchers could not draw conclusions on the basis of this

finding. Similarly, another Education report on four case studies found that

while administrators and faculty in schoolwide schools praised the

schoolwide option, “the schools were not producing remarkable levels of

student achievements.”3 Similar to what we discussed in chapter 2,

evidence on the effectiveness of schoolwide programs was often based

solely on the assertion of the school’s principal or Title I coordinator,

according to a recent research synthesis.4 Several studies of schoolwide

programs within individual school districts (including Philadelphia,

Houston, and Minneapolis) have provided mixed results. For example, in

Minneapolis the gap in math achievement between lower achieving and

other students was smaller in schoolwide than in targeted assistance

schools, but the opposite was true in Houston.5 Education has an ongoing

study of Title I programs which may shed some light on factors affecting

student achievement in high-poverty schools; however, it has a limited

scope and is not nationally representative.6

3

Sam Stringfield, Mary Ann Millsap, Rebecca Herman, and others, Urban and

Suburban/Rural Special Strategies for Educating Disadvantaged Children: Final Report,

prepared for the U.S. Department of Education (Apr. 1997).

4

Kenneth K. Wong and Stephen J. Meyer, “Title I Schoolwide Programs: A Synthesis of

Findings From Recent Evaluation,” Educational Evaluation and Policy Analysis (Summer

1998), pp. 115–36.

5

Kenneth K. Wong, Gail L. Sunderman, and Jaekyung Lee, “Redesigning the Federal

Compensatory Education Program: Lessons From the Implementation of Title I Schoolwide

Projects,” National Center on Education in the Inner Cities Review, Temple University (Aug.

1997).

6

See Brenda Turnbull, Megan Welsh, Camilla Heid, and others, The Longitudinal Evaluation

of School Change and Performance (LESCP) in Title I Schools: Interim Report to Congress,

prepared for the U.S. Department of Education (July 1999).
Chapter 4

Evaluations of Title I and Schoolwide

Programs Have Been Limited

Page 48 GAO/HEHS-00-89 Title I Schoolwide Programs

Data Limitations

Present Challenges in

Evaluating Program

Effectiveness

The lack of comprehensive, detailed, and nationally representative data on

Title I schools and services has made program evaluation especially

challenging. For example, researchers have been generally unable to

clearly link Title I services to student achievement because the available

data are not sufficiently comprehensive. As we found in our site visits,

schools use the flexibility they have under Title I to spend funds on a wide

variety of goods and services, some of which are similar or identical to

goods and services offered at other schools but not supported with federal

funds. This makes it even more difficult to distinguish the effect of Title I

from the effect of the overall educational program of the school.

Researchers also face challenges linking differences in student

achievement to Title I rather than to other factors such as differences in the

resources available to schools. To link Title I to student achievement,

researchers need sufficiently detailed data to either select comparable

schools to study or to statistically estimate the effect of program services

on student achievement separately from other external factors. However,

available data do not contain the information needed to select comparable

schools or to statistically adjust for other factors. For example, Education’s

major database on U.S. public schools, the Common Core of Data (CCD),

does not currently provide information on whether a school receives Title I

funds or has a schoolwide program; the CCD provides only incomplete

information on school poverty rates. Without these data, researchers

cannot use the database to identify schoolwide and targeted assistance

schools with similar poverty rates.

Other challenges exist in evaluating the effectiveness of educational

services in both schoolwide and targeted assistance programs. One is

selecting an appropriate measure of student achievement. And, as

discussed in chapter 3, measures of student achievement are not

comparable across states.

In addition, no clear consensus has emerged on what criteria should be

used to determine whether Title I is effective. For example, the results of

one of the earlier evaluations indicated that Title I students learn at

approximately the same rate as their more advantaged peers, but that gaps

in initial achievement tend to remain.7 Some individuals have interpreted

results from the Prospects Study as evidence that the Title I program may

7

Abt Associates, Inc., Prospects: Final Report on Student Outcomes.
Chapter 4

Evaluations of Title I and Schoolwide

Programs Have Been Limited

Page 49 GAO/HEHS-00-89 Title I Schoolwide Programs

not be effective. Others disagree, pointing to work which suggests that

without intervention, gaps in achievement between disadvantaged and

other students tend to grow over time.

To meet some of these challenges, Education plans to expand its data

collection to facilitate research on the effectiveness of Title I, including

schoolwide programs. For example, Education officials plan to collect

information on the Title I and schoolwide program status of the nation’s

schools as part of its CCD. This information could be combined with

student assessment and services data to facilitate more comprehensive

analyses of Title I effectiveness. In addition, Education is planning to

expand its examination of the results of state assessments—including

disaggregated data—to assess students’ performance. While these

assessment data will not be comparable across states, an analysis of

student achievement on state assessments is important because it

represents the primary criterion the Title I legislation applies to define

school success.

Education’s efforts to extend its data collection and analysis show promise

for enabling better research, but these actions will still not fully address the

challenges to determining program effectiveness. Education’s analysis of

disaggregated state data, for example, may provide helpful information in

the future. However, the value of this type of analysis will be limited

because not all states have fully disaggregated their assessment data.

Moreover, Education’s planned efforts will only partially address the need

for more comprehensive data on school characteristics, resources, and

services provided. For example, Education’s plans to add Title I

information to its CCD will be valuable, but data quality and consistency

issues (such as missing data on poverty and other key variables) must be

addressed.8 Similarly, detailed data on educational services, from a

sufficiently large number of both schoolwide and targeted assistance

schools, will be needed to assess program effectiveness.

8

U.S. Department of Education, National Center for Education Statistics, Evaluation of the

1996–97 Nonfiscal Common Core of Data Surveys Data Collection, Processing, and Editing

Cycle (Washington, D.C.: Feb. 1999).
Page 50 GAO/HEHS-00-89 Title I Schoolwide Programs

Chapter 5

Conclusions, Recommendations, Matter for

Congressional Consideration, Agency

Comments, and Our Evaluation Chapte5r

Conclusions Educating children has traditionally been largely a state and local

responsibility. However, through the Department of Education, the federal

government has a role in supporting quality education for disadvantaged

children and the responsibility for overseeing the nearly $8 billion annual

federal investment in the Title I program. In its program monitoring,

technical assistance, and research roles, Education can help the states hold

schools and districts accountable for the academic achievement of

disadvantaged students. Since the 1994 reauthorization of Title I, states

have made some progress in improving their systems for holding districts

and schools responsible for the academic achievement of all students,

including disadvantaged students. However, some areas are still in need of

attention. Education, in its leadership role, has provided some assistance to

states in developing standards, assessments, and criteria for adequate

yearly progress; however, a majority of states appear to need additional

help, for example, in collecting and reporting disaggregated assessment

data.

Beginning in the 2000–2001 school year, states are required to collect and

report assessment data disaggregated by subgroups of students, including

the economically disadvantaged. Only about one-third of the states

currently collect this information, and it is unlikely that all of the states will

meet the deadline for collecting and reporting disaggregated assessment

data. Without disaggregated data, test results for the whole student

population can mask the results of disadvantaged students and prevent

states and districts from identifying schools that may not be meeting the

educational needs of disadvantaged students.

While the majority of states provided us with their criteria for adequate

yearly progress, some of the criteria were confusing or vague. Moreover,

only one state’s criteria for adequate yearly progress are based on

disaggregated data of subgroups of students. These criteria are needed to

hold schools and districts accountable for improving the performance of

disadvantaged students and to discern whether achievement gaps between

disadvantaged students and nondisadvantaged students are narrowing over

time.

Finally, Education’s research can provide states, districts, and schools with

important information to help them improve instruction. Comprehensive

evaluation efforts are needed to determine which services and service

delivery methods are most effective in closing the achievement gaps among

students.
Chapter 5

Conclusions, Recommendations, Matter for

Congressional Consideration, Agency

Comments, and Our Evaluation

Page 51 GAO/HEHS-00-89 Title I Schoolwide Programs

Recommendations We make two recommendations to the Secretary of Education. First,

Education should conduct additional activities to facilitate the exchange of

information and best practices among states so they can identify ways to

improve the timeliness and specificity of their assessment data, the

collection and reporting of disaggregated assessment data, and the clarity

of their criteria for adequate yearly progress. Second, Education should

implement additional measures to improve research on the effectiveness of

specific services in both schoolwide programs and targeted assistance

schools. Such measures could include expanding and improving current

data collection efforts so that comprehensive analyses could be conducted

linking program characteristics to services and student outcomes, and/or

developing an evaluation design for a study or set of studies of educational

services that would include national representation of both schoolwide and

targeted assistance schools.

Matter for

Congressional

Consideration

To hold schools and districts accountable for improving the performance of

disadvantaged students and to help educators, parents, and others discern

whether achievement gaps between disadvantaged and nondisadvantaged

students are closing, the Congress should consider requiring that states’

criteria for progress, as expressed in their definitions of adequate yearly

progress, apply specifically to disadvantaged children, as well as to the

overall student population.

Agency Comments and

Our Evaluation

We provided a draft of this report to the Department of Education for

review and comment. Without commenting on the recommendations,

Education agreed that annual progress criteria based solely on the

performance of the student population as a whole may be inadequate to

hold schools accountable for the performance of disadvantaged students.

However, Education said that such accountability does not necessarily

require separate criteria for each of the six subgroups for which states

currently must report assessment data. We did not say that separate criteria

should be used for each of the student subgroups and we have modified the

report to make this clear. Education also commented that some states may

respond to including specific groups of disadvantaged students in their

progress criteria by lowering standards. We continue to believe that it is

important for states to include subgroups in their adequately yearly

progress criteria because this information will help them hold schools and

districts accountable for the performance of disadvantaged students.
Chapter 5

Conclusions, Recommendations, Matter for

Congressional Consideration, Agency

Comments, and Our Evaluation

Page 52 GAO/HEHS-00-89 Title I Schoolwide Programs

Education also emphasized in its comments that states are not required to

publicly report disaggregated data until the 2000–2001 school year. We had

pointed this out several times in the report but had also found that many

states are not yet positioned to meet this requirement. The large number of

states that have not fully implemented assessments and disaggregated the

results raises serious concerns about whether states will be able to provide

the detailed data necessary to ensure accountability for the performance of

disadvantaged students by the 2000–2001 school year.

The Department agrees with our position that it is difficult to separate the

effects of Title I from the effects of state and local efforts. Education said

that reviewing how schoolwide programs spend Title I funds would be

insufficient to evaluate program effectiveness. We did not recommend this

approach. Rather, we recommended that Education improve research on

the effectiveness of specific services designed to address the needs of

disadvantaged students. Research that links these services to educational

outcomes can help determine which services are most effective for raising

student achievement. This approach does not preclude other types of

studies, such as Education’s National Assessment of Title I, that examine

the overall performance of students in high-poverty schools. However,

research on the effectiveness of specific services will be especially useful

to principals and school district officials, in our opinion, because it can

help them better leverage their Title I dollars to promote student

achievement.

We reported that the lack of comprehensive, detailed, and nationally

representative data on Title I schools and services has made program

evaluation especially challenging. Education said that the agency has

gathered detailed information on Title I through several nationally

representative studies, including the Public School Surveys on Education

Reform, the Study of Education Resources and Federal Funding, and the

National Longitudinal Survey of Schools. We reviewed these data and

found that the information they provide is valuable, but did not contain

sufficient detail for researchers to assess the effectiveness of specific

services. For example, while Education has collected some data on

services, its surveys generally did not include sufficient information to

allow researchers to control for differences in resources and student

populations across schools. Education’s current efforts to extend its data

collection and analysis show promise. However, additional steps will be

needed to improve research on the effectiveness of federally supported

services to address the educational needs of disadvantaged students.
Chapter 5

Conclusions, Recommendations, Matter for

Congressional Consideration, Agency

Comments, and Our Evaluation

Page 53 GAO/HEHS-00-89 Title I Schoolwide Programs

Finally, Education provided other comments regarding technical aspects of

the report, which we incorporated where appropriate. (See app. I for

Education’s comments.)
Page 54 GAO/HEHS-00-89 Title I Schoolwide Programs

Appendix I

Comments From the Department of Apendxies

Education ApendxiI
Appendix I

Comments From the Department of

Education

Page 55 GAO/HEHS-00-89 Title I Schoolwide Programs
Appendix I

Comments From the Department of

Education

Page 56 GAO/HEHS-00-89 Title I Schoolwide Programs
Page 57 GAO/HEHS-00-89 Title I Schoolwide Programs

Appendix II

Major Contributors to This Report ApendxiI

Harriet C. Ganson, Assistant Director, (202) 512-9045

Sarah L. Glavin, Senior Economist, (202) 512-7180

Timothy E. Hall

Sheila R. Nicholson

Ellen K. Schwartz

John G. Smale

James P. Wright

(104972) Letr
Ordering Information The first copy of each GAO report is free. Additional copies of

reports are $2 each. A check or money order should be made out to

the Superintendent of Documents. VISA and MasterCard credit

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any list

from the past 30 days, please call (202) 512-6000 using a touchtone

phone. A recorded menu will provide information on how to obtain

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet,

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at:

http://www.gao.gov

To Report Fraud,

Waste, or Abuse in

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00