**One System or Two?**

**Title I Accountability in the Context of High Stakes for Schools in
Local Districts and States**

Jennifer O'Day

University of Wisconsin -- Madison

With

Betheny Gross

University of Wisconsin -- Madison

Paper prepared for presentation at the American Education Finance
Association conference held March 18-21, 1999 in Seattle, Washington.
The research reported in this paper was supported by a grant from the
U.S. Department of Education, Office of Educational Research and
Improvement, National Institute on Educational Governance, Finance,
Policy-Making and Management, to the Consortium for Policy Research in
Education (CPRE) and the Wisconsin Center for Education Research, School
of Education, University of Wisconsin-Madison (Grant No.
OERI-R3086A60003). Supplemental funding was provided by the Planning and
Evaluation Service of the U.S. Department of Education. The opinions
expressed are those of the authors and do not necessarily reflect the
views of the National Institute on Educational Governance, Finance,
Policy-Making and Management, Office of Educational Research and
Improvement, U.S. Department of Education, the institutional partners of
CPRE, or the Wisconsin Center for Education Research.

**Executive Summary**

Background (Section I)

In the past decade, the problem of persistently low-performing schools
has garnered increasing attention from educators, policy makers, and the
general public. In response, over 22 states and several major districts
have instituted policies requiring school-level accountability for
student performance and sanctions for schools that fail to improve. In
1994, the federal government joined this trend by strengthening the
accountability provisions in the Elementary and Secondary Education Act
(ESEA) and imposing sanctions for Title I schools that fail to
demonstrate adequate progress on standards-based measures of student
performance.

This report addresses the relationship between the school accountability
measures required for Title I schools and those put in place by state
and local jurisdictions for all their schools. Our main focus was on the
degree of coherence and compatibility among the multiple accountability
systems, as well as the level and coordination of support provided to
schools in need of improvement. The sample selected for study included
two state-administered systems (Kentucky and Maryland), two
locally-developed systems (San Francisco and Chicago), and one large
urban district responding to the state-based accountability requirements
(New York City). Each of these jurisdictions had developed its
system-wide accountability policies independent of, and generally prior
to, the new Title I provisions.

# **Findings (Section II)**

Five major findings regarding the identification and support of low
performing schools in these jurisdictions emerged.

-   The majority of the low-performing schools identified by the state
    or local accountability systems are Title I schools.

```{=html}
<!-- -->
```
-   Considerable overlap exists between the schools identified for
    improvement under Title I and those identified in the jurisdictional
    systems. This overlap is not absolute, however, and varies
    considerably among the jurisdictions.

```{=html}
<!-- -->
```
-   Where identification of schools by the jurisdictional and Title I
    systems is not congruent, the discrepancy stems primarily from
    differences in the criteria used for identification. These
    differences in criteria, in turn, seem to derive from several
    inter-related factors: differences in the purposes of the two
    systems, differences in who (i.e., which unit or level) identifies
    the schools, and general problems in implementation and
    administrative coordination.

```{=html}
<!-- -->
```
-   Lack of specified consequences for Title I schools that fail to
    improve seems to contribute to a relatively low saliency for Title I
    accountability in these jurisdictions.

```{=html}
<!-- -->
```
-   Support provided to low-performing schools (identified by either
    Title I or jurisdictional accountability systems) varies
    substantially from jurisdiction to jurisdiction and often from
    school to school.

Conclusions (Section III)

Based on these findings, we drew several conclusions pertinent to Title
I accountability.

[Compliance]{.underline}: All of the jurisdictions were attempting to
comply with the requirements of Title I accountability and were doing so
primarily by trying to fold Title I accountability into
jurisdiction-wide systems in some way. Strategies ranged from complete
coherence of the systems (Kentucky), to negotiations for state Title I
acceptance of the jurisdictional system (San Francisco), to a division
of labor between the systems (Chicago).

[Coherence]{.underline}: Despite attempts to make the systems coherent,
Title I schools often appear to be subject to dual systems of
accountability. While not an immediate problem in most jurisdictions, we
expect the lack of congruence among the systems to increase in saliency
as the more severe sanctions for Title I accountability begin to be
implemented.

[The urban problem]{.underline}: The problem of low performance,
accountability, and assistance for schools is particularly acute in
urban areas. Schools in large urban districts are more likely to be
identified by state accountability systems, and the problem of low
performance is likely to be more pervasive in these districts. Special
provisions and attention will be required to address this problem.

Recommendations (Section III)

Although not a specific charge of this study, several recommendations
for reauthorization of ESEA emerged from this study. Briefly, these are:

1.  In reauthorizing ESEA, Congress should remove or revise those
    processes or provisions that foster the development of a dual system
    of school accountability for Title I schools.

2.  The reauthorized ESEA should place greater emphasis on ensuring that
    schools in need of improvement receive appropriate, consistent, and
    intensive assistance from districts and states.

### Several aspects of Title I accountability guidance regarding criteria and consequences should be strengthened, either by incorporating them directly into law or by working closely with states to foster their implementation. These include the use of multiple criteria for accountability decisions, the incorporation of disaggregated student assessment results into accountability criteria, and the implementation of corrective actions appropriate for the specific problems in identified schools. 

### Foreword

In 1996, the Office of Educational Research and Improvement (OERI) of
the U.S. Department of Education (ED) funded a study of reconstitution
as a remedy for school failure as part of the work of the Consortium for
Policy Research in Education (CPRE). During the first year of that
study, CPRE co-sponsored a national meeting of state and local policy
makers and educators who were actively engaged in designing or
implementing high stakes school accountability policies, including
school reconstitution and probation. Similar meetings of state level
educators had been convened or planned by other organizations, including
the Education Commission of the States and the Council of Chief State
School Officers. A pattern emerged in each of these meetings that caused
some concern among researchers and ED officials involved: throughout the
discussions, local and state policy makers and administrators made
little if any mention of Title I of the Elementary and Secondary
Education Act (ESEA). Criteria for school identification were not being
linked to emerging definitions of adequate yearly progress, nor was
Title I mentioned as a source of support in the low-performing schools
targeted for improvement. Given that Title I requires states and/or
districts to hold schools receiving Title I funds accountable for
improved student performance, and given the expectation that most of the
lowest performing schools identified by the states and districts
represented at these meetings were Title I schools, this lack of
explicit attention to Title I generated the following question for
investigation: what is the relationship between the systemwide
provisions for school accountability being developed by some states and
districts and the provisions for accountability required and being
implemented for Title I schools?

To address this question, the Planning and Evaluation Service (PES) of
the ED decided to extend the scope and funding for the CPRE study on
reconstitution. This extension was to focus on the congruence and
coordination of the state and local accountability systems with the
systems being established under the auspices of Title I, to look into
supports offered to schools identified by such systems, and to make
recommendations for strengthening accountability and support systems for
low-performing Title I schools. Originally, four jurisdictions -- two
states (Kentucky and Maryland) and two districts (Chicago and San
Francisco) -- were targeted for study. Each of these had established a
jurisdiction-wide, high stakes accountability system for its
low-performing schools. A fifth jurisdiction -- New York City-- was
added because of the relatively long history of the Schools Under
Registration Review (SURR) program in that state.

This report presents a brief review of the findings and recommendations
of the investigation of Title I accountability in relation to the state
and local accountability systems in these five jurisdictions. The
results of the parent study of school reconstitution, on which some of
the recommendations are based, will be available in Fall 1999.

**Contents**

[Page]{.underline}

Executive Summary i

Foreword iv

Section I. Context and Purposes 1

Study Purpose 2

Title I Accountability and IASA 3

High Stakes School Accountability: The Jurisdictional Systems 7

Section II. Title I Accountability in the Context of State and Local
Accountability Systems 18

Identifying Low-Performing Schools 18

Sanctions and Assistance for Low-Performing Schools 36

Section III. Discussion and Recommendations 43

Recommendations 45

Appendix A. Policy Profiles

Appendix B. Comparison Tables of Accountability Policies

.

## Section I. Context and Purposes

> "We cannot and must not tolerate failing schools. To meet the new
> expectations of the American people, we must confront the
> all-too-common problem of schools that are low achieving and even
> dangerous...If a school is bad and can't be changed, reconstitute it
> or close it down."
>
> Richard W. Riley, Feb. 18, 1997

In the mid 1990s, after over a decade of education reform, the public
and its policy-makers turned increasing attention to the problem of
persistently failing schools \-- schools that, even after multiple
reform efforts, remained at the bottom on measures of student
performance. Policy talk in states and districts began to focus on the
need for greater accountability and more drastic interventions in these
schools. Meanwhile, in educational and mass media, occasional reports of
such interventions, like the 1993 reconstitution of Rusk Elementary
School[^1] in Houston, began to appear.

As is often the case with education reform trends, the move from policy
talk and isolated action to full-blown policy systems occurred at
break-neck speed. San Francisco, which had reconstituted four schools as
part of a desegregation consent decree in 1983, began to do so on an
annual basis in 1994, preceding such action with a two-stage process of
identification and assistance. The following year, the Illinois
legislature approved a similar multi-stage process for Chicago, which in
turn shocked local and national educators by placing 109 of its 554
schools on probation in 1996, threatening them with reconstitution if
they did not show improvement.

Nor were state governments standing idly by while districts took up the
school accountability banner. By 1997, 22 states had passed "academic
bankruptcy laws" allowing them to intervene directly in low-performing
districts and schools. Kentucky was among the first such states, having
written a system of school rewards and sanctions for student performance
into their Kentucky Education Reform Act (KERA) as early as 1990; the
first

groups of schools were identified under those provisions in 1994. That
same year, Maryland started placing schools on an annual list of
reconstitution-eligible sites, while New York, Texas, and other states
developed their own versions of similar procedures.

Through these actions, states and districts were setting in motion
interventions and policies to attend to the problem of persistently
low-performing schools. By and large, those policies focus on holding
schools accountable for the performance of their students and
instituting consequences for continued failure to improve. In light of
this trend, it was not surprising when the federal government also upped
the ante on school accountability. It did so by instituting a key set of
provisions in the 1994 reauthorization of the Elementary and Secondary
Education Act requiring states to hold Title I schools and their
districts accountable for their students' progress toward meeting the
standards applicable for all students. While the notion that schools
receiving Title I funds should have to show progress in student
performance first appeared in the Hawkins-Stafford Amendments to ESEA in
1988; the 1994 reauthorization, with its emphasis on adequate yearly
progress and corrective actions for non-improving schools, was an
expansion and deepening of the earlier provisions.

## Study Purpose

Current attention to school-based accountability and the institution of
high stakes measures for low-performing schools thus derives from
several sources -- *state* legislation and policies; *local* education
agencies; and *federal* programs, particularly Title I. An obvious
question is to what extent these various systems for school
accountability are working together to improve achievement for all
students. That question was the motivation behind this study.

The study reported here focused on five jurisdictions that in the
previous few years had instituted high stakes school accountability
systems for the schools under their authority -- Kentucky, Maryland, San
Francisco, Chicago, and New York. Our goal was to see how these
jurisdictions were incorporating the Title I provisions into their own
systems.

-   Were the Title I systems and the local and state systems of school
    accountability coherent and compatible?

-   Was Title I being viewed and used as a resource for assisting
    low-performing schools -- or at least low performing Title I schools
    -- identified through the local and state systems?

-   And finally, what were some of the lessons we were learning about
    high stakes accountability systems that might be used to improve the
    accountability measures in Title I, particularly in preparation for
    reauthorization of ESEA in 1999?

Because policy coherence is a central goal of standards-based reform, we
paid particular attention to the degree of congruence between Title I
and the jurisdictional system.

[Data:]{.underline} This study was part of a larger investigation of
school accountability policies and practices in these and other
jurisdictions. Data collection for the larger study included document
review (policies, media and research reports, etc.), interviews and
meetings with state and local policy makers and administrators, site
visits and case studies of reconstituted schools. Additional interviews,
document review and analysis of school-level data were conducted for
this study in the five jurisdictions.

# Report organization: To address the questions above requires an understanding of the key provisions in Title I and in the jurisdictions in question. These provisions are outlined in the remainder of Section I. Section II then reports our findings on the congruence between the systems created by Title I and those established by the jurisdictions for all their schools. Section III discusses the findings and advances several recommendations for improving accountability provisions in the next reauthorization of IASA.

## Title I Accountability and IASA

Title I accountability can only be understood within the context of the
overall framework of the 1994 reauthorized ESEA. Four aspects of that
framework have particular significance for the purposes of this study.
They are: the centrality of standards, the emphasis on policy coherence,
the attention to professional development and capacity, and
performance-based accountability.

# Standards

Standards are the lynchpin of the 1994 reauthorization and represent a
major departure from previous practice in Title I/Chapter 1. The
underlying notion is that all children should have the opportunity to
achieve to the same high standards. This approach runs counter to the
conventional dual system with challenging curriculum and expectations
for some students and a watered-down remedial basic skills approach for
others. Too often in the past, Title I/Chapter 1 contributed to this
dual system by offering fragmented remedial instruction in pull-out
sessions taught by ill-prepared personnel[^2]. With the 1994
reauthorization, services to Title I students and schools are intended
to enable the children served by Title I to meet the challenging
standards established by the state for all its children. Moreover, the
legislation requires the states to institute such standards (content and
student performance standards) in at least reading and math by the
1997-98 school year; final state assessments aligned with those
standards are to be in place by 2000-01.

# Coherence

If standards are the lynchpin, coherence is the goal -- coherence among
the programs of the Act, coherence among the levels of governance
(federal, state, local), coherence of policy components (curriculum,
assessments, professional development and accountability), and coherence
in the educational program available for all students. Much of the
criticism of

Chapter 1 under previous versions of ESEA was the fragmentation that it
engendered, both in the actual program for students -- with the emphasis
on pull-outs and isolated skills -- and also the fragmentation in the
system as a whole\-- with all the separate accounting mechanisms and
personnel and the stove-pipe administration of services for Title I
students. The reauthorized ESEA was designed to address that
fragmentation and to promote coherence, primarily by helping to pull
together all aspects of the instructional program and institutional
organization to foster achievement of the state standards.

Several specific aspects of Title I reflect this emphasis on coherence.
One is the expansion of schoolwide programs and the move away from
pull-out services for individual students. Tied to this is the promotion
of consolidated planning, in which Title I and other federal monies are
to be seen as assistance for a school in developing its overall program
to meet the needs of its students and to foster their achievement of the
standards. The comprehensive assistance centers, which pull together
assistance for all the various titles in ESEA, are another indication of
this effort. And finally, the flexibility that states have in developing
measures and systems and programs that are consistent with their own
standards and the thrust of their educational program is yet another.
The availability of this flexibility is particularly evident in the
Ed-Flex states, which are able to waive a number of requirements in
federal statutes to enhance the coherence and effectiveness of their
overall efforts.

# Professional development and capacity building

A third integral aspect of ESEA is the importance placed on notions of
capacity-building and professional development in support for schools
and teachers. A key criticism of earlier versions of Chapter 1 was that
often the teachers responsible for instruction -- or in many cases the
aides responsible for instruction \-- to Chapter 1 students were
ill-prepared and unable to provide the kind and quality of instruction
that students needed. For this reason, an emphasis in recommendations
prior to the 1994 reauthorization and in the reauthorization itself was
on the need for professional development, encouraging schools and
districts to use Title I funds for professional learning -- not only
that of Title I teachers but of all teachers in the school, as any of
them may serve Title I students. The establishment of more comprehensive
technical support, and mechanisms like the school support teams to
assist schools in planning and developing their schoolwide programs is
another example of this emphasis. In addition, as will be noted,
provisions were included for providing additional assistance to Title I
schools that were experiencing difficulties. Indeed, schools identified
as in need of improvement are required to allocate at least ten percent
of their Title I funds over a two year period to professional
development.

# Performance accountability

It is the accountability provisions of Title I that are the most
directly central to this study. Unlike earlier iterations of Chapter
1/Title I, in which the thrust of the accountability provisions focused
on accounting for the use of Title I dollars, the thrust of
accountability in Title I since 1994 is on student performance. Thus,
states are required to hold districts (LEAs) and schools receiving Title
I funds accountable for ensuring that all students meet the state's
content and performance standards. The primary mechanism for doing so is
the state's definition of adequate yearly progress (AYP), which is based
on a model of continuous improvement toward the standards. According to
federal guidance for Title I, AYP "must be defined in a manner that --

-   Results in **continuous** and **substantial, yearly** improvement of
    each school and LEA sufficient to achieve the goal of all children
    served under Title I, particularly economically disadvantaged and
    limited-English proficient children, meeting the State's proficient
    and advanced levels of performance;

```{=html}
<!-- -->
```
-   Is sufficiently rigorous to achieve that goal within an appropriate
    time frame; and

```{=html}
<!-- -->
```
-   Links progress primarily to performance on the State's final
    assessment while permitting progress to be established in part
    through the use of other measures, such as dropout, retention, and
    attendance rates.[^3]"

The specific definitions of AYP in use by the five jurisdictions in this
study are summarized in text form in Appendix A of this report and in
tabular form in Appendix B.

![](media/image1.wmf){width="4.340972222222222in"
height="3.2583333333333333in"}Another central feature of Title I
accountability is the designation of schools and districts as "in need
of improvement" when they fail to achieve AYP for two consecutive years.
Such designation brings with it several requirements: the development of
an improvement plan, the allocation of funds for professional
development, provision of technical assistance, and \-- if failure to
improve continues \-- some form of corrective intervention. "Corrective
actions" are intended to take place in the third year following
identification. It should be noted, however, that the inclusion of
corrective actions is new to this reauthorization and the legislation
precludes states from taking the more severe actions listed in this
legislation\-- withholding funds, revoking schoolwide program authority
and decision-making structures, reconstitution, or authorizing student
transfers---until their final assessments and final definitions of AYP
are in place; that is, no later than 2000-2001. Again, the specific
features of the school improvement provisions in the five jurisdictions
are summarized in Appendices A and B.

High Stakes School Accountability: The Jurisdictional Systems

To examine the degree to which Title I is incorporated into the systems
in these five jurisdictions, it is necessary first to understand both
the general framework of accountability in the jurisdictions and some of
the particulars of the policies. Our larger study of reconstitution and
other high stakes accountability systems suggests the outlines of that
general framework. While noting that policies vary in important respects
from jurisdiction to jurisdiction across the country, we find them to
reflect some common assumptions about accountability and improvement
efforts in schools. Taken together, these assumptions form a generic
"theory of action" (Argyris and Schon, 1978; Weiss, 1995) common to all
jurisdictions in this study.

The elements of this theory of action are as follows:

-   *The accountability system should be aligned with educational
    goals*. If increased student achievement is the goal, then student
    achievement is what educators should be accountable for and what the
    system should monitor. The belief is that the accountability system
    will draw attention to the desired goals and focus effort on their
    achievement.

```{=html}
<!-- -->
```
-   *The school should be the basic unit of accountability*. Two beliefs
    support use of the schools as the unit of accountability. One is
    that student performance is cumulative and influenced by the entire
    school program and climate. A second is that a system of collective
    (school-based) accountability will encourage educators at the school
    to work together, thus increasing overall capacity and performance.

```{=html}
<!-- -->
```
-   *The threat of negative sanctions will motivate educators to work
    harder, thus improving student achievement*. The underlying belief
    here is that persistently low student performance stems primarily
    from educators' lack of will. Whether policymakers attribute this
    lack of will to institutional racism or to low expectations, to poor
    school leadership or to well-meaning teachers becoming discouraged
    by external conditions, the belief is that educators need some
    external push to do what is necessary to raise student achievement.
    Negative sanctions are to provide that push.

```{=html}
<!-- -->
```
-   *Accountability measures should be accompanied with capacity
    building*. Policy makers recognize that limited capacity (knowledge,
    skills, and resources) at the school also contributes to low
    performance. School probation policies, therefore, also provide
    assistance and professional development opportunities to help build
    needed capacity in the schools.

```{=html}
<!-- -->
```
-   *Goal-setting, planning, and monitoring results are critical to
    improvement*. The underlying belief is that very broadly defined
    goals do little to focus attention, suggest strategies, or provide
    feedback for improvement. Schools need specific, measurable goals
    and well-specified plans for achieving them or for modifying
    practice if they are not achieved. All high-stakes school
    accountability systems currently in use, therefore, have specified
    student achievement targets and include site based planning
    mechanisms for achieving those targets.

```{=html}
<!-- -->
```
-   *Some schools are simply beyond help and should be closed down,
    reconstituted, or taken over.* All the accountability systems
    discussed here incorporate some last resort measure to intervene and
    protect students when a school continues to fail after previous
    interventions. In Chicago and San Francisco, this last step involves
    the removal and replacement of school administrators and staff as
    well as some additional measures. In New York and Maryland, the last
    resort is mainly a governance change -- removing decision-making
    authority from the school and district (state takeover) -- or
    closure.

Reflecting these common assumptions, the policies in these five
jurisdictions nonetheless vary considerably in their particulars. We
summarize those particulars in the remainder of this section. It should
be noted that we chose these five jurisdictions in order to see both
local and state interaction with Title I. Two of the systems examined,
Chicago and San Francisco, are locally developed and implemented. Both
are relatively large urban districts with very diverse student
populations. Two of the others, Kentucky and Maryland, are statewide
systems of accountability in which the state holds individual schools
accountable for performance, with differing involvement of LEAs in the
process. The final jurisdiction, New York City, falls between these two
approaches in that it is a large urban district operating within a
statewide accountability system[^4].

The remainder of this section describes the school-based accountability
systems operating in each of the five jurisdictions. Each description
includes the designations of improvement and the sanctions used in the
jurisdictional system, the criteria employed for placing schools in
those designations, the support available to schools once they've been
identified, and the numbers of schools in the categories in 1997-98, the
year in which data for this study were collected.

# Kentucky

School-based accountability for student performance has been a central
component of the Kentucky system since passage of the Kentucky Education
Reform Act (KERA) in 1990. What is unique about the Kentucky system is
that for the past nine years schools have been judged solely on whether
their students were *making progress* (relative to the school's previous
performance) toward the achievement standards established by the state.
The model of continuous improvement underlying this system is highly
compatible with the concept of adequate yearly progress outlined in the
Title I legislation.

The instrument used for judging school performance in Kentucky is the
state assessment system, which until spring 1998 was KIRIS (Kentucky
Instructional Results Information System). (A new assessment \-- the
Kentucky Achievement Test \-- is currently under development.) The
system measures the progress of each school in the state, relative to
the school's earlier performance on KIRIS and to specified biennial
targets. Those targets are based on two assumptions: 1) that all
students in a school should be scoring at the "proficient" level; and 2)
that schools should be able to accomplish this goal within a twenty year
period. To achieve this ultimate goal the state established a series of
two-year performance targets for each cycle. These performance targets,
or "improvement goals," are calculated from a baseline score, defined as
the school's accountability index at the beginning of the two-year
cycle. The Improvement Goal is set at 10% of the difference between the
baseline and 100. Thus, the improvement goal for a school with a base
line accountability index of 50 would be 55 for that two-year cycle.

Obviously, under this system, low-performing schools have been expected
to make greater proportional gains in performance, as they were the
farthest from the goal at the outset of the program. Nevertheless, ALL
schools in Kentucky have been subject to accountability for their
individually specified targets. Those surpassing their improvement
targets have been eligible for rewards, regardless of their absolute
performance level. Similarly, both "high performing" and
"low-performing" schools have been subject to sanctions if they failed
to make the specified gains. Schools whose performance index falls
relative to their baseline at the beginning of the biennium, but whose
decline is less than five percentage points, are designated as "schools
in decline." "Schools in crisis" are those whose performance falls by
more than five points or who are in decline for two cycles. Schools are
re-designated every two years, based on their meeting or not meeting the
improvement targets for that biennium. Our focus for the purposes of
this study was schools in "decline" or "crisis."

Through spring 1998, assistance to all schools designated in decline or
crisis was provided in the form of a "Distinguished Educator," who
worked with the school over a two-year period. Distinguished Educators
trained the staff in how to use data for instructional decisions,
assisted in the school planning process, and helped to align curriculum
and instruction with state standards. Distinguished Educators were
highly effective teachers and administrators selected and trained by the
state and released from their regular duties for this two-year period.
In schools designated as "in crisis," Distinguished Educators also had
the authority to evaluate and replace school staff. This system of
support, like the state assessment, is currently being revised, based on
new state legislation in 1998.

At the time of this study (1997-98), 302 schools for the previous cycle
had failed to meet their improvement target. Eighty-two of these were
designated in decline, and nine were in crisis. The total of 91 decline
and crisis schools were located throughout the state and demonstrated
varying levels of absolute aggregate performance.

# Maryland 

Maryland's high stakes accountability system for low-performing schools
was instituted under the leadership and authority of state
superintendent Nancy Grasmick in 1994 but traces its origins to a 1989
gubernatorial commission that laid out a similar performance-based
system. Beginning with a belief that children have a right not to be
forced to attend a failing school by accident of where they live, state
policy allows the state Board to directly intervene in failed schools by
removing governance authority from the local district. This state
"take-over" of individual schools is called reconstitution. To date, no
schools in Maryland have been reconstituted in this manner. Instead, the
Maryland leadership have focused on declaring schools "Reconstitution
Eligible" and requiring local school systems to develop improvement
plans for those schools. A larger "alert list" of schools provides LEAs
with advance information about schools in need of assistance.

The criteria for identifying Reconstitution Eligible schools is the
state performance index (SPI) which is composed of aggregate assessment
scores on Maryland State Performance Assessment Program (MSPAP) or the
Maryland Functional Tests (high school), attendance rates, and drop out
rates (high school). The state has established a goal of 70% of students
at proficient levels of MSPAP and it targets for intervention those
schools that are well below that standard and declining in performance.
The number of schools targeted each year is based in part on the
perceived capacity of the SEA to provide assistance.

The assistance provided, however, is mainly the responsibility of the
local districts, each of which must also submit a plan for
improvement[^5]. Specific interventions for Reconstitution Eligible
schools vary and may include replacement of administrators and staff
(zero-based staffing) as well as restructuring and professional
development.

When the system was instituted in 1994, two Baltimore City high schools
were designated as Reconstitution Eligible. Since that time, 90 schools
have been named Reconstitution Eligible, 79 of them in Baltimore City.
The concentration of Reconstitution Eligible schools in Baltimore, in
conjunction with other legal and financial matters, contributed to a
legal settlement between the city and the state, known as the Baltimore
Partnership. The Partnership, which came into existence in 1997, infuses
additional funding into city schools and revamps the governance
structure of the school system, replacing the old Board of Commissioners
appointed by the mayor with a new board jointly appointed by the mayor
and the governor. At the time of this study Maryland had no established
criteria for removing schools from the Reconstitution Eligible
designation.

# Chicago

Prior to the 1995 revision of the Illinois School Code,[^6] the central
office of Chicago Public Schools lacked the authority to intervene in
low-performing schools. The education reform legislation passed by the
Illinois legislature in 1988 had decentralized that authority to Local
School Councils. Local School Councils are governing bodies elected by
the school community at each site and vested with the authority to hire
the principal and set the instructional and management policies for the
school. Reformers believed that such decentralization would create a
system that was more responsive to student needs and would lead to
higher achievement. For many Chicago schools, this governance change
worked well. Others, however, remained mired in failure. Faced with a
large number of persistently failing schools, the Illinois legislature
revised the statutes, establishing a Chief Executive Officer for Chicago
Public Schools, appointed by the mayor and vested with the authority to
intervene in low-performing schools. The resulting system of school
accountability was established under this legislation and the leadership
of the current CEO, Paul Vallas.

Chicago's accountability system involves three stages: remediation,
probation, and reconstitution. Reflecting the emphasis in Chicago, our
focus for this study was on the probation and reconstitution categories.
CPS places schools on probation if fewer than 15% of their students are
reading at grade level on the ITBS and removes them from probation when
20% of their students are reading at grade level. Reconstitution can
occur if a school on probation fails to improve performance. Actual
reconstitution decisions are made case by case, based on an evaluation
of progress and potential for progress in the school. If the school is
reconstituted, the school administration and staff are removed from
their positions. Given an opportunity to reapply for their jobs in that
school, the staff may be either rehired or reassigned and replaced.

CPS has established a fairly elaborate and multi-faceted system of
support and monitoring for probation and reconstituted schools. Each
school receives the services of a probation manager, who assists the
leadership in developing and monitoring the implementation of the school
improvement plan. In addition, each school contracts (with financial
assistance from CPS) with an external partner for on-going professional
development and capacity building services. The external partners are
mostly university-based groups with varying approaches to school
improvement. Reliance on external support providers derives, in part,
from a CPS desire to involve local universities in improvement efforts.
CPS administrators also believed, on the one hand, that support would be
better received from outside the system, and, on the other, that CPS
central office simply did not have the capacity to meet the needs of all
the schools requiring assistance.

In 1996, 109 (approximately one-fifth) of Chicago public schools were
placed on probation. In 1997-98 several schools were removed from the
list and others put on, so that during the academic year 115 schools
were on probation, while seven high schools were reconstituted[^7].

Responsibility for holding Chicago schools accountable for performance
does not rest solely with the district administration, however. Chicago
also exists within a larger state system of accountability, which
differs in important ways from the local. Like Chicago's system, the
state has three levels of designation for low-performing schools, but
the criteria for identification in the state system is the Illinois
Goals Achievement Program (IGAP), rather than the ITBS, and specific
consequences for continued failure to improve have not been spelled out.
The state requirements are summarized in Appendices A and B and will be
discussed in Section II as tied to definitions of adequate yearly
progress.

# San Francisco

San Francisco's system of school accountability derives from a 1983
desegregation consent decree and the 1993 amendment to that decree. In
1984, four San Francisco schools were "reconstituted" in an effort to
desegregate schools in the Bay View Hunter's Point section of the city
and to improve the education available to the city's African American
and Latino students. Based on advice from an expert panel convened by
the court to review the first ten years of the decree, the parties
amended the agreement to include a call for "annually reconstituting at
least three schools until the task is completed."

To implement the amended decree, the district, under the leadership of
the new superintendent Wildemar Rojas, introduced a two-stage process in
which low-performing schools were first identified for the Comprehensive
School Improvement Program (CSIP) and given a year to improve. At the
end of the academic year, CSIP schools could be continued in the
program, "graduated," or reconstituted.

The decision to place schools in CSIP was based on a 100-point index
comprised of 17 indicators of need and performance. Student achievement,
measured by the CTBS, played heavily in the identification process. The
decision to reconstitute a school after it had been in CSIP was based on
a set of evenly weighted quantitative and qualitative indicators,
including a visit to the school by the superintendent and his district
team. In San Francisco, when a school is reconstituted, the entire staff
is "vacated" and the new staff must agree to adhere to a set of
philosophical tenets designed to ensure high expectations for all
students. Emphasis in reconstituted schools is also placed on the use of
technology, flexible class sizes, professional development and parental
involvement to improve instruction and achievement.

Schools targeted by the consent decree are allocated additional
discretionary monies as part of the desegregation efforts. In addition,
some additional monies are available to CSIP and reconstituted schools,
whether or not they are among those targeted by the decree. Starting in
1997-98, these monies were held centrally and are allocated to the
schools for specific purposes, primarily professional development
activities. Schools have also had the opportunity to receive external
consulting, and in the 1997-98 academic year, the district instituted an
interdepartmental team to coordinate services and assistance to CSIP and
reconstituted schools.

Since 1994, 24 schools have been placed into CSIP; 14 of these have now
graduated from the list, 10 schools have been reconstituted. In May
1997, district Superintendent Rojas signed a tentative agreement with
the president of the local teachers union to develop alternatives to
reconstitution[^8]. Two schools have been reconstituted since that
initial agreement was signed; none have been placed in CSIP. A new
three-tiered accountability system has been planned for the 1998-99
school year, but to date no schools have been placed in that system.

# New York City and SURR

New York's state system of school accountability, Schools Under
Registration Review (SURR), was initiated in 1989, though a similar
program, the Comprehensive Assessment Report (CAR)[^9] had begun in
1985. The SURR program is predicated on a state requirement that all
schools must be registered to operate in New York and on the authority
vested in the state department of education to revoke the registration
of schools not meeting state standards. Currently, schools placed in the
SURR program have three years in which to meet state standards or be
de-certified (i.e., closed down or "redesigned" with a new school plan).
(Note: This time limit was instituted by State Commissioner Rick Mills
in 1996-97; schools identified before that time were given only two
years after this provision went into effect.)

Schools are identified for SURR primarily through test scores on state
assessments in reading and math in grades 3, 6, 8 (reading only), and
11. The state has established minimum performance standards on these
assessments and requires 90% test-takers in each school to score at or
above those standards. In addition, the schools' annual drop out rate
must be below 5%. As of 1996, the schools identified for SURR are those
determined to be "farthest from the state performance standards."
Additionally, schools that offer their students "a poor learning
environment" can also be identified for SURR. Three elements contribute
to the state's definition of a poor learning environment: poor
performance on state standardized tests; parental complaints to the
state; conditions such as violence, excessive absenteeism and
suspension; or a high percentage of uncertified teachers.

Once identified for SURR, schools are to receive both state and district
assistance in their improvement efforts. All schools are required to
develop a Comprehensive Education Plan (CEP). In an effort to promote
district responsibility and accountability for improvement in SURR
schools, the state has also required districts to sign off on the
school's CEP and, since 1996, to submit a district Comprehensive
Assistance Plan (CAP) for each SURR school in its jurisdiction. Support
for SURR schools varies substantially and has been characterized as a
"patchwork" consisting of a state liaison, district assistance, and
opportunities such as the AFT's Teacher Centers located in many New York
City schools or participation in various summer institutes and
initiatives[^10]. In New York City, twelve of the lowest performing SURR
schools receive concentrated assistance and monitoring through the
auspices of the Chancellor's District. Governance authority for these
schools was temporarily removed from the Community School Districts by
NYC Chancellor Rudy Crew under the authority granted to him by the state
legislature in 1996. The Chancellor's District has its own supervising
superintendent whose goal has been to get the schools off the SURR list
and to develop their capacity to stay off and continue improving.

Between the academic years 1989-90 and 1996-97, 139 schools had been
identified for SURR in New York state, with the vast majority of these
being located in New York City. In the last year of that period, 99
schools were still in SURR, of which 92 were in NYC. The 1997-98 school
year saw a similar pattern, with 98 schools throughout the state
identified for SURR, 94 of these in NYC. Of these 94, 28 had been
redesigned and were still designated SURR, and one was in the process of
closure. Meanwhile, 18 previously identified schools were removed from
the SURR list and 20 new schools added.

**Section II.** **Title I Accountability in the Context of State and
Local Accountability Systems**

This section presents the key findings of our investigation into the
relationship between Title I accountability and local/state
accountability systems in these five jurisdictions. Our findings fall
into three general categories: identification of low-performing schools;
assistance provided to identified schools; and consequences of continued
low performance. For each of these, our emphasis is on the issue of
coherence and coordination -- i.e., to what extent do state and local
systems work with Title I to produce a coherent approach to school
accountability and improvement? The section concludes with several
overarching observations about the degree of coherence and coordination
among the systems and presents recommendations for the ESEA
reauthorization in 1999.

## Identifying Low-Performing Schools

The criteria and processes for identifying and categorizing levels of
performance are central features of any performance-based accountability
system. Criteria serve to define not only the goals and targets of
performance, but success and failure as well. They communicate to
schools and the public the system's expectations and provide an
indication of who is to be held accountable. As discussed in Section I,
the primary unit of accountability in each of the systems in this study
is the school, though to varying degrees districts also enter in as
accountable units.

We were confronted with two main tasks in this analysis. One was to
examine the congruence among the policies governing Title I
accountability and those governing accountability for all schools in the
given jurisdiction. The second task was to look at the degree of
congruence in the results of those policies -- that is, in which schools
actually get identified. For the purposes of this report, we begin with
the second of these tasks. We then consider the criteria and processes
outlined in the policy as one set of explanatory variables for the
congruence or incongruence in the schools identified.

To address the issue of which schools get identified, we first obtained
lists of schools in each relevant category. For example, in Maryland, we
obtained from the state a list of all schools designated as
Reconstitution Eligible through spring 1998. We included the schools'
Title I status, grade range, location, and so forth. We then obtained
the names and locations of Title I schools and Title I schools
identified as in need of improvement. Since there is no consolidated
list of such schools for the state, this required compiling the data
from each county's Title I report. We followed a similar procedure for
each of the jurisdictions. From these data, we constructed a data set
for each jurisdiction that included Title I schools, schools identified
by the state and/or local accountability systems, and schools designated
as "program improvement" or "schools in need of improvement" by Title I.
Through this process, we were able to produce, we believe, a fairly
accurate picture of the incidence and overlap of schools in each
relevant category.

Two observations about these data are worth noting, however. First,
their collection was not what one might call straightforward. Title I
lists and state/local lists were not held in common in any jurisdiction
except Kentucky. Differing offices kept records of each group of
schools. Moreover, routes to individuals who could actually supply the
information was often circuitous -- particularly with respect to Title I
schools -- and these individuals were often unable to provide
information on policy processes or support systems. This pattern raises
concerns about the degree of administrative coordination of systems,
apart from or in addition to concerns about the particular schools
identified. Second, because information on the schools had to be
obtained from several sources, our conclusions may not jibe exactly with
the assumed relationship among the systems in any given jurisdiction.

Below, we present our findings from this analysis. We use Venn diagrams
to graphically (though approximately) display the patterns we found. We
address three central questions in this discussion:

1.  What proportion of schools identified in the state and local systems
    are Title I schools?

2.  What is the degree of overlap between schools identified by the
    Title I criteria and those identified by the state and local
    accountability systems?

3.  How might we explain the degree of congruence or non-congruence in
    the two systems?

**Finding One: The majority of the low-performing schools identified by
the state or local accountability systems are Title I schools. (And,
with the exception of Kentucky, the majority of identified Title I
schools were higher poverty schools implementing schoolwide programs.)**

The proportion of schools designated as low-performing (on probation,
Reconstitution Eligible, in decline, etc.) that are receiving Title I
funding is displayed graphically in

Figures 1a - 1e and in numerical form in Table 1. The proportion of
Title I schools ranges from 59% in Kentucky to 93% in Maryland.

Two observations are worth noting. The first is that the high proportion
of Title I schools identified by the state and local systems supports
the concern that instigated this study. That is, large numbers of Title
I schools are apparently subject to at least two systems of
accountability for student performance \-- that required by the Title I
provisions of ESEA and that of the state or local jurisdiction in which
the school is located \-- and those systems may or may not be sending
coherent messages to schools about the goals and targets of achievement.
Given the large numbers of Title I schools involved, it thus makes sense
to look further into the congruence and coordination among the systems.

**State Accountability and Title I Schools**

![](media/image2.wmf){width="4.001388888888889in"
height="3.0027777777777778in"}

![](media/image3.wmf){width="4.001388888888889in"
height="3.0027777777777778in"}

**Local Accountability and Title I Schools**

![](media/image4.wmf){width="4.001388888888889in"
height="2.995138888888889in"}

![](media/image5.wmf){width="4.001388888888889in"
height="2.995138888888889in"}

# ![](media/image6.wmf){width="4.001388888888889in" height="2.995138888888889in"}

#  {#section-1 .unnumbered}

# Table 1. Title I Schools Identified by Jurisdictional Accountability Systems 

+--------------------+-------+----------+--------+---------+---------+
| 1                  | 2     | 3        | 4      | 5       | 6       |
+--------------------+-------+----------+--------+---------+---------+
| [Jurisdic          | N     | Total    | Number | Pro     | Pro     |
| tion]{.underline}\ | umber | number   | of\    | portion | portion |
| \                  | of    | of       | Title  | ide     | of      |
| Accountability     | Title | schools  | I      | ntified | Title I |
| Category           | I     | id       | s      | by      | schools |
|                    | sc    | entified | chools | juris   | ide     |
|                    | hools | by       | iden   | diction | ntified |
|                    |       | juri     | tified | that    | by      |
|                    |       | sdiction | by     | are     | juris   |
|                    |       | (c)      | jurisd | Title I | diction |
|                    |       |          | iction | (e)     | (f)     |
|                    |       |          | (d)    |         |         |
+--------------------+-------+----------+--------+---------+---------+
| # Kentucky         | 897   | 523 (a)  | 302    | .58     | .34     |
|                    |       |          | (a)    |         |         |
| Improving, Decline |       | 91 (b)   |        | .59     | .06     |
| and Crisis (a)     |       |          | 53 (b) |         |         |
|                    |       |          | (9=sw) |         |         |
| Decline and Crisis |       |          |        |         |         |
| schools (b)        |       |          |        |         |         |
+--------------------+-------+----------+--------+---------+---------+
| # Maryland         | 415   | 90       | 84     | .93     | .20     |
|                    |       |          | (      |         |         |
| Recon              |       |          | 57=sw) |         |         |
| stitution-eligible |       |          |        |         |         |
+--------------------+-------+----------+--------+---------+---------+
| # Chicago          | 274   | 118      | 88     | .75     | .32     |
|                    |       |          | (      |         |         |
| Probation and      |       |          | 85=sw) |         |         |
| Reconstitution     |       |          |        |         |         |
+--------------------+-------+----------+--------+---------+---------+
| # New York City    | 884   | 94       | 86     | .91     | .10     |
|                    |       |          | (      |         |         |
| S.U.R.R.           |       |          | 47=sw) |         |         |
+--------------------+-------+----------+--------+---------+---------+
| # San Francisco    | 64    | 24       | 18     | .75     | .28     |
|                    |       |          | (      |         |         |
| CSIP and           |       |          | 13=sw) |         |         |
| Reconstitution     |       |          |        |         |         |
| (1993-98)          |       |          |        |         |         |
+--------------------+-------+----------+--------+---------+---------+

a.  Includes schools in Improving Categories 1 and 2 (schools that show
    improvement but do not reach their target), Schools in Decline, and
    Schools in Crisis

b.  Includes only schools in Decline or Crisis

c.  Total number of schools identified by the jurisdiction-wide
    accountability system as falling into low-performing categories
    named in column 1.

d.  Number of schools receiving Title I funds that are designated in
    categories named in column 1.

e.  Column 4 / column 3.

f.  Column 4 / column 2

sw = schoolwide

A second observation is that while the proportion of Title I schools is
substantial in all jurisdictions, it varies significantly from
jurisdiction to jurisdiction. Not surprisingly, Kentucky is the
jurisdiction with the lowest proportion of its identified schools
receiving Title I funds. The apparent reason for this is that Kentucky
bases its accountability system on a model of continuous improvement.
ALL schools are expected to show progress relative to past performance
and to established high standards. Since schools are evaluated based on
their own record, relatively high-performing, low poverty (and thus
non-Title I) schools in Kentucky are more likely to be picked up by the
accountability system and designated as needing improvement than are
such schools in systems emphasizing an absolute level of performance.

Meanwhile, San Francisco and Chicago have moderate proportions of Title
I schools identified by their local accountability systems. Here the
explanation seems to lie more with the level of schooling than the
identification criteria. In San Francisco, for example, of the six
non-Title I schools that were identified for CSIP between 1993 and 1997,
four were high schools. Proportionally, high schools in San Francisco as
in other parts of the nation, are less likely to be receiving Title I
funds than are elementary schools. In fact, only one of the 64 Title I
schools in San Francisco in 1997-98 was a high school, while 54 were
elementary schools. By contrast, of the elementary schools identified
for CSIP during this period, 100% were Title I.

Finding Two: There is considerable overlap between the schools
identified for improvement under Title I and those identified in the
jurisdictional systems. This overlap is not absolute, however, and
varies considerably among the jurisdictions.

Our next step in the analysis of the identification of schools was to
focus in on the overlap among Title I schools identified by the
jurisdictional systems and those identified for improvement in Title I.
Our findings are displayed graphically in Figures 2a-2d and in tabular
form in Table 2. Please note that there are no graph and no figures for
San Francisco. At the time of this study, San Francisco was still in
negotiation with the state about the criteria for identifying program
improvement schools in the district, so for 1997-98, there were no
schools identified for program improvement[^11].

For the other four jurisdictions, it is possible to consider the degree
of overlap among identified schools in two ways, both of which are
indicated in Table 2. One way is to consider the overlap (that is, the
intersection of schools identified by the jurisdictional and the Title I
systems) as a proportion of jurisdiction-identified Title I schools. The
question addressed by such an analysis is what proportion of Title I
schools identified in the jurisdictional accountability system were also
identified in the Title I system? Alternatively, one could consider the
overlap as a proportion of program improvement schools (i.e., what
proportion of schools identified for program improvement in Title I are
also identified for sanctions in the jurisdictional system?). Taking New
York City as an example, we see that in the first instance, 74% of the
Title I schools identified for SURR in New York City were also
identified for program improvement in 1997. From this perspective, the
overlap between the jurisdictional system and the Title I system is
quite large. However, if we take program improvement schools as our
point of reference, the overlap appears as a small sub-set; more
specifically, only 12% of program improvement schools in New York City
were also identified for SURR.

**State and Title I Accountability: The Overlap**

![](media/image7.wmf){width="4.001388888888889in"
height="2.995138888888889in"}

![](media/image8.wmf){width="4.001388888888889in"
height="2.995138888888889in"}

**Local and Title I Accountability: The Overlap**

![](media/image9.wmf){width="4.001388888888889in"
height="2.995138888888889in"}

![](media/image10.wmf){width="4.001388888888889in"
height="2.995138888888889in"}

![](media/image11.wmf){width="4.001388888888889in"
height="2.995138888888889in"}

Table 2. Identification of Schools by Jurisdictional and Title I
Accountability Systems

+--------------------+-------+----------+--------+---------+---------+
| 1                  | 2     | 3        | 4      | 5       | 6       |
+--------------------+-------+----------+--------+---------+---------+
| [Jurisdic          | N     | Pr       | Number | Pro     | Pro     |
| tion]{.underline}\ | umber | oportion | of\    | portion | portion |
| \                  | of    | of\      | Title  | of      | of      |
| Accountability     | sc    | Title I  | I      | impr    | ju      |
| Category           | hools | schools  | s      | ovement | risdict |
|                    | in    | id       | chools | schools | ion-ide |
|                    | Title | entified | in     | in both | ntified |
|                    | I     | for      | both   | prog    | Title I |
|                    | i     | improv   | pr     | rams^e^ | schools |
|                    | mprov | ement^d^ | ograms |         | that    |
|                    | ement |          |        |         | are     |
|                    |       |          |        |         | also in |
|                    |       |          |        |         | Title I |
|                    |       |          |        |         | improve |
|                    |       |          |        |         | ment^f^ |
+--------------------+-------+----------+--------+---------+---------+
| # Kentucky         | 302   | .34^a^   | 302^a^ | 1.0     | 1.0     |
|                    |       |          |        |         |         |
| Improving, Decline |       |          | 53^b^  | .18     | 1.0     |
| and Crisis ^a^     |       |          | (9=sw) |         |         |
|                    |       |          |        |         |         |
| Decline and Crisis |       |          |        |         |         |
| schools ^b^        |       |          |        |         |         |
+--------------------+-------+----------+--------+---------+---------+
| # Maryland         | 92    | .22      | 34     | .37     | .40     |
|                    |       |          |        |         |         |
| Recon              |       |          |        |         |         |
| stitution-eligible |       |          |        |         |         |
+--------------------+-------+----------+--------+---------+---------+
| # Chicago          | 65^c^ | .27      | 60     | .92     | .68     |
|                    |       |          |        |         |         |
| Probation and      |       |          |        |         |         |
| Reconstitution     |       |          |        |         |         |
+--------------------+-------+----------+--------+---------+---------+
| # New York City    | 552   | .62      | 64     | .12     | .74     |
|                    |       |          |        |         |         |
| S.U.R.R.           |       |          |        |         |         |
+--------------------+-------+----------+--------+---------+---------+
| # San Francisco    | 0     | 0        | 0      | N/A^g^  | N/A^g^  |
|                    |       |          |        |         |         |
| CSIP and           |       |          |        |         |         |
| Reconstitution     |       |          |        |         |         |
| (1993-98)          |       |          |        |         |         |
+--------------------+-------+----------+--------+---------+---------+

a.  Includes schools in Improving Categories 1 and 2, Schools in
    Decline, and Schools in Crisis

b.  Includes only schools in Decline or Crisis

c.  Number of Chicago Schools on *State* Title I Improvement List (State
    Warning List).

d.  Column 2 of table 2/ Column 2 of Table 1.

e.  Column 4/ Column 2, Table 2.

f.  Column 4 of Table 2/ Column 4 of Table 1.

g.  At the time of this data collection no San Francisco schools had
    been identified for Program Improvement due to continuing
    negotiations with the state.

sw = schoolwide

In Chicago the situation is somewhat reversed, with only 68% of
probation schools also on the state Title I improvement list (i.e., the
State Warning List)[^12] but 92% of Title I improvement schools on
probation. In Maryland the situation is different still, with only 37%
to 40% of either group being identified by the other. Compare this
limited degree of joint identification in Maryland with the total
congruence of the two groups in Kentucky; i.e., in Kentucky 100% of
program improvement schools are schools identified by the state system
as in the improving, decline or crisis categories, while 100% of Title I
schools identified as improving, decline, or crisis are also in program
improvement[^13]. When only decline and crisis schools are considered
(because of their similarity to probation or Reconstitution Eligible
designations), they form a completely circumscribed subset of the larger
group of Title I schools in need of improvement. Thus, 30% of schools in
need of improvement were in decline or crisis while 100% of decline or
crisis Title I schools were in school improvement.

How do we explain this significant variation among the jurisdictions?
What does it mean when a Title I school is identified for improvement by
one accountability system and not by the other? Understanding the
reasons behind this variation among jurisdictions can provide insights
into how the jurisdictions are approaching Title I accountability in the
context of their own accountability structures.

Finding Three: Where identification of schools by the jurisdictional and
Title I systems is not congruent, the discrepancy stems primarily from
differences in the criteria used for identification. These differences
in criteria, in turn, seem to derive from several inter-related factors:
differences in the purposes of the two systems, differences in who
(i.e., which unit or level) identifies the schools, and general problems
in implementation and administrative coordination.

Our understanding of the reasons for the discrepancies in schools
identified by the two types of systems is based on analyses of the
specific policies in each jurisdiction and on interviews with state and
district personnel. (Summaries of the policies may be found in Appendix
A; tables categorizing the criteria, support and consequences for Title
I and jurisdictional accountability in each site are in Appendix B.) The
most obvious and pervasive reason for the discrepancies in each case is
that the two systems are using differing criteria.

Chicago and San Francisco, for example, use completely different
assessments in their local accountability systems than those used to
determine adequate yearly progress for Title I in their states (ITBS vs.
IGAP and CTBS vs. Stanford 9 respectively). Moreover, even where the
same assessments are used, results may be defined and interpreted
differently in the two systems. In New York, for example, the main
criteria for SURR is that the schools are *farthest from* standard on
the state assessments; the criteria for Title I improvement is that they
are not making *adequate progress* on those same assessments. Finally,
systems may differ in the additional criteria they include with the
assessments. San Francisco, for example, bases its identification for
CSIP on a 100 point index derived from 17 indicators, including student
performance, need indicators, grade retentions, and staff stability.
According to one local official, the system is neither standards- nor
progress-based, both requirements for Title I accountability. While
other district administrators would no doubt disagree with that
characterization, there is no denying that the constellation of criteria
used by San Francisco is unique to that jurisdiction and differs
substantially from AYP in other parts of the state.

While the details and specific stories vary by jurisdiction, the use of
differing criteria can be traced to several underlying factors:
differing purposes of accountability systems, differing levels of
governance responsible for identification, and implementation issues.

# Differing purposes/targets of accountability

The most obvious example of differing purposes for Title I and
jurisdictional accountability is that of New York, where Title I SURR
schools (those farthest from state standards) are largely a subset of
the larger group of schools not making adequate yearly progress. This
pattern does not derive from a decision to relate the systems in that
way, however, as is the case in Kentucky. In Kentucky, Title I schools
in decline or crisis (the designations which bring to bear the
Distinguished Educators and the threat of more serious sanctions) are
the subset of the total group of Title I schools in need of improvement,
but they are so as part of a unified, coherent, and multi-level system.
In New York, there is no clear relationship between SURR and Title I
accountability; it is simply that the state has chosen to target the
schools in most severe straights for state intervention. Meanwhile,
Title I accountability remains a separate system, targeting schools on a
broader basis.

# Differing levels of governance

The most common contributing factor to the differing criteria among
systems is the level of governance at which identification is defined
and made -- that is, the state vs. the local district. Again, Kentucky
avoids this problem -- this time by essentially cutting the local units
out of the decision-making process, as all identification is made by the
state, based on state tests and state-defined criteria.

By contrast, local districts in New York, Maryland, and California play
a considerable role in identifying which Title I schools are in need of
improvement. This role is consistent with federal guidelines for Title
I: while states define adequate yearly progress, LEAs identify schools
in need of improvement. Moreover, the state may allow LEAs to add local
assessments or develop their own assessment systems to be incorporated
into the definition of AYP and designation of schools needing
improvement.

In these three states, the district role in identifying Title I schools
in need of improvement leads to some interesting patterns. Consider
Maryland (Figure 2b) as a more pronounced case, where the combination of
differing levels of analysis and differing criteria at the local level
results in a relatively small portion of schools being concurrently
identified by both accountability programs. In the first place, the
Maryland SEA only *recommends* that districts identify for improvement
those Title I schools with a negative change score on the
state-determined School Performance Index (SPI). In practice, while LEAs
place strong emphasis on MSPAP and the SPI, they also consider other
factors such as mobility or relative position among the county's
schools. This means that the criteria for Title I identification will
differ from county to county. No such variation occurs in the state
accountability system.

Secondly, *all* counties select at least some Title I schools for
improvement, generally the schools most in need *relative to others in
the district*. In contrast, Reconstitution Eligible schools are
concentrated almost exclusively in Baltimore City (79 of 90) and to a
lesser extent in Prince George's County (9 of 90). This is because the
Reconstitution Eligible designation is reserved for those schools that
are at the lowest levels of performance (and declining) *relative to all
schools in the state.* Since most such schools are in Baltimore City and
PG County, nearly all (88 out of 90) RE schools are located in these two
districts. This helps to explain the graph in Figure 2b, as Title I
improvement schools in other counties fall outside the RE circle on the
graph. In addition, Baltimore City has chosen to target only a subset of
its RE Title I schools for Title I program improvement. This explains
the large area of the Reconstitution Eligible circle in Figure 2b that
lies within Title I but outside program improvement. The end result is
that only 37% of school improvement schools are also Reconstitution
Eligible and only 40% of Reconstitution Eligible Title I schools are
designated as improvement schools.

Maryland is a case where the locally determined Title I accountability
system is only somewhat congruent with the statewide system. San
Francisco and Chicago represent a different type of local-state
disjuncture. In the case of these two cities, the jurisdiction-wide
accountability system is local, based on local needs, goals, and
context. The criteria for these systems grew out of those local contexts
and exists independently of Title I accountability. In San Francisco,
the system was created to respond to a court-ordered desegregation
consent decree. It pre-dates the 1994 reauthorization of Title I, and
district administrators have been negotiating with the state to accept
the local system for Title I requirements. Similarly, in Chicago,
assessments used for probation and reconstitution determination differ
from the state tests used for AYP. While the overlap of schools
identified locally for probation and schools identified by the state
Title I office is considerable, it is not absolute. Moreover, because
Chicago has a well-developed local improvement program for probation
schools, they have chosen to define higher-performing "borderline"
schools for improvement under Title I. Defined as a sort of "division of
labor" between the local accountability system and Title I
accountability, the accommodation results in the targeting of a much
larger number of schools for assistance efforts. It also results in the
set of "program improvement" schools being completely outside the set of
probation schools (see Figure 3).

![](media/image1.wmf)

# Implementation issues

# A final explanation of differing criteria and results rests in two sets of implementation issues. One issue is that of timing. On the one hand, Title I accountability is meant to be phased in as other provisions are phased in. For this reason, the definition of AYP in most jurisdictions has been in flux over the past few years[^14], and sanctions for persistent failure to improve have yet to be determined (see below). On the other hand, some state and local systems pre-date Title I accountability -- that is, they have an institutional history that may not be entirely compatible with the specific provisions of Title I.

Added to the timing issue is a second set of implementation problems
which center on the continuation of fragmented, stovepipe Title I
administration in most jurisdictions. While there

has clearly been progress in these five jurisdictions in making the
systems more coherent, there exists, as indicated in Section I, a
continuing division of responsibility and knowledge between Title I and
the regular program.

[In either case, the result is that schools may be subject to differing
tests and differing sets of criteria for Title I and for their
local/state accountability systems. To the extent that the legislation
and guidance specify a separate system and criteria for Title I, the
federal government appears to be contributing to such dual
systems.]{.underline}

## Sanctions and Assistance for Low-Performing Schools

Findings 4 and 5 focus on the consistency and cooperation in sanctions
and support available to schools identified in the jurisdictional and
Title I programs.

**Finding Four: Lack of specified consequences for Title I schools that
fail to improve seems to contribute to a relatively low saliency for
Title I accountability in these jurisdictions**.

Although Section 1116 of ESEA requires districts and states to take
corrective action in program improvement schools that fail to make
sufficient progress after three years, significant consequences are not
expected to take effect until after the next authorization of ESEA, or
at least not before 2000. As a result, many jurisdictions are still in
the process of defining what those consequences and corrective steps
will be.

When we asked respondents in these five sites about consequences, none
were mentioned apart from the requirement that schools identified as in
need of improvement develop a school improvement plan. Either the
respondent did not know that consequences were required or they did not
mention them because at the time they were not yet defined and in place.
In either case, we had no indication that the potential consequences
called for in the legislation had any salience, either for the
administrators or for the schools.

By contrast, the jurisdiction-wide systems of accountability in these
five sites had very clearly delineated consequences. Those consequences
varied for jurisdiction to jurisdiction, as indicated in section 1 and
the tables in Appendix A. But in all cases, the potential for
reconstitution, closure, or takeover loomed as a threat over
low-performing schools; what's more, the specific nature of the threat
in that jurisdiction was well known to the individuals with whom we
spoke. We concluded that the consequences outlined in the jurisdictional
systems carried with them a saliency that was lacking in the more
diffuse (and perceived low stakes)

Title I systems.

This difference underscored our impression of separate systems in all
but Kentucky (and, at the state-level only, Illinois). It is unclear,
how or when the Title I consequences will be defined. With the exception
of Kentucky and Illinois, there is no automatic movement of Title I
schools into the higher sanctions of the jurisdictional systems.

Finding five: Support provided to low-performing schools (identified by
either

Title I or jurisdictional accountability systems) varies substantially
from jurisdiction to jurisdiction and often from school to school.

Our data on specific support to Title I schools are limited, but overall
demonstrate the following patterns:

[1. Much of the support in each jurisdiction is focused on helping
schools develop and implement an improvement plan that is consistent
with relevant requirements and responsive to particular school
needs]{.underline}.

The most promising example of the planning process and assistance seemed
to be in Kentucky, where consolidated plans were instituted at all
levels in the 1997-98 school year. Schools identify needs based on
assessment results, analyze causes of problems, and specify actions to
be taken. Title I, along with other resources, is allocated
strategically to address the needs of the school and district, not to
establish a separate program. In addition, a major focus of the
Distinguished Educators working with schools in decline or crisis has
been to assist them in using data to assess needs and in developing
plans that will align curriculum and instruction with state standards.

Chicago and San Francisco also provide schools with assistance in the
planning process. In Chicago, this assistance comes in large part from
the probation manager assigned to each school on probation. An important
part of the probation manager's role is to help develop the corrective
action plan, followed by monthly monitoring of the plan's
implementation.

Similarly, all CSIP or reconstituted schools in San Francisco have an
assigned liaison from the central office, whose job is to serve as an
information conduit and support provider. In the case of the five
reconstituted elementary and middle schools, this liaison is the
principal of a successful reconstituted middle school. This individual,
who works with the schools both individually and collectively through
regular meetings of the principals, identified four central foci in his
work in 1997-98. One of these was helping with the school improvement
plans and another was budgeting. In both areas, Title I was an important
point of discussion and assistance. One indication of the importance
placed on Title I in this group came in just the second meeting of these
principals, held before the start of the 1997-98 school year. At this
meeting, the district Title I coordinator presented information on Title
I resources, regulations, and options, offering her assistance to the
identified schools. She reported to us that this was the first time she
had been invited to attend such a meeting of school principals in her
several decades as Title I coordinator.

While the planning process is also emphasized in Maryland and New York
policies, no specific or consistent forms of assistance in planning were
mentioned by respondents from these jurisdictions[^15].

It should also be noted that anecdotal information from each of the
jurisdictions, as well as our case studies of reconstituted schools,
suggests that the effectiveness of the assistance provided for the
planning process varies considerably from school to school, depending
upon the capacity of the specific assistance provider and the
relationship established with the school leadership. A more thorough
investigation of that variation, however, was beyond the scope of this
study[^16].

[2. A variety of other forms of professional development and capacity
building are available to schools identified through the accountability
systems. The primary source of the support, however, is the
jurisdiction, with Title I assistance appearing somewhat peripheral and
sporadic.]{.underline}

Again, Kentucky and Chicago have the most *extensive* support -- in
Kentucky through the Distinguished Educators (DEs) and in Chicago
through the external partners (EPs). In both cases, the primary support
is that provided by the jurisdiction-wide system; it is also site-based
and extensive. The DEs in KY work with schools over a two-year period.
The external partners have annual contracts, but schools are encouraged
to stay with their external partners for multiple years. In Kentucky,
the DEs have received common training and take a fairly common approach
to schools, particularly regarding standards and assessments. Chicago's
external partners, by contrast, represent a variety of approaches and
services from restructuring into small schools to curriculum development
to coaching with individual teachers on pedagogical strategies and basic
skills instruction. In neither case was there much specific mention of
Title I in the context of these assistance efforts, though in Kentucky
ESEA regional assistance centers are notified when Title I schools enter
decline or crisis. Meanwhile, CPS officials specifically downplayed the
usefulness of assistance from the regional comprehensive assistance
center or school support teams, primarily because of their limited
experience with Chicago's urban context.

This is not to say that Chicago does not have specific assistance for
schools in need of improvement. Part of the rationale for the decision
to designate "borderline" schools as Title I "program improvement"
schools while leaving probation schools to local efforts was to make
more rational use of assistance available through Title I resources.
Thus, CPS has a separate strategy of program improvement for
non-probation, low-performing Title I schools that includes
administrator training and degree programs as well as professional
development activities for teachers.

Two of the most *intensive* examples of assistance to identified schools
occurred in NYC. One of these was the assistance offered PS 198 by the
leadership of Community District 2, and the other was the capacity
building efforts of the Chancellor's District, composed entirely of SURR
schools. These two cases share many characteristics. In both, assistance
was multi-faceted and multi-level, involving coordinated and persistent
efforts from multiple providers for both administration and teachers.
Moreover, both are examples of the district leadership assuming
responsibility for the success of identified schools. Both placed
primary emphasis on developing the capacity of school leaders and
teachers to engage in high quality instruction. Both worked closely with
teacher union leaders, and in both cases, all involved schools were high
poverty, Title I schoolwide project schools. Yet these two examples also
demonstrate an important difference. In District 2, the strategy used
for PS 198 was simply an extension and intensification of the existing
approach to school improvement employed throughout the district. By
contrast, the Chancellor's District represents a change in governance,
initiated precisely because the existing strategies of the relevant
community school districts were judged insufficient to ensure success of
their SURR schools (see profiles below).

## 

## 

## 

## 

## 

## 

## 

## 

## 

## 

## 

## 

## Section III. Discussion and Recommendations

How should we interpret these findings on the congruence of Title I
accountability measures and those of states and local districts?

We would stress again the caveats we expressed earlier: in this study we
are looking at only a few jurisdictions and those with established
systems. We are not sure what our findings mean for states that had not
already established or were moving toward similar accountability systems
at the time ESEA went into effect. Nonetheless, with respect to these
five jurisdictions, several conclusions are worth noting.

[1. All of the jurisdictions were attempting to comply with the
requirements of Title I accountability and were doing so primarily by
trying to fold Title I accountability into jurisdiction-wide systems in
some way]{.underline}.

This was most easily and completely accomplished in Kentucky, which had
an established, multi-level system of school accountability based on
progress toward state standards and assessments. Since there already
were standards in place and the state used a model of continuous
improvement, Kentucky's system was completely compatible with Title I
legislation and guidance. Moreover, no district variation exists, as
each LEA uses exactly the same system as the state.

Other states, Maryland, New York, Illinois---and now California, used
the same assessments as the basis for both Title I and the statewide
accountability system. However, where districts in those state
incorporated local assessments or other measures into Title I
identification criteria, the result may be the targeting of different
schools for Title I and statewide accountability

With regard to the two district-based accountability systems in this
study (San Francisco and Chicago) we also see an attempt to fold in
Title I, in this case, by negotiating with the state to use the local
accountability criteria as the definition of AYP for their schools.
Chicago has experienced some success in this effort resulting in a
division of labor: the Chicago Public Schools Office of Accountability
provides assistance and monitoring for probation schools (many of which
are on the state Title I list) while the local Title I administrator
provides assistance for borderline and "remediation" schools, as part of
the locally defined "Program Improvement" program. San Francisco was
having more difficulty in their negotiations with the state, but this
appeared to be moving toward resolution at the time of data
collection[^17].

[2. Despite attempts to make the systems coherent, Title I schools often
appear to be subject to dual systems of accountability.]{.underline}
Thus far in the implementation of ESEA, the existence of these dual
systems does not seem to be a major problem, but this may be due in part
to the fact that consequences for Title I have not yet really kicked in.
Title I accountability and corrective action, therefore, do not have
high saliency in these sites and thus do not divide school or
administrators' attention. It is conceivable that if the dual systems
continue, and if corrective actions (such as alternative governance and
reconstitution) are applied to Title I schools based upon current
definitions of AYP, schools could face similar high stakes consequences
coming from differing sources using differing criteria. We could find no
clear advantages to such a situation and can anticipate substantial
disadvantages in the long run. Overall, this pattern of dual
accountability seems to run counter to the general intent of ESEA to
foster coherence in educational programming for all students and
schools. Moreover, to the extent that the legislation calls for a
specific system of accountability for Title I schools, one could argue
that it contributes to the creation of such dual systems.

[3. The problem of low performance, accountability, and assistance for
schools is particularly acute in urban areas.]{.underline} Consider the
following findings: in New York City in 1998, 552 Title I schools were
identified for program improvement and 96% of state SURR schools were
also in New York City. In Maryland, 88% of Reconstitution Eligible
schools are in Baltimore City, making up half of the total number of
Baltimore schools. In Chicago, one-fifth of the schools had fewer than
15% reading at grade level on a test of basic skills in 1996, and in San
Francisco, ten of its 114 elementary, middle and high schools have been
reconstituted in the past four years, while almost a quarter have been
identified for CSIP.

Recognizing this high need, urban districts have been developing their
own accountability systems -- or in the case of Baltimore City having
accountability thrust upon them. Any Title I system instituted,
therefore, must recognize the special problem of accountability and
performance in the urban areas.

# **Recommendations**

Neither our intent nor our charge in this study was to develop a
comprehensive set of recommendations for improving accountability
provisions in Title I. Nonetheless, two central recommendations from
this work are almost unavoidable; they concern the need to guard against
the establishment of dual (jurisdictional and Title I) systems of
accountability and the need to greatly improve the assistance provided
to schools in need of improvement. We have also included a third set of
suggestions for strengthening the implementation of several elements
already in the legislation or guidance but which are too rarely
implemented by the states.

1\) [Fostering coherence]{.underline}

Based on our findings and analysis for this study, our central
recommendation concerns the need to foster greater coherence in school
accountability requirements.

More specifically, we suggest the following:

-   To receive Title I funds, states must have in place (by a given
    time) a system for holding schools and districts accountable for all
    children making progress toward meeting challenging state standards.
    [This system must apply to all schools, including those covered by
    the act. Once that system is in place, no separate performance-based
    accountability system should be applied to Title I schools on the
    basis of their Title I status.]{.underline}

```{=html}
<!-- -->
```
-   In the interim (while statewide systems are being developed), a
    temporary accountability system for districts and schools receiving
    Title I funds should be employed, similar to the current provisions
    for accountability under ESEA.

```{=html}
<!-- -->
```
-   If districts develop their own performance-based accountability
    systems, these should replace or be incorporated into the state
    systems so that schools are not subject to multiple accountability
    criteria and processes stemming from implementation of Title I
    provisions.

2\) [Building Capacity]{.underline}

Based both on this investigation and on the parent study, we believe
that limited instructional capacity is a critical factor in persistent
low performance and we see a need to vastly improve assistance provided
Title I schools in need of improvement.

More specifically, we suggest the following:

-   Changes to technical assistance provisions of ESEA should be based
    on a comprehensive review of the technical assistance and
    professional development currently provided thorough Title I and
    ESEA (a task clearly beyond the scope of this study).

-   Assistance under ESEA should be consistent with and integrated into
    other local and state-based assistance efforts, rather than under
    separate authority and administration.

```{=html}
<!-- -->
```
-   Districts should be held accountable for ensuring the progress of
    low-performing schools in their jurisdiction. We recommend that LEAs
    be required to submit a plan for assisting Title I schools
    identified in need of improvement along with the improvement plan
    submitted by each school. The LEA plan should include an examination
    of district policies that contribute to school failure and a
    strategy for addressing them.

```{=html}
<!-- -->
```
-   Greater resources should be targeted toward ensuring the improvement
    of teaching and teacher quality in low-performing Title I schools --
    including allocating more resources for professional development of
    teachers, administrators, and paraprofessionals and for the
    recruitment of more qualified staff into these schools. Emphasis
    must be placed on *instructional* improvement consistent with state
    standards in these assistance efforts.

3\) [Strengthening criteria and consequences]{.underline}

More specifically, we recommend that the accountability systems
developed by the states, in accordance with Title I provisions:

-   Incorporate multiple criteria for decisions about schools and
    districts. While current law requires multiple measures in final
    state assessment systems, in practice many jurisdictions rely on a
    single test for identifying schools in need of improvement. This
    pattern weakens the legitimacy of the system and raises issues of
    consequential validity when used for high stakes decisions such as
    probation and reconstitution.

```{=html}
<!-- -->
```
-   Disaggregate results by relevant sub-groups or performance levels in
    the school and require improvement for sub-categories of students as
    well as for the school as a whole. One disturbing pattern noted by a
    number of observers and participants in these systems is a tendency
    for some schools to concentrate improvement efforts on a certain
    segment of the student population, such as those on the cusp of a
    proficiency category or those most easily reached. By doing so,
    school personnel may increase the relevant measure of aggregate
    school performance (proportion of students scoring at or above a
    certain cutoff in some jurisdictions, the mean level of achievement
    in others) and thus be removed from threat of sanctions. Such
    practice, however, does not help to ensure progress of *all*
    students in the school toward the state and local standards.
    Requiring improvement among relevant sub-groups of students would
    help to ensure equity in accountability provisions.

```{=html}
<!-- -->
```
-   Finally, we support the incorporation of corrective actions into the
    school accountability systems. We have seen in these jurisdictions
    how sanctions seem to increase the saliency of an accountability
    system and to focus attention on improvement targets. However, we
    would also urge that requirements for corrective actions allow for
    some degree of flexibility in their design and use, based on local
    evaluation of the causes of poor performance in a given school and
    determination of the most effective means of addressing those
    causes. We also suggest, as a way of increasing district
    accountability, that districts provide a clear rationale for how the
    given action responds to the needs of a particular school and its
    students.

**Appendix A:**

##### Policy Profiles

**California**

# *Adequate Yearly Progress and State Accountability (AYP)*

The definition of adequate yearly progress has been in flux for the past
few years.

1996-97 was the first year the state imposed requirements on AYP. The
state required all schools to have 40% of its students at grade level to
satisfy adequate yearly progress. However, the schools were not advised
of the criteria until after the school year was out. To meet the state's
demand the districts had to reconstruct scores from the assessments they
did perform to evaluate the grade level status of their students. Due to
the late information and lack of appropriate assessments, several
districts did not report any information on the adequate yearly progress
of their schools.

In April 1998 the state established a new set of accountability
requirements. The state indicated that the Stanford 9 assessment must be
a component in the evaluation of grade level. With this exam the 50^th^
percentile will be the cut off for proficient. However, schools are
allowed to consider alternative assessments for students that score
below the 50^th^ percentile. Districts could also establish their own
measures as long as they were approved by the state.

Additionally, the state decided to include an improvement component to
the accountability system. They established 90% meeting or exceeding
grade level on assessments based on standards as the target to be met in
eight years. Schools are required to meet the average yearly improvement
necessary to reach the goal in eight years. A member of the state
department of compensatory education indicated that the interim state
assessments (Stanford 9) need to be changed. This year the Stanford 9
was used with the 50^th^ percentile mark as grade level. Schools were
still offered the option to use other assessments to evaluate the grade
level of students that fell below the 50^th^ percentile. The state
published guidelines for the use of alternative measures. Next year the
state will likely use the augmented SAT/9 with some alignment to state
content standards. She also remarked, "It is likely that our state board
of education will assign NAEP proficiency levels to scores on the
augmented SAT/9."

# *Support for Program Improvement Schools*

Schools can apply for funding or receive technical assistance from a
variety of sources. First, schools can apply for Title 1 set aside funds
for school improvement. To be eligible, the school must submit a plan
for improvement. Schools can receive assistance in professional
development from the regional network, a statewide system of school
support. Also, schools can receive assistance from the federally
supported Comprehensive Assistance Center.

**Illinois**

### *Adequate Yearly Progress and State Accountability*

Illinois instituted the Academic Early Warning List as a system of
school improvement for all schools in the state. This list identifies
schools in two categories: "Does Not Meet", also referred to as Level 1,
and "Declining." If a school has more that 50% of all Illinois Goal
Assessment Program (IGAP) student scores in the school not at standard
for two years consecutively, the school is identified in the "Does Not
Meet" category. If a school increases the number of below standard
students by 20% or more three years following the benchmark year, they
are classified as "Declining."(Note: schools can be identified as
declining even though more than 50% of the school's students meet
standard.)

Once identified as a Level 1 "Does Not Meet" school the state requires
the school to meet or exceed standards, or continue to make adequate
yearly progress until the goal is met. Schools that meet the standard
are removed from the warning list. Otherwise, the school will remain on
the list as long as the school makes adequate yearly progress. If a
school fails to meet the standard and does not make adequate yearly
progress after two years on the warning list, the school is placed on
the academic watch list.

Schools classified as "Declining" can be placed on the academic watch
list if the school has declined such that more than 50% of its students
do not meet standard for two consecutive years. Declining schools lose
this designation when the percent of IGAP scores at or above standard
equals the average percent of scores at standard for the previous 4
years.

Adequate yearly progress is defined in terms of progress toward a goal
of 50% IGAP scores at standard over five years. Therefore, adequate
yearly progress is satisfied if a school increases the percent of IGAP
scores meeting standard by one fifth of the difference between the
benchmark and 50%. Despite this seemingly strict formula, a
representative in the state board of education indicated that in
practice meeting adequate yearly progress is less strict. She explained
that the state performs a rather subjective evaluation of progress.
Schools generally just need to report their progress and what the school
has done to raise scores.

Probation/Reconstitution policy in the Chicago Public Schools (CPS) also
relies heavily on test scores. However, Chicago also examines attendance
rates and drop out rates in high schools. Identification of failing
schools is made at the district level. Schools are first identified for
remediation based on IGAP scores. The second level of identification is
probation. The basic criterion for probation is performance on the Iowa
Test of Basic Skill (ITBS) and Tests of Academic Proficiency (TAP). If
less than 15% of a school's students achieve national norms, the school
is subject to probation. The district also considers attendance rates,
dropout rates, and IGAP scores for borderline schools. Finally, schools
that the district determines have not progressed, or declined during
probation are identified for reconstitution.

State and CPS criteria for entering the accountability process appear to
be aligned in two respects. First, IGAP scores are used to identify
schools for academic warning and remediation. Second, CPS recognizes
removal from the state academic watch list as one criterion for removal
from the probation list.

## *Support for Program Improvement Schools*

The state requires all schools on the academic early warning list to
develop a School Improvement Plan (SIP) that must be submitted to the
local school board or local school council (LSC) and the state
superintendent. To assist schools in this process the state board of
education provides a staff member to work with the school. In Chicago
the probation member serves as the contact with the state instead of an
Illinois state board of education (ISBE) staff member. Schools also
qualify for targeted school improvement funds from the state. Finally,
every two years a panel of outside educators, parents, and community
members will evaluate the schools' services, the impact of these
services on student learning, student work, and achievement data. This
process is known as a Quality Assurance External Review.

Title I schools that fall on the Academic Warning list receive all of
these services as program improvement schools. There was very little
discussion of school support teams (SSTs).

## Kentucky

### *Adequate Yearly Progress and State Accountability*

Title I accountability is based on the statewide accountability system.
All Title I schools identified as improving, improving category 2,
decline, and crisis are identified as improvement schools. Until this
year only crisis schools had staff placed on probation.

The schools are rated by the accountability index. The index score of
100 implies that 100% of students are proficient, there was no
retention, no drop-outs, and 100% of students successfully transitioned
to adult life. Clearly, 100 is the optimal goal. In reality, schools are
indexed at the beginning of a biennium and this index is identified as
the baseline. Schools then identify an improvement goal. Schools that
meet or exceed the improvement goal are identified as successful or
reward schools. Schools that do not meet the goal but improve are
identified as improving or improving category 2. Schools that decline
are identified in decline or crisis. Prior to the most recent
legislation, schools that fell more than five points on the index were
identified in crisis.

### *Assistance to Improvement Schools*

All schools in decline, in crisis and improving category 2 receive the
assistance of state Distinguished Educators (DEs). DEs provide
assistance in developing and implementing the SIP. Also, these schools
qualify for Commonwealth School Improvement Funds. Title I schools
receive all of these services but also regional school support teams are
notified of their status. These SSTs assist in developing the SIP and
improvements. However, their involvement is less intensive that DEs.
Other than the SST services, Title I schools receive the same assistance
as non-Title I schools.

**Maryland**

# *Adequate Yearly Progress and State Accountability*

The state relies on the School Performance Index (SPI) to track the
success of schools. Elementary Schools are at standard when on average
70% of the school's students score satisfactory on the content exams and
attendance is at 94%. The standards for a middle school are 94%
attendance, 70% satisfactory on the Maryland School Performance and
Assessment Program (MSPAP), and 95%, 85%, and 90% on reading, math, and
writing respectively on the Maryland Functional Test (MFT). The
standards for a high school are 94% attendance, 97% retention, and 95%,
80%, 90%, 85%, 90% scoring satisfactory on the MFT reading, math
writing, citizenship and all 11^th^ grade exams respectively. It should
be noted that attendance, retention, and average outcomes on exams are
weighted equally in the SPI.

The state suggests that all districts identify schools at standard and
schools showing substantial or sustained progress toward the standards
as meeting adequate yearly progress. The state also recommends that all
schools with a negative change in the index should be identified for
school improvement. However, Local education agencies (LEAs) identify
schools in their districts. Discussions with some LEAs revealed that not
all schools with a negative change index are identified. Also, some
schools that show a positive change index may be identified if the
school is particularly needy. In identification LEAs appear to place a
strong emphasis on the MSPAP and SPI. However, they may choose other
variables such as mobility, relative position among the county's
schools, or severity of decline. The limited sample suggests that a
uniform standard across the state does not exist.

Baltimore City schools focus on the School Performance Index as the
basis for identifying schools that fail to meet adequate yearly
progress. The director of compensatory and funded programs for the
Baltimore City Public Schools stated that " All schools with a negative
SPI are identified as potential needs improvement sites." Furthermore,
he indicated that only schools with a negative SPI are considered
potential "Title I schools in need of improvement." However, schools
with a negative SPI may not be identified if indicators from other
achievement data or special circumstances suggest that the school should
not be identified.

Prince George's County's process of identifying schools for Title I
school improvement makes direct reference to the state's reconstitution
policy. The primary criteria determining adequate yearly progress are
the MSPAP and the Comprehensive Test of Basic Skills. All Reconstitution
Eligible schools are identified for Title I school improvement. Schools
with a negative SPI are identified for improvement if they are
identified as Reconstitution Eligible, 21^st^ Century, Alert, or Title I
schools not meeting adequate yearly progress..

Identification as a Reconstitution Eligible school is made at the state.
Schools that may be identified as Reconstitution Eligible are below
standard and declining even after developing and implementing a SIP.
Currently, only 90 schools are identified as Reconstitution Eligible
schools.

# *Support for Identified Schools*

Apparently, districts in the sample discussed above help schools develop
and implement SIPs. St. Mary's county also provides resource people to
help in the schools. Counties such as Anne Arundel have identified a
county alert list on which all the improvement schools fall. All schools
on this alert list receive assistance. There was almost no talk of
school support teams. The former director of compensatory programs
explained that the state is developing a new school support team
comprised primarily of retired teachers and administrators. However,
this SST is not yet established and is only intended to work with
schoolwide programs whether they are improvement schools or not.

# **New York**

#### Adequate Yearly Progress and State Accountability

In 1996 the state targeted 2001 as the year all schools should reach the
regent's standard. This standard is 90% of students in a school reaching
the state benchmark. Given this state target, adequate yearly progress
is defined as progress to reduce the gap between the 1996 mark and 2001.
Schools with base line in 1996 and 1995 are expected to reduce the gap
by 20% each year beginning in the 1996-97 school year. Schools with a
base line from 1997 must reduce the gap by 25% each year. All schools
that achieve the 90% standard are expected to maintain the standard to
satisfy adequate yearly progress. A publication from the Division of
Assessment and Accountability states that " schools which fail to meet
their adequate progress target or fall below 90% threshold are to be
identified in need of improvement."

The Schools Under Registration Review (SURR) process also targets 90% of
students achieving the state standards. However, the state also places a
minimum requirement on drop out rates, and considers the whole school
environment. First, the state expects that the drop out rate be no
higher than 5%. Second, the state identifies schools with a poor
learning environment. Three elements contribute to the state's
definition of a poor learning environment. These elements include 1)
poor performance on state standardized tests, 2) parent complaints to
the state, 3) conditions such as, high violence, excessive absenteeism
and suspension, or high percentage of uncertified teachers. Schools
identified for SURR are those furthest from the minimum standards or
schools identified as a poor learning environment.

SURR and Title I accountability processes incorporate the same state
assessments. However, New York City (NYC) uses local assessments for
Title I identification. Since SURR schools are identified by the state,
local assessments are not incorporated.

## *Support for Program Improvement Schools*

Title I improvement schools receive assistance from the community school
district and NYC Title I school support teams (SSTs). However, SSTs
focus their attention on schoolwide programs. Schoolwide programs
identified for improvement receive a very high priority for intensive
assistance. This assistance comes in the form of:

1.  Schoolwide planning

2.  Developing assessments aligned with state standards

3.  Capacity building

4.  Intensive on site assistance

**Appendix B:**

**Comparison Tables of Accountability Policies**

**Illinois**

STATE IDENTIFICATION

+-----------------+-----------------+-----------------+-----------------+
|                 | Academic\       | Declining\      | Academic\       |
|                 | Warning List    | Schools         | Watch List      |
+-----------------+-----------------+-----------------+-----------------+
| Criteria        | Less than 50%   | The percent of  | Failure to meet |
|                 | of students in  | students        | adequate yearly |
|                 | the school meet | meeting         | progress for    |
|                 | state           | standard        | two consecutive |
|                 | standards.      | declined by 20  | years           |
|                 |                 | or more percent |                 |
|                 |                 | over three      |                 |
|                 |                 | years.          |                 |
+-----------------+-----------------+-----------------+-----------------+
| Consequences    | Required to     | Required to     |                 |
|                 | develop a plan  | develop a plan  |                 |
|                 | for             | for             |                 |
|                 | improvement.    | improvement.    |                 |
|                 | Schools are     | School is       |                 |
|                 | expected to     | expected to     |                 |
|                 | make adequate   | raise the       |                 |
|                 | yearly progress | percent of      |                 |
|                 | or meet the 50% | students        |                 |
|                 | level.          | meeting         |                 |
|                 |                 | standard to a   |                 |
|                 |                 | level equal to  |                 |
|                 |                 | the average     |                 |
|                 |                 | level for the   |                 |
|                 |                 | previous 4      |                 |
|                 |                 | years.          |                 |
+-----------------+-----------------+-----------------+-----------------+
| Support         | 1.  State board | 1.  State board |                 |
|                 |     provides a  |     provides a  |                 |
|                 |     staff       |     staff       |                 |
|                 |     member to   |     member to   |                 |
|                 |     assist in   |     assist in   |                 |
|                 |     developing  |     developing  |                 |
|                 |     improvement |     improvement |                 |
|                 |     strategies. |     strategies. |                 |
|                 |                 |                 |                 |
|                 | 2.              | 2.              |                 |
|                 |   Participation |   Participation |                 |
|                 |     in a        |     in a        |                 |
|                 |     Quality     |     Quality     |                 |
|                 |     Assurance   |     Assurance   |                 |
|                 |     External    |     External    |                 |
|                 |     Review      |     Review      |                 |
+-----------------+-----------------+-----------------+-----------------+

**Illinois cont.**

CHICAGO IDENTIFICATION

+-----------------+-----------------+-----------------+-----------------+
|                 | Remediation     | Probation       | Reconstitution  |
+-----------------+-----------------+-----------------+-----------------+
| Criteria        | Low test scores | Less than 15%   | Failure to      |
|                 | on IGAP         | meeting         | improve after   |
|                 |                 | national norms  | two years on    |
|                 |                 | on ITBS and     | probation       |
|                 |                 | TAP, and        |                 |
|                 |                 | failure to      |                 |
|                 |                 | improve under   |                 |
|                 |                 | remediation     |                 |
+-----------------+-----------------+-----------------+-----------------+
| Consequences    | Must develop a  | Must develop a  | 1.  School is   |
|                 | plan for school | plan for school |     re-staffed. |
|                 | improvement     | improvement     |                 |
|                 |                 |                 | 2.  Must        |
|                 |                 |                 |     develop a   |
|                 |                 |                 |     plan for    |
|                 |                 |                 |     school      |
|                 |                 |                 |     improvement |
+-----------------+-----------------+-----------------+-----------------+
| Support         | Assistance      | Provided a      | Provided a      |
|                 | developing the  | probation       | probation       |
|                 | improvement     | manager and     | manager and     |
|                 | plan            | external        | external        |
|                 |                 | partner to      | partner to      |
|                 |                 | assist in       | assist in       |
|                 |                 | developing and  | developing and  |
|                 |                 | implementing    | implement the   |
|                 |                 | the improvement | improvement     |
|                 |                 | plan            | plan            |
+-----------------+-----------------+-----------------+-----------------+

# 

# **Kentucky**

+-----------+-------------+-------------+-------------+-------------+
|           | Improving   | Improving   | Decline     | Crisis\*    |
|           |             | Category 2  |             |             |
+-----------+-------------+-------------+-------------+-------------+
| #         | Improvement | The         | The         | The         |
|  Criteria | was less    | school's    | school's    | school's    |
|           | than the    | improvement | index fell  | index fell  |
|           | goal for    | in the      | below the   | more than 5 |
|           | the present | present     | benchmark   | points      |
|           | cycle.      | cycle was   | for the     | below the   |
|           |             | less than   | current     | index for   |
|           |             | the goal    | cycle, but  | the present |
|           |             | AND the     | the decline | cycle.      |
|           |             | school was  | was less    |             |
|           |             | identified  | than 5      |             |
|           |             | improving,  | points.     |             |
|           |             | or decline  |             |             |
|           |             | in the      |             |             |
|           |             | previous    |             |             |
|           |             | cycle.      |             |             |
+-----------+-------------+-------------+-------------+-------------+
| Con       | 1.          | 1.          | 1.          | 1.          |
| sequences |  Identified |  Identified |  Identified |  Identified |
|           |     for     |     for     |     for     |     for     |
|           |     Title I |     Title I |     Title I |     Title I |
|           |     school  |     school  |     school  |     school  |
|           |             |             |             |             |
|           | improvement | improvement | improvement | improvement |
|           |             |             |             |             |
|           | 2           | 2           | 2           | 2           |
|           | .  Required | .  Required | .  Required | .  Required |
|           |     to      |     to      |     to      |     to      |
|           |     develop |     develop |     develop |     develop |
|           |     a SIP   |     a SIP   |     a SIP   |     a SIP   |
|           |             |             |             |             |
|           |             |             |             | Teachers    |
|           |             |             |             | are placed  |
|           |             |             |             | on          |
|           |             |             |             | probation.  |
+-----------+-------------+-------------+-------------+-------------+
| A         | Regional    | 1           | 1           | 1           |
| ssistance | school      | .  Regional | .  Regional | .  Regional |
|           | support     |     SST     |     SST     |     SST     |
|           | teams       |             |             |             |
|           | (SSTs) are  |    notified |    notified |    notified |
|           | notified of |     for     |     for     |     for     |
|           | Title I     |     Title I |     Title I |     Title I |
|           | schools     |     schools |     schools |     schools |
|           | identified  |             |             |             |
|           | for         | 2           | 2           | 2           |
|           | i           | .  Assigned | .  Assigned | .  Assigned |
|           | mprovement. |     a DE    |     a DE    |     a DE    |
|           | SSTs        |             |             |             |
|           | provide     | 3           | 3           | 3           |
|           | assistance  | .  Eligible | .  Eligible | .  Eligible |
|           | with SIP    |     for     |     for     |     for     |
|           | and         |     School  |     School  |     School  |
|           | im          |             |             |             |
|           | provements. | Improvement | Improvement | Improvement |
|           |             |     Funds   |     Funds   |     Funds   |
+-----------+-------------+-------------+-------------+-------------+

\*Note: Crisis category no longer exists under the new legislation.

SIP = School Improvement Plan

DE = Distinguished Educator

**Maryland**

+-----------------+-----------------+-----------------+-----------------+
|                 | Title I School  | Reconstitution  | State Take-over |
|                 | Improvement     | Eligible        |                 |
+-----------------+-----------------+-----------------+-----------------+
| Criteria        | 1.  Title I     | Identified by   |                 |
|                 |     school      | the state as a  |                 |
|                 |                 | school that is  |                 |
|                 | 2.  Failure to  | below standard  |                 |
|                 |     meet or     | and fails to    |                 |
|                 |     show        | make progress   |                 |
|                 |     substantial | or declines     |                 |
|                 |     and         | after           |                 |
|                 |     sustained   | developing and  |                 |
|                 |     progress    | implementing a  |                 |
|                 |     toward the  | SIP             |                 |
|                 |     standard of |                 |                 |
|                 |     70%         |                 |                 |
|                 |                 |                 |                 |
|                 |    satisfactory |                 |                 |
|                 |     on MSPAP    |                 |                 |
|                 |     and other   |                 |                 |
|                 |     elements    |                 |                 |
|                 |                 |                 |                 |
|                 |    incorporated |                 |                 |
|                 |     in the SIP  |                 |                 |
+-----------------+-----------------+-----------------+-----------------+
| Consequences    | Required to     |                 | Operation of    |
|                 | develop a SIP   |                 | the school is   |
|                 |                 |                 | handed over to  |
|                 |                 |                 | an external     |
|                 |                 |                 | agent.          |
+-----------------+-----------------+-----------------+-----------------+
| Assistance      | Varies by       |                 |                 |
|                 | district        |                 |                 |
+-----------------+-----------------+-----------------+-----------------+

###### SIP = School Improvement Plan.

**New York**

+------------+----------------+-------------------+------------------+
|            | School         | SURR              | Redesign         |
|            | Improvement    |                   |                  |
+------------+----------------+-------------------+------------------+
| Criteria   | Failing to     | 1.  Schools       | Failure to       |
|            | meet the 90%   |     identified by | Improve after    |
|            | standard and   |     the state as  | SURR             |
|            | failure to     |     farthest from | identification   |
|            | make 20%       |     the standard. |                  |
|            | improvement    |                   |                  |
|            | toward the 95  | 2.  Schools       |                  |
|            | standard       |     identified as |                  |
|            |                |     a "poor       |                  |
|            |                |     learning      |                  |
|            |                |     environment"  |                  |
+------------+----------------+-------------------+------------------+
| Co         | Required to    | 1.  Districts are | 1.  School must  |
| nsequences | develop a      |     required to   |     close and    |
|            | Comprehensive  |     develop a     |     re-open      |
|            | Action Plan    |     corrective    |     under a new  |
|            |                |     action plan   |     school plan. |
|            |                |     that outlines |                  |
|            |                |     their support |                  |
|            |                |     of SURR       |                  |
|            |                |     schools.      |                  |
|            |                |                   |                  |
|            |                | 2.  Schools must  |                  |
|            |                |     develop an    |                  |
|            |                |     improvement   |                  |
|            |                |     plan with the |                  |
|            |                |     district.     |                  |
|            |                |                   |                  |
|            |                | 3.  District is   |                  |
|            |                |     required to   |                  |
|            |                |     designate a   |                  |
|            |                |     special       |                  |
|            |                |     liaison to    |                  |
|            |                |     work with the |                  |
|            |                |     SURR schools. |                  |
|            |                |                   |                  |
|            |                | 4.  Schools are   |                  |
|            |                |     identified as |                  |
|            |                |     Corrective    |                  |
|            |                |     Action        |                  |
|            |                |     Schools       |                  |
|            |                |     following     |                  |
|            |                |     once the      |                  |
|            |                |     plans have    |                  |
|            |                |     been          |                  |
|            |                |     approved.     |                  |
+------------+----------------+-------------------+------------------+
| Support    | 1.  Assistance | 1.  State         |                  |
|            |     for        |     provides      |                  |
|            |     community  |     registration  |                  |
|            |     school     |     review teams  |                  |
|            |     district   |     to assess     |                  |
|            |     to develop |     needs of the  |                  |
|            |     a local    |     school and    |                  |
|            |     assistance |     resources     |                  |
|            |     plan (LAP) |     needed.       |                  |
|            |                |                   |                  |
|            | 2.  NYC school | 2.  State         |                  |
|            |     support    |     provides a    |                  |
|            |     teams      |     set of        |                  |
|            |     assist     |     liaisons to   |                  |
|            |     with       |     help          |                  |
|            |                |     interpret     |                  |
|            |    school-wide |     regulations.  |                  |
|            |     programs.  |                   |                  |
|            |                | 3.  Continued     |                  |
|            |                |     monitoring of |                  |
|            |                |     progress for  |                  |
|            |                |     up to 3       |                  |
|            |                |     years.        |                  |
+------------+----------------+-------------------+------------------+

[^1]: Olson, Lynn (1995). Starting Over. *Teacher Magazine*, 6. 38-43.

[^2]: According to the National Assessment for Chapter 1.

[^3]: Taken from the Title I Guidance on Standards and Accountability
    section III provided by the U.S. Department of Education

[^4]: Note Illinois and California have or are developing statewide
    systems of school accountability which will interact with the
    locally designed systems in Chicago and San Francisco. The focal
    systems for this study, however, were those devised and implemented
    by the local district. Interactions with state systems will be noted
    as appropriate. New York City by contrast developed specific
    measures for responding to the designation of its schools as SURR
    schools. Thus the primary accountability system for New York City
    was that of the *state.*

[^5]: A policy slated for implementation in July 1999 will provide
    greater guidance and assistance to schools that have been in Program
    Improvement for three consecutive years. These schools will be
    required to develop and implement a program that incorporates the
    nine points of a researched based design introduced by the
    Comprehensive School Reform Demonstration Program. Schools will be
    provided assistance from the state Title I office and can solicit
    assistance from external agents.

[^6]: 1995 Amendatory Act to the Illinois School Code; Public Act 89-15.

[^7]: Currently 91 schools are on probation and no new schools have been
    reconstituted.

[^8]: The Trust Agreement remained as of March 1999.

[^9]: Ascher, C., Ikeda, K., and Fruchter, N. *School on Notice: A
    policy study of New York State's 1996-97 Schools Under Registration
    Review Process*, Institute for Education and Social Policy, New York
    University, 1998.

[^10]: See Ascher et al. 1998

[^11]: From statements made in March 1998 by Dr. Mary Welsh Byrd,
    Director of State and Federally Funded Projects of the San Francisco
    Unified School District. See footnote 17.

[^12]: As we will see, the *district* list for program improvement is
    another matter entirely (see Figure 3).

[^13]: The "improving category" is comparable to a "watch list" in many
    states. These are schools that failed to meet their improvement
    targets but did not decline in performance. By definition, Title I
    program improvement schools are all Title I schools falling in the
    improving, decline or crisis categories.

[^14]: In fact, the Title I guidance suggests that states use the
    transitional period to experiment with different definitions of AYP.

[^15]: See note 3 for changes in Maryland's assistance and improvement
    planning policy.

[^16]: Several studies of such assistance are currently underway. One is
    an OERI field initiated study of School Improvement Plans in schools
    on probation(or a similar designation) in three jurisdictions. That
    study is housed at the University of Maryland and is under the
    leadership of Rich Mintrop and James Cibulka.

[^17]: No resolution emerged, however, and as of March 1999 no plan for
    identifying program improvement schools in San Francisco had been
    approved by the state. Nonetheless in summer 1998 the Title I office
    designated fourteen schools for program improvement for the upcoming
    school year and a preliminary list for 1999-2000 shows 32 candidates
    for program improvement assistance.