Archived Information

  ------------------------ ------------------------------------------------------
  ![](media/image1.jpeg)   [Planning and Evaluation Service]{.smallcaps}

                           
  ------------------------ ------------------------------------------------------

[The Longitudinal Evaluation]{.smallcaps}

[of School Change and]{.smallcaps}

[Performance (LESCP)]{.smallcaps}

[in Title I Schools]{.smallcaps}

[FINAL REPORT]{.smallcaps}

[Volume 1: Executive Summary]{.smallcaps}

2001

![](media/image2.png){width="1.2361111111111112in" height="1.25in"}

  -----------------------------------------------------------------------
  [U.S. Department of Education \~ Office of the Deputy
  Secretary]{.smallcaps}

  [Doc #2001-20]{.smallcaps}
  -----------------------------------------------------------------------

the longitudinal evaluation of school change\
and performance (LESCP)\
in title I schools

FINAL REPORT

VOLUME i: EXECUTIVE SUMMARY

Prepared for:

U.S. Department of Education

Office of the Deputy Secretary

Contract No. EA 96008001

Prepared by:

Westat, Rockville, Md.

and

Policy Studies Associates, Washington, D.C.

2001

This report was prepared for the U.S. Department of Education under
Contract No. EA96008001. The project monitor was Daphne Hardcastle in
the Planning and Evaluation Service. The views expressed herein are
those of the contractor. No official endorsement by the U.S. Department
of Education is intended or should be inferred.

**U.S. Department of Education**

Rod Paige

*Secretary*

**Office of the Deputy Secretary**

William D. Hansen

*Deputy Secretary*

**Planning and Evaluation Service**

Alan L. Ginsburg

*Director*

**Elementary and Secondary Education Division**

Ricky Takai

*Director*

July 2001

This report is in the public domain. Authorization to produce it in
whole or in part is granted. While permission to reprint this
publication is not necessary, the citation should be the following: U.S.
Department of Education, Office of the Deputy Secretary, Planning and
Evaluation Service, *The Longitudinal Evaluation of School Change and
Performance in Title I Schools, Volume 1: Executive Summary*,
Washington, D.C., 2001.

To order copies of this report, write

ED Pubs

Editorial Publications Center

U.S. Department of Education

P.O. Box 1398

Jessup, MD 20794-1398;

via fax, dial (301) 470-1244;

or via electronic mail, send your request to <edpubs@inet.ed.gov>.

You may also call toll-free 1-877-433-7827 (1-877-4-ED-PUBS). If 877
service is not yet available in your area, call 1-800-872-5327
(1-800-USA-LEARN); Those who use a telecommunications device for the
deaf (TDD) or a teletypewriter (TTY), should call 8-800-437-0833.

To order online, point your Internet browser to
[www.ed.gov/pubs/edpubs.html](http://www.ed.gov/pubs/edpubs.html).

This report is also available on the department's web site at
[www.ed.gov/offices/OUS/PES/eval.html](http://www.ed.gov/offices/OUS/PES/eval.html).

On request, this publication is available in alternative formats, such
as Braille, large print, audiotape, or computer diskette. For more
information, please contact the Department's Alternate Format Center at\
(202) 260-9895 or (202) 205-8113.

Contents

[Acknowledgments]{.smallcaps} v

[Overview]{.smallcaps} 1

[Background]{.smallcaps} 3

[Key Findings]{.smallcaps} 8

[Conclusions]{.smallcaps} 16

Figures

1 Poverty level of schools in the LESCP sample (n=71) 4

2 LESCP scores relative to national and urban norms for closed-ended

reading 8

3 Effect of poverty on LESCP student achievement on the SAT-9 in

reading, third to fifth grade 9

4 Effect of poverty on LESCP student achievement on the SAT-9 in

mathematics, third to fifth grade 10

5 LESCP student achievement gains on the SAT-9 closed-ended reading

test, third to fifth grade 11

6 LESCP-predicted score on the SAT-9 closed-ended reading test,

third grade 12

7 LESCP student achievement gains on the SAT-9 closed-ended

mathematics test, third to fifth grade 14

8 LESCP-predicted score on SAT-9 closed-ended mathematics test,

third grade 15

9 Relationship between policy environments and favorable instructional

practices 17

Acknowledgments

We are indebted to many individuals whose contributions made the
Longitudinal Evaluation of School Change and Performance (LESCP)
possible. The project extended over 5 years and had many components and
phases. Here we can only mention a few of the individuals who had a role
in the design, conduct, and reporting of LESCP. We are especially
grateful to our federal Project Officers Elois Scott and Daphne
Hardcastle and to Audrey Pendleton and Jeffery Rodamar who served in an
acting capacity. They provided invaluable substantive guidance, as well
as support on the administrative and operational side. We wish to thank
Alan Ginsburg, director of the Planning and Evaluation Service (PES),
whose ideas and questions helped us formulate the research design, study
questions, and analyses for LESCP. Ricky Takai, director of the
Elementary and Secondary Division of PES, asked the hard questions as
the study and analyses progressed and kept us on course. Other staff of
PES who contributed in various capacities over the life of LESCP are
Stephanie Stullich, Joanne Bogart, and Barbara Coates. From the
Compensatory Education Programs office, Mary Jean LeTendre and Susan
Wilhelm provided thoughtful advice. We also thank Valena Plisko and
Michael Ross of the National Center for Education Statistics for their
helpful expertise.

Members of the LESCP Technical Work Group were active contributors to
the study effort from the very beginning. They provided advice on the
conceptual framework and research design, reviewed and advised on plans
for analysis, and ultimately on the analysis itself. We wish especially
to thank them for their reviews and thoughtful comments and
recommendations during the development of this report. Members of the
Technical Work Group are identified below. Each member served for the
entire study period except where noted.

Technical Work Group Members

Dr. David Cordray, Department of Psychology and Human Development,
Vanderbilt University

Dr. Judith McDonald, (1998--2000) Team Leader, School Support/Title I,
Indian Education, Oklahoma State Department of Education

Dr. Andrew Porter, Wisconsin Center for Education Research, School of
Education, University of Wisconsin-Madison

Dr. Margaret Goertz, Consortium for Policy Research in Education

Dr. Mary Ann Millsap, Vice President, Abt Associates

Dr. Jim Simmons, Program Evaluator for the Office of Student Academic
Education, Mississippi Department of Education

Dr. Joseph F. Johnson, Charles A. Dana Center, University of Texas at
Austin

Ms. Virginia Plunkett, (1997--98) Colorado Department of Education

Dr. Dorothy Strickland, Graduate School of Education Rutgers University

A large number of Westat and subcontractor staff contributed to LESCP.
Linda LeBlanc of Westat served as project director and Brenda Turnbull
of Policy Studies Associates (PSA) as co-project director. From Westat,
Alexander Ratnofsky, Patricia Troppe, William Davis, Ann Webber, and
Camilla Heid served on the analysis and reporting team. Raymond Olsen,
Stephanie Huang, Bahn Cheah, Alan Atkins, Sandra Daley, and Cyril
Ravindra provided systems support for analysis and survey operations.
Juanita Lucas-McLean, Therese Koraganie, and Dawn Thomas handled all
survey data collection and processing. Heather Steadman provided
secretarial support, Carolyn Gatling provided word processing support,
and Arna Lane edited this report.

At PSA, Megan Welsh played a major role in the study's analysis and
reporting. Other PSA staff members who also made substantial
contributions to the study are Joelle Gruber, Ellen Pechman, Ullik Rouk,
Christina Russell, and Jessica Wodatch.

Jane Hannaway of the Urban Institute coordinated the teams of site
visitors from the Urban Institute and along with Nancy Sharkey assisted
with analyses of some of the early study data.

Everett Barnes and Allen Schenck organized and oversaw the work of RMC
Research Corporation staff from that company's Portsmouth, Denver, and
Portland offices over three cycles of site visits and data collection
for LESCP.

The analyses of student achievement benefited from the help of Aline
Sayer, of the Radcliffe Institute for Advanced Study, Harvard
University, who lent her expertise in the statistical technique of
hierarchical linear modeling.

The Longitudinal Evaluation of School Change and Performance would not
have been possible without the support and participation of school
principals and teachers who welcomed us into their schools and provided
the heart of the information on which this report is based. We are
particularly indebted to the school administrators who took on the task
of coordinating all LESCP activities at their schools. Our greatest
thanks go to the 12,000 students who took part in the study and did
their best on the assessments we administered each spring and to their
parents for allowing their children to contribute to the body of
knowledge on school reform.

the longitudinal evaluation of school change\
and performance (LESCP)\
in title I schools

# [Overview]{.smallcaps}

The Longitudinal Evaluation of School Change and Performance (LESCP)
followed the progress of students in 71 high-poverty schools as they
moved from third to fifth grade. The study was designed to investigate
the impact on student achievement of specific classroom practices
fostered by school-, district-, and state-level policies. This
longitudinal analysis, conducted between 1996 and 1999 as part of the
National Assessment of Title I, especially was intended to test the
effects of changes in curriculum and instruction called for by advocates
of standards-based reform. Many of these changes were being made in the
wake of the 1994 Amendments to Title I of the Elementary and Secondary
Education Act, which called on states to adopt challenging academic
standards and assessments aligned with these standards by 2001.

Like previous research on Title I, this study clearly demonstrated that
student and school poverty adversely affected student achievement in
both reading and mathematics. Although most Title I research has sought
to identify school practices that can improve student achievement, this
was the first major study to examine the impact of standards-based
reform practices on student achievement. Students in the LESCP schools,
on average, did not catch up with national norms during the course of
the study. The data analysis, however, revealed certain school practices
and standards-based policies that were more likely to result in student
achievement gains as students moved from the third grade through the
fourth and fifth grades. Not all school practices and standards-based
policies studied were clearly linked with student achievement gains.

The study found that reading achievement improved faster when two
factors were present:

-   **Teachers gave high ratings to their professional development in
    reading.** The growth in student test scores between grades three
    and five was about 20 percent greater when teachers rated their
    professional development high than when they gave it a low rating.

-   **Third-grade teachers were especially active in outreach to parents
    of low-achieving students.** Growth in test scores between third and
    fifth grade was 50 percent higher for those students whose teachers
    and schools reported high levels of parental outreach early than
    students whose teachers and schools reported low levels of parent
    outreach activities for the third grade.

Conversely, reading achievement was less likely to improve when
fifth-grade teachers spent considerable time engaged in basic
instruction such as filling out worksheets or reading aloud. Growth in
test scores was 10 percent lower when teachers spent a lot of time on
basic instruction than when they spent little time engaged in these
activities.

Factors that positively affected mathematics achievement gains were the
following:

-   **Teachers who highly rated their professional development in
    mathematics.** Growth in test scores between grades three and five
    was 50 percent higher for those students whose teachers and schools
    rated their professional development high than when they gave it a
    low rating.

-   **Early teacher outreach to parents of students who initially showed
    low achievement.** Test scores in mathematics grew between the third
    and fifth grade at a 40 percent higher rate for students in schools
    whose teachers reported high levels of parental outreach than
    students in schools whose teachers reported low levels of parental
    outreach activities

-   **Instructional practices that involved students in more exploration
    in upper grades.** Growth in test scores between the third and fifth
    grades was about 17 percent greater for students whose fifth grade
    teachers reported relatively very high usage of exploration in
    instruction versus students whose fifth grade teachers reported
    relatively very low usage.

On the other hand, students' mathematics scores fell further behind when
their schools had disproportionately more low-achieving students and
teachers were relatively satisfied with their own instruction skills.
Teachers\' use of standards and assessments had inconsistent effects on
mathematics achievement. Students' gain scores were lower than the LESCP
average in schools where teachers reported knowing and using standards
and assessments. One interpretation of these findings is that schools
that initially paid the highest level of attention to standards and
assessments might have been the schools where poor student performance
was a problem.

Achieving higher levels of implementation on the significant variables
can have important impacts for student achievement growth. For example,
in mathematics, the 2-year gain would be 18 points higher if both the
student\'s own teacher and the average of all teachers in the school
gave very high ratings to professional development (at the 90th
percentile for all teachers and schools in the sample) as opposed to
very low ratings (at the 10th percentile). An 18-point differential is
sizable compared with a 46-point average, 2-year gain in mathematics for
all schools. For the variable of outreach to the parents of low
achievers, the 90th to 10th percentile differential would translate into
a 17-point gain for students who were initially low achieving.

# [Background]{.smallcaps}

Title I is the major federal program designed to improve learning and
achievement among elementary and secondary students in high-poverty
schools. Where poverty is at least 50 percent, the school may use Title
I funds for schoolwide improvements benefiting its at-risk students. The
1994 amendments to Title I were enacted to help students who are
failing, or most at risk of failing, to meet challenging academic
standards. The amendments charged states with developing or adopting
challenging standards for what students need to learn and at what level
of proficiency. To measure how well students and schools are meeting
these standards, states also must administer assessments aligned with
these standards. Like other federal education policies, Title I does not
prescribe how to raise student achievement; rather, its impact depends
on how states and districts reform curriculum and instructional
practices to meet their academic standards. When this study began in
1996, few states, districts, or schools had begun to implement
standards-based reform programs, much less demonstrate their
effectiveness. Consequently, measurable effects on student achievement
could only be expected in schools and classrooms that had instituted
reforms at least several years earlier.

Because of this, the schools selected for this study were not intended
to be statistically representative of high-poverty schools at the
national or the local level. The 71 schools, all of which received Title
I funds as high-poverty schools, were in 18 school districts in 7 states
where standards-based reforms, including assessments and accountability
provisions, either were under way before 1996 or began while the study
was in progress. Although all of the schools were affected by reform
policies involving standards, assessments, and accountability, the
policies and the speed and thoroughness of implementation varied.
Nonetheless, because these schools had been putting reforms into
practice for some time, their experiences provided a good picture of how
standards-based reform has played out in high-poverty schools and offer
valuable lessons about standards-based practices that can improve
student achievement.

Of the 71 schools, 59 were operating schoolwide programs in 1998--99, in
which Title I funds can be used for the benefit of all students rather
than being targeted to selected students. This

reflected, in part, the high-poverty levels of participating schools
(see Figure 1): 15 schools had more than 90 percent of their students
living in poverty, 25 schools had between 75 percent and 90 percent,
21 schools between 50 percent and 75 percent, and 10 schools fewer than
50 percent. In all schools, the poverty rate was higher than 35 percent.

Figure 1. Poverty level of schools in the LESCP sample (n=71)

![](media/image3.wmf){width="4.031944444444444in"
height="2.4743055555555555in"}

Percentage of students eligible for free or reduced-price lunch

> \_\_\_\_\_\_\_\_\_\_\_\_\_\_\_
>
> \*In all LESCP schools, the poverty rate was higher than 35 percent.

## School and Classroom Variables

This study drew on several sources of data: standardized reading and
mathematics achievement test scores,[^1] teacher surveys, district
administrator and principal interviews, classroom observations, focus
groups of school staff and parents, and documents regarding school
districts' policies related to standards-based reform. It traced the
students' achievement scores and examined the effect of a number of
student and school-level variables involving school practices, teacher
preparation, and reform policies on both initial achievement and changes
over time.[^2] Data were analyzed for both individual teachers and
entire schools. Information about teachers' curriculum and instructional
practices can be held up against student achievement to show which
practices may have an effect on achievement gains. School-level data are
important because some potentially effective instructional influences on
learning may come from aspects of the whole school environment or from
the broader district- or state-level policy environment. Students'
achievement, when examined against particular practices in their
classrooms and schools, provides valuable evidence about the impact of
these practices.

Most variables were derived from teachers' survey responses about their
familiarity, beliefs, practices, and preparation related to
standards-based reform. The questions addressed specific parts of an
overall vision of standards-based reform: a framework of content and
performance standards, together with assessments and curriculum keyed to
those standards, that would command attention and guide classroom
practice; curriculum and instruction designed to engage students in
relatively advanced academic tasks rather than on rote drill and
practice; teachers prepared to teach in new ways, having participated in
professional development geared to the standards and assessments; and
active communication between school and home.[^3] The researchers
organized the responses into index variables---combinations of the
teacher's answers to survey questions that were closely related to each
other, statistically as well as conceptually. (Appendixes B and D in
Volume 2 of the report describe the statistical properties of each
index.)

Several variables that were intended to measure the level of activity in
standards-based reform were based on an analysis of district and school
policy documents. Although the study was not designed to investigate
policy environments intensively, and no data were collected at the state
level, it was still possible to use this documentary evidence to
identify significant differences among the districts. Based on the
policies described in these documents and data published in *Quality
Counts*, the annual *Education Week* status report on the implementation
of standards, districts could be classified as falling at the high or
low end on specific aspects of standards-based reform.

The study used a statistical technique called hierarchical linear
modeling (HLM), which makes it possible to isolate and measure the
effect of each independent variable on an outcome. (Appendix A in Volume
2 describes the HLM analysis.) This statistical technique enabled
researchers to identify whether each variable had a significant,
independent relationship to achievement. The technique also allowed them
to estimate how many points higher or lower a student would score if a
variable were present to an especially great degree in a classroom or
school, relative to the rest of the LESCP sample. Although the effect of
each variable was isolated for the study, in actual schools and
classrooms, the different variables occur together, creating overall
instructional conditions in which students learn. The bottom line for
students, then, is a combination of all the positive or negative
influences on achievement.

The school and classroom practice variables tested as possible
contributors to students' test scores and gains were the following:

-   **Visibility of Standards and Assessments.** This variable was based
    on responses regarding teachers' familiarity with their state or
    district content standards, curriculum frameworks, student
    assessments, and performance standards for mathematics and reading,
    as well as their perceptions about how their curriculum reflected
    each of these elements.

-   **Basic Instruction in Upper Elementary School Grades.** This
    variable drew on responses regarding how much time students spent in
    more basic, rather than advanced, activities in reading instruction,
    such as reading aloud, completing workbooks or skill-sheet
    assignments, practicing phonics, and practicing word attack.

-   **Preparation for Instruction.** This variable drew on responses
    about how well prepared teachers considered themselves to use
    small-group reading and mathematics instruction; take into account
    students' existing skill levels when planning curriculum and
    instruction; integrate reading or mathematics instruction into other
    content areas; use a variety of assessment strategies; and teach
    groups that are heterogeneous in ability.

-   **Exploration in Instruction.** This variable, specific to
    mathematics, was developed from teachers' responses about how much
    they used manipulatives to demonstrate concepts in mathematics and
    discuss multiple approaches to solve a problem. The variable also
    covered the frequency with which their students worked on problems
    in small groups, had whole-class discussions about solutions
    developed in small groups, had student-led whole group discussions,
    represent and analyze relationships using tables and graphs,
    responded to questions or assignments that required writing at least
    a paragraph, worked with manipulatives, and worked on projects and
    assignments that take a week or more to finish.

-   **Presentation and Practice in Instruction.** This variable, also
    specific to mathematics, was based on responses about the extent to
    which teachers used relatively conventional instructional
    activities, such as lectures or presentations, whole-group
    discussions, demonstrations of exercises on the board, and tests.
    This variable also addressed how often teachers had students respond
    orally to questions, work individually on written assignments or
    worksheets in class, and practice or drill on computational skills.

-   **Rating of Professional Development**. This variable drew on
    teachers' opinions about the quality of their professional
    development in reading and mathematics content and in instructional
    strategies. This variable also covered teachers' opinions about how
    well their professional development reflected their school's or
    department's plans to change practice; supported reform efforts in
    their school; supported state or district standards or curriculum
    frameworks; supported state or district assessments; bolstered their
    confidence in using new pedagogical approaches; and enhanced their
    ability to adapt their teaching to state assessment requirements,
    state standards, or curriculum framework requirements.

-   **Outreach to Low-Achievers' Parents.** This variable measured the
    extent to which teachers communicated with parents of low-achieving
    students through face-to-face meetings, sent them materials on ways
    to help their child at home, and telephoned them when their child
    was having problems and, more routinely, when there were no
    problems.

-   **Focus on Assessment and Accountability.** This variable, which
    characterized districts that were especially active in
    measurement-related reforms, combined several items concerning the
    reporting and the consequences of assessments: whether the district
    reported assessment data in terms of proficiency levels; whether the
    district publicly reported school and district achievement status in
    readable and understandable formats using data disaggregated for at
    least two demographic groups by 1998; and whether schools in the
    state or district were subject to sanctions or rewards based on
    student performance as of 1997.[^4]

-   **Standards, Consequences, and Plans.** This somewhat broader
    variable combined the enactment of standards and accountability
    policies with the citation of these policies in school and district
    improvement plans. It summarized a number of district-level items:
    whether the state or district had enacted content standards in
    reading and mathematics by 1997; whether schools were subject to
    state or district sanctions or rewards based on student performance
    as of 1997; whether district and school improvement plans
    consistently referred to content standards in all 3 years of the
    study; and whether district and school improvement plans
    consistently set specific goals linked to required assessments in
    all 3 years.[^5]

# [Key Findings]{.smallcaps}

The findings of this study are presented here in four sections. The
first section reviews findings having to do with the effects of poverty
on students\' mathematics and reading achievement. The next two sections
summarize the effects of school and classroom practices on reading and
mathematics achievement. A final section presents findings on the
effects of the policy environment of instructional practices.

## Effects of Poverty on Students' Reading and Mathematics Achievement

Individual and school poverty had a clear, negative effect on student
achievement.

-   On average, students in the LESCP sample scored below national norms
    and urban norms in all years and grades tested (see Figure 2).

Figure 2. LESCP scores relative to national and urban norms for
closed-ended reading\*

![](media/image4.wmf){width="6.197916666666667in"
height="2.7680555555555557in"}

\* This figure presents two different groups of students who were tested
as part of LESCP. The "full LESCP" refers to all students tested in a
particular grade for a particular year (e.g., third graders in 1997).
The "longitudinal LESCP" refers to only those students who were tested
three times: as third graders in 1997, fourth graders in 1998, and fifth
graders in 1999.

-   Students in the LESCP schools had average reading scores of 602 in
    third grade and 640 in fifth grade, compared with national norms of
    614 in third grade and 654 in fifth grade.[^6]

```{=html}
<!-- -->
```
-   Students in the LESCP schools had average mathematics scores of 593
    in third grade and 640 in fifth grade, compared with national norms
    of 600 and 646.

```{=html}
<!-- -->
```
-   Students who **lived in poverty** (i.e., were eligible for free or
    reduced-price lunch) did significantly worse initially on the
    reading and mathematics tests than other students in this
    sample.[^7] The analysis showed that a student whose teachers and
    school were at the average for every variable measured, but who came
    from a poor family, would

```{=html}
<!-- -->
```
-   Have a third-grade reading score 6.1 points below the average for
    the sample. Such a student then would make gains at an average pace
    (if no other variable departed from the average value), neither
    closing the gap nor falling further behind in reading (see Figure
    3).

Figure 3. Effect of poverty on LESCP student achievement on the SAT-9 in
reading, third to fifth grade

![](media/image5.wmf){width="6.1375in" height="3.8722222222222222in"}

[Figure reads]{.underline}: A student who experiences average conditions
on all variables has a predicted third-grade score of 608.6 and a
predicted fifth-grade score of 646.9 (gain of 38.3). Both scores are
below the national norms for that grade. A student who is eligible for
free or reduced-price lunch has a predicted third-grade score of 602.5
and a predicted fifth-grade score of 640.8 (gain of 38.3).

-   Score 5.8 points below the average third-grade student in this
    sample on the mathematics test, with all other variables held
    constant (see Figure 4).

```{=html}
<!-- -->
```
-   Students who attended **schools with the highest percentages of poor
    students performed**

```{=html}
<!-- -->
```
-   Worse initially on the reading test than their peers.[^8] In a
    school where 100 percent of students were eligible for free or
    reduced-price lunch, a student's third-grade reading score would be
    11.8 points below the average for the sample, holding all the other
    variables constant. However, school poverty by itself did not affect
    the pace at which students gained in reading. These students would
    neither close the gap nor fall farther behind if all other variables
    were at average values for the sample.

```{=html}
<!-- -->
```
-   8.9 points below the average on the mathematics test in third grade.
    However, the students in these schools partially closed this initial
    gap, gaining 5.8 more points than an average LESCP student by the
    end of fifth grade.

Figure 4. Effect of poverty on LESCP student achievement on the SAT-9 in
mathematics,\
third to fifth grade

![](media/image6.wmf){width="6.141666666666667in"
height="3.8243055555555556in"}

[Figure reads:]{.underline} A student who experiences average conditions
on all variables has a predicted third-grade score of 597.1 and a
predicted fifth-grade score of 643.3 (gain of 46.2). Both scores are
below the national norms for that grade. A student who is eligible for
free or reduced-price lunch has a predicted third-grade score of 591.3
and a predicted fifth-grade score of 637.5 (gain of 46.2).

## Effects of School and Classroom Practices on Reading Achievement

**When third-grade teachers were especially active in outreach to low
achievers' parents, students made faster gains** in reading over the
next 2 years, gaining 4.6 points more than students whose teachers made
only an average amount of outreach.[^9] In addition, in schools where
all third-grade teachers were especially active in outreach to low
achievers' parents, students gained an additional 3.7 points by fifth
grade. Third-grade classrooms with generally low achievement tended to
be the ones in which teachers reported contacting more parents of low
achievers.

**When teachers gave higher ratings to their professional development in
reading, students gained 3.1 points more than average** during 2 years
in school (see Figure 5). This occurred when teachers described their
professional development as matching the school's reform plan, focusing
on standards and assessments, and boosting their confidence in using new
approaches.

Figure 5. LESCP student achievement gains on the SAT-9 closed-ended
reading test, third to fifth grade

![](media/image7.wmf){width="6.61875in" height="4.029166666666667in"}

^1^ Teachers emphasizing basic instruction spend more time with
worksheets, reading aloud, and other types of relatively routine skill
practice.

^2^ Outreach to low achievers\' parents measured in third grade was
associated with both the predicted third-grade score and the score gain
from third to fifth grade. For that variable, the figure reflects a
significant difference in the predicted third-grade score (603.5) rather
than the average third-grade score (608.6).

**Students whose fifth-grade teachers reported spending more time
instructing the class at a basic level in reading with worksheets,
reading aloud, and other types of relatively routine skill practice
gained 1.9 points less on average** than those whose teachers spent an
average amount of time working at that level. This result appears to
support the theory that continuous instruction in basics at the upper
elementary level impedes students' progress. Another possible
explanation for the result is that teachers whose students showed weaker
reading skills in fifth grade felt the need to reinforce some basic
skills such as phonics.

**Students' initial reading scores tended to be higher in classrooms
where teachers reported they were aware of and implementing the policies
of standards-based reform** (see Figure 6). When third-grade teachers
reported very high visibility of standards and assessments and said that
they believed their curriculum reflected these policies, their students
scored 2.8 points above the LESCP average in the third grade.

Figure 6. LESCP-predicted score on the SAT-9 closed-ended reading test,
third grade

![](media/image8.wmf){width="6.477777777777778in"
height="3.470138888888889in"}

[Figure reads]{.underline}: A student who experiences average conditions
on all variables has a predicted third-grade score of 608.6, below the
national norm. For a student whose teacher reports high visibility of
standards and assessments, the HLM model predicts a third-grade score of
611.4 (2.8 points higher).

**The best combination of circumstances for reading achievement gains
was the following:**

-   Less use of basic instruction in upper elementary school grades,

-   Higher teacher ratings of professional development,

-   More intense outreach to parents of low achievers, and

-   Higher visibility of standards and assessments in third grade.

The sample included 110 students whose classrooms and schools offered
exactly this combination of circumstances. These students, who started
out 10.6 points behind their peers and 18 points below national norms,
narrowed the gap to end up 4.9 points behind their peers and 12 points
below national norms.

## Effects of School and Classroom Practices on Mathematics Achievement

**Students of fifth-grade teachers who reported higher levels of
exploration in mathematics instruction gained 3.6 points more than the
average on mathematics tests over 2 years**.[^10] Implementation of
reforms that call for more student-initiated activities and more
complicated assignments in mathematics appeared to have a positive
relationship to student gains. However, instruction that involved more
exploration may have been offered to classes that showed greater skill
in mathematics (see Figure 7).

**Teachers' ratings of their professional development consistently were
related to greater student gains in mathematics.** When all fourth-grade
teachers gave high ratings to professional development in mathematics
(saying it was well matched to reform plans, standards, and assessments
and that they had learned from it), students' gains exceeded the average
by 3.9 points. A similarly high rating from a fifth-grade teacher was
associated with an additional boost of 3 points.

**Outreach to parents was related to mathematics achievement gains for
one subgroup of students, those who initially showed low achievement.**
It is possible that when teachers involved low-achieving students'
parents early on, students were able to resolve their learning
difficulties before they multiplied and went on to raise their test
scores.

**Schools with more low-achieving third graders relative to the rest of
the sample fell further behind in mathematics over the course of the
study.** With all other variables held constant, the schools at the low
end on student performance in third grade had gains that were 10.8
points below the average for the sample. Apparently, the presence of
large numbers of low-achieving students either depressed gains for all
students or was an indicator of generally unfavorable educational
conditions in the school.

**Students of teachers who expressed high confidence in their
preparation for specific kinds of mathematics instruction gained 5.2
points less than average.** To explain this surprising finding, one can
speculate that the teachers who had not changed their practices in some
years would report greater levels of comfort about their own
preparation, whereas ambitious changes in instruction might have given
teachers doubts about their own skills at the same time as they provided
good environments for student learning.

Figure 7. LESCP student achievement gains on the SAT-9 closed-ended
mathematics test,\
third to fifth grade

![](media/image9.wmf){width="6.4944444444444445in"
height="4.722222222222222in"}

[Figure reads:]{.underline} A student who experiences average conditions
on all variables has a predicted gain of 46.2 points from third to fifth
grade, resulting in a predicted fifth-grade score of 643.3. A student
whose teacher reports a large amount of exploration in instruction has a
predicted gain of 49.8 points, with a predicted fifth-grade score of
646.9.

**The visibility of standards and assessments had an inconclusive effect
on students' mathematics achievement.** On the one hand, students in
schools where teachers reported knowing and using standards and
assessments showed student gains that were 7.6 points less than average.
On the other hand, third-grade mathematics scores were 6.5 points higher
in those schools where third-grade teachers reported the highest
visibility of standards and assessments in 1997 (see Figure 8). This
puzzling combination of findings might reflect changes in the policy
climate during the late 1990s. In 1997, the schools adhering most
closely to standards and assessments might have included those that were
especially alert to professional trends. As standards-based reform
became more prominent, schools with the highest levels of attention to
standards and assessments might have been, increasingly, the schools
where poor student performance was a problem.

Figure 8. LESCP-predicted score on SAT-9 closed-ended mathematics test,
third grade

![](media/image10.wmf){width="6.495833333333334in"
height="3.2263888888888888in"}

[Figure reads:]{.underline} A student who experiences average conditions
on all variables has a predicted third-grade score of 597.1, below the
national norm. For a student whose school reports high visibility of
standards and assessments, the HLM model predicts a third-grade score of
603.6 (6.5 points higher), above the national norm.

**The best combination of circumstances for mathematics gains was the
following:**

-   Relatively higher amounts of exploration in instruction,

-   A teacher who believed he or she had more to learn in mathematics
    instruction, and

-   Higher teacher ratings of professional development.

Students who experienced this combination of instructional variables,
starting out 18 points behind their counterparts in the LESCP sample and
20 points below national norms, would close 40 percent of the gap
between themselves and the longitudinal sample and about 50 percent of
the gap between their scores and national norms.[^11]

##  Effects of the Policy Environment on Instructional Practices

**In general, instructional practices associated with student
achievement gains were equally accessible to all students in the LESCP
sample, including those from relatively lower income families, lower
achieving students, and students in schools serving comparatively
worse-off families.** Thus, within the limited range of poverty and
achievement found in this sample, there were no systematic differences
in equity of access to these instructional practices.

**Teachers in districts and states that emphasized assessment and
accountability were more likely to report having received high-quality
professional development in mathematics**, which was associated with
greater student gains in mathematics (see Figure 9).

**Teachers in districts and states that combined reforms in standards,
consequences, and plans reported more visible standards and assessments
in mathematics and rated professional development in both reading and
mathematics highly.**

-   In reading, the favorable rating for professional development worked
    in favor of student achievement gains.

-   In mathematics, the visibility of standards and assessments had a
    mixed effect on achievement, but the rating of professional
    development was associated with greater gains.

# [Conclusions]{.smallcaps}

This study identified school practices and state or local policies that
contributed to higher student achievement and greater achievement gains
in Title I elementary schools in states and districts that had been
engaged in standards-based reform for some time. Several school
practices−ranging from high-quality professional development and highly
visible standards and assessments to less basic instruction and more
intense outreach to low-achievers' parents−had positive effects on
student achievement in reading and mathematics.

The larger education policy environment appeared to contribute to the
ways in which teachers experienced elements of reform. In those states
and districts that combined several standards-based reforms−the
enactment of standards, accountability for schools, and plans that
referred specifically to standards−teachers described their professional
development as well matched to standards and assessments, and said that
they had learned from that professional development. Their students, in
turn, made greater gains in reading and mathematics.

The study's findings lend some support to the policy position that a
framework, including standards, assessments, and professional
development, can improve student achievement when teachers are engaged
with that framework. In particular, students did better when teachers
reported that their professional development was coordinated with
standards and assessments. Teachers' reports that they knew and used
standards and assessments had a mixed relationship to student
performance. Outreach to the parents of low-achieving students was of
long-term benefit to reading achievement for all students and to
mathematics achievement for low-achieving students. More challenging
curriculum and instruction were associated, in general, with greater
student gains in both subjects. All these instructional conditions could
combine to help mitigate the serious negative effects of poverty, at
both the student and school levels, on achievement. An additional
investigation along these lines is being conducted through the
development of intensive case studies of high-performing LESCP schools.
These cases studies will focus on teachers' curriculum and instruction
in two sets of schools: high-poverty schools that were consistently high
performing throughout the LESCP study and high-poverty schools that
showed significant gains in student performance during the LESCP study.

It bears repeating that the data were drawn from a select sample of
high-poverty schools that was neither representative nor random.
However, because the schools in this sample were among the earliest to
begin implementing elements of standards-based reform, they offer a
robust set of data for policymakers considering how to advance
standards-based reform on a national level. Drawn from several different
states, the sampled schools reflect a range of conditions and are likely
to illustrate relationships that would be found around the country. As
such, this study's findings can help suggest ways that all students can
achieve high standards.

[^1]: The SAT-9 scores provided a uniform measure of student scores and
    gains across the whole sample, enabling the study team to make
    comparisons among all the schools and classrooms. Because Title I
    emphasizes attainment of state standards, however, the SAT-9 would
    not be a good choice for this purpose if it measured content that
    was not closely related to the states' own expectations for
    students. Therefore, the study team used data from state assessments
    to gauge whether student performance on the SAT-9 had a reasonably
    good relationship to performance on the knowledge and skills that
    the states were measuring. The researchers correlated schools\'
    performance on the SAT-9 with their performance on the state
    assessment by comparing how the school did on each measure in
    relation to the other LESCP schools in the state. In five of the
    seven states, there was good correlation between relative rankings
    by SAT-9 and relative rankings by state assessments. Thus, while
    SAT-9 does not map perfectly onto the content and skills measured by
    all states, it appeared to provide a good substitute for state
    assessments in the analysis of student achievement.

[^2]: The study did not focus on other kinds of school variables such as
    very small class size (because the sample did not include many very
    small classes) or the use of comprehensive models for school reform
    (because only a few schools in the sample were using them).

[^3]: The interrelated parts of a standards-based reform vision are
    described in J. O'Day and M.S. Smith. (1993). "Systemic reform and
    educational opportunity," in S. Fuhrman (Ed.) *Designing coherent
    education policy.* San Francisco: Jossey-Bass, pp. 250-312, and in
    L.B. Resnick and K.J. Nolan. (1995). "Standards for education," in
    D. Ravitch (Ed.), *Debating the future of American education: Do we
    need national standards and assessments?* Washington, D.C.:
    Brookings Institution. Other sources for the variables measured in
    this study included work on curriculum and instruction, such as A.
    Porter. (1998). *A curriculum out of balance: The case of elementary
    school mathematics*. East Lansing, MI.: Institute for Research on
    Teaching, College of Education, Michigan State University; and M.S.
    Knapp and P.M. Shields, (Eds.), (1990). *Better schooling for the
    children of poverty: Alternatives to conventional wisdom, Volume II:
    Commissioned papers and literature review*. Washington, D.C.: U.S.
    Department of Education; and work on parent and family involvement
    such as S.L. Dauber and J.L. Epstein. (1993). "Parent attitudes and
    practices of involvement in inner-city elementary and middle
    schools," in A.T. Henderson and N. Berla (Eds.), *A new generation
    of evidence: The family is crucial to student achievement*,
    Washington, D.C.: Center for Law and Education; and U.S. Department
    of Education. (1994). *Strong families, strong schools: Building
    community partnerships for learning.* Washington, D.C.: Author.

[^4]: A total of 34 schools were in "high" districts, and 20 were in
    "low" districts.

[^5]: Twenty-six schools were in high districts, and eight were in low
    districts.

[^6]: Because these are scale scores, which generally rise with each
    year that a student advances, a typical student would start out with
    a score of around 614 and gain about 40 points over 2 years.

[^7]: This finding is consistent with other research on Title I schools.
    See, for example, M.J. Puma, N. Karweit, C. Price, A. Ricciuti, W.
    Thomspon, and M. Vaden-Kiernan. (1997). *Prospects: Final report on
    student outcomes*. Washington, D.C.: U.S. Department of Education.

[^8]: Again, this finding is consistent with the earlier work of Puma
    and colleagues (1997), among others.

[^9]: We use the 90th percentile of teacher responses to illustrate the
    size of the effect associated with each instructional variable.

[^10]: Again, we use the 90th percentile of teacher responses to
    illustrate the size of the effect associated with each instructional
    variable.

[^11]: For simplicity, the variable of visibility of standards and
    assessments is left out of this analysis because its effects were
    mixed. Strictly interpreted, the data suggest that students would do
    better if their schools were initially very attuned to standards and
    assessments (relative to the rest of the sample) but then did not
    keep pace with the other schools in intensifying their attention to
    these policies.