POLICY AND PROGRAM STUDIES SERVICE

Evaluation of the 

Magnet Schools Assistance Program, 

1998 Grantees 

FINAL REPORT

2003 

U.S. DEPARTMENT OF EDUCATION ~ OFFICE OF THE UNDER SECRETARY

DOC #2003-15

Evaluation of the 

Magnet Schools Assistance Program, 

1998 Grantees 

Prepared for: 

U.S. Department of Education 

Office of the Under Secretary 

Policy and Program Studies Service 

Bruce Christenson 

Marian Eaton 

Michael S. Garet 

Luke C. Miller 

Hiroyuki Hikawa 

Phyllis DuBois

American Institutes for Research 

Washington, D.C. 

2003 
This report was prepared for the U.S. Department of Education under Contract No. ED-98-CO-0067. The views 

expressed herein are those of the contractors. No official endorsement by the U.S. Department of Education is 

intended or should be inferred. 

U.S. Department of Education 

Rod Paige 

Secretary 

Office of the Under Secretary 

Eugene Hickok

Under Secretary

Policy and Program Studies Service 

Alan L. Ginsburg 

Director 

Program and Analytic Studies Division 

David Goodwin 

Director 

November 2003 

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While permission to 

reprint this publication is not necessary, the citation should be: U. S. Department of Education, Office of the Under 

Secretary, Evaluation of the Magnet Schools Assistance Program, 1998 Grantees, Washington, D.C., 2003. 

To order copies of this report, write: 

ED Pubs 

Education Publications Center 

U.S. Department of Education 

P.O. Box 1398 

Jessup, MD 20794-1398 

Via fax, dial (301) 470-1244. 

Or via electronic mail, send your request to: edpubs@inet.ed.gov. 

You may also call toll-free: 1-877-433-7827 (1-877-4-ED-PUBS). If 877 service is not yet available in your area, 

call 1-800-872-5327 (1-800-USA-LEARN). Those who use a telecommunications device for the deaf (TDD) or a 

teletypewriter (TTY) should call 1-800-437-0833. 

To order online, point your Internet browser to: www.edpubs.org. 

This report is available on the Department's Web site at: http://www.ed.gov/about/offices/list/ous/ppss/index.html 

On request, this publication is also available in alternative formats, such as Braille, large print, audiotape, or 

computer diskette. For more information, please contact the Department's Alternate Format Center at (202) 260-

9895 or (202) 205-8113.
 Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

 iii

Contents

I. Introduction and Overview of the Magnet Schools Assistance Program (MSAP) and 

Evaluation..................................................................................................................................I-1

The Magnet Schools Assistance Program.........................................................................I-2 

Evaluation of the Magnet Schools Assistance Program ...................................................I-3 

Evaluation Questions ........................................................................................................I-4

Studies Comprising the Evaluation...................................................................................I-5 

Data Collected in 1999–2000 through 2001–2002 ...........................................................I-5 

Organization of the Report................................................................................................I-6

II. Characteristics of MSAP-Supported Schools and Programs..........................................II-1

Grade Levels of Schools Served by Magnet Programs................................................................ II-1

Types of Magnet Program Structures .......................................................................................... II-2

Funding of MSAP-Supported Schools......................................................................................... II-4

Recruitment of Students............................................................................................................... II-5

Program Enrollment..................................................................................................................... II-8

Characteristics of MSAP and Non-MSAP Students and Schools.............................................. II-10

Summary.................................................................................................................................... II-14

III. Support for Systemic Reform and Innovative Educational Practices in MSAP 

Schools....................................................................................................................................III-1

Data Sources ...............................................................................................................................III-3

External Supports for Change.....................................................................................................III-4

Characteristics of Magnet School Programs...............................................................................III-6

Supports for Teaching and Learning.........................................................................................III-10

Classroom Instructional Practices.............................................................................................III-15

Sharing Innovative Practices with Others.................................................................................III-17

Summary...................................................................................................................................III-17

IV. Desegregation Objectives and Outcomes ..................................................................... IV-1

Desegregation Objectives of MSAP’s Targeted Schools............................................................IV-1

Desegregation Outcomes at MSAP’s Targeted Schools.............................................................IV-3

Factors That Influence MSAP Targeted Schools’ Ability to Reduce Minority 

 Group Isolation ..........................................................................................................................IV-7

Summary...................................................................................................................................IV-11

V. Student Achievement Outcomes in MSAP-Supported Schools..................................... V-1

Achievement Objectives Study....................................................................................................V-2

Study of Achievement Trends in MSAP and Comparison Schools...........................................V-18

Summary....................................................................................................................................V-24

VI. Putting the Results in Context ........................................................................................ VI-1

Implementation and Achievement Outcomes .............................................................................VI-2

District Context and Desegregation Outcomes...........................................................................VI-9

Status of MSAP Projects and Schools after the Grant Period Ended .......................................VI-13

Other Outcomes of MSAP Projects ..........................................................................................VI-15

Summary...................................................................................................................................VI-17
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

iv

VII. Conclusions .................................................................................................................... VII-1

Innovative Educational Methods and Practices ........................................................................ VII-1

Magnet Schools and Systemic, Standards-Based Reform ........................................................ VII-2

Minority Group Isolation .......................................................................................................... VII-3

Student Achievement Objectives and Outcomes ...................................................................... VII-5

Strengths and Limitations of the Study..................................................................................... VII-7

Implications .............................................................................................................................. VII-7

Glossary

References
 Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

 v

Exhibits 

Exhibit I-1. Response rates for years 1 through 3 data collection...........................................................I-6 

Exhibit II-1. Comparison of grade levels of MSAP magnet schools with magnet schools 

nationally that promote desegregation .............................................................................. II-2 

Exhibit II-2. Percent of MSAP schools in with PWS programs compared with percent magnets 

nationwide in with PWS programs by grade level, 2000–2001* ...................................... II-4 

Exhibit II-3. Mean budget amounts for 226 MSAP-supported schools in 46 MSAP projects, by 

year.................................................................................................................................... II-5 

Exhibit II-4. Percentage of districts with specific types of recruitment efforts by school staff, 

students, and parents ......................................................................................................... II-6 

Exhibit II-5. Percentage of districts using specific types of outreach to recruit students....................... II-7 

Exhibit II-6. Percent minority enrollment in PWS and in whole school programs, by grade level: 

2000–2001....................................................................................................................... II-10 

Exhibit II-7. Percentage of MSAP-supported schools with targeted and schoolwide Title I 

programs compared with non-MSAP schools in districts and schools nationally .......... II-13 

Exhibit II-8. Comparison of average pupil-teacher ratio in MSAP and non-MSAP schools in the 

same district by grade level............................................................................................. II-14 

Exhibit III-1. Conceptual framework: Connections between improved student achievement and 

various aspects of magnet school educational programs and contexts.............................III-2 

Exhibit III-2. Percent of MSAP project directors providing MSAP schools with technical 

assistance on curriculum and instruction from never to once a week or more, 

1998–1999 and 2000–2001 ..............................................................................................III-6 

Exhibit III-3. Percentage of MSAP schools with selected themes..........................................................III-8 

Exhibit III-4. Sense of professional community and school climate in MSAP and comparison 

schools, 2000–2001........................................................................................................III-14 

Exhibit III-5. Percent of surveyed case study teachers in MSAP and matched comparison schools 

who use computers for specific instructional purposes at least once a week, 2000–

2001................................................................................................................................III-16 

Exhibit IV-1. Desegregation objectives of MSAP’s targeted schools, overall and by type of 

desegregation plan............................................................................................................IV-3 

Exhibit IV-2. Percent of MSAP targeted schools making progress on its desegregation objectives 

by grade level after adjusting for districtwide changes in proportion of minority 

students in public schools.................................................................................................IV-6

Exhibit IV-3. Percent of MSAP targeted schools reducing minority group isolation by extent of 

progress and type of district desegregation plan, after adjusting for districtwide 

changes in proportion of minority students in public schools..........................................IV-7 

Exhibit IV-4. Summary of factors influencing the average annual change in percent minority 

enrollment.......................................................................................................................IV-10 

Exhibit V-1. Project-level objectives by subject domains ......................................................................V-3 

Exhibit V-2. Project-level objectives by type of measure.......................................................................V-4 

Exhibit V-3. Percentage of project-level objectives with and without analyzable data each year.............. 5 

Exhibit V-4. Description of the school-level goals by subject domain....................................................... 7 

Exhibit V-5. Types of measures used for school-level goals...................................................................... 8 

Exhibit V-6. Number of language arts and mathematics goals per school (1999–2000)............................ 9 

Exhibit V-7. Percentage of schools that met half or more of their English language arts goals...........V-12 

Exhibit V-8. Percentage of schools that met half or more of their mathematics goals.........................V-12 

Exhibit V-9. Percentages of schools that met half or more of their final goals ....................................V-13 

Exhibit V-10. Percentage of schools that made progress toward half or more of their English 

language arts goals ..........................................................................................................V-14 
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

vi

Exhibit V-11. Percentage of schools that made progress toward half or more of their mathematics 

goals ................................................................................................................................V-15 

Exhibit V-12. Percentage of schools that made progress toward half or more of their final goals.........V-16 

Exhibit V-13. Correlations between magnet school achievement and school/program 

characteristics..................................................................................................................V-23 

Exhibit VI-1. Tensions between magnet programs and reform efforts...................................................VI-5 

Exhibit VI-2. 2001–2002 Status of magnet schools in 57 MSAP projects funded in 1998 ..................VI-14

Appendixes to the Chapters 

Appendix I ................................................................................................................................A-I-1 

Appendix II ............................................................................................................................. A-II-1 

Appendix III...........................................................................................................................A-III-1 

Appendix IV...........................................................................................................................A-IV-1 

Appendix V............................................................................................................................. A-V-1 
 Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

 vii

Acknowledgements 

This study has benefited from the efforts of a host of collaborators and colleagues over 

the past four years, and we wish to express our gratitude for their contributions. First, we thank 

the project staff, principals, and teachers of the Magnet Schools Assistance Program in our eight 

case study sites for welcoming us into their offices and classrooms and taking time out of their 

busy schedules to discuss their programs and answer our surveys. We especially appreciate the 

efforts of the MSAP project directors that made our visits so interesting and productive. 

We also are grateful to the members of our technical work group for their expert 

guidance and invariably stimulating discussions over the past four years. Panel members 

include: Douglas A. Archbald (University of Delaware), Charles Cassidy (Connecticut State 

Department of Education), Adam Gamoran (University of Wisconsin), Norris Hill (San Jose 

Unified School District, retired), Horace Leake (independent evaluator), Claire Smrekar 

(Vanderbilt University), and Barbara Camacho Benton (Tucson Unified School District). 

The authors also owe a tremendous debt of gratitude to colleagues from two 

organizations who collected data in the eight MSAP projects described in the case studies. Our 

collaborators at The McKenzie Group, Sophia King, Dara Seybold, Edith Stevens, and Nicole 

Warner, under the direction of Scott Joftus, collected the data upon which two of the case studies 

are based. AIR site visitors who made substantial contributions to the other six case studies 

include Elizabeth Ballesteros, Andrew Davis, Diane Garavaglia, Muna Shami, and Michael 

Simone. Thanks also Michelle Bullwinkle, Phil Esra, and Jean Wolman for their diligent and 

careful editing and production of these reports. 

Finally, we would like to express our appreciation to Meredith Miller, Dena Patterson, 

and Daphne Hardcastle at the Department of Education for seeing us through the latter years of 

this project, and to Steve Brockhouse for his many years of cooperation and support. 
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

viii
 Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

 ix

Executive Summary 

For nearly four decades, magnet schools have been an important element of American 

public school education. They have offered innovative programs not generally available in local 

schools and provided opportunities for students to learn in racially diverse environments. Magnet 

schools have been particularly important in districts that are trying to desegregate. 

Congressional support for desegregation first came in the form of the Emergency School 

Aid Act (ESAA), enacted in the spring of 1972 to “encourage the voluntary reduction, 

elimination, or prevention of minority-group isolation.”1 Legislation specifically authorizing 

grants to support the planning and implementation of magnet programs in school districts 

attempting to desegregate was passed in 1976 as an amendment to ESAA,2 and again in 1984, 

with the enactment of the Magnet Schools Assistance Program (MSAP). MSAP grants are 

intended to support magnet schools that are part of an approved desegregation plan and that are 

designed to bring students from different socioeconomic, ethnic, and racial backgrounds 

together. Beginning in 1985, MSAP has offered multiple-year grants to school districts through a 

competitive process administered by the U.S. Department of Education (ED). 

In 1998, the American Institutes for Research (AIR), with the McKenzie Group serving 

as subcontractor, was awarded a contract to evaluate the MSAP. This is the final report for our 

evaluation. It is based on data collected from the 57 projects that received three-year MSAP 

awards in the summer of 1998. 

The Magnet Schools Assistance Program 

During the period covered by this study, the Magnet Schools Assistance Program was 

authorized under the Improving America’s Schools Act (IASA) of 1994. Under this legislation, 

the program had four purposes: to support, through financial assistance to eligible school districts 

or consortia of school districts, the following: 

• The elimination, reduction, or prevention of minority group isolation in elementary 

and secondary schools with substantial portions of minority students. 

• Courses of instruction within magnet schools that will substantially strengthen the 

knowledge of academic subjects and the grasp of tangible and marketable vocational 

skills of students attending such schools. 

• The development and design of innovative educational methods and practices. 

• The development and implementation of magnet school projects that will assist local 

education agencies (LEA) in achieving systemic reforms and providing all students 

the opportunity to meet challenging state content and performance standards.3

 

1

 P.L. 92–318, Title VII, Sec. 701–720, June 23, 1972. 

2

 ESAA was eliminated as a separate program in 1981, when it was consolidated along with more than 30 other 

programs as part of a block grant program under the Omnibus Budget Reconciliation Act of 1981. 3

 20 U.S.C. 7202 
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

x

In each school district receiving MSAP funds, MSAP projects are developed to support 

these four purposes in one or more magnet schools. In this report, we refer to the MSAP (the 

U.S. Department of Education source of funding and assistance), the 57 districts receiving 

MSAP grants in 1998, the projects that the districts developed with MSAP funds, and the 

MSAP schools and programs supported by the projects. 

This report examines the progress MSAP projects made in meeting the legislative 

purposes of the Magnet Schools Assistance Program. Particular emphasis is given to program 

outcomes in reducing minority student isolation and improving student achievement. The results 

show that program outcomes varied within and across school districts. While MSAP schools 

adopted innovative practices and worked to align their programs with state and district systemic 

reforms, overall they made only modest progress in reducing minority group isolation and 

improving student achievement during the three-year funding period. A major factor contributing 

to these findings may have been the length of the grant period: three years may not allow 

sufficient time for MSAP projects to fully implement their programs and show substantial 

change in school enrollment and achievement patterns. 

In the following sections, we outline the data sources that informed the evaluation and 

provide general information about the operation of the program during the 1998–2001 funding 

cycle. In four subsequent sections, we examine the extent to which the 1998 cohort of MSAP 

grantees fulfilled the program’s legislative purposes of reducing, eliminating, or preventing 

minority isolation; increasing student achievement; promoting innovative practices; and 

supporting systemic reforms. In the last section, we discuss strengths and limitations of the 

study, as well as implications of our findings for the administration of the MSAP. 

Data Sources 

Data for the National Population of MSAP Projects and Schools 

To assess the progress made by MSAP projects and schools with respect to the four main 

purposes of the program, AIR collected survey data from the full set of projects and schools 

funded by MSAP in 1998–2001. During 1999–2000, 2000–2001, and 2001–2002, we collected 

and analyzed data on all 57 projects through telephone interviews with the MSAP Project 

Directors; in 1999–2000 and 2001–2002, we also conducted mailed surveys, completed by the 

Project Directors. In addition, we gathered data on the 292 MSAP schools through Principal 

Surveys in both 1999–2000 and 2000–2001. Finally, we obtained school-level data from the 

National School-Level State Assessment Score Database (for use in comparative analyses of 

student achievement in MSAP and non-MSAP schools), from the Common Core of Data of the 

National Center for Education Statistics (for use in describing characteristics of MSAP schools 

and districts and analyzing desegregation outcomes), and from the 1999–2000 Schools and 

Staffing Survey. 

Case Studies of Eight Projects 

In both spring 2000 and spring 2001, we visited eight MSAP projects selected for indepth case studies. Although too few to be a representative sample of the national population, the 
 Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

 xi

case studies were chosen to include major dimensions on which MSAP districts and schools 

vary: regions of the U.S., urbanicity, grade levels served, and whether the district’s desegregation 

effort was voluntary or required by a court or other agency. In each case study district, we 

conducted interviews at the district level and observations and interviews in four MSAP schools, 

as well as in one or two comparison schools. We also surveyed a sample of teachers in each case 

study school. Case study data were used to contextualize the findings of our more quantitative 

analyses of data from the national sample.

Characteristics of the MSAP-Supported Schools and Programs 

In 1998, three-year MSAP grants were awarded to 57 districts and 292 schools within those 

districts. Some magnet programs did not become fully operational during the first year of the 

grant, and a few schools discontinued their programs before 2000–2001. Descriptive information 

about the MSAP-funded districts and schools follows: 

• MSAP-funded magnet schools. A total of 285 of 292 magnet schools funded by the 

1998 MSAP grants operated magnet programs in 2000–2001.4

 These MSAPsupported schools represent about 9 percent of all magnets in the United States 

promoting desegregation.5

 MSAP-supported magnet schools are similar in terms of 

grade level distribution to the pool of all magnets in the U.S. promoting 

desegregation. (Approximately 60 percent of the MSAP programs are in elementary 

schools.) 

• MSAP funding for the magnet schools. While on average MSAP-supported schools 

received about $300,000 per year, grant amounts varied considerably across projects. 

Support for personnel constituted the largest budget expenditure. 

• Whole school programs and programs-within-schools (PWSs). Most MSAPsupported schools (88 percent) operate whole school programs that offer the magnet 

curriculum to all students in the school who are in the grades served by the program. 

PWSs, which offer magnet curricula to some, but not all, students in a school, 

comprise a small proportion of MSAP’s magnet programs (12 percent). This pattern 

differs somewhat from the national pool of magnet schools promoting desegregation, 

in which about one-third of the schools operate PWSs. 

• Characteristics of students in MSAP-supported schools. MSAP magnet school 

enrollments include a high proportion of minority students (73 percent on average) 

and students in poverty (an average of 60 percent are eligible for free or reduced price 

meals). Approximately 70 percent of the MSAP schools operate Title I programs. 

 

4 Initially 292 schools were funded, but the magnet programs in six schools were not operational and two MSAP 

elementary magnet schools combined into a single school. Of the six schools in which the magnet programs were 

not operational, two schools were closed, two schools have programs that never became operational during the 

1998–2001 funding cycle, one school was removed from the district and taken over by the state, and one had its 

magnet status removed by the district after the first year for low performance. Four of the schools were 

elementary and two were middle schools. 5

 The estimates of magnet schools are based on data from the 1999–2000 Schools and Staffing Survey of the 

National Center for Education Statistics. 
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

xii

Results for Minority Student Isolation 

A major objective of the MSAP is to prevent, eliminate, or reduce minority group 

isolation in MSAP schools. Minority group isolation (MGI) refers to schools in which minority 

group students constitute more than 50 percent of school enrollment. 

• Most of the 294 MSAP schools targeted for desegregation6 (77 percent) sought to 

reduce MGI, rather than prevent or eliminate it. 

• Less than one-tenth (8 percent) aimed to prevent MGI by keeping the school’s 

minority enrollment from exceeding 50 percent. 

• Less than one-sixth (16 percent) aimed to eliminate MGI by reducing their minority 

enrollment to 50 percent or less. 

This pattern reflects the fact that most of the targeted schools are in districts in which 

minority students constitute more than 50 percent of public school enrollment. 

Overall, the MSAP-supported magnets had a modest impact on preventing, eliminating, 

or reducing MGI at the 294 desegregation-targeted schools from 1997–1998 (the year prior to the 

initiation of the three-year MSAP projects) through 2000–2001 (the final year of the MSAP 

grant), with some variation by project, program, and school features. 

• Adjusting for districtwide demographic trends, the MSAP-supported magnet 

programs for 57 percent of the desegregation-targeted schools succeeded in 

preventing, eliminating, or reducing MGI, while 43 percent did not show progress.

• The proportion of targeted schools in districts with voluntary desegregation plans that 

prevented, eliminated, or reduced MGI was slightly higher than the proportion of 

such schools in districts with desegregation plans required by a court or other agency 

(60 percent vs. 53 percent). 

• A larger proportion of elementary schools prevented, eliminated, or reduced MGI 

compared with middle schools or high schools (60 percent vs. 54 percent and 48 

percent, respectively).

• Whole school programs, which are offered to all students in the school, were more 

likely to prevent, eliminate, or reduce MGI than were programs-within-schools, 

which are offered to only some of the students attending a school (59 percent vs. 49 

percent).

 

6

 In most cases, the school in which an MSAP-supported program is located is also the school that is targeted for 

desegregation. In a few cases, the magnet program is designed to draw students out of one or more other 

desegregation-targeted schools that may or may not receive MSAP funding. Therefore, there are 292 MSAPfunded magnet schools but 294 targeted schools. 

The impact of MSAP-supported programs on school desegregation was modest. Adjusting for 

districtwide demographic trends in minority enrollment, 57 percent of the desegregationtargeted schools succeeded in preventing, eliminating, or reducing minority group isolation, 

while 43 percent did not succeed. 
 Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

 xiii

Analysis of the amount of progress desegregation-targeted schools made in reducing 

MGI indicated that a small proportion of targeted schools prevented or eliminated minority 

group isolation, while the successful schools mostly reduced minority group isolation. 

• One in six (17 percent) of the targeted schools experienced a reduction of 5 

percentage points or more in MGI relative to the district. About one-quarter (28 

percent) experienced a reduction of 1 to 5 percentage points, and one in fifteen (7 

percent) experienced a reduction that was less than 1 percentage point. 

• An additional one in twenty (5 percent) of the targeted schools prevented or 

eliminated MGI. 

• In the remaining 43 percent of schools that did not make progress, MGI increased or 

remained constant. 

Analysis of the factors that influence the ability of targeted elementary schools to reduce 

minority isolation pointed to several school features: 

• Schools are more likely to experience decreasing minority isolation when the school 

has a racially and ethnically mixed group of minority students.

• Schools are more likely to experience decreasing minority isolation when parents are 

involved in school events and activities. 

• Schools with larger numbers of students per teacher are more likely than those with 

lower student-to-teacher ratios to experience increases in minority group isolation. 

Some of the challenges facing MSAP projects that may explain the modest impact that 

MSAP magnet programs have had on MGI include such factors as the decreasing number of 

nonminority students in many districts, a need for more effective recruitment, the need to support 

parents and retain students in the magnet programs, limitations on factors that are used in the 

selection of students, and inability to conduct recruitment efforts for the first year of the project 

because of the timing of grant awards. Our case study data suggest that districts are 

experimenting with strategies to address some of these challenges. Additional research into these 

and other challenges is needed to better understand the processes that influence the success of 

programs in meeting their desegregation objectives. 

Results for Student Achievement Objectives and Outcomes 

Two studies of student achievement in MSAP-supported schools were conducted. The first showed 

that MSAP-supported schools were most successful in meeting or making progress toward their 

student achievement goals they had set for the first year of magnet program operation, but 

continued improvement over longer time periods proved more difficult. The second, an analysis of 

statewide test data, showed that MSAP-supported elementary magnet schools made noticeable 

progress in reading and mathematics during the grant period. However, when the analysis 

controlled for changes in the demographic composition of the schools, the gains exhibited by 

MSAP schools were not significantly different from those exhibited by non-MSAP schools with 

similar characteristics. 
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

xiv

We examined student achievement in MSAP schools using two types of data: information 

provided in MSAP projects’ annual reports concerning the progress that the schools had made 

toward meeting achievement goals set by the project, and data obtained from statewide testing 

programs for MSAP magnet schools and a matched sample of non-MSAP schools. 

Progress Toward Meeting Achievement Goals 

MSAP projects were required to establish goals for student achievement for each funded 

school. MSAP projects set a wide variety of achievement goals for their schools. 

• Most achievement goals related to student performance on standardized tests in 

English language arts and mathematics. 

• Overall, the MSAP-supported schools were most successful in meeting or making 

progress toward goals set for the first year of magnet program operation, but 

continued improvement over longer time periods proved more difficult. Overall, 

about 51 percent of the schools met half or more of the benchmarks set for the final 

year of the grant in language arts, and approximately 36 percent did so for 

mathematics. 

• The availability of achievement data was a concern in these analyses. MSAP projects’ 

annual progress reports provided usable school-level data for about one-third of the 

general achievement objectives that had been described in MSAP applications. For 

each grant year, outcome data were available for between half and two-thirds of the 

specific school-level goals we tracked. A major reason for the lack of outcome data 

was that the state had revised or discontinued the assessments upon which the 

projects had based their objectives. 

Comparative Analysis of Achievement in MSAP and non-MSAP Elementary Schools 

In addition to analyzing objectives and data provided by the MSAP projects, AIR used 

school-level state assessment scores to examine gains in reading and mathematics performance 

of MSAP-supported magnet elementary schools and a sample of matched comparison schools 

without MSAP funding. 

• Both the MSAP-supported magnet schools and the comparison schools made 

noticeable progress in reading and mathematics during the grant period. However, 

when changes in the demographic composition of the schools were controlled for, the 

gains exhibited by MSAP schools were not significantly different from those 

exhibited by non-MSAP schools with similar characteristics. 

• The measure most strongly associated with achievement growth in both reading and 

mathematics in MSAP-supported schools was the overall strength of the professional 

community of the school (i.e., the extent to which teachers in a school are reported to 

share a common set of goals and beliefs, and to have frequent opportunities for 

collegial interaction). Additional factors were associated with growth in only one of 

the subjects. Greater progress in reading was associated with professional 

development related to standards-based reform. Greater progress in mathematics was 

associated with stronger influence of state or district standards and frameworks on 
 Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

 xv

curriculum and instructional decisions, and with longer periods of magnet operation 

(i.e., magnet programs that had been in operation prior to the 1998 MSAP grant). 

Influence of Implementation Time and District Context on Achievement Results 

In interpreting the similarity of achievement results for MSAP and non-MSAP schools, it 

is important to consider the conditions under which dramatic differences might occur. One 

prerequisite for improved achievement is sufficient time for a program to be fully implemented, 

for teachers to change their methods, and for students to respond with improved performance on 

achievement tests. Information from surveys and case studies suggests that these conditions were 

not always met. Implementation time was particularly problematic for new MSAP-supported 

magnet schools, which had three years to design programs, acquire materials, train teachers, and 

implement new methods well and consistently enough to affect student performance. Due to late 

notification or funding, the use of a planning year, or the opening of a school during the second 

or third year of the grant, some schools had even less than three years in which to produce 

measurable results. 

Furthermore, for the performance of MSAP and non-MSAP students on standardized 

tests to differ appreciably, one would expect instructional programs addressing content covered 

by the tests to differ in the two types of schools. Given the context of high-stakes accountability 

in which all schools operate, case data indicate that non-MSAP as well as MSAP schools focus 

their efforts on improving teachers’ practice and student performance using a combination of 

regular and special funding sources (of which MSAP is only one), thus diminishing the 

differences between MSAP and non-MSAP instructional programs. 

Results for Innovative Educational Methods and Practices 

Magnet schools are expected to adopt distinctive themes and innovative programs, 

designed to promote a positive school climate and professional community among teachers. 

These conditions, in turn, are expected to lead to effective instructional practices and ultimately 

to improved student achievement. Findings, based on interview and survey responses, include 

the following: 

• Magnet themes. MSAP schools have adopted a wide variety of themes. Over onethird of MSAP schools include technology among their themes, and more than a 

quarter of MSAP schools include a science theme. Arts, communication, and 

mathematics are also common themes. 

• Adoption of comprehensive school reform models. More than half of the MSAP 

schools have adopted comprehensive school reform models, such as Success for All. 

MSAP schools adopted a variety of themes and innovative practices, focusing especially on 

technology and science, and they differed from comparison non-magnet schools in their 

districts in several ways. In particular, a higher proportion of MSAP than comparison schools 

adopted comprehensive school reform models; they had somewhat more positive school 

climates; and teachers reported giving more emphasis to higher-order thinking skills. 
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

xvi

This is a substantially higher proportion than is observed among the full national 

population of Title I schools. It is also higher than among comparable non-MSAP 

schools in the MSAP districts. 

• School climate and community. MSAP schools differ in some organizational and 

instructional features from comparison schools in the same district. MSAP schools on 

average have a somewhat more positive sense of professional community than 

comparison schools. For example, principals of MSAP schools are more likely to 

report that staff members are supportive and encouraging of each other than are 

principals in non-MSAP schools. MSAP principals are also more likely to report that 

administrators and teachers collaborate to help make the school run effectively than 

are principals in non-MSAP schools. MSAP schools also have a somewhat more 

positive school climate (i.e., fewer student disengagement and behavior problems), 

although there is considerable variation among schools. 

• Instructional practices. According to teacher survey responses, MSAP schools make 

somewhat more use of technology in instruction than do comparable schools, and 

place more emphasis on instructional methods designed to elicit higher-order thinking 

skills, such as open-ended projects and presentations. 

Results for Supporting Systemic, Standards-Based Reform 

While MSAP schools are intended to adopt innovative themes and practices, they are also 

expected to ensure that these programs are aligned with state and district standards and 

assessments. We hypothesized that magnet schools would be more likely to flourish if their 

themes and programs were aligned with state and district standards and assessments, but we also 

anticipated that innovative magnet themes might at times conflict with the emphases in many 

state and district assessments. Findings, drawn from surveys and in-depth case studies, include 

the following: 

• Respondents to the MSAP Principal Surveys report a high degree of familiarity with 

standards and assessments, and indicate that the content of state standards and 

assessments match the goals of their magnet programs. 

• The case data support the conclusion that magnet themes are generally consistent with 

the content emphasized in state standards. 

• Case data also provide some evidence of tension between the goals of innovative 

instruction and systemic reform. Staff in some MSAP schools reported feeling 

pressured to learn how to teach a new theme/curriculum while simultaneously being 

mindful of state content standards and assessments. It also appears that some MSAP 

schools altered their initial plans in order to bring the curriculum more in line with 

standards and assessments, or reduced their emphasis on novel programs to increase 

the time for work more directly related to state standards and assessments. 

The goals of federally supported magnet programs are generally consistent with the content of

state standards and assessment. However, there is evidence from case studies that tension may

exist between the goals of innovative instruction and systemic reform. 
 Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

 xvii

Strengths and Limitations of the Study 

The results we have obtained must be understood in terms of the strengths and 

weaknesses of the evaluation design. Two strengths of the design are: 

• The study made use of data on the full population of MSAP-funded projects and 

schools. We conducted telephone interviews with all of the local MSAP project 

directors in the first year following implementation and during the second and third 

years of implementation. We also conducted surveys of MSAP principals during the 

second and third years of implementation, achieving more than a 90 percent response 

rate each year. These survey data for the full population are accompanied by data on 

school-level demographics and achievement, drawn from national databases and 

MSAP annual progress reports. 

• Detailed case study and teacher survey data were collected in a sample of MSAP and 

matched comparison schools in eight in-depth case study districts. This information 

allowed us to contextualize findings from our more quantitative analysis of 

desegregation and achievement outcomes. 

The evaluation data, however, have some important limitations: 

• Our results on school practices in the full population of magnet schools are based 

entirely on principal self-reported data. Analyses comparing principal and teacher 

reports for schools in the case study districts indicate that conclusions based on 

principal and teacher reports are reasonably similar for questions that were asked of 

both sets of respondents. Nonetheless, it would be preferable to have data from 

teachers for a larger sample of MSAP schools. 

• Our data on achievement are restricted to aggregate school data. We had initially 

planned to conduct a more powerful study to gauge the impact of magnet schools on 

student achievement using linked-longitudinal student-level data. This approach 

would have allowed analyses of student growth in magnet and comparison schools 

that took account of prior achievement and other student background characteristics. 

We were prevented from doing so by a Department of Education moratorium placed 

on the collection of individual student-level data pending resolution of issues 

pertaining to the Family Educational Records and Privacy Act (FERPA). Given the 

increasing emphasis on providing evidence on the achievement effects of school 

interventions, it would be useful to explore ways in which individual-level data might 

be made available for evaluations without compromising important privacy concerns. 

• The evaluation was not able to make as much use as planned of student achievement 

and enrollment data collected from the annual performance reports of the MSAP 

projects. The information on student achievement provided usable school-level data 

for about one-third of the general achievement objectives described in MSAP 

applications. Some of the reporting problems included lack of clarity of the objective 

and its measurement, changes in the assessment or the metric in which it was 

reported, changes in objectives, and other inconsistencies in reporting. Mathematics 

and English language arts outcomes could be assessed for about half of the schools in 

the first year of the grant and just under two-thirds for the latter two years. 
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

xviii

Additionally, the enrollment data provided by grantees was not sufficiently 

standardized for the purposes of analyzing progress in reducing minority group 

isolation, so we made use of the Common Core of Data from the National Center for 

Education Statistics. 

Implications 

Overall, our results indicate that MSAP projects and schools achieved mixed results over 

the three years that are the focus of our evaluation. In particular, most MSAP schools appear to 

have adopted innovative themes, and there is some evidence, based on both survey and case 

study data, that MSAP schools on average were able to establish a somewhat more positive 

school climate and level of professional community than other schools serving similar students 

in the same districts. In addition, our survey and case study data indicate that MSAP schools 

made efforts to align their programs with state and district standards and assessments. 

At the same time, MSAP schools made only modest progress in preventing, reducing, or 

eliminating minority student isolation and improving student achievement. There is some 

indication, however, that MSAP schools with more positive program features (e.g., school 

climate) outperformed comparison schools in student achievement. 

These results suggest a number of approaches that might be taken to enhance the 

prospects that MSAP schools will produce positive desegregation and achievement outcomes. 

• Narrowing the focus. It may be useful to permit districts to prioritize the goals or 

purposes of MSAP projects to achieve one or two focused outcomes rather than 

attempting to pursue multiple, sometimes competing, goals with limited resources. 

Case study data suggest that projects sometimes had difficulty balancing the four 

goals required of them under MSAP’s authorization under IASA. As reauthorized 

under NCLB, MSAP projects are now expected to focus on six goals, potentially 

making it even more difficult to achieve these goals. Narrowing the focus—and 

evaluating the grantees on this narrower focus—might improve the chances for 

positive results. 

• Re-examining the definition of minority student isolation. Federal regulations 

currently define minority group isolated schools as those in which more than 50 

percent of the students are minority group members. In light of the high proportion of 

minority students in urban school districts, federal policymakers might wish to reexamine the meaning and utility of equating minority group isolation with a single 

fixed percentage. Given the high percentages of minorities in the large urban districts 

that MSAP typically serves, it was not surprising to find that the desegregation 

objective of targeted schools was overwhelmingly to reduce, rather than eliminate or 

prevent, MGI. As the proportion of minorities in schools generally continues to rise, 

there would seem to be a diminishing opportunity for schools to prevent or eliminate 

MGI as it is currently defined without adversely impacting other schools in the 

district. Policymakers might wish to examine more broadly the meaning of minority 

group isolation in an increasingly diverse population. 
 Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

 xix

• Awarding MSAP grants in a more timely manner. The timing of a grant award is 

critical to the first year implementation of recruitment efforts and magnet programs. 

Inability of districts to begin recruitment efforts and delays in first year programming 

are particularly likely when funds are not received until June or later of the year the 

program is expected to begin operation. If funds were awarded by March, or even 

earlier, of the calendar year the project were to begin, districts might be able to 

implement more effective recruitment efforts for the first year of magnet programs, 

and schools would have more time to secure materials, training, and personnel for the 

first year of the program. 

• Extending the period of funding. The MSAP program might provide more than 

three years of funding. Three years may not be sufficient to plan, develop, and 

implement curriculum and expect to see change in enrollment and student 

performance around a new theme. It may take several years for magnet school 

programs to build a strong reputation before they can attract students from outside the 

immediate neighborhood. Whether or not the period of funding is extended, it would 

be helpful to continue to examine school outcomes for a period longer than three 

years. 

• Improving the use of annual performance benchmarks. Districts receiving MSAP 

funds are required to set annual performance benchmarks for improvements in 

minority student isolation and student achievement, and to report on their success in 

meeting these benchmarks. The benchmarks districts set for their schools varied 

considerably in ambition and plausibility. If districts are required to set benchmarks, 

additional technical assistance may be required to ensure that the benchmarks are 

meaningful and that outcomes are monitored consistently. 

As we have indicated, the data collection for this evaluation was conducted prior to the 

enactment of No Child Left Behind (NCLB). Our results do, however, suggest that the 

provisions of NCLB may have special implications for magnet schools. 

• Providing assistance in the use of disaggregated achievement data. By 2005–

2006, all states must conduct annual testing in grades 3–8, and at least one grade from 

10–12. In addition, achievement scores must be reported disaggregated by ethnicity 

and other subgroups. The availability of these data will enhance the capacity to 

examine achievement outcomes, and, in particular, to determine whether magnet 

schools are effective in closing the achievement gap between minority and 

nonminority students. As such data are increasingly available, it may be useful to 

provide technical assistance to MSAP grantees to encourage appropriate uses of 

disaggregated achievement data in evaluating magnet schools. 

• Supporting district choice systems. Under the provisions of NCLB, districts are 

expected to offer a choice of schools to students enrolled in schools that fail to meet 

adequate yearly progress standards for two consecutive years. Magnet schools may be 

particularly attractive options for families with students in failing schools, and thus 

MSAP projects may have particular strengths in assisting districts to build 

coordinated district choice systems. 
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

xx

• Providing support for magnet schools that fail to meet adequate yearly progress 

requirements. Under NCLB, schools that fail to make adequate yearly progress for 

two consecutive are expected to revise their plan for the use of Title I funds and 

engage in other interventions. Magnet schools confronting a failure to meet adequate 

yearly progress standards may face the additional challenge of maintaining the 

continuity and integrity of the school’s distinctive mission or theme while 

incorporating changes in curriculum and instruction to improve test scores. 
 I. Introduction 

 I-1

I. Introduction and Overview of the Magnet Schools 

Assistance Program (MSAP) and Evaluation 

For nearly four decades, magnet schools have been an important element in American 

public school education, offering innovative programs not generally available in local schools 

and providing opportunities for students to learn in racially diverse environments. The number of 

school districts offering magnet schools expanded rapidly during the 1980s: from 138 districts in 

1982 to 230 districts in 1991. The number of schools offering magnet programs more than 

doubled during this time from 1,019 to 2,433 schools, while the number of students participating 

in magnet programs nearly tripled from 441,000 to over 1.2 million (Steel and Levine, 1994). 

The number of magnet schools continued to increase during the 1990s to an estimated 3,026 

schools enrolling 2.5 million students in 1999–2000.1

 Some studies have credited magnet schools 

with contributing to school desegregation, and with improving educational quality.2 

Magnet schools have been particularly important in districts that are trying to 

desegregate. Congressional support for desegregation first came in the form of the Emergency 

School Aid Act (ESAA) enacted in the spring of 1972 to “encourage the voluntary reduction, 

elimination, or prevention of minority-group isolation,”3 and the ESAA was amended in 1976 to 

authorize grants to support the planning and implementation of magnet programs in school 

districts attempting to desegregate. ESAA was eliminated as a separate program in 1981 and 

consolidated along with more than 30 other programs as part of a block grant under the Omnibus 

Budget Reconciliation Act of 1981. 

Congress resumed specific support for magnet schools in 1984 with the enactment of the 

Magnet Schools Assistance Program (MSAP). MSAP grants are intended to support magnet 

schools that are part of an approved desegregation plan and that are designed to bring students 

from different socioeconomic, ethnic, and racial backgrounds together. Since 1984, MSAP has 

offered multiple-year grants to school districts through a competitive process administered by the 

U.S. Department of Education (ED). Three-year grants were awarded in 1998 to 57 school 

districts across the country, and 64 school districts were awarded grants in 2001. 

In 1998, the American Institutes for Research (AIR), with the McKenzie Group serving 

as subcontractor, was given a contract to evaluate the MSAP. This is the final report for our 

evaluation. It is based on data collected from all 57 projects and 8 case study districts during the 

1999–2000 and 2000–2001 school years, from applications and progress reports that the projects 

have submitted to ED, and extant data from national, state, and district sources. This overview 

chapter provides a brief description of the MSAP, the evaluation design for this study, and the 

organization of this report. 

 

1 Estimates for 1999–2000 refer to schools operating programs intended to promote desegregation. The 1999–2000 

estimate for all magnet schools, including those without a desegregation objective, is 5,576 schools that enroll 

approximately 4.5 million students. Source: Schools and Staffing Survey of the National Center for Education 

Statistics. 2

 See the section on Evaluation Questions below for a discussion of the research literature on magnet schools. 

3

 P.L. 92–318, Title VII, Sec. 701–720, June 23, 1972. 
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

I-2

The Magnet Schools Assistance Program 

The results reported here concern MSAP projects that were awarded MSAP funds in the 

summer of 1998 and received funding during the 1998–1999, 1999–2000, and 2000–2001 school 

years. At the time of the study, the program operated as authorized by the Improving America’s 

Schools Act (IASA) of 1994 (20 USC 7201–7213).4

 

During the period under study, the Magnet Schools Assistance Program had four 

purposes. The program was to support the following purposes, through financial assistance to 

eligible LEAs5

 or consortia of LEAs: 

• The elimination, reduction, or prevention of minority group isolation in elementary 

and secondary schools with substantial portions of minority students. 

• The development and implementation of magnet school projects designed to assist 

LEAs in achieving systemic reforms and providing all students the opportunity to 

meet challenging state content standards and challenging state performance standards. 

• The development and design of innovative educational methods and practices. 

• Courses of instruction within magnet schools designed to substantially strengthen the 

knowledge of academic subjects and the grasp of tangible and marketable vocational 

skills of students attending such schools.6

Districts (LEAs) applying for MSAP grants had to describe the ways in which they 

planned to support these purposes in their MSAP projects. 

The first MSAP grants were for two-year periods, with second-year funding contingent 

on an ED review; grants were awarded in 1985, 1987, 1989, 1991, and 1993. In 1995, the grant 

period was extended to three years, to allow districts more time to implement their programs. 

 

4

 MSAP was recently reauthorized as part of the No Child Left Behind Act of 2001 (NCLB), 20 U.S.C. 7231-

7331. The major provisions of the program as authorized under NCLB are similar to the provisions under IASA. 5

 LEAs are local education agencies—public school districts or education centers serving students in some or all 

of grades K–12. This definition and others are included in the glossary at the end of this volume. 6

 20 U.S.C. 7202 (1994). These purposes were revised under the No Child Left Behind Act of 2001. Under NCLB, 

MSAP has the following six purposes: “(1) the elimination, reduction, or prevention of minority group isolation 

in elementary schools and secondary schools with substantial proportions of minority students which shall 

include assisting in the efforts of the [United States] to achieve voluntary desegregation in the public schools; (2) 

the development and implementation of magnet school programs that will assist local educational agencies in 

achieving systemic reforms and providing all students the opportunity to meet challenging State academic 

content standards and student academic achievement standards; (3) the development and design of innovative 

educational methods and practices that promote diversity and increase choices in public elementary and public 

secondary schools and public educational programs; (4) courses of instruction within magnet schools that will 

substantially strengthen the knowledge of academic subjects and the attainment of tangible and marketable 

vocational, technological, and professional skills of students attending such schools; (5) improving the capacity 

of local educational agencies, including through professional development, to continue operating magnet schools 

at a high performance level after Federal funding for the magnet schools is terminated; and (6) ensuring that all 

students enrolled in magnet school programs have equitable access to high quality education that will enable the 

students to succeed academically and continue with postsecondary education or productive employment.” 20 

U.S.C. 7231(b). 
 I. Introduction 

 I-3

During the period from 1985 through 1998, 379 MSAP grants were awarded to a total of 171 

school districts7

 in 35 states and the District of Columbia. 

The 57 MSAP grants for 1998 totaled almost $96.5 million and ranged in size from 

$350,000 to $2,856,392. The average amount was $1,692,982 per district. Awards in 1999 and 

2000 were for similar amounts, with some variation based on grantee plans (e.g., the timing of 

heavy equipment purchases or the use of a planning year for some schools in 1998). 

 

In each school district receiving MSAP funds, MSAP projects were developed to support 

the four purposes in one or more magnet schools. In this report we refer to the MSAP (the U.S. 

Department of Education source of funding and assistance), the 57 districts receiving MSAP 

grants in 1998, the projects that the districts developed with MSAP funds, and the MSAP 

schools and programs supported by the projects. 

Evaluation of the Magnet Schools Assistance Program 

To determine whether the MSAP is fulfilling its purposes and to provide information on 

the uses, successes, and problems associated with federal funding of magnet schools, AIR began 

a comprehensive, four-year evaluation of the MSAP in September 1998. Conducted under the 

authority of the Magnet Schools Assistance Program, Title VI of the Improving America’s 

Schools Act (IASA)8 and sponsored by ED’s Policy and Program Studies Service, the evaluation 

focuses on the 57 school districts that received funding for MSAP projects in 1998 for a threeyear grant cycle. 

In applying for MSAP grants, each of the 57 MSAP projects set desegregation objectives 

for reducing, eliminating, or preventing minority group isolation in each of their MSAP schools 

and achievement objectives for strengthening student knowledge and skills in those schools. As 

funded projects, they submitted annual performance reports to the Department of Education in 

which they documented their progress toward meeting these objectives, as well as other aspects 

of implementing their magnet programs. Our Year 1 Interim Report (issued in 2000) was based 

on information gathered from the 57 projects’ grant applications and 1998–1999 performance 

reports to ED, as well as through initial interviews and surveys conducted by AIR during 1999–

2000. It provided district context and described the objectives that the projects had set and the 

strategies they were implementing to support systemic reform, implement innovative practices, 

and attain their desegregation and achievement objectives. This final report updates information 

in the first report and addresses the extent to which the projects met their objectives. Data are 

drawn from the grantees’ 1999–2000 and 2000–2001 performance reports; from interviews, 

surveys, and site visits that AIR conducted during 2001 and 2002; and from extant data sources. 

 

7

 In a few cases, the grantee was a consortium of school districts. 

8

 20 U.S.C. 7212 
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

I-4

Evaluation Questions 

Our evaluation is guided by these major evaluation questions: 

I. What are the characteristics of MSAP projects? 

II. What are the characteristics of MSAP districts? 

III. To what extent are federally funded magnet projects reducing the incidence 

or degree of minority isolation in their programs? 

IV. To what extent are federally funded magnet projects promoting systemic, 

standards-based reform? 

V. To what extent do federally funded magnet projects feature innovative 

educational methods and practices that meet identified student needs and 

interests? 

VI. To what extent do federally funded magnet projects strengthen students’ 

knowledge of academic subjects and skills needed for successful careers in 

the future? 

VII. How has the MSAP contributed to the development and implementation of 

magnet projects? 

While there has been continuing policy interest in magnet schools, there is surprisingly 

little systematic research on the effects of magnet schools in reducing minority student isolation 

or improving student achievement. 9 One of the claims for the support of magnet schools is that 

they promote desegregation of schools by increasing the school choices afforded to parents. 

Although the literature examining desegregation outcomes is relatively small, synthesizing the 

available studies is challenging, in part because studies differ in their approaches to the 

measurement of desegregation outcomes. While some researchers have found that magnet 

schools have succeeded in creating desegregated schools (Yu et al., 1997) or significantly 

contributing to desegregation of schools (for example, Blank et al., 1983), other researchers have 

been more cautious about the role that magnet schools have had in promoting desegregation 

(Henig, 1996). In a previous evaluation of MSAP between 1989 and 1991, researchers concluded 

that the magnet programs had a relatively modest impact on desegregation of schools (Steel and 

Eaton, 1996). 

Several studies have reported positive effects of magnet schools on student achievement 

outcomes. (See, for example, Archbald, 1995; Blank, 1990a; Hill, Foster, and Gendler, 1990; 

Gamoran, 1996.) Some show mixed or negative effects. For example, Crain et al. (1999) report 

positive effects of high school career magnet schools on a variety of achievement outcomes, but 

negative effects on student graduation rates; and Adcock and Phillips (2000) report no effects of 

magnet school participation. 

 

9

 For studies examining the impact of magnet schools on minority student isolation or student achievement, see 

Archbald (1988 and 1995); Blank (1990a); Yu, Taylor, Goldring, Smrekar, and Piche (1997); Blank et al. (1983); 

Steel and Eaton (1996); Henig (1996); Rossell (2003); Gamoran (1996a&b); Crain, Allen, Thaler, Sullivan, 

Zellman, Little, and Quigley (1999); Hill, Foster and Gendler (1990); Adcock and Phillips (2000); and Steel and 

Levine (1994). For a bibilography of research on magnet schools, see Magnet Schools of America (2003). For 

reviews of the literature on magnet schools, see ERIC Clearinghouse on Educational Management (2002); Blank 

(1990b), and Goldhaber (1999). 
 I. Introduction 

 I-5

One potential reason for the mixed results that appear in the empirical work on magnet 

schools is that, by design, magnet schools are quite varied. Apart from differences in grade level 

served and instructional focus, magnet schools may also vary in terms of desegregation 

objectives and strategies. There is very little research on the features of magnet schools 

associated with positive outcomes or the role of the contexts in which magnet schools operate. 

The 57 districts receiving MSAP funds beginning in 1998–1999 used the funds to support 

nearly 300 magnet schools. Thus the evaluation makes it possible to describe the variation in 

magnet school characteristics and outcomes, as well as variation in the contexts in which the 

magnet schools operate. While our primary focus in the work reported here is to address the 

major evaluation questions, where possible, we draw connections between the results we obtain 

and the findings in the broader research literature.10

Studies Comprising the Evaluation 

To provide answers to the evaluation questions, we conducted four interrelated studies:11

• Study 1, Profile of All 57 MSAP Projects—descriptive analyses of program 

context, program characteristics, and enrollment and achievement outcomes for all 57 

of the MSAP projects funded in 1998. 

• Study 2, Profile of All MSAP-supported Schools—a focus on the nearly 300 

schools that received program funds in MSAP-funded districts: the school context, 

program characteristics, and enrollment and achievement outcomes. 

• Study 3, In-depth Case Studies—extensive reviews of eight selected MSAP projects 

to illuminate the aggregate results obtained from the national data collection (Studies 

1 and 2). case study districts were not drawn at random, but rather were selected to 

reflect the variety among the 57 projects (e.g., geographic region, size). 

• Study 4, In-depth Case Studies of MSAP Schools—detailed reports on four MSAPsupported schools in each of the eight case study MSAP projects, selected to include 

elementary, middle, and high school levels and a variety of program themes. Also 

included in each district case study are one or two non-magnet schools that serve 

students with racial-ethnic backgrounds similar to those in the MSAP schools, to 

permit comparisons of school programs in general and student achievement in 

particular. 

Data Collected in 1999–2000 through 2001–2002 

During 1999–2000 and 2000–2001, we collected and analyzed data on all 57 projects 

through telephone interviews with the MSAP project directors and through project surveys, also 

completed by the project directors. We also interviewed MSAP project directors (or in some 

 

10 This report focuses primarily on questions III, IV, V, and VI. Questions I, II, and VII were addressed in the 

Year 1 Interim Report (U.S. Department of Education, 2001). 11 A fifth study, a review of the role of the U.S. Department of Education in promoting high quality magnet schools 

and providing technical assistance to grantees, was completed during the first year of the evaluation and is not 

discussed further in this report. 
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees

I-6

cases other knowledgeable district representatives) in fall 2001, after the federal grant period had 

ended. In addition, we gathered data on the 292 MSAP schools through principal surveys in both 

1999–2000 and 2000–2001. Response rates for these data collections are shown in Exhibit I-1 

below.12

In both spring 2000 and spring 2001, we made site visits to eight MSAP projects selected 

for in-depth case studies. The site visits entailed interviews at the district level and observations 

and interviews in four MSAP schools and one or two comparison schools in each district. We 

also surveyed a sample of teachers in each case study school. In addition, we obtained a variety 

of school- and district-level descriptive data (e.g., on student and staff characteristics) from 

districts and state education agencies, as well as school-level achievement data from the National 

School-Level State Assessment Score Database. 

Exhibit I-1. 

Response rates for years 1 through 3 data collection 

Data Collection Instrument N Responses 

Response 

Rate 

MSAP Project Director Interview (1999–2000) 57 57 100% 

MSAP Project Director Interview (2000–2001) 57 57 100% 

Final Project Director Interview (2001–2002) 57 57 100% 

Project Survey (1999–2000) 57 56 98% 

Project Survey (2000–2001) 57 54 95% 

MSAP School Principal Survey (1999–2000) 284 267 94% 

MSAP School Principal Survey (2000–2001) 267 266 93% 

Comparison School Principal Survey (1999–2000) 21 20 95% 

Comparison School Principal Survey (2000–2001) 21 17 81% 

Exhibit reads: The response rate for the MSAP Project Director Interview was 100 percent in 1999–

2000, 2000–2001, and 2001–2002. 

Organization of the Report 

The subsequent chapters in this report are designed to update the description of the 

MSAP projects and to answer the evaluation questions that pertain to the four main legislative 

purposes of MSAP. Chapters II-V present the results of quantitative analyses of data on student 

enrollment and achievement, as well as data drawn from surveys administered as part of the 

evaluation. Chapter II describes the characteristics of the MSAP-supported schools and 

programs. Chapter III discusses innovative educational practices and systemic reform in MSAPschools. Chapter IV examines the desegregation objectives of schools targeted for prevention, 

elimination, or reduction of minority group isolation as defined by MSAP statutes and 

regulations, and assesses whether the schools have met or made progress in meeting those 

objectives. Chapter V assesses the student achievement outcomes in MSAP-supported schools 

between 1998–1999 and 2000–2001. 

 

12 The number of principal surveys distributed was smaller than 292 because not all schools were operational every 

year of the grant. For more information, see the methodology section in the introduction to the Case Studies 

Appendix. 
 I. Introduction 

 I-7

Chapter VI draws on data from the case studies of MSAP projects and schools to place 

the outcomes presented in Chapters II-V in context, and it examines the broader impact of the 

MSAP program. Finally, Chapter VII summarizes the evidence concerning the major research 

questions addressed in this report. 



 II. Characteristics of MSAP-Supported Schools

 II-1

II. Characteristics of MSAP-Supported Schools and Programs 

In 1998 the federal Magnet Schools Assistance Program (MSAP) provided funds for 

programs in 292 schools in the 57 school districts that were awarded MSAP grants. In this 

chapter, we present a picture of the MSAP magnet schools by describing the following: 

• Grade levels of schools served by magnet programs. 

• Types of magnet program structures. 

• Amount and use of funds received by MSAP-supported schools. 

• Recruitment efforts. 

• Enrollment levels and trends in MSAP-supported schools. 

• Characteristics of MSAP and non-MSAP students and schools. 

 Where data are available, we compare the characteristics of 1998 federally funded 

magnet schools to estimates for all federal and nonfederal magnets nationwide in 1999–2000. 

 

Grade Levels of Schools Served by Magnet Programs 

A total of 285 of the 292 magnet schools funded by the 1998 MSAP grants operated 

magnet programs in 2000–2001.1 These schools represent about 5 percent of all magnets 

nationwide, and 9 percent of the magnet schools promoting desegregation. Recent estimates for 

1999–2000 indicate that there are approximately 5,576 magnet schools nationwide, of which 

3,026 have programs intended to promote desegregation.2

The MSAP-supported schools are predominantly located in elementary schools,3 which 

comprise 60 percent of all MSAP schools. Middle schools represent another quarter (23 percent), 

and high schools account for less than one-seventh (14 percent) of the schools at which MSAP 

programs are operating. A handful of programs (3 percent) are in schools that serve a 

combination of grade levels (Exhibit A-II-1). 

 

1

 Initially 292 schools were funded, but the magnet programs in six schools were not operational and two MSAP 

elementary magnet schools combined into a single school. Of the six schools in which the magnet programs were 

not operational, two schools were closed, two schools have programs that never became operational during the 

1998–2001 funding cycle, one school was removed from the district and taken over by the state, and one had its 

magnet status removed by the district after the first year for low performance. Four of the schools were 

elementary and two were middle schools. All operated whole school programs. 2

 The estimates of magnet schools are based on data from the 1999–2000 Schools and Staffing Survey of the 

National Center for Education Statistics. 3 See glossary at the end of this document for definitions used to distinguish elementary, middle, high, and 

combined-level schools. 

MSAP-supported magnet schools are predominantly elementary schools (60 percent), as is 

the case for all magnet schools nationwide (63 percent). 
II. Characteristics of MSAP-Supported Schools

II-2

Comparison of MSAP-supported schools to all magnet schools indicates that both have 

similarly high proportions of elementary schools (Exhibit II-1). Elementary schools represent the 

largest proportion of magnet schools for both federal and nonfederal programs intended to 

promote desegregation.4

 Nationally, a smaller proportion of magnets overall serve middle 

schools compared with MSAP-supported magnets, and a somewhat larger proportion serve high 

schools compared with MSAP-supported magnets. 

Exhibit II-1. 

Comparison of grade levels of MSAP magnet schools with magnet schools nationally 

that promote desegregation 

0%

10%

20%

30%

40%

50%

60%

70%

Elementary Middle High School Combined Levels

Percent of magnets

MSAP

Nationally

Exhibit reads: Elementary schools represent the largest proportion of MSAP magnets and magnet schools 

nationally. 

Source: 1999–2000 NCES: Schools and Staffing Survey: Public School Questionnaire, item 15a, b, and c 

N = 285 MSAP-supported schools in 2000–2001 

Types of Magnet Program Structures 

 

4

 This preponderance of programs in elementary schools is consistent with earlier patterns of magnet school 

programs: a national survey of schools in 1991–1992 indicated that 53 percent of magnet programs are located in 

elementary schools. The percentages are not strictly comparable as the percentages in the study are based on 

programs rather than schools and a school may have more than one program (Steel and Levine, 1994). 

MSAP-supported schools overwhelmingly manage whole school programs (88 percent) that offer the 

magnet to all students in the school who are in the grades at which the program operates. 

Programs-within-schools (PWS) offering magnet curricula to some, but not all, students in a school 

are a small proportion of MSAP’s magnet programs (12 percent) compared with all magnet 

programs (i.e., nonfederal and federal) nationwide (32 percent). PWS programs are most prevalent 

in high schools for MSAP-supported schools (30 percent) and magnet schools generally (56 

percent). 
 II. Characteristics of MSAP-Supported Schools

 II-3

Magnet schools differ in the manner in which they structure enrollment in their programs. 

Some schools offer a magnet program to all students in the school who are in the grades at which 

the program operates. These are referred to as whole school magnet programs. They are 

distinguished from programs within schools (PWSs) that offer magnet curricula to some but not 

all of the students in the school. Of the schools supported by 1998 MSAP grants, almost nine out 

of ten MSAP schools (88 percent) are whole school magnets. PWSs operate in only 33 out of the 

285 MSAP magnet schools (12 percent). PWSs represent a larger proportion of the secondary as 

compared to elementary school programs. Nearly one-third (30 percent) of MSAP high schools 

are PWSs as compared with about one in twenty (6 percent) MSAP elementary schools. 

Over the past decade, the percentage of MSAP supported schools utilizing PWSs has 

decreased. Among magnet schools supported under the FY 1989 and FY 1991 MSAP grants an 

estimated 39 percent operated programs within schools.5

The percentage of PWS magnets is lower among current (i.e., FY 1998) MSAP schools 

compared with estimates for all federal and nonfederal magnet schools in 1999–2000, as shown 

in Exhibit II-2 below. The proportion of MSAP magnets operating a PWS is lower at every grade 

level compared to all magnet schools. However, the pattern across grade levels is similar, with 

PWSs being most prevalent in high schools. 

 

5

 Percentage estimates of PWSs for magnet schools supported by FY 1989 and FY 1991 MSAP grants reported in 

Steel and Eaton (1996), p. 17. 
II. Characteristics of MSAP-Supported Schools

II-4

Exhibit II-2. 

Percent of MSAP schools in with PWS programs compared with percent magnets 

nationwide in with PWS programs by grade level, 2000–2001* 

Exhibit reads: Overall and at every grade level, a higher percentage of magnet schools nationwide in 1999–2000 

have PWS programs compared with MSAP magnets. 

*Nationwide estimates for 1999–2000 based on magnet schools with programs intended to promote desegregation. 

Source: 1999–2000 NCES: Schools and Staffing Survey: Public School Questionnaire, item 15a, b, and c 

N = 285 MSAP-supported schools 

Funding of MSAP-Supported Schools 

The amount of funds received by MSAP-supported schools averaged about $300,000 per 

school year, but the range was considerable across the projects. For example, in 1998–1999, the 

school budgets ranged from $11,000 (for a school with a planning year) to $844,000 (for a 

school establishing a technology-based program). To provide an indication of what grants mean 

at the school level, Exhibit II-3 focuses on school-level MSAP budgets for the 226 MSAPsupported schools (79 percent of all MSAP-supported schools) for which such budgets are 

available. As it indicates, on average, MSAP-supported schools spent about $120,000–$140,000 

on personnel (staff and fringe), about $70,000 on equipment, and about $60,000 on supplies per 

year. 

While the average amount of funds received by MSAP-supported schools is about $300,000 

per year, the range varies considerably across projects. Support for personnel constitutes 

the largest mean budget expenditure. 

0%

10%

20%

30%

40%

50%

60%

70%

Elementary Middle High School All

Percent PWS magnets

MSAP Schools

All Magnet

Schools: 2000-01
 II. Characteristics of MSAP-Supported Schools

 II-5

Exhibit II-3. 

Mean budget amounts for 226 MSAP-supported schools in 46 MSAP projects, by year 

Exhibit reads: Support for personnel constitutes the largest mean budget expenditure of grant funds by MSAPsupported schools, followed by expenditures for equipment and supplies. 

Source: U.S. Department of Education 

N = 226 MSAP-supported schools 

Recruitment of Students 

Recruitment of students is intrinsic to the concept of a “magnet” school because magnet 

schools must try to attract students from outside the immediate neighborhood who would 

otherwise not attend the school. In this section, we describe district efforts to recruit applicants to 

MSAP programs. 

Personnel Resources 

The personnel that districts can use to assist in recruiting efforts include the MSAP 

Project Director and other district staff, along with principals and teachers at the school level. In 

addition, magnet programs often involve students and their parents in outreach activities that 

publicize the school’s program. 

Efforts to recruit students for MSAP programs are coordinated by the MSAP project. While 

MSAP-funded personnel are responsible for most of the recruitment effort, district-funded 

recruitment specialists also provide support. In most MSAP districts, the principals, teachers, 

and students at MSAP-supported schools participate in the recruitment effort through 

presentations, school tours, or visits to other schools. MSAP districts typically engage in over 

10 types of outreach activities. One-third of districts do not focus their outreach on any 

particular group of students, and less than half report that they focus most of their outreach 

materials and activities on targeted groups of students. 

$0

$20,000

$40,000

$60,000

$80,000

$100,000

$120,000

$140,000

$160,000

Personnel

& Fringe

Travel Equipment Supplies Contractual Constructn Other Direct Training

Stipends

Indirect

Costs

Budget Category

Dollars per year

1998-1999

1999-2000

2000-2001
II. Characteristics of MSAP-Supported Schools

II-6

Estimates from MSAP Project Directors indicate that an average full-time equivalent 

(FTE) of 2.1 staff is devoted to the MSAP projects’ recruitment efforts. MSAP-funded staff are 

responsible for most of that recruitment effort, although district-funded recruitment specialists 

also provide support. 

Exhibit II-4 indicates the percentage of MSAP districts that engage in various types of 

recruitment strategies. In almost all of the districts, principals and assistant principals organize 

school tours, and teachers make presentations to visiting parents and students. In more than twothirds of the districts, teachers also assist by visiting other schools to meet with potential 

students. Students play a role in the recruitment effort in more than 80 percent of the districts by 

conducting school tours; however, only one-third of the districts have students visit other schools 

to help recruit students. Parents are the least used resource for recruitment. Just half of the 

districts report that parents have been enlisted to make presentations to other parents about 

magnet schools. In one-third of the districts, parents make telephone calls to provide information 

to others about the magnet schools. 

Exhibit II-4. 

Percentage of districts with specific types of recruitment efforts by school staff, 

students, and parents 

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Other

Parents make telephone calls

to provide information

Parents make presentations

to other parents

Students visit other district

schools to recruit

Students help conduct school

tours

Magnet school teachers visit

other schools to recruit

students

Magnet teachers make

presentations to visitors

Principals/assistant principals

organize school tours

Percent of Districts

Exhibit reads: Nearly all MSAP districts utilize magnet school principals and teachers to assist with on-site 

recruitment of students, and most involve magnet teachers in off-site recruitment of students through visits to other 

schools. Most MSAP districts also utilize magnet students in their recruitment effort, while parents of magnet 

students are less likely to be included in the district's recruitment strategy. 

N = 56 districts 

Source: Project Survey, 1999–2000, Item 6 
 II. Characteristics of MSAP-Supported Schools

 II-7

Outreach Activities 

The typical MSAP district engages in more than 10 outreach activities to provide 

information to students, parents, and the community about its magnet programs. The number of 

activities is one indicator of the importance that most districts place on outreach. Exhibit II-5 

displays the wide range of outreach activities that districts report using. The most frequently used 

strategies include making printed brochures available and distributing information and 

applications to students. Nine out of ten districts also plan tours of their MSAP schools for 

visiting parents and students, mail information and applications to parents upon request, and 

advertise in the media and at fairs or forums. Eight out of ten schools take their presentations on 

the road and make presentations to church congregations or other community organizations and 

schools. More than two-thirds canvass their districts by mailing information and applications to 

the parents of all eligible students. 

Exhibit II-5. 

Percentage of districts using specific types of outreach to recruit students 

Exhibit reads: While the most frequently used forms of outreach for recruiting students include printed brochures 

and information distributed directly to students, telecommunications also plays a role in the outreach activities of 

most districts. Outreach activities that require additional personal services or attention, such as home visits or 

provision of transportation, are used by a smaller percentage of districts. 

N = 56 districts 

Source: Project Survey, 1999–2000, Item 7 

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Other

Home visits to parents/students

Tours of school with transportation provided

Babysitters provided for school events

Telephoning parents to explain program

Information on an Internet site

Information mailed to all parents

Presentations at other schools

Presentations at churches or community organizations

Advertising at fairs or other forums

Media advertisements

Information mailed to parents who request it

Tours of school without transportation provided

Information distributed to students

Brochures

Percent of Districts
II. Characteristics of MSAP-Supported Schools

II-8

Telecommunications plays a role in the outreach activities of most districts. Two-thirds 

of districts use an Internet site to convey information about their programs. A similar proportion 

of districts report telephoning parents of potential students to explain their MSAP programs. 

Strategies that require personal service or attention are used by a much smaller 

percentage of districts. For example, one-quarter to one-third of the districts provide 

transportation or babysitting services to allow parents to tour the MSAP school or participate in a 

school event. Less than one-fifth of the districts make home visits to parents of potential 

students. 

Targeted Recruitment 

MSAP districts vary considerably in the extent to which their recruitment efforts focus on 

targeted groups of students. One-third of districts recruit students from throughout the district 

and make no effort to target any particular group of students. Slightly less than half of the 

districts indicate that they focus most of their efforts on targeted groups of students. Less than 

one-fifth of the districts focus all of their outreach on targeted groups of students (Exhibit A-II9). 

Reserve Pool of Applicants 

The success of recruitment efforts is indicated to some extent by applicants waiting to get 

into magnet program. While exact counts of reserve applicants are not available, nearly twothirds (63 percent) of MSAP Project Directors report that they have one or more schools for 

which there are more applicants than seats available. Almost all of the directors reporting an 

excess of applicants indicated that students are admitted after the start of the school year if space 

becomes available. For programs in which there is not a single grade for entry, applicants who 

are not admitted in one year typically have to reapply for entry in a subsequent year.6

Program Enrollment 

Total Enrollment 

The 285 MSAP schools enrolled nearly 184,000 students, including the students in PWS 

magnets who were enrolled in the magnet curricula and all students in schools with whole school 

programs (Exhibit A-II-11). About half of all these MSAP students (52 percent) were enrolled in 

 

6

 While project directors responded to a specific question about waiting lists, we learned that many districts no 

longer maintain waiting lists, per se, but instead maintain a reserve pool of applicants from which additional 

names may be drawn as seats become available. 

MSAP-supported school programs enrolled nearly 184,000 students in 2000-2001, about 

one-half of whom were in elementary schools, one-quarter in middle schools, and one-fifth 

in high schools. The vast majority of MSAP students at all grade levels were enrolled in 

whole school programs. Minority students represent three-quarters of students in whole 

school programs and slightly less than two-thirds of students in PWS programs. 
 II. Characteristics of MSAP-Supported Schools

 II-9

elementary schools, one-quarter (25 percent) in middle schools, and about one-fifth (20 percent) 

in high schools. The remaining students (4 percent) were enrolled in schools serving combined 

grade levels. 

The MSAP enrollment data indicate that the vast majority of MSAP students (94 percent) 

are enrolled in whole school programs (Exhibit A-II-12). Although only 6 percent of MSAP 

students are enrolled in PWS magnets overall, the percentage ranges from 2 percent of 

elementary school students to 9 percent of middle school students to 12 percent of high school 

students. 

The number of schools operating MSAP programs increased from 263 in 1998–1999 to 

285 in 2000–2001.7

 The number of students offered magnet curricula in MSAP-supported 

schools increased by more than 18,000 students, or 11 percent, between the first and third year of 

the projects. Three-quarters of the increase (76 percent) reflected the greater number of MSAPsupported schools in operation in 2000–2001 compared with the number operating in 1998–

1999. The remainder of the increase reflected a slight increase in the overall average enrollment 

in MSAP-supported magnet schools. The distribution of students by grade level and type of 

program is quite constant in both the first and third years of the programs. 

Minority Enrollment 

MSAP magnets are intended to bring students from different socioeconomic, ethnic, and 

racial backgrounds together and to improve the educational opportunities of students. In MSAP 

programs, minority students constitute three-quarters (75 percent) of student enrollments and 70 

percent or more of students enrolled at every grade level (Exhibit A-II-13). 

Minority students represent three-quarters (75 percent) of students in whole school 

programs, compared with slightly less than two-thirds (63 percent) of PWS magnet students. As 

shown in Exhibit II-6, the difference in representation of minority students in PWS and whole 

school programs is more apparent in MSAP high schools than in middle and elementary schools. 

 

7

 Data for 1998–1999 reported in the Year 1 Interim Report showed magnet program enrollment of 165,789 

students including an estimated 157,013 in whole schools and 8,776 in PWS programs. U.S. Department of 

Education (2001), p. I-3 and Appendix I-4. 
II. Characteristics of MSAP-Supported Schools

II-10

Exhibit II-6. 

Percent minority enrollment in PWS and in whole school programs, by grade level: 2000–

2001 

Exhibit reads: Across all grades, minority students constitute 75 percent of the enrollment in whole school magnet 

schools, compared to about 63 percent of the enrollment in programs within schools (PWS). The gap is larger for 

high schools than elementary schools. 

N = 285 MSAP schools for all grades, including: 33 PWS and 252 whole school magnets overall; 11 PWS and 161 

whole school magnets at the elementary level; 10 PWS and 56 whole school magnets at the middle school level; and 

12 PWS and 28 whole school magnets at the high school level. There are 7 whole school magnets with combined 

grade levels with an average minority enrollment of 70 percent. 

Source: MSAP 2000–2001 performance reports; MSAP Principal Survey, 2000–2001, Item 24; and NCES 2000–

2001 Common Core of Data 

Characteristics of MSAP and Non-MSAP Students and Schools 

Student Characteristics 

How do students attending MSAP-supported schools compare to students attending nonMSAP schools? We examine this question by comparing the racial-ethnic, economic, and other 

While on average minority students represent a larger percentage of students in a district’s 

MSAP schools (73 percent) than in the non-MSAP schools (63 percent), the average in both 

exceeds the minority enrollment threshold by which the federal government defines minority 

group isolation. Available data indicate that on average, students in MSAP and non-MSAP 

schools within a district are similar in terms of the proportions who are eligible for free and

reduced price lunches (60 vs. 58 percent), of Limited English Proficiency (14 vs. 12 

percent), and who have Individualized Education Plans (13 vs. 12 percent). However, a 

larger proportion of MSAP-supported schools operate Title I programs. 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Elementary Middle High School All Grades

Percent of Students

PWS

Whole
 II. Characteristics of MSAP-Supported Schools

 II-11

characteristics of students attending MSAP and non-MSAP schools within the same district and 

comparing the average across districts for each group. 

Comparisons of the racial and ethnic composition of MSAP and non-MSAP schools in 

MSAP districts for 1999–2000 indicated the following:8 

• On average, minority students represent a larger percentage of the student enrollment 

in a district’s MSAP schools (73 percent) than in non-MSAP schools (63 percent) at 

every grade level (Exhibit A-II-15) as might be expected since MSAP programs tend 

to be placed in schools that are initially more minority isolated.9

• The average proportion of minority students for both MSAP and non-MSAP schools 

exceeds the 50 percent minority enrollment threshold by which the federal 

government defines minority-isolated schools. 

• Black students average a larger percentage of students in MSAP schools (42 percent) 

than in non-MSAP schools (34 percent); the average percentage of students from 

other racial-ethnic minorities was similar for MSAP and non-MSAP schools (Exhibit 

A-II-15). 

Comparison of the economic status of students in MSAP and non-MSAP schools showed 

the following: 

• Although the average proportion of MSAP students eligible for free or reduced price 

lunches (60 percent) is slightly higher than that of non-MSAP students (58 percent), 

more than half of the students in both MSAP and non-MSAP schools are eligible for 

free or reduced price lunch (Exhibit A-II-16). 

Data available for some of the MSAP districts provide evidence for similarity in the 

composition of students attending MSAP and non-MSAP schools regarding the percentages with 

Limited English Proficiency (LEP) and Individual Education Plans (IEPs). Counts of the number 

of LEP students for each school within a district were obtained from districts and state agencies 

for 24 of the 57 districts (42 percent) that represent 140 (49 percent) of the MSAP schools. 

Counts of the number of students with IEPs were obtainable for a third of the districts (33 

percent), also representing about one-third (35 percent) of the MSAP schools. The percentages of 

MSAP students who are LEP or who have IEPs were compared with the percentage of LEP and 

IEP students attending non-MSAP schools that serve a similar grade level to those grade levels 

served by each district’s MSAP schools. The results of this analysis indicated that: 

• The mean proportion of students who are LEP in MSAP schools (14 percent) is 

similar to the mean proportion in non-MSAP schools (12 percent) (Exhibit A-II-17). 

 

8

 Results are based on comparisons of MSAP and non-MSAP schools in 55 of the 57 MSAP districts for which 

data were available in the 1999–2000 CCD. Of the two districts not included, data were not available for one 

district and there were no comparison schools at the same grade level for the second district. The results of these 

analyses are quite similar to the analyses of 1998–1999 CCD data reported in the Year 1 Interim Report. 9

 Data from the 1999–2000 Schools and Staffing survey indicate that the average proportion of minority students 

in magnet schools with an explicit desegregation objective is 66 percent, and that the average for all magnet 

schools, including those without a desegregation objective, is 57 percent. 
II. Characteristics of MSAP-Supported Schools

II-12

• The mean proportion of IEP students in MSAP schools (13 percent) is similar to the 

mean proportion in non-MSAP schools (12 percent) (Exhibit A-II-18). 

Title I Programs in MSAP-Supported and Non-MSAP Schools

Title I programs serve disadvantaged students who are considered to be at risk of poor 

academic achievement in school because they come from low-income families or lack 

proficiency in English. Schoolwide programs that offer Title I assistance to all students in a 

school are reserved for schools in which 50 percent or more of the students qualify to receive 

support from a Title I program. For schools in which less than 50 percent of students qualify, 

Title I assistance is targeted to only those students who qualify. 

The percentage of MSAP-supported schools operating a Title I program is higher than the 

national average and higher than non-MSAP schools within the district that serve the same 

grades as the MSAP schools. As seen in Exhibit II-7, about 71 percent of MSAP schools receive 

Title I program funds, compared to 50 percent of the non-MSAP schools in those districts and 59 

percent of schools in the United States overall.10 

The high proportion of MSAP-supported schools with a schoolwide Title I program is 

evidence of the prevalence of disadvantaged students in MSAP schools. More than half of the 

MSAP schools (60 percent) report operating a schoolwide Title I program, compared with 43 

percent of non-MSAP schools. The proportion of MSAP schools with schoolwide Title I 

programs is twice the national percentage (27 percent) (Exhibit A-II-19). 

 

10 Data for schools in 32 of the 57 MSAP districts were obtained from the 1999–2000 CCD, while data for the 

United States were obtained from a national survey as receiving Title I funds. 
 II. Characteristics of MSAP-Supported Schools

 II-13

Exhibit II-7. 

Percentage of MSAP-supported schools with targeted and schoolwide Title I programs 

compared with non-MSAP schools in districts and schools 

nationally

60%

43%

27%

7%

32%

11%

0%

10%

20%

30%

40%

50%

60%

70%

80%

MSAP Non-MSAP Nationally

Percent of schools

Targeted

Schoolwide

 

Exhibit reads: About 60 percent of MSAP schools have Title I schoolwide programs, compared with 43 percent of 

non-MSAP schools in the same districts, and 27 percent of schools nationally. 

N = 32 out of 57 MSAP districts 

Source: Data for schools in MSAP districts are from NCES 1999–2000 Common Core of Data. National data are for 

the 1997-1998 school year as reported from the Follow-up Public School Survey on Education Reform by Heid and 

Webber (1999). 

Pupil-Teacher Ratios in MSAP and Non-MSAP Schools 

Pupil-teacher ratios provide some insight into the way a district uses its educational 

resources. How does the staffing of MSAP-supported schools compare with non-MSAP schools? 

Exhibit II-8 shows that the average pupil-teacher ratio falls within a range of 16 to 18 students 

per teacher for both MSAP and non-MSAP schools at all grade levels. At the elementary grade 

level, MSAP schools average one less pupil per teacher than non-MSAP schools in their district. 

At the high school level, MSAP-supported schools average two fewer students per teacher. In 

middle schools, the average is about one pupil more per teacher. 
II. Characteristics of MSAP-Supported Schools

II-14

Exhibit II-8. 

Comparison of average pupil-teacher ratio in MSAP and non-MSAP schools in the same 

district by grade level 

0.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

16.0

18.0

20.0

Elementary Middle High School

Pupil Teacher Ratio

MSAP

Non-MSAP

Exhibit Reads: Average pupil teacher ratios of MSAP and non-MSAP schools at every grade level are between 16 

and 18 students per teacher. 

N = 51 districts 

Source: NCES 1999–2000 Common Core of Data 

Summary 

MSAP-supported magnet schools are predominantly elementary schools (60 percent), as 

is the case for all magnet schools nationwide (63 percent). Almost one-quarter (23 percent) of 

MSAP-supported magnet schools are middle schools, while high schools account for about oneseventh (14 percent) of MSAP schools.

MSAP-supported schools overwhelmingly manage whole school programs (88 percent) 

that offer the magnet to all students in the school who are in the grades at which the program 

operates. Programs-within-schools (PWS) offering magnet curricula to some, but not all 

students, in a school are a small proportion of MSAP’s magnet programs (12 percent) compared 

with all magnet programs nationwide (32 percent). PWS programs are most prevalent in high 

schools for MSAP-supported schools (30 percent) and magnet schools generally (56 percent). 

While the average amount of funds received by MSAP-supported schools was about 

$300,000 per year, the range varied considerably across projects. Support for personnel 

constituted the largest mean budget expenditure. 

Recruitment efforts for MSAP programs are coordinated by the MSAP project. While 

MSAP-funded personnel are responsible for most of the recruitment effort, district-funded 

recruitment specialists also provide support. In most MSAP districts, the principals, teachers, and 

students at MSAP-supported schools participate in the recruitment effort through presentations, 
 II. Characteristics of MSAP-Supported Schools

 II-15

school tours, or visits to other schools. MSAP districts typically engage in more than 10 types of 

outreach activities. One-third of districts do not focus their recruitment effort on any particular 

group of students, and less than half report that they focus most of their efforts on targeted 

groups of students. 

MSAP-supported school programs enrolled nearly 184,000 students in 2000–2001, about 

one-half of which were in elementary schools, one-quarter in middle schools, and one-fifth in 

high schools. The vast majority of MSAP students at all grade levels were enrolled in whole 

school programs. Minority students represent three-quarters of students in whole school 

programs and slightly less than two-thirds of students in PWS programs. 

While on average minority students represent a larger percentage of students in a 

district’s MSAP schools (73 percent) than in the non-MSAP schools (63 percent), the average in 

both exceeds the minority enrollment threshold by which the federal government defines 

minority group isolation. Available data indicate that on average, students in MSAP and nonMSAP schools within a district are similar in terms of the proportions who are eligible for free 

and reduced price lunches (60 vs. 58 percent), who are of limited English proficiency (14 vs. 12 

percent), or who have an Individualized Education Plan (13 vs. 12 percent). However, a larger 

proportion of MSAP-supported schools operate Title I programs. 

 III. Support for Systemic Reform 

 III-1

III. Support for Systemic Reform and Innovative Educational 

Practices in MSAP Schools 

Two of the purposes of MSAP projects are to “assist local educational agencies in 

achieving systemic reforms and providing all students the opportunity to meet challenging State 

content standards and challenging State performance standards,” and to engage in “the 

development and design of innovative educational methods and practices” in public schools.1

This chapter examines the extent to which MSAP schools are achieving these legislative 

purposes. In examining state and district systemic reform efforts, we focus on a set of external 

supports for change, including alignment with standards and assessments and coordination with 

other reforms. In examining innovative practices in MSAP projects, we take a broad view, 

focusing on innovation in a number of central elements of the school program. These include 

innovation in the overall school program or mission (e.g., a new school focus or theme); in 

supports for teaching and learning (e.g., new professional development activities for teachers, or 

improvements in the school climate); and in instructional practices (e.g., new curriculum content 

or pedagogy). 

Drawing on recent research on school effectiveness, we hypothesize that changes in these 

areas may lead to improved student achievement in several ways. In particular, as shown in 

Exhibit III-1, we anticipate that adopting a clear school mission or focus and implementing 

strong supports for teaching and learning will help improve the quality of instruction (Bryk, Lee, 

and Holland, 1993; Coleman and Hoffer, 1987; Newman and Associates, 1996; Purkey and 

Smith, 1983; Sebring and Bryk, 2000). Improvements in instructional quality, in turn, should 

lead to gains in student achievement (Brophy and Good, 1986; Carpenter, Fennema, and Franke, 

1996; Gamoran, Porter, Smithson, and White, 1997; Knapp, Shields, and Turnbull, 1992; 

National Reading Panel, 2000; Taylor, Pearson, Clark, and Walpole, 1999; Teddlie and 

Stringfield, 1993; Wong, Hedges, Borman, and D’Agostino, 1996). 

 

1 As described in Chapter I, MSAP was recently reauthorized under No Child Left Behind (NCLB, 20 U.S.C. 

6301, Section 5301(b)(2–3)). During the period covered by the work reported here, MSAP was operating as 

authorized under the Improving America’s Schools Act (IASA) of 1995. The two purposes discussed in this 

chapter were included in both IASA and NCLB. 
III. Support for Systemic Reform

III-2

Exhibit III-1. 

Conceptual framework: Connections between improved student achievement and 

various aspects of magnet school educational programs and contexts2

 

2

 Italicized topics have been addressed in this report and/or the year 1 report. For several reasons, a few topics 

have not been addressed in either report. Although measures of all of the topics pertaining to characteristics of 

the magnet program, external supports for change, and supports for teaching and learning were included on the 

principal survey, an examination of the data suggested that a few of the measures were not sufficiently strong or 

reliable to include in our reported results. Data on classroom instructional practices were gathered primarily on 

the teacher survey. Because the teacher survey was administered only in MSAP case and matched non-MSAP 

comparison schools, we have given data from the teacher survey somewhat less emphasis than the universe data, 

which are based on the principal and project surveys. 

Classroom instructional 

practices

• Challenging content 

• Appropriate instructional 

methods 

• Appropriate approaches to 

assessment 

• Use of diverse instructional 

materials including technology

• Classroom composition 

• Instructional time

Characteristics of magnet 

school program

• Reach (percent of students 

participating) 

• Depth (extent involves whole 

day)

• Subject focus of magnet theme 

• History (new program vs. 

renewal)

External supports for 

change

• Alignment with standards and 

assessments 

• Coordination with other reforms

• Accountability for achievement

• District MSAP support for 

schools

• Autonomy and flexibility

• Performance on standardsbased assessments 

• Performance of specific 

vocational skills 

• Improved attendance rates

• Improved graduation rates 

Student achievement

Supports for teaching and 

learning

• Ongoing professional 

development for teachers 

• Professional community among 

teachers 

• Positive school climate

• Mechanisms for student 

academic support and 

personalization 

• Mechanisms for parental 

involvement

• High standards for all students 

• Organization of school day and 

staff configuration focused on 

learning

• School leadership focused on 

teaching and learning 

• Continuous improvement

processes
 III. Support for Systemic Reform 

 III-3

Data Sources 

The results on innovative practices and systemic reform reported in this chapter are based 

primarily on a set of surveys administered to project directors, school principals, and teachers. 

These survey data are complemented by interview and observational data we collected from 

selected MSAP and matched comparison schools in a sample of eight case study districts that 

received MSAP funds. Data from the case studies are reported in detail in the Case Studies 

Appendix. In general, conclusions based on the principal and teacher survey data are consistent 

with information drawn from the case studies. 

Most of the results reported in this chapter are based on data gathered from the full

population of MSAP projects and schools. We surveyed the full population of 57 MSAP project 

directors in the second and third years of the MSAP grants (1999–2000 and 2000–2001), and we 

also surveyed the principals of the full population of MSAP schools in the same years. 

In addition to these full population surveys, we conducted several data collection 

activities in the purposive case study sample of eight MSAP projects. In each of these eight case 

study projects, we selected a set of MSAP schools and matched non-MSAP comparison schools 

for study. In both 1999–2000 and 2000–2001, we surveyed the principals in the non-MSAP 

comparison schools, using a survey parallel to the form used in the national survey of MSAP 

principals. In addition, we surveyed a sample of teachers in both the MSAP and comparison 

schools.3

 The data collected in the case study districts enable us to compare practices in MSAP 

schools with practices in other schools in the same districts. While these comparisons allow us to 

examine possible effects of MSAP support on school practices, the results must be interpreted 

with care. Although we made an effort to choose comparison schools that are well matched to 

the MSAP schools in terms of student composition, size, and other features, the schools may 

differ in ways we could not take into account in the matching process. Thus, when we find 

differences in practices between MSAP and comparison schools, we cannot be sure that these 

practices are due to MSAP support.4 The qualitative case study data reported in the Case Study 

Appendix cast further light on the extent to which the differences we observed between MSAP 

and comparison schools can be attributed to MSAP support. 

This chapter is written as a companion to Chapters IV and V in the Evaluation of the 

Magnet Schools Assistance Program, 1998 Grantees: Year 1 Interim Report, which reported the 

 

3

 See Appendix A-I for a description of the case study sample and the sampling plan for teachers. See the Case 

Study Appendix for a detailed description of each case study project and district. 4

 To assess whether the case MSAP schools are representative of the full population of MSAP schools, we 

conducted a number of analyses to test for differences in key measured variables between the MSAP schools in 

the case sample and the full population of MSAP schools. Overall, few differences were observed. No significant 

differences were found between the MSAP-funded schools included in the case study sample (N = 30) and those 

not included (N = 256) in Title I status (e.g., not eligible, targeted assistance, or schoolwide assistance) or in 

principal-reported measures of ongoing professional development for teachers, professional community among 

teachers, positive school climate, mechanisms for parental involvement, high standards for all students, and 

alignment with standards and assessments. We conducted a number of analyses to examine possible 

demographic differences between the MSAP and non-MSAP matched comparison schools in the case study 

sample. No significant differences were found in the school level served, Title I status, or years of teaching and 

administrative experiences of the principals. 
III. Support for Systemic Reform

III-4

results of the 1999–2000 project director and principal surveys. The organization of the chapter 

is based on the conceptual framework presented in Exhibit III-1. In each section, we focus on 

three types of analyses. First, we provide descriptive information on MSAP schools in 2000–

2001, the third year of program support, drawing on the principal surveys and on surveys of 

teachers in the case study schools. Data for some topics in the conceptual framework were only 

collected in 1999–2000, and for these topics we briefly review the results for 1999–2000. 

Second, we examine whether practices in MSAP schools improved between the second and third 

years of the project for those topics asked about in both years. Finally, we examine whether 

differences are observed between MSAP and comparison schools in the case study districts for 

those topics covered with both groups of schools. Appendix Exhibit A-III-44 provides a 

summary of the measures we draw on in the chapter. 

In Chapter IV, we examine the relationship between variables reported here and changes 

in schools’ minority student isolation during the period 1998 to 2001. In Chapter V, we examine 

the relationship between these variables and improvement in schools’ student achievement. 

External Supports for Change 

We examined several aspects of external support for change in MSAP schools, including 

the influence of state curriculum frameworks and assessments, and district support for MSAP 

schools (see Exhibits A-III-1 through A-III-14). 

Familiarity with and Influence of State Curriculum Frameworks and Assessments 

 State curriculum frameworks and assessment systems have come to be a major source of 

external support for systemic reforms designed to improve student achievement. In the 2000–

2001 principal survey, principals were asked a series of questions about their familiarity with the 

mathematics and language arts frameworks and assessments and the influence of these 

frameworks and assessments on their instructional programs. Principals report a high degree of 

familiarity with standards and assessments, and they indicate that the content of state standards 

and assessments match the goals of their magnet programs. The majority of principals report that 

they are “very familiar” with both sets of curriculum frameworks (72 percent for mathematics 

and 78 percent for language arts) and their school’s scores on the statewide assessment (92 

percent for both subjects). Additionally, a majority (ranging from 88 to 91 percent) reported that 

the frameworks and assessments guided “to a great extent” decisions regarding curriculum and 

instruction. These results are very similar to the 1999–2000 survey responses of the project 

directors, who reported being “quite familiar” with the frameworks, assessments, and 

performance standards for both mathematics (70 percent) and language arts (77 percent) and that 

the frameworks, assessments, and performance standards match “to a great extent” the MSAP 

schools’ instructional goals in mathematics and language arts (89 percent each) (U.S. 

Department of Education, 2001). 

MSAP principals report a high degree of familiarity with state standards and assessments, and 

they indicate that the content of state standards and assessments guided decisions regarding 

curriculum and instruction in their magnet programs. 
 III. Support for Systemic Reform 

 III-5

A majority of MSAP principals also reported that the state assessments reflect the goals 

of their schools’ magnet program (69 percent for mathematics and 71 percent for language arts). 

These percentages were slightly higher than those of the project directors in the 1999–2000 

survey where a majority (55.6 percent) reported that the state frameworks and assessments 

influenced “to a great extent” their MSAP themes and goals. 

The influence of state systemic reforms, as reported by principals in 2000–2001, differed 

by school level and subject focus. Elementary principals reported greater familiarity with both 

the mathematics and language arts curriculum frameworks and assessment scores than high 

school principals (p<0.01). Additionally, elementary principals reported that the language arts 

assessments guide curriculum and instruction decisions to a greater extent than did high school 

principals (p<0.05). 

Our case study data on the effects of systemic reform on magnet schools are consistent 

with the survey data in that they indicate that local schools are highly familiar with state 

standards and assessments, and magnet programs are generally aligned with state standards. The 

case data provide a somewhat more complex picture, however. In particular, the data show that 

some MSAP schools altered their initial plans to bring their curriculum more in line with 

standards and assessments, or reduced their emphasis on novel programs to increase the time for 

work directly related to standards and assessments. We consider these issues in more detail in 

Chapter VI. 

Change in District-level Supports between the First and Third Year of the MSAP 

Grant 

District-level support from the MSAP project director is another key source of external 

support for the implementation and maintenance of magnet programs. In our analysis of data 

from the 1999–2000 project director survey, we found that a majority of project directors 

provided technical assistance to MSAP schools on issues relating to curriculum and instruction 

more than once a month during the first year of the MSAP grant (U.S. Department of Education, 

2001). By the third year of the grant, MSAP project directors were spending less time providing 

technical assistance to MSAP schools with regard to both curriculum and instruction and 

leadership than they did during the first year of the grant (p<0.10), but were still providing such 

technical assistance at least once a month on average. (See Exhibit III-2.) These findings indicate 

that project directors were heavily involved during the start-up phases of the new magnet 

program—crafting and implementing a specific academic curriculum and establishing schoollevel support for the magnet program. With this accomplished, as one might expect, they spent 

less of their time on these matters in subsequent years. 
III. Support for Systemic Reform

III-6

Exhibit III-2. 

Percent of MSAP project directors providing MSAP schools with technical assistance on 

curriculum and instruction from never to once a week or more, 1998–1999 and 2000–2001 

0

5

10

15

20

25

30

35

40

45

50

Never Less than once a

month

About once a

month

About once

every 2 weeks

Once a week or

more

Percent of MSAP Project Directors

1998-1999

2000-2001

Exhibit reads: 28 percent of MSAP project directors provided their MSAP schools with technical assistance on 

curriculum and instruction once a week or more in 1998–1999, but only 14 percent of them offered such technical 

assistance at that frequency in 2000–2001. 

N = 51 projects 

Source: MSAP Project Survey, 1999–2000, Item 13; MSAP Project Survey, 2000–2001

External Supports in MSAP and Comparison Schools 

Based on the 2000–2001 principal survey data, there were almost no differences in the 

external supports for MSAP and non-MSAP matched comparison schools in the case study 

sample. There were no differences between the two groups in their familiarity with assessments 

and standards or their reported influence on curriculum and instruction. Interestingly, 

comparison schools reported that both the state mathematics and language arts assessments 

reflect their instructional programs to a greater extent than at the MSAP schools. However, this 

difference was only significant for language arts (p<0.05). 

Characteristics of Magnet School Programs 

MSAP schools have adopted a diverse set of themes. Over a third of MSAP schools included 

technology among their themes, and more than a quarter of MSAP magnet schools included a 

science theme. Arts, communication, and mathematics were also common themes. Survey data 

indicate that the programs adopted in MSAP schools include specialized elective courses and 

added program requirements in secondary schools; and the magnet programs have led to 

changes in English and language arts and mathematics instruction in both elementary and 

secondary schools. 
 III. Support for Systemic Reform 

 III-7

In our initial conceptualization of the key organizational and instructional features of 

magnet schools, we defined four features that we believed would be particularly important in 

describing the quality of the implementation of magnet programs: the theme or focus area of the 

program, the program’s reach and depth, and the school’s prior experience with magnet 

programs. Brief descriptions of each of these characteristics follow. 

• Magnet school programs select themes or focus areas (e.g., science and technology, 

arts and communication) that are intended to guide curricula and instruction. To 

describe the themes, we drew on data from MSAP applications and other materials. 

Some schools adopt a comprehensive school reform (CSR) model to reinforce their 

theme (e.g., the “Co-NECT” school reform model emphasizes the use of technology 

in instruction to improve student achievement). Therefore, principals were asked 

what, if any, CSR models their school had adopted. 

• The reach of a magnet school program is the proportion of students involved in the 

magnet program. To measure reach, we included items on the principal survey asking 

whether the magnet program was available to all students in the school, or was 

instead intended for a subgroup of students. 

• The depth of a magnet school program is the proportion of a student’s instruction 

affected by the magnet program. To measure depth we asked secondary school 

principals a series of questions about how the magnet program influenced which 

courses students took. We also asked teachers if the magnet program led them to 

make changes in their instruction. 

• A school’s previous experience or history with magnet programs prior to the 1998 

MSAP grant was measured through a question on the principal survey. 

Themes of Magnet School Programs 

In the Year 1 Interim Report, we provided descriptive information on a number of 

characteristics of the magnet programs in MSAP schools, focusing in particular on the theme or 

focus area and the adoption of comprehensive school reform models. As discussed in the earlier 

report, the most popular MSAP themes were science and technology.5

 More than a third of 

MSAP magnet schools included technology among their themes, and more than one-quarter 

included science. The arts, communication, and mathematics were also common themes. (See 

Exhibit III-3.) Also about 54 percent of principals of MSAP schools reported that they had 

selected or developed a comprehensive school reform model.6

 Analysis of principal survey data 

 

5

 See Exhibits A-V-1 and A-V-2 in U.S. Department of Education, Office of the Under Secretary, Planning and 

Evaluation Service, Elementary and Secondary Education Division. Evaluation of the Magnet Schools Assistance 

Program, 1998 Grantees: Year 1 Interim Report Appendices, Washington, D.C., 2001, Chapter V, pp. 1–2. 6

 Externally developed CSR models adopted by MSAP schools include Accelerated Schools, Audrey Cohen 

Purpose-Centered Education, Coalition of Essential Schools, Comer School Development Program, Core 

Knowledge, Different Ways of Knowing, Direct Instruction, the Foxfire Fund, High Schools That Work, HighScope Primary Grades Approach to Education, HOSTS, Integrated Thematic Instruction, International 

Baccalaureate, Junior Great Books, the Learning Network, Lightspan, Literacy First, Microsociety, Modern Red 

Schoolhouse, Montessori, Paideia, Success For All-Roots and Wings, Success-in-the-Making, and Talent 

Development High School. 
III. Support for Systemic Reform

III-8

from the case study sample indicates that MSAP schools were much more likely to have adopted 

a school reform model than the non-MSAP matched comparison schools (47 to 25 percent). 

Exhibit III-3. 

Percentage of MSAP schools with selected themes 

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Technology Science Math Arts Communication

Percent of MSAP Schools

Elementary

Middle

High

Exhibit reads: 33 percent of elementary schools, 31 percent of middle schools, and 45 percent of high schools had 

technology as one of their themes. (Schools could, and many do, have multiple themes.) 

N = 173 elementary schools, 70 middle schools, and 40 high schools 

Source: Grantee applications; MSAP Principal Survey, 1999–2000, Item 49 

Reach and Depth of Magnet School Programs 

We found very little variation in the reach of the magnet programs, but greater variation 

in program depth. As we reported in Chapter II, in almost all magnet schools, magnet programs 

were whole school programs offering magnet curricula to all students enrolled. Only 12 percent 

of the programs targeted specific students within a school. 

To measure depth in the full population of MSAP secondary schools, we included a small 

set of items on the principal survey. In particular, we asked the secondary principals whether the 

school provided special elective courses as part of the magnet program; whether students were 

able to select a theme or area of specialization, and, if so, if the selection of a theme or area was 

required; and if students were required to complete courses in addition to those required in other 

high schools in the district. We reasoned that high school magnet programs with these 

characteristics would have greater curricular depth than schools without these features.7

Overall, the 2000–2001 principal survey data indicate that most MSAP secondary schools 

offer special courses and permit students to select areas of specialization. More than three 

 

7

 Because elementary schools generally offer instruction in self-contained classrooms rather than through electives 

and other courses, we did not include items on electives, areas of specialization, and requirements for elementary 

school principals. 
 III. Support for Systemic Reform 

 III-9

quarters of MSAP secondary schools offer special electives as part of the MSAP program. In 

more than half of the of MSAP schools, students can pick an area of specialization, and, in about 

two-thirds of these schools, students are required to select an area of specialization. In about half 

of the schools, students are required to complete courses in addition to those required by other 

schools in the district. 

The 2000–2001 survey data indicate that MSAP high school magnet programs have 

somewhat greater depth, based on these measures, than middle school programs. More high 

school principals (91 percent) than middle school principals (74 percent) reported that they 

offered special or elective courses as part of their magnet program (p<0.10). Also, students in 

high school magnet programs are also more likely than middle school students (70 vs. 39 

percent) to pick a theme or area of specialization (p<0.005). However, high schools do not differ 

from middle schools in whether or not they require students to select a theme or area of 

specialization (67 vs. 79 percent). Students in magnet programs at high schools were more likely 

than those in middle school programs (68 vs. 45 percent) to be required to complete courses in 

addition to those required by other schools in the same district (p<0.05) (see Exhibit A-III-15). 

 

As another way of gathering information on the depth of the magnet programs in MSAP 

schools, we included items on the teacher survey administered in 2000–2001 in the case schools. 

In particular, we asked a sample of English and language arts and mathematics teachers who 

indicated that they had taught at the school prior to the start of the MSAP grant (43 percent of the 

sample) whether or not their instruction had changed as a result of the introduction of the magnet 

program. Those who had changed were asked about various types of changes in their instruction, 

such as adding new topics, adding more advanced work, changing teaching methods, changing 

assessment methods, increasing the use of technology, and increasing material that connects 

English and language arts or mathematics with other subjects. 

Most (81 percent) of the surveyed teachers at all grade levels who had taught English 

language arts and mathematics prior to the start of the MSAP grant reported that magnet school 

programs encouraged or facilitated changes in their instruction, particularly in terms of increased 

use of technology and the increased material that connects English and language arts or 

mathematics with other subjects. The data indicate that elementary teachers made somewhat 

more changes than did secondary teachers in these areas. For example, elementary teachers were 

more likely than secondary teachers to report that they added more new topics to their 

curriculum (p<0.005), changed teaching methods (p<0.05), and incorporated technology into 

their instruction (p<0.005). There were no differences between mathematics and reading and 

language arts teachers in reported changes introduced as a result of the magnet program (Exhibit 

A-III-16). 

Overall, the 2000–2001 survey data provide some indication that in many MSAP schools, 

the magnet program had some impact on curriculum throughout the whole day, but there is 

clearly variation among MSAP schools in program depth. Our case study data, reported in 

Chapter V and the Case Study Appendix, provide examples of the range of variation in the depth 

of magnet programs. 
III. Support for Systemic Reform

III-10

History with Magnet School Programs 

We hypothesized that a school’s experience implementing some form of a magnet 

program prior to receiving the MSAP award might have a major influence on the quality of the 

school’s program. On the 1999–2000 principal survey, we asked each principal to indicate 

whether the school had operated a magnet program prior to the MSAP award, and if so, whether 

the pre-MSAP magnet program was still operating. (The item did not ask specifically about 

whether the earlier magnet program was MSAP supported.) Overall, about 47 percent of 

principals indicated that their school was operating a magnet program immediately prior to the 

MSAP award (i.e., 1997–1998), and another 7 percent reported that their school had once had a 

magnet program, but it was no longer operating at the time of the MSAP award (i.e., it was not a 

magnet school in 1997–1998). Our case data provide more detailed information on the various 

ways schools with prior magnet experience made use of MSAP resources. The case data also 

suggest that at least some magnet schools add or eliminate magnet themes relatively frequently. 

This is a topic to which we will return in Chapter VI. 

Change in Program Characteristics between the Second and Third Year of the 

MSAP Grant 

Only the principal measures of depth were asked in both waves of survey data collection, 

and there was little change between the second and third year of the MSAP grant in the depth of 

the magnet programs. The only significant difference (p<0.05) was that more high schools 

allowed students to select their theme or area of specialization in 2000–2001 than 1999–2000. 

Among MSAP middle schools, there was no significant change either in the measures of 

program depth or in requiring students to select a theme or area of specialization.

Supports for Teaching and Learning 

In the Year 1 Interim Report, we provided descriptive information on several supports for 

teaching and learning, including professional community among teachers, positive school 

climate, mechanisms for student academic support and personalization, and high standards for 

learning. In this report, we provide additional results for these and two additional supports: 

ongoing professional development for teachers and mechanisms for parent involvement. A brief 

description of each follows. 

• Professional Community Among Teachers describes the extent to which the 

teachers in a school share a common set of goals and beliefs, and have frequent 

Survey data indicate that MSAP schools differ from non-MSAP schools serving similar 

students in several key supports for teaching and learning. In particular, they have somewhat 

more positive school climates and a somewhat stronger sense of professional community 

among staff. There is some evidence that these supports for teaching and learning improved 

during the three years of program support. For example, MSAP elementary schools reported 

that their professional community improved between 1999-2000 and 2001-2002, and both 

elementary and middle MSAP schools reported improvements in school climate.
 III. Support for Systemic Reform 

 III-11

opportunities for collegial interaction. To measure professional community in the full 

set of MSAP schools, we included items on the principal survey asking about the 

presence of a common focus about the central mission of the school, collegiality, 

collaboration among teachers, and collaboration between teachers and the 

administration.8

• Positive School Climate concerns the level of academic engagement and safety in 

the school. To measure school climate, we included survey items asking principals 

about the extent to which various problems exist at the school, including student 

disengagement (e.g., tardy, transient, and unprepared to learn), student misbehavior 

(e.g., physical conflicts among students, verbal abuse of teachers, and student 

disrespect for teachers), and other severe student problems (e.g., alcohol and drug 

abuse).9

• Mechanisms for Student Academic Support and Personalization include 

strategies for taking individual differences among students into account in the 

educational program. We measured two types of school practices in this domain. 

First, we asked about schools’ use of parent satisfaction and student surveys to 

identify student needs and interests. We reasoned that the use of such surveys might 

reflect an interest in providing services tailored to parent and student needs and 

concerns. Second, we asked about the ways in which students with individual 

education plans (IEPs) and limited English proficiency (LEP) participated in the 

magnet program, reasoning that encouraging participation of all students might be an 

indication of efforts to respond to students with special needs and interests. 

• High Standards for Learning concern the specific instructional activities required of 

students in MSAP schools. For example, we asked each elementary school principal 

whether the school required a minimum number of hours of instruction in reading and 

math each day, and we asked each middle and high school principal if students were 

required to complete a portfolio of work.10

• Ongoing Professional Development for Teachers concerns the extent to which 

teachers participate in sustained and intensive professional development related to 

instruction. To measure the availability of professional development, we asked 

principals to report on the degree of emphasis given in the school to professional 

development related to standards-based reform, such as state or district academic 

performance standards and addressing the needs of students with limited English 

proficiency or disabilities.11

• Mechanisms for Parental Involvement include mechanisms used to facilitate parent 

participation in the instruction of their children (as measured by parental participation 

 

8

 For more information on the importance of teacher professional community for effective schools, see Marks, 

Secada, and Doane (1996), Sebring, et al. (2000), Sebring and Bryk (2000), and Wenzel, et al. (2001). 9

 For more information on the importance of school climate for effective schools, see American Association of 

University Women (1992), Lee, Chen, and Smerdon (1996), Sebring and Bryk (2000), and Wenzel, et al. (2001). 10 For more information on the importance of high standards for learning for effective schools, see Lee and Smith 

(1999). 11 For more information on the importance of professional development for effective schools, see Cohen and Hill 

(1998), Kennedy (1998), and Corcoran, Wang, and Foley (1999). 
III. Support for Systemic Reform

III-12

in special events such as an open house or science fair); and mechanisms for 

involvement in school governance (e.g., instructional issues or budget decisions).12

Professional Community among Teachers and Positive School Climate 

 

Based on data from the 2000–2001 surveys, most MSAP schools appeared to exhibit 

relatively strong levels of professional community and positive school climate, although we 

found some variation. A majority of MSAP principals “agreed” or “strongly agreed” that their 

schools had a strong sense of professional community, with elementary MSAP principals 

reporting stronger professional community than either middle or high school MSAP principals 

(p<0.05). We found somewhat more mixed results concerning school climate. While MSAP 

schools tended to report only minor problems, many—especially high schools—reported 

“moderate” or “serious” problems in areas relating to student disengagement. Fewer schools 

reported the existence of more serious problems, although these were reported as occurring at a 

moderate level in some MSAP schools. MSAP high school principals reported more problems 

(i.e., student disengagement, misbehavior, and severe student problems) than MSAP elementary 

school principals (p<0.05). 

Mechanisms for Student Support and Personalization 

With respect to mechanisms for student support and personalization, we found that about 

61 percent of MSAP principals reported using a parent satisfaction survey for school selfassessment, and about 55 percent reported using a student survey. MSAP principals also reported 

that on average 87 percent of both students with IEPs and LEP students are included in the same 

magnet activities as other students. They also reported that 30 percent of students with IEPs and 

32 percent of LEP students take part in magnet-related activities that accommodate their special 

needs. 

High Standards for Learning 

With respect to academic standards, our 2000–2001 data showed substantial variation 

across schools in the extent to which high standards for all students have been implemented. We 

asked principals which of a variety of academically demanding activities (e.g., year-long 

projects, reading a minimum number of books in a grading period, completing a portfolio of 

work in one or more academic subjects) their students were required to undertake. For example, 

on average more than 90 percent of MSAP elementary schools required an hour of math 

instruction and an hour of reading instruction for students each day; however, only 57 percent 

required students to give an oral presentation at regular intervals. Similarly, 84 percent of MSAP 

middle schools require students to complete an individual or small group project during the year, 

but only 54 percent require students to give an oral presentation at regular intervals (U.S. 

Department of Education, 2001, Chapter V). 

 

12 For more information on the importance of parental involvement for effective schools, see Wasley, Hampel, and 

Clark (1997), Newmann and Wehlage (1995), and Haynes (1998). 
 III. Support for Systemic Reform 

 III-13

Ongoing Professional Development for Teachers 

The 2000–2001 principal survey data indicate that MSAP elementary, middle, and high 

schools placed substantial emphasis on professional development focused on state or district 

content and performance standards, as well as student performance assessments. For example, 76 

percent of the principals in MSAP schools reported that the school had placed a major emphasis 

on providing professional development for teachers on state or district curriculum and 

performance standards, and 62 percent had placed a major emphasis on student performance 

assessment. 

Mechanisms for Parental Involvement 

Our survey data indicate wide variation across MSAP schools in the extent to which 

parents were involved in school activities. For example, about 56 percent of elementary MSAP 

principals reported that most parents participated in regularly scheduled schoolwide parentteacher conferences, but only 16 percent reported that most parents were involved in subject area 

events such as science fairs or concerts. Middle school and high school principals reported lower 

levels of parental involvement in most events. For instance, only 23 percent of middle school 

principals and 12 percent of high school principals reported that most parents attended parentteacher conferences. While 16 percent of both elementary and 21 percent of middle school 

principals reported that most parents attended special subject area events, only 9 percent of high 

school principals did so. Similarly, while 38 percent of elementary principals reported that half 

or more of the parents participated in at-home learning activities to support school objectives, 

only 21 percent of middle school principals and 6 percent of high school principals did so. 

Change in Supports for Teaching and Learning between the Second and Third 

Year of the MSAP Grant 

Data from the 2000–2001 principal survey show improvements in some supports for 

teaching and learning during the third year of the MSAP grant (see Exhibits A-III-17 through AIII-35). For example, the sense of professional community among teachers at elementary schools 

was strengthened between 1999–2000 and 2000–2001, the second and third years of MSAP 

support (p<0.01). School climate in elementary schools also improved (p<0.01), as evidenced by 

fewer reports of student behavior problems such as physical conflict among students. Middle 

schools reported improvements in their school climate in the area of student disengagement 

(p<0.10). 

Apart from the changes observed in professional community and school climate, a few 

other differences were observed in the areas of professional development and parental 

involvement. For example, both elementary and high school principals reported significantly 

more offerings for professional development on curriculum frameworks and assessments in 

2000–2001 than in 1999–2000 (p<0.05 for elementary principals and p<0.10 for high school 

principals). Parental involvement in special events, such as open houses and parent-teacher 

conferences, increased at elementary schools (p<0.01). Additionally, parental involvement in 

school governance issues increased in both elementary and middle school (p<0.01). 
III. Support for Systemic Reform

III-14

In other areas, however, little change was observed. There were no differences in the 

average number of academic requirements for elementary, middle, or high school students. Nor 

were there significant changes in the reported inclusion of students with IEPs or limited English 

proficiency in regular or modified magnet activities. 

Supports for Teaching and Learning in MSAP and Comparison Schools 

Analysis of principal survey data from schools in the case study sample indicates stronger 

supports for teaching and learning at MSAP schools than at non-MSAP comparison schools. 

Principals in MSAP schools reported a stronger sense of professional community among teachers 

(p<0.01) and a more positive school climate with fewer problems with student disengagement 

(p<0.05) and misbehavior (p<0.01). (See Exhibit III-4.) Parental involvement in special events 

was also greater at MSAP than comparison schools (p<0.05) (see Exhibits A-III-21, A-III-24, AIII-25, and A-III-34). 

Exhibit III-4. 

Sense of professional community and school climate in MSAP and comparison schools, 

2000–2001 

1

1.5

2

2.5

3

3.5

4

Professional Community Student Behavior Problems Student Disengagement

Problems

Mean

MSAP

Comparison

Exhibit reads: On a scale of “strongly disagree” to “strongly agree,” MSAP principals report “strongly agree” 

(mean=3.5) that there is a strong sense of professional community in their school while comparison principals 

“agree” (mean=3.2) that their schools have a strong sense of professional community. Also, on a scale of “not a 

problem” to “serious problem,” MSAP principals report that student disengagement problems are a “minor problem” 

(mean=2.2) at their school, while comparison principals state that these problems are a “moderate problem” 

(mean=2.7) at their schools. 

N = 29 MSAP schools and 16 matched comparison schools 

Source: MSAP Principal Survey, 2000–2001, Item 17 
 III. Support for Systemic Reform 

 III-15

Classroom Instructional Practices 

To examine classroom instructional practices, we analyzed survey responses from the 

case study schools, focusing on measures of the use of different types of instructional methods 

(e.g., use of long-term projects), instructional use of computers, and use of assessment tools (see 

Exhibits A-III-36 through A-III-42). As these data were collected with the primary purpose of 

providing insight on how MSAP schools differ from comparison schools in their same district, 

we highlight these results below. 

Classroom Instructional Practices in MSAP and Comparison Schools 

Survey results suggest that MSAP teachers have gone further than teachers in the 

matched comparison schools in incorporating into their classroom practices instructional 

methods that require higher-order thinking skills. Examples of such instructional methods 

include assignments that challenge students to provide verbal or written explanations of their 

understanding of a mathematical problem, give their own interpretation of something they have 

read, or apply recently acquired knowledge to a similar problem arising in a new situation. 

Mathematics teachers in MSAP schools, both elementary and secondary, use activities that 

require higher-order thinking skills significantly more often than their non-MSAP counterparts 

(p<0.05), as do MSAP elementary reading and languages arts teachers (p<0.10). 

Both MSAP and non-MSAP teachers reported using computers in their instruction 

between 11 and 20 times on average during the 2000–2001 school year. According to responses 

on both the mathematics and language arts teacher surveys, students in MSAP and non-MSAP 

schools used computers for the same general purposes. However, while there are no significant 

differences in the frequency with which MSAP and non-MSAP students used computers for 

some purposes, such as mastering new skills or remediating skills not learned well, MSAP 

students were reported to use computers more frequently to do the following: 

• Express themselves in writing (p<0.01). 

• Find out about ideas or information (p<0.01). 

• Engage in simulations or explorations of ideas (p<0.05). 

• Present information to an audience (p<0.05). 

Survey data indicate that teachers in MSAP schools are more likely than teachers in nonMSAP schools to use teaching methods that focus on higher-order thinking skills, such as 

project-based learning. MSAP teachers also make more use of technology in instruction in 

ways that support higher-order thinking, and make more use of varied assessment strategies.
III. Support for Systemic Reform

III-16

Exhibit III-5. 

Percent of surveyed case study teachers in MSAP and matched comparison schools who 

use computers for specific instructional purposes at least once a week, 2000–2001 

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Expressing

themselves in

writing

Finding out about

ideas or

information

Engaging in

simulations or

explorations of

ideas

Presenting

information to an

audience

Percent of Teachers

MSAP

Comparison

Exhibit reads: 39 percent of surveyed case study teachers in MSAP schools have their students express themselves 

in writing on a computer at least once a week whereas only 28 percent of matched comparison case study teachers 

use computers for this purpose at least once a week. 

N varies between 216 and 219 MSAP teachers and 132 and 133 matched comparison teachers. 

Source: Elementary Mathematics Teacher Survey, 2000–2001, Item 30; Middle and High School Mathematics 

Teacher Survey, 2000–2001, Item 30; Elementary Reading Teacher Survey, 2000–2001, Item 37; Middle and High 

School Language Arts Teacher Survey, 2000–2001, Item 31; Elementary Mathematics Teacher Survey (nonMSAP), 2000–2001, Item 24; Middle and High School Mathematics Teacher Survey (non-MSAP) Survey, 2000–

2001, Item 24; Elementary Reading Teacher Survey (non-MSAP), 2000–2001, Item 31; Middle and High School 

Language Arts Teacher Survey (non-MSAP) Survey, 2000–2001, Item 25 

These survey results are consistent with the data we collected as part of the case studies. 

For example, in a MSAP middle school at one of the case study sites (see District B case study), 

students in the digital photography class created slide show presentations on iMac laptops, and 

students in a technology-based history class worked on an Internet-based project in which they 

researched and produced museum displays that other students across their state could view on the 

Internet. 

Additionally, MSAP teachers used a more varied approach to assessing the academic 

progress of their students than non-MSAP teachers. Mathematics teachers made significantly 

(p<0.01) greater use of short or long written responses and individual or group projects or 

presentations. 
 III. Support for Systemic Reform 

 III-17

Sharing Innovative Practices with Others 

MSAP principals were asked if other educators, the media, and other members of the 

general public had approached them for information about their magnet programs. The results 

indicate that MSAP schools, especially at the elementary level, were sharing their experiences 

with innovative practices with other educators as well as the general public. In 2001, more 

MSAP schools than in 2000 reported that they had provided information, hosted visits, or 

provided professional development to educators from other schools. Additionally, more schools 

reported receiving requests for information or permission to visit from reporters, researchers, or 

other individuals or groups besides educators or parents (see Exhibit A-III-43). These individuals 

and groups may have learned about how the MSAP schools have increased their emphasis on 

professional development, strengthened their professional communities, improved school 

climates, and increased parental involvement. 

Summary 

Magnet schools are expected to adopt distinctive themes and innovative programs, and 

are designed to promote a positive school climate and professional community among teachers. 

These conditions in turn are expected to lead to effective instructional practices. The survey data, 

as well as the case data discussed in Chapter VI, indicate that MSAP schools have done just this. 

They have adopted a wide variety of themes. More than half of the MSAP schools have adopted 

comprehensive school reform models, which is higher than the proportion reported among the 

full national population of Title I schools or among comparable schools in the MSAP districts.13

MSAP schools on average have a somewhat more positive sense of professional community and 

school climate than comparable schools, although there is considerable variation. Survey data 

indicate that MSAP schools make somewhat more use of technology in instruction than do 

comparable schools, and place more emphasis on instructional methods designed to elicit higher 

order thinking skills, such as open-ended projects and presentations. We found some evidence of 

improvement in school climate and instructional practice between the second year of program 

implementation (1999–2000) and the third year of implementation (2000–2001). Principals’ 

responses indicated that there was improvement in professional community and parental 

involvement. Other elements, including the level of challenge reflected in the number of 

academic requirements for students, showed little improvement. We theorize that with these 

 

13 Reported estimates of the national percentage of Title I schools adopting reform models based on data from the 

National Longitudinal Survey of Schools range from 31 percent in 1998–1999 to 43 percent in 1999–2000. See 

U.S. Department of Education, Office of the Under Secretary, Planning and Evaluation Service, Elementary and 

Secondary Education Division. The Same Standards for Migrant Students: Holding Title I Schools Accountable. 

Volume I: Title I Schools Serving Migrant Students: Recent Evidence from the National Longitudinal Survey of 

Schools. Washington, D.C.: Author, 2002; and U.S. Department of Education, Office of the Under Secretary, 

Planning and Evaluation Service, Elementary and Secondary Education Division. Schools Identified as in Need 

of Improvement under Title I: Recent Evidence from the National Longitudinal Survey of Schools, Washington, 

D.C.: Author, 2002. Nationally representative data are not currently available on the percentage of non-Title I 

schools that have adopted comprehensive models. 

Survey data indicate that many MSAP schools shared their experiences with innovative 

practices with other educators and with the general public. 
III. Support for Systemic Reform

III-18

elements in place, MSAP schools will realize improved student achievement. We examine this 

connection in Chapter V. 

Furthermore, while MSAP schools were intended to adopt innovative themes and 

practices, they were also intended to ensure that these programs were aligned with state and 

district standards and assessments. We found that the relationship between magnet schools and 

state and district standards is complex. Principals report that they are familiar with standards and 

assessments and that the content of state standards and assessments match the goals of their 

magnet programs. As we consider in more detail in Chapter VI, our case data support the 

conclusion that magnet themes are generally consistent with the content emphasized in state 

standards, although staff in some MSAP schools indicated that they feel pressured to learn how 

to teach a new theme or curriculum while simultaneously being mindful of the state content 

standards and assessments. 

 
 IV. Desegregation Objectives and Outcomes

 IV-1

IV. Desegregation Objectives and Outcomes 

Since the 1970s magnet schools have been used by school districts to promote 

desegregation efforts by increasing school choices. Public officials have sought to increase racial 

and ethnic integration of schools within and sometimes between districts by structuring the 

incentives for parents to choose particular schools for their children. The federal government 

policy as represented in Improving America’s Schools Act (IASA) and more recently in No 

Child Left Behind (NCLB) supports these efforts through the Magnet Schools Assistance 

Program (MSAP). One of the federally mandated objectives of MSAP is to foster desegregation 

of schools. This chapter does the following: 

• Describes the desegregation objectives of schools that have been targeted for 

desegregation under the 1998 MSAP grants. 

• Evaluates the desegregation outcomes for those schools based on changes in 

percentage of minority enrollments. 

• Discusses the factors that influence a school’s ability to desegregate. 

 Desegregation Objectives of MSAP’s Targeted Schools 

The MSAP was enacted in 1984 to provide federal support for magnet schools that are 

part of an approved voluntary or required (i.e., required by court or other government agency) 

desegregation plan. Through the MSAP, funds are provided to support programs that will 

promote the reduction, elimination, or prevention of minority group isolation (MGI) in 

elementary or secondary schools. Minority group isolation refers to schools in which minority 

group children constitute more than 50 percent of school enrollment.1

 Minority group is defined 

to include children from the following racial-ethnic backgrounds: American Indian or Alaskan 

Native, Asian or Pacific Islander, Hispanic, and black (not of Hispanic origin).2

The categories of desegregation objectives are the following: 

• Schools with the objective of reducing minority group isolation have more than 50 

percent minority enrollment before a program starts, and their objective is to lower 

the proportion of minority enrollment (although they may still remain minority 

isolated). 

 

1

 34 CFR §280.4 

2

 In four districts with required desegregation plans, a court or other government agency has defined minority as 

black. In a fifth district with a required desegregation minority plan, a court order defines minority as black or 

Hispanic. 

More than 90 percent of schools targeted by MSAP for desegregation are minority 

group isolated; that is, more than one-half of their students are minority (i.e. black, 

Hispanic, Asian-Pacific Islander, or American Indian-Alaska Native). These schools 

are seeking to reduce or eliminate minority group isolation. Less than 10 percent of 

schools targeted by MSAP for desegregation are seeking to prevent minority group 

isolation by keeping the number of minority students in the school from exceeding 

more than half of all student enrollment. 
IV. Desegregation Objectives and Outcomes

IV-2

• Schools with the objective of eliminating minority isolation are also minority-isolated 

before a program begins, and their objective is to lower the percent of minority 

enrollment level below 50 percent during the grant period (i.e., for the 1998 grantees, 

between 1998–1999 and 2000–2001). 

• Schools with the objective of preventing minority isolation are in danger of becoming 

minority isolated and aim to keep minority enrollments from rising above 50 percent 

of enrollment during the three-year project period. These schools’ minority 

enrollment is expected to remain below 50 percent with the magnet program. 

Districts with required desegregation plans may have requirements established by a court or 

other agency that vary from those defined above. In general, however, the grantees in districts 

with required desegregation plans have also identified goals consistent with the desegregation 

objectives defined above. The purpose of the analysis reported in this chapter is limited to 

examining the extent to which schools targeted for desegregation in MSAP-funded districts are 

preventing, eliminating, or reducing minority group isolation.3 

In the 2000–2001 school year that ended the 1998 MSAP funding cycle, a total of 294 

schools were “targeted” for the reduction, elimination, or prevention of minority group isolation 

in connection with an MSAP-supported program. Typically the school that is targeted for 

desegregation is the same school in which the magnet programs are implemented. However, 

school districts may establish a magnet program in one or more schools with the aim of drawing 

students from higher minority enrollment schools into those programs. In these situations, the 

objective is to reduce, eliminate, or prevent minority group isolation in the “feeder” schools from 

which students are being drawn. Among the 294 schools that were targeted for desegregation, 30 

were targeted feeder schools. 

The overwhelming objective of the 294 desegregation-targeted schools was to reduce 

minority group isolation. Fully three quarters (77 percent) of targeted schools sought to reduce 

MGI, while about one-sixth (16 percent) aimed to eliminate MGI by reducing their minority 

enrollment to 50 percent or less. Less than one-tenth (8 percent) of targeted schools were aiming 

to prevent MGI isolation by keeping the school’s minority enrollment from exceeding 50 

percent. The general pattern of objectives is similar for schools in voluntary and required 

districts (see Exhibit IV-1). 

The small proportion of schools seeking to prevent or eliminate MGI is understandable 

given the large percentage of schools that are located in districts with high minority enrollment. 

Three-quarters of the targeted schools are in districts in which minority students constitute more 

than 50 percent of public school enrollment. Two-thirds of the targeted schools are in districts 

with over 60 percent minority student enrollment in public schools (Exhibit A-IV-2). 

 

3

 Compliance with a court order is determined by a judge, not the Department of Education (likewise an agency 

determines compliance with an agency order). Therefore, if a judge or agency deems a magnet school operation 

and enrollment to be in compliance with an order, the Department of Education will defer to that interpretation. 

See 1998 Magnet School Application Notice 63 Federal Register 8021 (February 17, 1998). (See also 34 CFR 

280.2(a)(1).)
 IV. Desegregation Objectives and Outcomes

 IV-3

In their applications, the MSAP grantees were required to include benchmarks or goals in 

the form of projected enrollments for schools targeted for desegregation. Those projections 

typically indicated the number and percent of minority students for each year of the grant. The 

benchmarks were intended to reflect the changes in enrollment that grantees expected to result 

from implementation of the district’s MSAP project. Some districts were more ambitious than 

others in setting their benchmarks. Thus rather than focusing on whether or not targeted schools 

met their benchmarks, this evaluation focuses on the extent to which targeted schools reduced, 

prevented, or eliminated MGI. 

Exhibit IV-1. 

Desegregation objectives of MSAP’s targeted schools, overall and by type of 

desegregation plan 

Exhibit reads: The predominant objective for MSAP’s desegregation-targeted schools in both districts with 

required and voluntary desegregation plans was to reduce minority group isolation. 

N = 294 targeted schools 

Sources: Grantee applications, performance reports, NCES 1997–1998 and 1998–1999 Common Core of Data, 

and U.S. Department of Education personnel 

Desegregation Outcomes at MSAP’s Targeted Schools 

The success of magnet programs in making progress toward their desegregation objective 

is based on the extent to which MSAP’s desegregation-targeted schools have reduced, prevented, 

or eliminated MGI between 1997–1998 and 2000–2001. Targeted schools have succeeded in 

Adjusting for districtwide demographic trends, the magnet programs for 57 percent 

of MSAP’s desegregation-targeted schools made progress in preventing, 

eliminating, or reducing minority group isolation, while 43 percent did not show 

progress. 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Voluntary Required Total

Percent of Schools

Prevent 

Eliminate

Reduce
IV. Desegregation Objectives and Outcomes

IV-4

preventing or eliminating MGI if the school’s minority student attendance in 2000–2001 does 

not exceed 50 percent of student enrollments, and targeted schools have made progress toward 

reducing MGI isolation if their minority enrollment is less in 2000–2001 than in 1997–1998 even 

though it still exceeds 50 percent.4

 Specifically: 

• Targeted schools have prevented minority group isolation if their minority enrollment 

did not exceed 50 percent prior to the 1998 funding cycle (i.e., 1997–1998) nor at the 

end of the funding cycle (i.e., 2000–2001).

• Targeted schools have eliminated minority group isolation if their minority student 

enrollment exceeded 50 percent prior to the 1998 funding cycle, but was 50 percent 

or less by the end of the funding cycle.

• Targeted schools whose minority enrollment exceeds 50 percent in 1997–1998 and 

2000–2001 have reduced minority group isolation if the proportion of minority 

students is lower at the end of the funding cycle compared with enrollment prior to 

the 1998 funding cycle. 

In determining the progress of a targeted school in reducing MGI, the Department of 

Education takes into account the changing demographic conditions in a district. If an increase in 

MGI at a desegregation-targeted school is less than the districtwide increase in MGI of students 

at the same grade level (i.e., elementary, middle, or high school) as the targeted school, the 

Department considers the school as having made progress in reducing MGI.5 

Taking into account demographic trends by adjusting for districtwide changes in minority 

enrollment indicates the increase or decrease in percent minority enrollment at a school relative 

to the percent minority enrollment districtwide. For example, if minority enrollment at an 

elementary school targeted for desegregation under MSAP increases by 1 percent, but the 

percent minority enrollment in elementary schools districtwide increases by 2 percent, the 

magnet program is considered to have made progress in meeting its desegregation objectives. 

Even though the targeted school has a higher percent minority enrollment at the end of the 

program than at the beginning, the difference between the proportion of minority enrollment in 

the targeted school and the district is reduced because of the district’s larger percentage increase. 

Adjusting for demographic trends is an important consideration for the schools that have 

been targeted for desegregation by MSAP programs. More than three-quarters of targeted 

schools were in districts in which the proportion of minority enrollment at their grade level 

increased by at least 1 percent between 1997–1998 and 2000–2001 (Exhibit A-IV-3). 

 

4

 Grantees provided estimates of minority enrollment levels for schools that were not open in the year prior to the 

start of the funding cycle. 5

 One of the roles of the Office of Civil Rights (OCR) and ED is to provide technical assistance when districts 

have trouble making progress in the statutory objective of reduction, elimination, or prevention of minority group 

isolation. 
 IV. Desegregation Objectives and Outcomes

 IV-5

Progress in Meeting Desegregation Objectives 

This analysis focuses on whether and to what extent schools targeted for desegregation 

prevented, eliminated, or reduced minority group isolation.6

 After adjusting for district trends in 

percent minority enrollment, MSAP’s magnet programs have evidenced progress in reducing, 

preventing, or eliminating minority group isolation in 57 percent of the schools targeted for 

desegregation.7

 The other 43 percent of targeted schools did not show progress in reducing 

MGI.8 

Were some targeted schools more likely to make progress in meeting their desegregation 

objective than others? For example, were targeted schools in districts operating with a 

voluntarily initiated desegregation plan more or less likely to make progress than those in 

districts with desegregation plans required by an external authority? How similar were the 

desegregation outcomes for elementary schools as compared to secondary schools? The evidence 

indicates that the proportion of targeted schools in districts with voluntary desegregation plans 

that made progress in meeting their desegregation objectives was slightly higher compared to 

districts with required desegregation plans (60 percent vs. 53 percent). (Exhibit A-IV-5). 

However, overall differences were more substantial between grade levels, with a larger 

proportion of elementary schools making progress compared with high schools (60 percent vs. 

48 percent). This difference was observed in districts with voluntary as well as required 

desegregation plans (Exhibit IV-2). The proportion of middle schools making progress toward 

desegregation (54 percent) generally fell between that of elementary and high schools. However, 

in districts with voluntary desegregation plans, the proportion of middle schools making 

desegregation progress was close to that of elementary schools, whereas in districts with required 

desegregation plans, a lower proportion of middle schools made progress compared with 

elementary schools. 

Progress in meeting desegregation objectives also differed by type of magnet program. 

Overall, a larger proportion of whole school programs (59 percent) showed progress toward 

meeting their desegregation objectives compared with PWS magnet schools (49 percent). The 

proportion of targeted feeder schools that showed progress (47 percent) was similar to that of 

PWSs (Exhibit A-IV-6).9

 

 

6

 Whether or not a district with a required desegregation plan met the desegregation goals set out in a court or 

agency order is not being assessed in this study. Compliance with a court or agency order is determined by a 

judge or agency, not by the Department of Education. (See footnote 3 in this chapter). 7

 Without adjusting for district trends in minority enrollment, 39 percent of desegregation-targeted schools made 

progress in reducing, preventing, or eliminating MGI. Thus adjusting for district trends increases the proportion 

of targeted schools making progress by 18 percentage points. That increase consists of schools whose percent 

minority enrollment is actually greater in 2000–2001 than in 1997–1998. They are, however, considered to have 

made progress because the increases are less than the districtwide increase at comparable grade levels. 8

 For purposes of this evaluation, the standard MSAP definition of minority is applied to all districts to maintain 

comparability of analysis with respect to reducing, preventing, and eliminating MGI. The count of targeted 

schools making progress in reducing MGI increases by one school if the definition of minority is modified for 

the five districts in which a court or other government agency order limits the definition of minority to blacks or 

to blacks and Hispanics. 9

 We have not drawn comparisons of progress by program type for districts with required and voluntary 

desegregation plans, given the small number of PWS magnets (N = 8) and targeted feeder schools (N = 10) in 

districts with voluntary desegregation plans. 
IV. Desegregation Objectives and Outcomes

IV-6

Exhibit IV-2. 

Percent of MSAP targeted schools making progress on their desegregation objectives by 

grade level after adjusting for districtwide changes in proportion of minority students in 

public schools 

Exhibit reads: A higher proportion of desegregation-targeted elementary schools made progress in meeting their 

desegregation objectives compared with high schools in both districts with required and voluntary desegregation 

plans. Evidence of progress toward desegregation for middle schools was similar to elementary schools in voluntary 

desegregation plan districts, but less than elementary schools in districts with required desegregation plans. 

N = 294 targeted schools 

Source: Grantee applications, performance reports, NCES 1997–1998 and 2000–2001 Common Core of Data, and 

U.S. Department of Education personnel 

Extent of Reduction in MGI 

The previous section focused on the percent of schools making progress. In this section, 

we turn to the amount of progress schools made in reducing MGI between 1997–1998 and 2000–

2001. One in twenty (5 percent) of the targeted schools prevented or eliminated MGI. One in six 

(17 percent) experienced a reduction of 5 percentage points or more in MGI relative to the 

district (see Exhibit IV-3). About one-quarter (27 percent) experienced a reduction of 1 to 5 

percentage points. One in fifteen (7 percent) experienced a reduction that was less than 1 

percentage point. In the remaining 43 percent of schools that did not make progress, MGI 

increased or remained constant during this period. 

 

Does the amount of progress differ for targeted schools in districts with voluntary 

desegregation plans as compared to districts with required desegregation plans? The differences 

and similarities are shown in Exhibit IV-3. A higher proportion of targeted schools in voluntary 

districts experienced reductions in MGI of 1 to 5 percentage points relative to the district 

minority enrollment, whereas a higher proportion of targeted schools in districts with required 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Voluntary Required All Schools

Percent of Schools

Elementary

Middle School

High School
 IV. Desegregation Objectives and Outcomes

 IV-7

desegregation plans prevented or eliminated MGI. Similar proportions of targeted schools in 

both voluntary and required desegregation plan districts reduced MGI by smaller amounts of less 

than 1 percentage point, as well as larger amounts of 5 percentage points or more relative to the 

percent minority enrollment in the district at comparable level schools. 

Exhibit IV-3. 

Percent of MSAP targeted schools reducing minority group isolation by extent of 

progress and type of district desegregation plan, after adjusting for districtwide changes 

in proportion of minority students in public schools

Exhibit reads: If allowances are made for districtwide changes in the percent minority, one-half of MSAP’s 

desegregation-targeted schools would have either prevented or eliminated MGI or experienced a decrease of at least 

1 percentage point in the proportion of minorities in their school. 

N = 294 targeted schools 

Source: Grantee applications, performance reports, NCES 1997–1998 and 2000–2001 Common Core of Data, and 

U.S. Department of Education personnel 

Factors That Influence MSAP Targeted Schools’ Ability to Reduce 

Minority Group Isolation 

Among the schools targeted for desegregation, what are the characteristics of those that 

have been more successful in reducing MGI? Whereas the preceding analysis evaluates the 

Schools are more likely to experience decreasing minority isolation when they have a 

racially and ethnically mixed group of minority students and when parents are involved in 

school events and activities. On the other hand, schools with larger numbers of students per 

teacher are more likely than those with lower student to teacher ratios to experience 

increases in minority group isolation. 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Voluntary Required All Schools

Percent of Schools

Reduced Proportion

Minority by < 1%

Reduced Proportion

Minority by 1% to < 5%

Reduced Proportion

Minority by 5% or more

Prevented or Eliminated

MGI
IV. Desegregation Objectives and Outcomes

IV-8

extent to which schools targeted for desegregation met or made progress in meeting their 

objectives, this section examines factors related to changes in the percent minority enrollment at 

MSAP’s desegregation-targeted elementary schools between the 1997–1998 and 2000–2001 

school years. 

The framework for assessing the factors that might influence a school’s ability to reduce 

minority group isolation considers both district setting and school-level characteristics. Within 

each of these categories, contextual or general characteristics may be distinguished from MSAP 

project or program features.10 The factors that are considered in this analysis include the 

following: 

District context: 

• Changes in percent minority enrollment districtwide. 

• Percentage change in the number of nonminority students in district. 

• Non-MSAP school choices (e.g., non-MSAP magnet schools, charter schools, private 

religious and sectarian schools). 

District’s MSAP project characteristics: 

• Type of desegregation plan (i.e., voluntary vs. required). 

• Complexity of program (i.e., number of grade levels and number of schools). 

School characteristics: 

• Pupil-teacher ratio. 

• Attendance zone school (i.e., school giving enrollment priority to students living in 

school’s neighborhood attendance area). 

• “Mixed” minority enrollment (i.e., no single race-ethnic minority represents more 

than half of the minority students in the school). 

• Poverty level (i.e., percent of students eligible for free or reduced-price lunch). 

• School climate and other organizational features (e.g., student disengagement or 

behavioral problems, sense of community among staff, school’s emphasis on 

professional development, and parental involvement in governance and special 

events). 

School’s MSAP program characteristics: 

• PWS vs. whole school program. 

• Targeted feeder program. 

 

10 In the previous section we examined overall differences in progress based on the type of program (i.e., PWS, 

whole school, or targeted feeder school) and type of desegregation plan (i.e., voluntary vs. required). The 

analysis in this section permits us to see the extent to which reductions in minority isolation differ with respect to 

these district and program characteristics while controlling for other factors that may influence changes in 

minority isolation. 
 IV. Desegregation Objectives and Outcomes

 IV-9

• Performance-based admissions standards (i.e., based on test performance, auditions, 

academic record). 

• Desegregation objective (i.e., prevent, reduce, or eliminate MGI). 

• Magnet history (i.e., school was magnet prior to 1998 funding). 

Analysis was conducted using a hierarchical linear model to examine annual changes in 

minority enrollment in 172 elementary schools targeted for desegregation. The model was 

developed incrementally beginning with analysis of the district context variables, followed by 

MSAP project characteristics, general school characteristics, and finally MSAP program 

characteristics. At each stage, only those characteristics that contributed to an improvement in 

the overall model were retained. 

As anticipated, we found that increases in the district minority enrollment had a 

significant effect on the progress schools made in reducing minority group isolation. For targeted 

schools in districts in which the percent minority rose over the period from 1997–2000, the 

schools also tended to become more minority group isolated. For every percentage point growth 

in percent minority in the district, MSAP’s targeted schools tended to grow 0.41 percent per year 

in percent minority (see Exhibit IV-4). Thus, the average change in percent minority in targeted 

schools was proportionately half of the change occurring in their district. 

We found that several school features are also related to improvement in MGI. On 

average attendance zone schools showed 1.17 percent per year greater decrease in percent 

minority than nonattendance zone schools.11 The direction of the effect is the opposite of what 

we expected. We had anticipated attendance zone schools would show less improvement because 

by giving priority to the students in the neighborhood, the magnet has fewer opportunities to 

recruit nonminority students from outside the neighborhood compared to nonattendance zone 

schools. 

 

Nonminority parents may be more willing to choose a predominantly minority school 

when there is a mix of race-ethnic groups in that school. In our analysis, mixed minority schools 

experienced 2 percent more improvement in MGI per year compared to schools in which any 

single race-ethnic minority represents over half of the students in the school. 

Schools in which parents are involved in special events experienced more progress in 

reducing MGI compared with schools in which parents are not involved. On a zero-to-four scale 

summarizing principals’ reporting on the level of parental involvement in special events, each 

unit of increase in parental involvement resulted in a 0.67 percent per year average decrease in 

MGI. 

 

The failure of schools to reduce minority group isolation is related to high student-toteacher ratios. That is, more students per teacher in a school tended to result in increases rather 

than decreases in the percent minority enrollment. A difference of one student per teacher is 

 

11 While attendance zone schools showed more improvement in MGI than non-zoned schools, attendance zone 

schools started the period with an average proportion minority that was 3 percentage points higher than nonzoned schools. See Exhibit A-IV-8 in Appendix IV. 
IV. Desegregation Objectives and Outcomes

IV-10

associated with a 0.19 percent per year increase in MGI. This result may indicate that schools 

with smaller class sizes, as indicated by fewer students per teacher, are able to attract a larger 

ratio of white applicants relative to minority applicants. This result is consistent with evidence 

reported by Henig (1996), based on an analysis in Montgomery County magnet schools. In that 

study, the ratio of transfer requests into a school by white families to transfer requests by 

minority families decreased as the student-teacher ratio increased. 

Of the MSAP program features examined, schools with PWSs on average showed 1.34 

percent per year greater decrease in minority isolation of students compared with whole school 

programs. Schools with PWSs started the period from 1997–1998 with an average percent 

minority enrollment that was 14 percentage points higher than whole schools. It is only when we 

control for the fact that PWS schools begin at a higher level of MGI that we are able to see that 

on average these schools experience a greater improvement in MGI. 

Exhibit IV-4. 

Summary of factors influencing the average annual change in percent minority 

enrollment 

District Characteristics Effect on Annual Change 

in School’s Percent 

Minority 

Change in % Minority 1997–1998 to 2000–2001 0.41*** 

School Level Characteristics 

Attendance Zone School -1.17*** 

Pupil-Teacher Ratio 0.19*** 

Mixed Minority Group -1.98*** 

Parental Involvement in Special Events -0.67** 

Program-within-a-School (PWS) -1.34* 

Exhibit reads: Progress of schools in reducing MGI is greater in schools with neighborhood 

attendance zones, a mixed race-ethnic makeup of minority students, greater parental 

involvement in a school’s special events, and schools with a PWS program. 

 ***p <.01; **p<.05; *p<.10 

Source: Appendix IV, Exhibit A-IV-8 

Other factors examined in this analysis did not significantly improve our model. For 

example, neither the complexity of the MSAP project nor the district’s desegregation plan was 

related to changes in minority enrollment in targeted elementary schools. Among school features, 

for example, parental involvement in school governance was not significant. Likewise, none of 

the other measures of school climate and organization (e.g., student problems, sense of 

community among staff, support of professional development) were related to change in the 

percent minority at targeted schools. Similarly, several of the magnet program features, such as 

the use of performance-based admissions standards and prior history as a magnet school, did not 

have a significant effect on changes in minority enrollment. 
 IV. Desegregation Objectives and Outcomes

 IV-11

 While we found several features of magnet schools related to an improvement in 

minority student isolation, the variables we tested explained only a modest part of the variation 

across districts and schools in the progress they made. (See Exhibit A-IV-8 for coefficient 

estimates.) We are not sure what factors explain this variation, because we lack good measures. 

One possibility is that the variation across districts may be due to differences in the quality of the 

information and other supports for choice. In a similar vein, variation across schools may be due 

to differences in the “identity” schools are able to communicate. These possibilities should be 

explored in future research. 

Other Factors That May Explain Desegregation Outcomes 

The preceding analysis of district and school characteristics associated with success in 

reducing minority groups isolation examines differences among targeted schools and their 

magnet programs. Two other potentially important factors might also help explain why more 

than 40 percent of desegregation-targeted schools were not successful in making progress on 

their desegregation objective. One factor is related to the timing of the MSAP grant awards. 

Grantees were limited in their ability to conduct first year recruitment efforts in the spring of 

1998 as they did not receive funding until June 1998 or later. The second factor is related to the 

processes by which MSAP schools may select students. Limitations placed on the use of race as 

a factor in selection of students may make it more difficult to achieve desegregation objectives. 

We discuss these issues further in Chapter VI. 

Summary 

The desegregation objective of targeted schools was overwhelmingly aimed at reducing 

MGI. The small proportions of schools seeking to eliminate or prevent minority isolation is 

understandable, because most of the targeted schools are in districts that have high proportions of 

minority students. 

Adjusting for districtwide demographic trends, the magnet programs for 57 percent of 

MSAP’s desegregation-targeted schools were successful in preventing, eliminating, or reducing 

minority group isolation, while 43 percent did not show progress. The extent to which MGI was 

reduced, prevented, or eliminated differed somewhat for targeted schools in districts with 

required and voluntary desegregation plans. The proportion of targeted schools preventing or 

eliminating MGI was greater in districts with required desegregation plans, while a larger 

proportion of targeted schools in voluntary districts reduced MGI by 1 to 5 percentage points. 

Almost one-fifth of targeted schools in districts with voluntary and required desegregation plans 

experienced reductions in MGI of more than 5 percentage points. 

Analysis of the factors influencing an elementary school’s ability to reduce minority 

group isolation revealed several features of schools that were related to reduction in minority 

isolation. Schools with a racially and ethnically mixed group of minority students and schools in 

which parents are reported to be involved in school events and activities were the schools that 

saw the greatest decrease in minority isolation, whereas high pupil-teacher ratios were related to 

increases in minority group isolation. Two unexpected findings were the reduction in minority 

isolation of schools with neighborhood attendance zones as compared to those without 
IV. Desegregation Objectives and Outcomes

IV-12

attendance zones, and the improvement of schools with PWS programs as compared to those 

with whole school programs. Our analysis accounted for only a small portion of the variation 

across schools and districts in changes in minority isolation. 
 V. Student Achievement Outcomes 

 V-1

V. Student Achievement Outcomes in MSAP-Supported 

Schools 

Strengthening students’ knowledge of academic subjects and skills needed for successful 

future careers is a primary legislative purpose of MSAP. Accordingly, student achievement 

outcomes are a primary focus of the MSAP performance indicators and of this evaluation. All 

applicants for MSAP grants must describe project objectives that address this federal legislative 

purpose, and all grantees must annually provide data on the progress their projects have made 

toward meeting the objectives. 

As we have found through our surveys and case studies, MSAP projects also operate 

within local contexts of increasing accountability for improving student achievement, 

particularly in the areas of English language arts and mathematics. Over the past few years, 

many districts have adopted or revised their standards and curricula, as well as invested in 

professional development to strengthen instruction in these (and other) content areas. MSAPsupported magnets, like other schools in their districts, are expected to show improved 

performance over time on state assessments. In some cases, magnet schools have been in the 

vanguard of their district’s reform efforts, thanks to the additional resources provided by the 

MSAP grant that have supported professional development training, release time for planning, 

and the purchase of new instructional materials and technology. While magnet schools enjoy the 

advantages of augmented resources and engaging themes to capture students’ interest and 

increase their learning, they also face substantial challenges. In particular, many serve substantial 

numbers of children whose circumstances—poverty, mobility, and limited proficiency in 

English—may increase the difficulty of showing dramatically improved performance in the short 

term. Implementation of programs may be impeded by unexpected circumstances or mid-course 

corrections, which may lead, in turn, to delays in the appearance of the program’s effects on 

student performance. Thus, while substantial improvement in student performance within the 

three-year span of an MSAP grant is an objective for MSAP projects, universal attainment of that 

objective is not guaranteed. 

In the nationwide MSAP evaluation, we investigated student achievement outcomes 

during the 1998–2001 grant period in two studies that are reported in this chapter. First, we 

examined the achievement objectives that projects adopted as MSAP performance indicators in 

their grant applications. In our Year 1 Interim Report (2000), we provided a detailed description 

of these objectives. In this final report, we update the description and discuss the degree to which 

the MSAP-funded magnet schools met the objectives their projects set for them. The data used in 

this analysis were drawn from projects’ annual performance reports to the U.S. Department of 

Education (ED). The second study is a series of detailed analyses comparing the performance of 

students in similar MSAP and non-MSAP schools on state assessments in reading and 

mathematics. While the first study evaluates the projects in accordance with the standards they 

set for themselves (which vary considerably in scope and sophistication), the second gauges the 

performance of MSAP schools in different districts using a common standard—the improvement 

in performance of the MSAP schools relative to that of non-MSAP schools serving similar 

student populations. 
V. Student Achievement Outcomes 

V-2

Achievement Objectives Study 

Data Sources 

Information about student achievement objectives and corresponding student 

achievement outcome data were obtained primarily from the grantees’ applications and their 

annual and supplementary performance reports. We also contacted project directors to request 

data that were missing from their reports, and in a few cases we extracted information from state 

Department of Education Internet sites. As explained in more detail later in this section, data 

needed to determine the extent to which goals were met were available for some but by no means 

all of the objectives set forth in projects’ applications. 

Project-Level Objectives and School-Specific Goals 

In this study, we describe grantees’ expectations for student achievement at two levels of 

aggregation. First, each of the MSAP grantees stated one or more project-level objectives in their 

applications. These objectives typically applied to all of the MSAP-supported schools in the 

district, or to all of the MSAP schools at a given grade level (for example, “Each year of the 

grant, the average mathematics scores of MSAP magnet schools will increase by 3 percentile 

points over the previous year’s average score.”); some applied only to one or a few schools. In 

total, we identified 564 project-level objectives that the 57 MSAP funded districts had set at or 

near the beginning of their three-year grant period. Most objectives specified a benchmark for 

each of the three grant years, while a few called for one or two measurements during the grant. 

While project objectives are stated in general terms, the unit of analysis for determining 

the extent to which those expectations are met is the school. Each project objective translates into 

one or more school-specific goals for the performance of students at a particular grade level on a 

particular measure of achievement. For example, in a district that administers a mathematics test 

to students in grades 1 through 5, a project-level objective calling for improved mathematics 

scores of elementary school students implies five school-level goals (one for each grade) for each 

MSAP elementary school in the district. When an objective takes account of additional factors 

such as students’ minority status and gender, it translates into even more school-level goals. 

The first part of this exploration of student achievement objectives describes the range of 

project-level objectives set by the grantees. The second part summarizes the extent to which 

schools met or made progress toward meeting the specific goals that had been set for them. 

Types of Objectives Set by MSAP Grantees 

Approximately half of the project-level objectives set by MSAP grantees are either for 

reading, writing, or English language arts (30 percent), or for mathematics (19 percent). A 

little more than a quarter of the objectives focus on more general outcomes, such as rates of 

school attendance, course-taking, dropping out, or acquiring applied learning skills. 

Standardized tests are by far the most common measures upon which objectives are based. 
 V. Student Achievement Outcomes 

 V-3

An achievement objective consists of several components: the content domain assessed, a 

description of the students to whom it applies (e.g., school, grade level, minority status, and 

possibly English proficiency), the achievement measure used, the magnitude of the change in 

performance that is expected, the time frame within which change will occur, and the external 

reference group (if any) with which the performances of magnet students will be compared. 

Exhibits V-1 and V-2 describe project-level objectives in terms of the subject areas they address, 

and the measures used to track them. 

Subject Area 

Exhibit V-1 shows the subject domains addressed by the 564 project-level objectives. 

(See Exhibit A-V-1 for the corresponding table.) Approximately half of the objectives are either 

for reading, writing, or English language arts (30 percent)1

 or for mathematics (19 percent). The 

prevalence of these two subjects reflects the fact that MSAP projects base most of their 

objectives on state or district assessment results, and these are the subjects most commonly 

assessed in these tests. A little more than a quarter of the project-level objectives are not related 

to one specific academic subject. Typically, these objectives focus on more general outcomes 

such as rates of school attendance, course taking, dropping out, or acquiring applied learning 

skills, or they refer to test scores in multiple academic subjects. 

Exhibit V-1. 

Project-level objectives by subject domains 

Exhibit reads: About 30 percent of the achievement objectives set by MSAP projects were in language arts, 

compared with about 19 percent in mathematics, and 5 percent in science and social studies. About 25 percent were 

not subject-specific. 

N = 564 project-level objectives 

Source: MSAP grant applications and annual performance reports 

 

1

 Throughout this report, the combined subjects of reading, writing, and English language arts are considered 

together and referred to as “language arts” or “English language arts.” 

0%

5%

10%

15%

20%

25%

30%

35%

Language Arts Math Science Social Studies Computer/ 

Vocational

Skills

Career

Awareness

Arts and

Performances

Other Not subjectspecific
V. Student Achievement Outcomes 

V-4

Type of Measure 

Standardized tests are by far the most common measures upon which objectives are 

based; as shown in Exhibit V-2, about three-quarters of all project-level objectives use them. 

(See Appendix Exhibit A-V-2 for the corresponding table.) Virtually all of the 57 MSAPsupported projects have at least one objective based on standardized test scores. The next most 

commonly used measures are other assessments (approximately 15 percent) that document 

student performance in academic knowledge and skill areas not covered by district or state 

assessments. For example, one project planned to devise an Internet research task and a rubric to 

assess students’ mastery of computer research skills; several projects planned to assess students’ 

career awareness, work-related habits, or attitudes toward school using surveys or teacher 

ratings. 

Exhibit V-2. 

Project-level objectives by type of measure 

Exhibit reads: About 75 percent of the measures used to set achievement objectives were standardized tests, 

compared with about 15 percent that were other assessments. Other types of measures were less commonly used. 

N = 564 project-level objectives 

Source: MSAP grant applications and annual performance reports 

Frequency of Measuring Outcomes 

About 87 percent of the project-level objectives specify annual benchmarks that are to be 

reached by the end of each of the three grant years, while about 11 percent are to be measured 

only at the end of the grant period. The rest are to be measured at only two points during the 

grant period (either at the end of each of the first two years or each of the last two years of the 

project). 

Although these data suggest that the MSAP-supported districts expected to collect and 

report annual outcome data for at least 87 percent of the 564 project-level objectives, the number 

of objectives for which analyzable annual school-level outcome data were available is 

considerably smaller. In the next section, we discuss the factors that reduced the number of 

objectives that could be included in the analysis of school-level outcomes. 

0%

10%

20%

30%

40%

50%

60%

70%

80%

Standardized Test Other

Assessments

Course-taking and

Grades

Educational

Trajectory

Benchmark

Preparation for

Postsecondary

Education

Advanced

Vocation-Related

Outcomes

Other
 V. Student Achievement Outcomes 

 V-5

Project-Level Objectives for Which School-Level Outcome Data Are Available 

Exhibit V-3 below summarizes the availability of school-level data that addressed the 

project-level objectives for each of the three years of the grant period. The total number of 

objectives tracked varies slightly by year due to the fact that a few project-level objectives called 

for progress to be measured in only one or two years. For each grant year, usable school-level 

data were available for about one-third of the objectives, with the most data available for 1999–

2000 (38 percent). 2 Five to six percent of the objectives did not have analyzable data because 

their projects dropped them during the grant period. Some were dropped formally—that is, 

project staff explicitly stated (in a performance report or an interview with AIR) that they had 

dropped the objective. The most common reason given was that the state had revised or 

discontinued its assessments so that the projects could no longer track the test scores that they 

had planned to use in their evaluations.3

 In addition, a few objectives were described in grant 

applications but never mentioned again, even though project staff did not formally acknowledge 

dropping them. Another small group of objectives (3 to 6 percent) could not be evaluated at the 

school level because project reports aggregated achievement results across magnet schools. 

Because the unit of analysis for our study of goal attainment was schools, these district-level 

outcome data could not be included in our analyses. 

Exhibit V-3. 

Percentage of project-level objectives with and without analyzable data each year 

Data Availability 1998–1999 1999–2000 2000–2001 

Analyzable 30.3% 38.3% 29.2% 

Objective dropped 6.5% 4.8% 5.3% 

Analyzable at the district level 4.7% 5.6% 2.8% 

Missing/Not usable 58.5% 51.3% 62.6% 

Total number of annual 

objectives 

489 499 562 

Exhibit reads: Analyzable school-level data were available for about a third of project-level objectives each year. 

The proportion of objectives with analyzable data was highest in 1999–2000 (38 percent). 

N = 564 project-level objectives (some pertain to only one or two project years) 

Source: MSAP grant applications and annual performance reports 

 

2

 In many cases, the lack of information can be attributed to factors beyond the control of the MSAP projects—

most notably changes in their state tests or the format in which test results were reported. Thus, lack of 

information cannot be construed as lack of progress in improving student achievement. 3

 A few objectives that were revised to take account of changes that occurred during the grant period are included 

among the 564 project level objectives described in this section. 

For each grant year, usable school-level data were available for about one-third of the 

objectives. The most common reason given for the lack of outcome data was that the state 

had revised or discontinued the assessments upon which the projects had based their 

objectives. 
V. Student Achievement Outcomes 

V-6

Between 51 and 63 percent of the objectives could not be evaluated each year either 

because projects submitted no results for them or provided information in a form that did not 

allow us to determine whether the specific goal set by the project had been attained (for example, 

a district might report schools’ average test scores to track an objective that called for increases 

in the percentage of students who scored at proficiency level on the state test). Outcome data for 

the first grant year (1998–1999) were unavailable for some magnet schools because they did not 

go into operation until 1999–2000. In the final grant year, a few schools had ceased to be MSAPfunded magnets, and some projects were unable to obtain 2000–2001 assessment results from 

their state testing agencies in time to include them in their final performance reports.4 We will 

discuss the issue of data availability further in Chapter VII. 

School-Level Goals Included in the Analysis 

In order to assess the degree to which individual schools met the goals set for them in the 

project-level objectives, we separated each objective for which any usable outcome data had 

been reported into a series of year-specific school-level goals (e.g., goals pertaining to spring 

2000 and 2001 mathematics scores for grades 3, 4, and 5 for School A and for School B). This 

process yielded a total of 2,894 school-level goals that are included in our analysis.5

 

Subject Domain 

As shown in Exhibit V-4, more than 80 percent of the school-level goals that could be 

tracked involve student performance in language arts (46 percent) or mathematics (36 percent). 

As stated earlier, these are the subjects that are most frequently measured by state assessment 

programs and thus, subjects for which data are most easily available to projects. The remaining 

20 percent of the school-level goals are distributed in small numbers among many other areas of 

endeavor. “Non-subject-specific” objectives involve a variety of behavioral outcomes associated 

with student achievement such as end-of-year promotions or retentions, high school graduation, 

taking college aptitude tests, and participating in community-based experiences. “Other” goals 

are based on a variety of measures including composite scores on multiple subject assessments 

(e.g., high school competency examinations), enrollment in honors courses, and teachers’ ratings 

of students’ work habits. 

 

4

 About three quarters of schools with little or no usable outcome data were in districts in which little or no usable 

data were reported for any of the MSAP schools, or any of the schools at a particular grade level. The remaining 

one-quarter included new schools that opened during the grant period or were discontinued as magnets during the 

study. Overall, about one-third of MSAP-funded elementary and middle schools, and half of the high schools 

had missing or unusable data for two or three of the three years included in our analyses. 5

 An additional 184 goals for which projects reported only district-level data are not included in the analyses. 

More than 80 percent of the school-level goals that could be tracked involve student 

performance in language arts (46 percent) or mathematics (36 percent). By far the most 

common measures used for evaluating student performance are standardized tests. 
 V. Student Achievement Outcomes 

 V-7

Exhibit V-4. 

Description of the school-level goals by subject domain 

Exhibit reads: Forty-six percent of school-level goals involve student performance in language arts, while 36 

percent of the goals involve student performance in mathematics. 

N = 2,894 school-level goals 

Source: MSAP grant applications and annual performance reports 

Type of Measure 

As may be seen in Exhibit V-5, by far the most common measures used for evaluating 

student performance are standardized tests, which account for approximately 88 percent of the 

school-level goals. (These account for 93 percent of the goals for mathematics achievement and 

92 percent of the goals for language arts, and these two subject areas account for 87 percent of 

goals based on standardized tests.) Other measures are less common and are distributed across a 

variety of categories such as alternative (non-standardized) assessments, enrollments and grades 

in particular courses, completion of requirements for college admission, and educational 

attainments such as promotion or graduation. (Most of the objectives whose measurement type is 

categorized as “other” undertook to relate student achievement outcomes and measures of 

program implementation and curriculum alignment with state standards.) 

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Language Arts

Math

Science

Social Studies

Computer/Vocational Skills

Not subject-specific

Arts and Performances

Career Awareness

Other

Missing
V. Student Achievement Outcomes 

V-8

Exhibit V-5. 

Types of measures used for school-level goals 

Exhibit reads: Standardized tests are the measures used for approximately 88 percent of the school level goals for 

which data are available. Other types of measures are much less common. 

N = 2,894 school-level goals 

Source: MSAP grant applications and annual performance reports 

Exhibit V-6 shows that the numbers of goals pursued by the MSAP-supported schools 

varied considerably. (The exhibit summarizes the numbers of goals per school reported in 1999–

2000; Appendix Exhibits A-V-5 and A-V-6 provide detailed information for all three years). A 

majority of schools have multiple goals for both subjects, usually because multiple grades within 

the school are tested. (In addition, some schools specify goals for both minority and nonminority 

students separately rather than just for “all students.”) Schools tend to have more goals in 

language arts than in mathematics because many states assess students in both reading and 

writing. The average number of goals per school is between 3 and 4 for each project year (not 

shown), but a small percentage of schools had more than 10 goals per subject. 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Standardized

Test

Course-taking

and Grades

Alternative

Assessments

Preparation for

College

Educational

Attainments

Other Missing
 V. Student Achievement Outcomes 

 V-9

Exhibit V-6. 

Number of language arts and mathematics goals per school (1999–2000) 

Exhibit reads: In 1999–2000, 23 percent of the MSAP magnet schools had one goal for language arts achievement, 

in contrast to 36 percent that had between 4 and 10 goals and 12 percent that had 11 or more goals. 

N = 185 schools with language arts and 183 schools with mathematics goals in 1999–2000 

Source: MSAP grant applications and annual performance reports 

Number of mathematics goals per school (1999-2000)

1 goal

31%

2-3 goals

35%

4-10 goals

25%

11+ goals

9%

1 goal

2-3 goals

4-10 goals

11+ goals

Number of language arts goals per school (1999-2000)

1 goal

23%

2-3 goals

29%

4-10 goals

36%

11+ goals

12%

1 goal

2-3 goals

4-10 goals

11+ goals
V. Student Achievement Outcomes 

V-10

Attainment of School-Level Goals 

To what extent did MSAP-funded schools meet, or make progress toward meeting, the 

student achievement goals that their projects set for them? Of necessity, the analysis undertaken 

to answer this question is limited to the school-level goals for which projects supplied usable 

outcome data in documenting their annual progress. In this section, we examine the percentage 

of year-specific goals that schools met during each year of the grant, as well as the percentage of 

“final” (end-of-project) goals they met. In the next section we examine the extent to which 

schools made progress toward meeting these goals, even when they did not meet them outright. 

As explained above, the vast majority of the school-level goals for which outcome data are 

available relate to students’ performances on standardized tests in language arts and 

mathematics, and these are the focus of this study. Goals of other types are too few and diverse 

to support detailed analyses. 

Our analysis addresses two related questions concerning annual and end-of-project goals. 

First, what proportion of the benchmarks that were set for them at the beginning of the project 

did each school meet? This question is an important one, but sets a very high standard: it does 

not take into account the fact that schools faced different levels of challenge in meeting their 

goals. Schools differed both in the number of goals they had to meet and in the amount of 

improvement required to meet them. Thus, one school might make laudable progress and yet not 

meet its benchmarks, while another school made minimal progress but met a higher proportion 

of goals that were less demanding or fewer in number. To take into account this variation in 

challenge, we explored a second question: even if the schools did not meet their benchmarks, 

what proportion of their goals did they make progress toward meeting? Not only is this question 

sensitive to any improvement in performance, but it can take into account some results that were 

reported in a format that did not match the format specified by the objective. (For example, if the 

objective called for improvements in the percentage of students passing a test, but the project 

only reported average student scores, we could not determine whether the objective was met, but 

we could determine whether progress had been made.) 

Meeting Annual Goals 

We examined the percentage of year-specific goals in language arts and mathematics that 

schools met in each year and also whether they met their final goals by the end of the third year. 

A year-specific goal sets a benchmark to be measured in a particular grant year. The nature of 

the change specified by these goals are of several types. Most common are goals that require 

schools to meet increasingly demanding performance benchmarks in successive years. For 

instance, a school may be required to show annual increases in performance relative to its 

Outcomes could be assessed for about half of the MSAP-supported schools in 1998-1999 

and for just under two-thirds of the schools for the latter two years of the study. Overall, 

schools were most successful in meeting or making progress toward their initial goals, but 

continued improvement over longer periods of time proved more difficult. Overall, about 51 

percent of all schools met half or more of their final goals for language arts and 

approximately 36 percent did so for mathematics. About one in five magnet schools met 

virtually all of their end-of-project goals for language arts and mathematics, and two in five 

made progress toward meeting all of their end-of-project goals for these subjects. 
 V. Student Achievement Outcomes 

 V-11

baseline year scores (“The average mathematics score for third grade will increase by 2 

percentile point in the first year of the grant, 3 additional points in the second year of the grant, 

and an additional 4 points in the final year of the grant.”); to meet an escalating standard that is 

not tied specifically to a baseline year (“The proportion of students scoring at proficient or above 

in reading will be at least 75 percent in the first year, 80 percent in the second year, and 85 

percent in the third year.”); or to narrow the performance gap between its students and the 

district average (“The school’s average score will be no more than 5 percentile points below the 

district average in the first year, will equal the district average in the second, and exceed it in the 

third.”). In all of these cases, first year goals are less demanding than second and third year 

goals. A second type of goal requires the magnet school’s annual performance or gains over 

time to compare favorably with those of a comparison group (“Each year, magnet students’ gains 

will equal or exceed those of the district as a whole.”). The challenge set by such goals depends 

on the school’s initial level of performance relative to the reference group and the reference 

group’s subsequent increases or decreases in performance. Finally, some goals set an unchanging 

standard of performance to be met each year (“Each spring, at least 85 percent of students will 

score at mastery level on the district’s computer skills test.”). In evaluating the attainment of 

year-specific goals, we classified a goal as having been met if the school’s performance in that 

year equaled or exceeded the amount set in the objective. (In the case of objectives calling for 

cumulative growth relative to a baseline level, to continue the first example, a school that 

increased by 1 percentile point in the first year and an additional 3 percentile points in the second 

would be counted as having met the second year goal but not the first year goal.) 

Final goals set a level of performance that the school is expected to reach by the end of 

the grant. Some call explicitly for cumulative growth (a school meeting the objective described 

above would have to show 10 percentile points of improvement between 1998 and 2001), while 

others simply set a level to be met (for example, “by the end of the final project year, 85 percent 

of students will demonstrate awareness of science careers”). 

Although MSAP-supported programs operated in a total of 292 schools over the course 

of the 1998–2001 grant period, the number of schools for which analyzable outcome data were 

available each year was considerably smaller. Outcomes could be assessed for about half of the 

schools in 1998–1999 and for just under two-thirds of the schools for the latter two years of the 

study (detailed information for all three years is included in Appendix Exhibits A-V-5 and 

A-V-6). In view of the number of schools for which outcome data were unavailable, the results 

presented below must be interpreted with caution. 

Exhibits V-7 and V-8 display the percentages of schools at each grade level and overall6

that met half or more of their school-level goals for language arts and mathematics, respectively 

(see Exhibits A-V-7 through A-V-14 for more details). The figures show that overall, schools 

were most successful in meeting their initial goals, but meeting benchmarks over longer periods 

of time proved more difficult. About 57 percent of the schools met half or more of their 1998–

1999 English language arts goals, whereas only 42 percent met half or more of their 2000–2001 

goals. The corresponding percentages for mathematics goals were 57 percent for 1998–1999 and 

33 percent for 2000–2001. This decline in goal attainment is most pronounced for high schools 

and least pronounced for elementary schools. As will be seen in the second part of this chapter, 

 

6

 The analysis for “all schools” includes the elementary, middle, and high schools displayed in the charts as well as 

a few additional schools whose wide grade spans did not allow us to assign them to a single grade level. 
V. Student Achievement Outcomes 

V-12

the downward trend in the proportion of goals met does not reflect an actual decline in the 

schools’ performance over time. It does indicate that schools had more difficulty in meeting 

continuous improvement goals later in the grant. In some cases, schools failed to sustain initial 

gains in later years, while in others they were unable to meet goals that called for levels of 

student performance to increase each year. 

Exhibit V-7. 

Percentage of schools that met half or more of their English language arts goals 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1998-1999 1999-2000 2000-2001

Percent met half or more goals

All schools

Elementary schools

Middle schools

High schools

Exhibit reads: The percentage of all schools that met half or more of their English language arts goals declined 

from about 57 percent in 1998–1999 to 42 percent in 2000–2001. 

N = 147 (1998–1999), 185 (1999–2000), and 182 (2000–2001) schools with English language arts outcome data 

Source: MSAP annual performance reports 

Exhibit V-8. 

Percentage of schools that met half or more of their mathematics goals 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1998-1999 1999-2000 2000-2001

Percent met half or more goals

All schools

Elementary schools

Middle schools

High schools

Exhibit reads: The percentage of all schools that met half or more of their mathematics goals declined from about 

57 percent in 1998–1999 to about 33 percent in 2000–2001. 

N = 134 (1998–1999), 183 (1999–2000), and 190 (2000–2001) schools with mathematics outcome data 

Source: MSAP annual performance reports 
 V. Student Achievement Outcomes 

 V-13

Meeting End-of-Project Goals 

Thus far, we have seen that percentages of schools meeting goals set for them by the 

MSAP projects dropped over the three-year grant period. We also examined the extent to which 

magnet schools had reached their end-of-project goals. Summary results are presented in the 

following two figures, and detailed information is provided in Appendix Exhibits A-V-7 through 

A-V-14. Exhibit V-9 shows that a slight majority (51 percent) of all schools met half of their 

language art achievement goals, while about one-third (36 percent) of schools met half their 

mathematics achievement objectives. At all three grade levels, schools tended to meet a greater 

proportion of their language arts goals than their mathematics goals. This result may reflect a 

greater or longer-term emphasis on improving literacy instruction during the grant period or a 

greater challenge involved in bolstering the effectiveness of mathematics instruction.7 Overall, 

about one in five schools met virtually all of their end of project goals for language arts and 

mathematics (see Appendix Exhibits A-V-7 and A-V-8). 

Exhibit V-9. 

Percentages of schools that met half or more of their final goals 

Exhibit reads: Overall, about 51 percent of all schools met half or more of their final goals for language arts and 

approximately 36 percent did so for mathematics. 

N = 184 schools with English language arts outcome data and 191 schools with mathematics outcome data in 2000–

2001 

Source: MSAP annual performance reports 

 

7

 We explored the possibility that schools with fewer goals in a subject were more successful in meeting half or 

more of them than schools with more goals. We found that schools with the largest and smallest number of goals 

(more than ten or only one) were least successful in meeting half or more of them, but for schools in the middle 

range (two to ten goals) there was no consistent relationship between numbers of goals and success in meeting 

them. It should be noted that the number of goals pursued by a school was largely a function of the number of 

grades assessed by the state rather than of multiple goals being set for a particular set of students. Thus, the 

inconsistent relationship between number of goals and proportion met is not particularly surprising. 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

All schools Elementary

schools

Middle schools High schools

Percent met half or more goals 

Mathematics

Language Arts
V. Student Achievement Outcomes 

V-14

Making Progress toward Meeting Annual Goals 

Exhibits V-10 and V-11 display the percentages of schools that made progress toward 

meeting half or more of their school-level goals (see Appendix Exhibits A-V-15 through A-V-22 

for more details). The decline in the percentage of schools making progress each year is less 

precipitous than that of schools meeting their goals outright. Thus, some schools had difficulty in 

meeting the specific targets set for them even though their overall test scores may have been 

increasing. In some cases, schools failed to sustain initial gains in later years while in others they 

were unable to meet goals that called for levels of student performance to increase each year. 

Exhibit V-10. 

Percentage of schools that made progress toward half or more of their 

English language arts goals 

Exhibit reads: The percentage of schools that made progress toward half or more of their English language arts 

goals declined from about 70 percent in 1998–1999 to about 58 percent in 2000–2001. 

N = 147 (1998–1999), 185 (1999–2000), and 182 (2000–2001) schools with English language arts outcome data 

Source: MSAP annual performance reports 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1998-1999 1999-2000 2000-2001

Percent made progress toward half or more goals

All schools

Elementary schools

Middle schools

High schools
 V. Student Achievement Outcomes 

 V-15

Exhibit V-11. 

Percentage of schools that made progress toward half or more of their mathematics 

goals

Exhibit reads: The percentage of schools that made progress toward half or more of their mathematics goals 

decreased from about 72 percent in 1998–1999 to 53 percent in 2000–2001. 

N = 134 (1998–1999), 183 (1999–2000), and 190 (2000–2001) schools with mathematics outcome data 

Source: MSAP annual performance reports 

Making Progress toward End-of-Project Goals 

Exhibit V-12 presents the percentages of schools that made progress toward their end-ofproject goals. Approximately 67 percent and 59 percent of the magnet schools made progress 

toward half or more of their final goals for language arts and mathematics, respectively. (See 

also Appendix Exhibits A-V-15 through A-V-22.) At all grade levels, more schools made 

progress toward goals for language arts than for mathematics. About two in five magnet schools 

made progress toward meeting all of their end-of-project goals for language arts and 

mathematics (see Appendix Exhibits A-V-15 and A-V-16). 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1998-1999 1999-2000 2000-2001

Percent made progress toward half or more goals

All schools

Elementary schools

Middle schools

High schools
V. Student Achievement Outcomes 

V-16

Exhibit V-12. 

Percentage of schools that made progress toward half or more of their final goals 

Exhibit reads: Overall, about 67 percent of schools made progress toward half or more of their final goals for 

language arts and approximately 59 percent did so for mathematics. 

N = 184 schools with English language arts outcome data and 191 schools with mathematics outcome data in 2000–

2001 

Source: MSAP annual performance reports 

Discussion of Achievement Objective Results 

In response to the federal mandate to strengthen students’ knowledge and skills, MSAP 

projects pursued many and varied student achievement objectives over the three-year grant 

period. Although these objectives involved accomplishments in several academic subjects and 

performance domains such as computer literacy and performing arts, the largest proportion of 

these objectives focused on achievement in English language arts and mathematics. For 

individual MSAP-supported schools, these objectives translated into a profusion of specific 

goals. The vast majority of those goals for which outcome data are available involve students’ 

performances on standardized tests in language arts and mathematics. It is noteworthy that the 

projects were unable to report three years of outcome data pertaining to a considerable fraction 

of the goals they set at the beginning of their grants, often due to problems in developing local 

measures and to discontinuities in the assessment programs operating in their states. Overall, 

only about one-third of the objectives adopted by the 57 MSAP projects were represented by 

analyzable outcome data in annual performance reports, and the annual goal attainment analyses 

reported here are based on test results from about one-third of the 292 MSAP schools that 

operated between 1998 and 2001. 

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

All schools Elementary

schools

Middle schools High schools

Percent made progress toward half or more goals

Mathematics

Language Arts
 V. Student Achievement Outcomes 

 V-17

In general, it appears that MSAP projects set ambitious performance benchmarks for their 

magnet schools. Overall, about half of the MSAP schools met a substantial proportion (half or 

more) of their annual English language arts and mathematics goals outright, and about two-thirds 

of them made progress toward doing so. Schools were most successful in meeting their 1998–

1999 goals, somewhat less so in meeting the goals set for the later years. This pattern was 

evident in the results for schools at all three grade levels. Overall, elementary schools were 

somewhat more successful in meeting their goals, and the decrease in the proportion of schools 

meeting or progressing toward their goals in the later years was somewhat smaller for them. 

Although these results are not dramatically positive, it is important to keep in mind that MSAPsupported programs are usually located in schools with a substantial need for improvement that 

may not be easy to accomplish over the short term, and the goals set for schools varied in the 

magnitude of change they required. Goals for later years frequently required sustained growth 

over three years, and some required magnet schools to show higher scores or greater growth than 

other schools in their districts. 

A number of factors may account for the grade level differences observed in these results. 

The smaller size and simpler organization of elementary schools and the greater uniformity of 

content covered in the elementary grades may make it easier for them than for the 

departmentalized middle and high schools to design and implement coherent educational 

interventions that yield discernable school-level performance gains within the timeframe of the 

MSAP grants. Given the focus of most state accountability systems and instructional 

improvement initiatives on the performance of elementary students, it is also possible that the 

non-MSAP resources to help schools improve student achievement are more substantial for 

elementary than secondary schools. It is also possible that districts have greater experience with 

setting realistic growth targets for elementary than secondary schools. 

It is important to keep in mind that the downward trend observable in these results does 

not indicate that students’ average scores were declining over this period, but rather that they 

were not meeting goals that were set for them in applications for MSAP grants. The context in 

which these goals were set—applying for a magnet school grant—may well have encouraged 

optimistic estimates of schools’ potential for improved performance. Without considerable 

knowledge of the conditions existing in each school, it would be difficult to assess how realistic 

these goals were. In view of the fact that some projects revised or abandoned some of their 

objectives over the course of the grant, it seems likely that grant writers were overly optimistic 

about the districts’ ability to collect achievement data and schools’ ability to meet the goals set 

for them. 

Because most of these goals focused on changes in MSAP-supported schools’ 

performance without reference to the performance of comparison schools facing similar 

challenges, these results do not tell us how the magnet schools’ performance compared to that of 

similar schools that did not have MSAP grants. The study described in the next section addressed 

this question with data provided by state testing agencies. These data allowed us to examine the 

performance of MSAP elementary schools relative to that of matched comparison schools in the 

same states. 
V. Student Achievement Outcomes 

V-18

Study of Achievement Trends in MSAP and Comparison Schools 

The preceding analysis provides information about the MSAP projects’ success in 

meeting objectives they set for themselves, but has a number of limitations. First, the projects set 

different standards for themselves (different numbers of objectives with varying degrees of 

challenge), making it difficult to generalize about the overall improvement in student 

achievement accomplished by the MSAP-supported schools as a group. In addition, most 

achievement objectives set by districts were made without reference to the performance of 

comparable students who had not been involved in MSAP magnet programs. Because it is 

common for all schools’ test scores to rise over time (as instruction becomes oriented to the state 

assessments and students become more familiar with what is expected of them), comparative 

data from similar schools that did not receive MSAP support are needed to gauge the degree to 

which MSAP students made progress over and above the general trend in their states. Finally, 

districts were unable to provide much of the outcome data needed to determine whether they had 

met their objectives. Consequently, we could calculate rates of meeting or making progress 

toward meeting objectives for only a fraction of the MSAP-supported schools. 

This section presents the results of a study that avoided some of these problems by using 

a common standard and comparison groups to gauge the accomplishments of the MSAPsupported schools. The study used average school scores on state assessments in reading and 

mathematics administered between 1997–1998 and 2000–2001. These data were obtained from 

the National School-Level State Assessment Score Database, which contains test scores and 

student demographic data for virtually all of the schools in the United States. Therefore, we were 

able to identify non-MSAP-supported schools in each state to serve as comparisons for the 

MSAP magnet schools. Furthermore, we were able to convert multiple years of data from 

different tests (or in different metrics) to a common scale for purposes of analysis. A major 

limitation of these data for the period of the study is that they are overall school averages that 

cannot be disaggregated by student characteristics such as minority and poverty status, and thus 

do not allow us to determine the degree to which different student groups are showing similar 

trends. Nonetheless, they represent a substantial advance over the self-reported data analyzed in 

the first part of this chapter. 8

 

8

 The original plan of this study called for an analysis of longitudinal achievement data of individual students in 

MSAP-supported and comparison schools in selected districts. Changes in policy regarding the use of studentlevel data in federally sponsored research required us to abandon this plan in favor of an analysis of school-level 

data for a larger number of districts. 

AIR used school-level state assessment scores to examine gains in reading and mathematics 

performance of MSAP-supported magnet schools and a sample of matched comparison 

schools without MSAP funding. Both the MSAP-supported magnet schools and the 

comparison schools made noticeable progress in reading and mathematics during the grant 

period. However, when changes in the demographic composition of the schools were 

controlled, the gains exhibited by MSAP schools were not significantly different from those 

exhibited by non-MSAP schools with similar characteristics. 
 V. Student Achievement Outcomes 

 V-19

The analyses reported in this section are based on outcomes from 135 MSAP-supported 

elementary magnet schools from 14 states.9

 We focused on elementary schools both because 

there are more MSAP-supported schools at this level than at higher grades and because more 

state assessment data are available for these schools than for middle and high schools. Two 

major questions are addressed. First, did achievement trends for MSAP schools from the spring 

of 1998 (prior to the MSAP award) through spring 2001 differ by a statistically significant 

amount from the trends for matched non-MSAP schools? We used hierarchical linear modeling 

(HLM) regression techniques to compare the changes in achievement of MSAP and comparison 

schools, while controlling for other factors that might account for differences in their 

performance. Second, among the MSAP magnet schools, were particular aspects of the schools 

or their projects associated with greater student achievement growth? To explore this question, 

we conducted correlational analyses using achievement trend measures and indices of school and 

magnet project characteristics derived from responses to AIR’s Principal and MSAP Project 

Director surveys. 

Did achievement trends for the MSAP schools differ by a statistically significant 

amount from the trends for matched non-MSAP schools? 

To address this question, we used hierarchical linear modeling to compare trends in 

reading and mathematics achievement of MSAP-supported and comparison schools, while 

controlling for additional factors that might account for differences in their students’ 

achievement. For each MSAP-supported school, we selected 10 additional schools in the same 

state that were most like the MSAP school in 1997–1998 (the year before the MSAP grants were 

awarded) in terms of: 

• Urbanicity of the community served. 

• Number of students in the school. 

• Percentages of Asian-Pacific Islander, Black, Hispanic, and Native American 

students. 

• Percentage of students in poverty (i.e., eligible for free or reduced-price lunches). 

• Average test scores in reading and mathematics in the baseline year (spring 1998).10

Thus, schools were matched on their demographic and achievement characteristics.11

MSAP schools that had relatively low performance on statewide tests were matched with 

comparison schools with similarly low performance.

 

9

 The study includes MSAP elementary schools that had usable score data in both 1997–1998 (or 1998–1999) and 

2000–2001 for at least one grade and were located in states that had such data for at least two MSAP-supported 

schools. Fifty elementary schools were excluded. See Appendix Exhibit A-V-23 for an explanation of the 

exclusions. 10 Matches for each MSAP school were made using the Mahalanobis’s distance measure to identify its 10 

statistically “closest” neighbors within the state. The statistic takes account of two schools’ similarity on all of 

the selection characteristics simultaneously. Descriptive data comparing characteristics of MSAP to matched 

non-MSAP elementary schools are provided in Appendix V. See Exhibit A-V-24. 11 Demographic and achievement characteristics were matched for the year prior to grant awards, except in the case 

of free or reduced price lunches where the best available data for matching were from 1999-2000. 
V. Student Achievement Outcomes 

V-20

The outcome measures for each school were the trend (amount of change) in their reading 

and mathematics scores between their baseline year and the last year of the MSAP grant (2000–

2001). Because testing conditions and outcome measures differ among states (and sometimes 

within states in different years), it was necessary to standardize all the school-level scores before 

calculating the school trends (see Appendix V for a discussion of the standardization method). 

We conducted a multilevel regression analysis that modeled the schools’ achievement 

trend as a function of the change in minority composition between the baseline year and 2000–

2001, the percentage of students in poverty, and the school’s MSAP status. By restricting the 

analysis to the 135 MSAP magnet schools and their matched comparison schools, we also 

controlled for the effects of demographic composition and performance on the state assessment 

at the beginning of the MSAP grant period. (For details of the regression analysis, see Appendix 

Exhibit A-V-25.) 

Results for both reading and mathematics showed that both the MSAP magnets and their 

non-MSAP comparison schools made noticeable progress during the grant period (an average of 

0.5 standard deviation units). Changes in enrollment composition were associated with small but 

statistically significant increases (an increase in nonminority students was associated with higher 

achievement gains). When changes in demographic composition were controlled, however, the 

gains exhibited by MSAP schools were not significantly different from those exhibited by nonMSAP schools with similar characteristics. 

The analysis described above compared the achievement of MSAP magnet schools and 

the 10 most similar non-MSAP schools in the same state. Because differences between districts 

might themselves play a role in the academic success of students, we also conducted an analysis 

that compared the achievement of MSAP-funded and non-MSAP schools located in the same

district. (Whereas each MSAP-supported school had 10 comparison schools selected from 

throughout its state, the number of these comparison schools that were located in the same 

district ranged from 0 to 10.) The results of the full analysis were reproduced in the same-district 

analysis: that is, the gains of the MSAP-funded schools still did not differ significantly from 

those of non-MSAP schools once changes in demographic composition were taken into 

account.12

These results warrant some comment. First, they argue against the common assumption 

that improvement in MSAP schools’ test scores over time is merely a reflection of the different 

mix of students enrolled in them as a result of the magnet program. When changes in 

demographic composition are controlled for, the MSAP schools still show growth equivalent to 

the comparison schools. Second, it is possible that the magnet schools do not show greater 

growth than their counterparts because the three-year time frame of this study may have been too 

 

12 We also considered the possibility that the lack of difference between MSAP-supported and comparison schools 

was due to the inclusion of magnet schools supported with state or local funds among the randomly selected 

comparison schools. We found that information about magnet programs available in NCES’s school census (the 

CCD) is too incomplete to allow identification of all magnet schools in the comparison sample. However, it 

appears that the proportion of such schools is quite small—not enough to account for the similarity of 

achievement results in the MSAP and non-MSAP schools. 
 V. Student Achievement Outcomes 

 V-21

short to allow instructional improvements to be fully implemented and the benefits of these 

efforts to be realized to their full potential. 

Third, and perhaps most importantly, magnet schools may not stand out from their 

counterparts because both sets of schools have been working to improve their students’ 

achievement. Systemic reform is intended to be systemic—that is, to improve all schools through 

standards-aligned curricula, targeted professional development for teachers and administrators, 

and resources to make instruction more effective and to support struggling students. While 

MSAP grants helped to fund such efforts in the magnet schools, district and school 

administrators sought funds from other sources to support similar work in non-MSAP schools. 

This pattern was evident in a number of our case study sites, where we found substantial 

similarities between the language arts and mathematics instruction in magnet and comparison 

schools. For instance, District G’s school reform program provided a common, districtdeveloped curriculum and professional development for all of its schools. In District A, the 

magnet and comparison schools had adopted the same accelerated reading program and deployed 

newly acquired computers and software for instructional purposes. It is likely that what occurred 

in the microcosm of the case study districts reflects statewide trends affecting the non-MSAP 

comparison schools in our statewide samples. We consider these issues in more detail, using data 

from our eight case studies, in Chapter VI. 

Among the MSAP magnet schools, were achievement gains associated with 

particular characteristics of the school or the MSAP project? 

Although overall results showed no significant differences between MSAP and 

comparison schools, some magnets made greater achievement gains than others. To investigate 

the extent to which these differences were associated with school and project characteristics, we 

used regression methods to create a “residual achievement measure” that indicated the degree to 

which each school’s actual achievement trend from 1998 to 2001 diverged from the trend that 

would be predicted based on its demographic composition and prior performance on state tests. 

Correlation methods were then used to gauge the relationships between this performance 

indicator and the school and project characteristics measures derived from responses to AIR 

surveys (see Chapter III). 

As shown in Exhibit V-13, for both reading and mathematics, the measure most strongly 

associated with achievement growth in the MSAP magnet schools was the overall strength of the 

professional community in the school. This measure combines principals’ responses to 15 items 

concerning the extent to which the school staff share a common vision for the school, support 

The measure most strongly associated with achievement growth in both reading and 

mathematics in MSAP-supported magnet schools was the overall strength of the professional 

community of the school. Greater progress in reading was also associated with professional 

development related to standards-based reform. Greater progress in mathematics was also 

associated with schools in which state or district standards and frameworks reportedly had a 

strong influence on curriculum and instruction decisions, and with programs in schools 

whose magnet programs were in operation prior to the 1998 grant award. 
V. Student Achievement Outcomes 

V-22

and respect each other, coordinate instruction and enforce school rules, continue to learn and 

seek new ideas, and care for students. 

Two other factors correlated (p<.05) with growth in one of the subject areas. Greater 

progress in reading was associated with more professional development related to standardsbased reform (that is, training about state or district standards, assessment of student 

performance, and addressing the needs of students from diverse cultural backgrounds, students 

with limited English proficiency, and students with disabilities). Greater progress in mathematics 

was associated with principals reporting that mathematics frameworks and assessments had a 

strong influence on curriculum and instruction decisions in the school. Progress in mathematics 

was less strongly associated (p<.10) with two additional measures: (1) the degree to which state 

curriculum frameworks and assessments were seen to align with the goals of the school’s 

program, and (2) the salience of factors associated with student disengagement with school (e.g., 

tardiness, absenteeism, lack of preparation to learn, apathy, poor health, lack of parent 

involvement). 

Finally, another factor correlated with achievement growth in mathematics (p<.05) was 

the length of time that the magnet school program had existed. Of the 135 schools included in the 

study, 61 (45 percent) had been operating a magnet program in 1997, the year prior to the MSAP 

award. The other 74 schools initiated new magnet programs in 1998. Continuing a magnet 

program that had been established prior to 1998 was associated with higher gains in mathematics 

achievement between 1998 and 2001. This suggests that “mature” magnet schools may be more 

successful in improving achievement than newer schools. 
 V. Student Achievement Outcomes 

 V-23

Exhibit V-13. 

Correlations between magnet school achievement growth and school or program 

characteristics 

Correlation with 

Achievement Growth 

School/Program Characteristic13 Reading Mathematics 

Overall strength of professional community as measured in 1999–2000 0.20** 0.17** 

Overall strength of professional community as measured in 2000–2001 0.24*** 0.22*** 

Professional development related to standards-based reform (1999–2000) 0.23** (n.s.) 

Extent to which subject area frameworks and assessments guide 

curriculum and instruction decisions (n.s.) 0.21** 

Extent to which state assessment and standards relate to magnet program 

goals (n.s.) 0.17* 

Extent of student disengagement (1999–2000) (n.s.) -0.16* 

Extent of student disengagement (2000–2001) (n.s.) -0.16* 

Magnet program new in 1998–1999 (n.s.) -0.17** 

* = p:<.10 

**= p <.05 

***=p <.01 

Exhibit reads: The correlation between the overall strength of a school’s professional community as measured in 

1999–2000 and the school’s gains in reading achievement is 0.20, indicating that schools with stronger professional 

community tend to have higher achievement growth. 

N = 135 MSAP elementary schools 

Source: National School-Level State Assessment Score Database 

We explored a variety of other factors that were potentially related to the magnet schools’ 

academic performance, but none of them correlated significantly with the degree of achievement 

growth. Specifically, we constructed several measures of project features based on project 

director responses to the 1999–2000 and 2000–2001 Project Surveys. These scales indicated the 

extent to which state frameworks, assessments, and performance standards were judged to have 

influenced MSAP themes and goals, the degree to which the MSAP project’s district officers 

provided technical assistance to the schools on curriculum and instruction, and the degree to 

which they provided technical assistance on leadership. None of these scales correlated at a 

statistically significant level with the degree of student achievement growth among MSAPsupported schools. Furthermore, the following school-related scales also did not show 

statistically significant correlations: 

• Extent of student misbehavior (e.g., disrespect or abuse of teachers, physical conflict 

among students, vandalism). 

• The number of academic activities (e.g., books read) the magnet program required of 

students. 

 

13 See Chapter III and Appendix III for a discussion of the school characteristic scales and the survey items on 

which they are based. 
V. Student Achievement Outcomes 

V-24

• Structures for involving parents in school activities (e.g., regular parent conferences, 

parent education workshops). 

• Structures for involving parents in school governance. 

• Principal’s familiarity with frameworks and assessments in mathematics and 

language arts. 

Summary 

The results presented in the first part of this chapter showed a decline in the proportion of 

MSAP-supported schools meeting student achievement goals between 1998 and 2001. In 

contrast, the results of the second study show that overall the students in MSAP-supported 

elementary schools accomplished gains in reading and mathematics that were commensurate 

with those of their counterparts in non-MSAP schools. Information from surveys, interviews and 

site visits indicate that schools varied substantially in the speed with which they implemented 

their programs, the strength of the professional community that operated in the schools, the 

challenges presented by the student population, and the alignment of school programs with state 

standards and assessments. The analysis of factors associated with gains in student achievement 

suggests that magnet schools in which faculty members participated in a strong professional 

community were more likely than others to show gains in student achievement in both subjects. 

Gains in mathematics were associated with schools reporting alignment of the instructional 

program with state standards and assessments, while gains in reading were associated with 

standards-based professional development—that is, student achievement on state tests was 

associated with program features that supported teachers in aligning their instruction with state 

standards.14 The results also provide some evidence that the three-year grant period did not 

provide new magnet schools sufficient time to get their programs in place and show measurable 

gains in student achievement that those programs were expected to produce. 

In sum, the results for student achievement are mixed. On the one hand, the overall 

achievement of students attending MSAP-supported magnet schools (as measured by 

standardized test scores in elementary schools) increased between 1998 and 2001, and many 

schools at all grade levels met or made progress toward meeting a complex array of achievement 

goals that had been set for them. On the other hand, the gains of the MSAP schools were not 

significantly different from those of similar non-MSAP schools. In addition, schools were less 

able to meet the (usually more demanding) goals set for them in later years than initially. Finally, 

there was some evidence that variations in program features and implementation conditions were 

related to achievement gains. Chapter VI will address in more detail the program implementation 

issues and other factors that affected projects’ success in meeting the legislative purposes of the 

MSAP. 

 

14 Differences in the types of standards alignment associated with reading and mathematics may relate to 

differences in content area differences. For instance, state standards may specify the sequence in which students 

encounter mathematical topics, making alignment of curricula with standards a more salient issue in mathematics 

than in reading. 
 VI. Putting the Results in Context 

 VI-1

VI. Putting the Results in Context 

In Chapters III, IV, and V, we examined the success of MSAP projects and schools in 

meeting the four main goals set by the program: supporting systemic reform; promoting 

innovative educational practices; reducing, preventing, or eliminating minority student isolation; 

and improving student achievement. We reported results based primarily on quantitative 

demographic, survey, and achievement data for the full population of MSAP districts and 

schools. Based on these data, we reported that MSAP projects made progress toward meeting the 

program objectives in some areas, but less progress in others.1

In brief: 

• With respect to the support of systemic reform, we found that MSAP projects 

generally sought to align their programs with state systemic reform efforts but, at 

times, experienced some conflict between the instructional goals of the magnet 

programs and state or district standards and assessments. 

• With respect to the adoption of innovative practices, we found that MSAP schools 

adopted a wide range of innovative themes and programs. According to our survey 

data, MSAP schools have somewhat more positive school climates and other 

attributes generally associated with effective schools than non-MSAP comparison 

schools. 

• With respect to desegregation outcomes, we found that after adjusting for districtwide 

changes in minority enrollment, magnet programs for 57 percent of MSAP’s 

desegregation-targeted schools made progress in preventing, eliminating, or reducing 

minority group isolation, while 43 percent did not make progress. Although just 5 

percent of the targeted schools succeeded in preventing or eliminating MGI, 52 

percent reduced MGI by lowering the percent minority enrollment at the school 

relative to the percent minority enrollment in their districts. The extent of reductions 

varied, with 17 percent of schools reducing MGI by more than five percentage points, 

28 percent reducing MGI by one to five percentage points, and 7 percent reducing 

MGI by less than a percentage point. A more detailed analysis at the elementary level 

identified some features of schools and their programs associated with reductions in 

MGI, including a racially and ethnically mixed group of minority students, parents’ 

involvement in school activities, and lower student-to-teacher ratios (i.e., smaller 

class sizes). 

• With respect to student achievement, the results are mixed. Overall, relatively few 

MSAP schools met all of the achievement goals that had been set for them; however, 

many met a substantial proportion of them, and more made progress toward meeting 

them. We found that on average, MSAP elementary schools had about the same 

achievement trends over the period from spring 1998 through spring 2001 as a set of 

matched comparison schools, but the MSAP schools with more positive 

organizational and instructional features tended to show somewhat better 

achievement results than did other schools. Furthermore, schools with new magnet 

 

1

 A complete summary of the evaluation results with respect to the four MSAP goals is presented in Chapter VII. 
VI. Putting the Results in Context

VI-2

programs (initiated in 1998 or later) tended to show smaller gains in mathematics 

than schools whose programs had been in operation longer. 

In this chapter, we attempt to put these results in context, drawing on data from the case 

studies conducted in eight MSAP districts. (The case studies are presented in detail in the Case 

Studies Appendix.) First, we draw on the case study data to offer some tentative explanations for 

the relatively weak evidence of improved achievement outcomes. Our explanations focus 

primarily on the challenges of implementing magnet schools. Most if not all of the MSAP 

schools funded in 1998 selected magnet themes and adopted some innovative instructional 

programs, and there is some evidence that magnet schools showed improvements in climate and 

professional community. Nonetheless, the programs the schools adopted may not have been 

sufficiently well implemented to show effects on reading and mathematics achievement. In 

addition, the innovative themes adopted by many magnet schools were not directly focused on 

boosting reading and mathematics achievement. If such themes diverted the school’s focus from 

reading and mathematics instruction, the school’s likelihood of showing gains in reading and 

mathematics achievement test scores would be reduced. We consider these issues in the first part 

of the chapter. 

Next, we offer some tentative explanations for the modest progress in reducing minority 

student isolation. Our explanations center on the changing demographic composition of the 

MSAP districts, and on difficulties in disseminating information about magnet programs and 

providing support to parents in choosing to apply to and stay with a magnet school. 

 

We then turn to the broader impact of the MSAP program on the districts that received 

MSAP grants in 1998. We begin this discussion by describing the status of the MSAP schools in 

the first year after the end of the MSAP grant period. We conclude by considering some possible 

benefits districts derived from the MSAP grants, apart from those captured in the four main 

program goals. 

Implementation and Achievement Outcomes 

The in-depth case study data, as well as comments made by the MSAP project directors 

in telephone interviews, suggest that one explanation for the modest improvement in 

achievement in MSAP schools may be that magnet programs in many schools were not fully 

implemented during the grant period. Regardless of an innovative program’s potential for 

One explanation for the modest improvement in achievement in MSAP schools may be that 

magnet programs in many schools were not fully implemented or not implemented early 

enough for achievement effects to be manifest during the grant period. Several factors may 

have influenced the progress schools made in implementing MSAP programs: turnover in 

district and school leadership, tension between MSAP projects and other reform efforts, the 

absence of sufficient time to implement new programs, and the need to revise themes to 

attract students; and teacher resistance to changes being implemented. 
 VI. Putting the Results in Context 

 VI-3

fostering students’ achievement, program effects will not be manifest if the program itself has 

not been sufficiently implemented.2

One challenge we confront in discussing the implementation of magnet programs is that 

MSAP schools varied widely in the programs they attempted to implement. A uniform measure 

of implementation is thus difficult to apply across schools. In Chapter III, we described various 

organizational and instructional features that the literature indicates are associated with 

achievement—for example, professional community among teachers, positive school climate, 

parent involvement, and the use of specific approaches to instruction and assessment. We 

hypothesized that well-implemented magnet programs would have these features, which would, 

in turn, promote student achievement. Our survey data indicated considerable variation across 

schools in the extent to which these features were present in MSAP schools, which in turn 

suggests wide variation in implementation. 

The more nuanced information in our case study data provides further evidence that the 

quality and depth of implementation varied widely across schools. For instance, a program with 

deep implementation may be found in District G’s Character First High School. The program is 

infused throughout the curriculum and shapes the school schedule, which features weekly 

schoolwide meetings, as well as a class period devoted to character-building activities such as 

team sports and elementary school tutoring. Similarly, in a Montessori school in District E, 

students spend most of the day working individually and in small groups in multiage classrooms 

with distinctive curriculum and materials. By contrast, shallow implementation is exemplified by 

a middle school in District F, where a MicroSociety program operated much like an elective 

rather than as a theme running throughout the school day; the school continues to strive to 

connect the program with other school activities. In District C, we interviewed students in a 

school for accelerated studies in science, mathematics, and technology who were unaware that 

their school had a magnet theme. 

Several factors may have played an especially important role in influencing the progress 

schools made in implementing MSAP programs: turnover in district and school leadership; 

tension between MSAP projects and other reform efforts; the absence of sufficient time to 

implement new programs—especially programs that were locally developed or required 

 

2

 It must be stressed that in the following discussion, incomplete program implementation and insufficient time 

after implementation for desired results to become manifest are offered as tentative explanations of the modest 

improvement in achievement reported in previous chapters, and not necessarily the only ones (e.g., some 

programs may have greater potential to boost academic achievement than others). While we have substantial 

evidence that in many schools implementation took time to unfold, with the data we have available we can only 

speculate that the achievement patterns we observed were related to the degree of program implementation. As 

reported in Chapter V, we found that several features of MSAP schools, including strength of professional 

community and problems with student disengagement were associated with achievement. As shown in Exhibit 

III-1, we view these variables as mediating the effect of implementation on achievement, not as direct measures 

of implementation. The finding that some of these features improved between the second and third year of 

implementation (see Chapter III) lends support to the notion that these features are related to implementation. 

Finally, as reported in Chapter V, we found that mathematics achievement improved more for schools that had 

been magnets prior to the MSAP award than those that were new magnets. This, too, suggests, but does not 

demonstrate, that the time available for implementation may have been a factor affecting achievement outcomes. 

It clearly would be worthwhile to develop direct measures of the implementation of magnet programs, so that the 

relationship between implementation and achievement could be tested more rigorously.
VI. Putting the Results in Context

VI-4

extensive new staff, new equipment, new materials, or new modes of instruction; the need to 

revise themes to attract students; and teacher resistance to changes being implemented. 

Turnover in District and School Leadership 

Turnover in district and school leadership often made initial MSAP plans difficult to 

implement once the grant was received. As described in Chapter I, we surveyed principals in the 

full population of MSAP schools in 1999–2000 and 2000–2001, the second and third years of 

program funding in the 57 MSAP projects. Data from the surveys indicate that one in six of the 

MSAP school principals changed during this one-year period. Our case study data illustrate some 

of the difficulties that turnover in key leaders may generate. In District F, for instance, the 

project was directed by a different person each year of the three-year grant, and the district’s 

superintendent changed once as well. Consequences of this turnover included delays in the 

opening of two new magnet schools and in the implementation of magnet themes and 

technological infrastructure. In another district, the MSAP project director commented that one 

magnet school’s advanced technology program had taken at least a year and a half of the threeyear grant period to implement. One difficulty the school had faced was the loss of its original 

principal and the hiring of a new principal who “had no idea of what was going on.”3

 Our case 

study data on the effects of turnover are consistent with the results of several other recent studies 

of the implementation of comprehensive school reform. For example, Berends, Bodilly, and 

Kirby (in press) found that turnover of the principal had a negative effect and continuity of 

leadership had a positive effect on implementation of the New American Schools design, a 

national initiative to develop replicable schoolwide reform programs. 

Tension between MSAP Projects and Other Reform Efforts 

Tensions between MSAP projects and other reform efforts also created challenges for 

some projects and schools. To some extent, our data on the role of MSAP projects in supporting 

systemic reform efforts, reported in Chapter III, cast some light on these tensions. But our 

telephone interviews with project directors and case study data indicate that tensions may have 

occurred even in situations in which MSAP project directors, principals, and teachers viewed 

their programs as aligned with state standards and assessments. 

As one way of exploring the frequency of tensions at the project level, we asked MSAP 

project directors if they experienced tensions between what the magnet project was trying to 

accomplish and what the state and district were asking schools to do in terms of standards and 

reform. Their responses provide a crude indicator of the frequency of tension, but the results, 

shown in Exhibit VI-1 indicate that some projects experienced tensions with various types of 

reforms. 

 

3

 It should be noted, however, that not all staff changes caused delays. Some were attempts to remediate other 

problems that were held responsible for implementation delays. 
 VI. Putting the Results in Context 

 VI-5

Exhibit VI-1. 

Tensions between magnet programs and reform efforts 

0%

10%

20%

30%

40%

50%

60%

70%

Little or none With state only With district only With state &

district

With other 

Tensions Cited by Project Directors

% of MSAP projects

Exhibit reads: About 63 percent of project directors reported little or no tension with other reforms, while about 12 

percent reported tensions with state reforms. 

N = 56 projects 

Source: MSAP Project Director Interview, 2000-2001, Item 6 

As the graph indicates, 63 percent of the project directors (representing 35 projects) 

reported little or no tension. These directors often commented that as their districts had 

developed their applications for MSAP funding, they had aligned their programs with state and 

district standards. Throughout the course of the grant period, for example, the MSAP project in 

District F worked to align the curriculum in magnet schools with state and district frameworks in 

math, language arts, science and social studies, and to use the state and district content and 

performance standards in the magnet programs. 

Of the 12 percent (7 project directors) reporting tension with state reforms, the most 

common source was the emphasis on state test scores. In a typical response, a project director 

noted, “There is an overemphasis on test preparation during the school day, instead of an 

emphasis on good education.” Another said that “The state is emphasizing testing facts and 

magnet programs are more about testing facts and conceptual understanding.” This feeling was 

also noted among the 11 percent (6 project directors) reporting tension from both state and 

district reforms. For example, in District D the state’s emphasis on test scores and the 

implementation of districtwide reforms in literacy and mathematics curriculum impeded the 

project’s plan to have the faculty in all of the MSAP-funded magnet schools produce 

comprehensive curriculum maps representing all of the content areas. 

Of the 5 percent (3 project directors) citing tensions caused by district reforms, one 

project director said that his district administration viewed the magnet schools as “fun-to-be 

places” and did not see the connection between the state standards and the magnet objectives. 

Like several other project directors who felt that the magnet programs were not understood by 

the district, he addressed the problem by making School Board presentations to show the links 
VI. Putting the Results in Context

VI-6

between the magnet program and state and district standards. The remaining 11 percent (6 

project directors) reported tensions from a variety of other sources (for example, between magnet 

and non-magnet schools). 

Our case study and project director interview data suggest that in at least some schools, 

the tension between the content emphasized in state standards and assessments and the content 

emphasized in some magnet programs may have slowed implementation or caused changes in 

key program elements. In addition, this tension may also have had a more direct effect on 

achievement, at least as reflected in the statewide assessment scores that were the main outcome 

measures that MSAP programs used to report achievement outcomes, as well as the measures 

used in our analyses of achievement trends in MSAP schools and non-magnet comparison 

schools (see Chapter V). To the extent that magnet programs emphasized content other than that 

emphasized in statewide assessments, the chance of showing improved outcomes on these tests 

was probably reduced. For instance, a school in District D developed a science theme that was 

dismantled halfway through the grant period when the district required schools with low reading 

scores to devote several hours a day to an intensive literacy program. In another district, a school 

that began as a language immersion magnet was required to add mathematics and science 

elements in order to prepare students to take the state’s standardized tests. 

Time Required for Implementation 

In many ways, magnet programs resemble comprehensive school reform efforts, in that 

they are designed to affect multiple features of the school, including school organization, 

curriculum, and instruction. The literature on comprehensive school reform indicates that the 

implementation of whole school reform can take considerable time—frequently from 5 to 10 

years. (For a review of the literature on implementing comprehensive school reform, see 

Desimone, 2002.) In their quasi-experimental study of a comprehensive school reform model, 

Bloom and his colleagues discuss the time required to detect achievement effects in the schools 

they followed: 

[T]he pattern of test score changes tracked the pattern of program implementation. 

During the first two years of implementation, when schools focused on changing 

their organizational cultures and decision-making processes, there was no 

systematic change in scores. In the third year, as the schools struggled to 

implement instructional changes with the limited guidance afforded by the early 

Accelerated Schools model, test scores dropped somewhat. In the final two years 

of the follow-up period, test scores gradually surpassed the baseline level. 

(Bloom, et al., 2001, p. 75) 

Our case study evidence suggests that three years of MSAP funding may not provide 

sufficient time for a new magnet theme to take hold and become deeply integrated into the life of 

a school. This is especially true if delays in notification and funding reduce the implementation 

time even further. Several project directors alluded to delays in implementing programs—for 

instance, in acquiring equipment and hiring staff—because grants were awarded in late summer 

and early fall. One director commented that her project had not been fully operational during 

1998–1999 because MSAP funding was received so late (six weeks into the school year) that the 

first year had been dedicated to acquiring technology and hiring new staff. New principals, 
 VI. Putting the Results in Context 

 VI-7

recruited through a national search, were not in place until January, and additional staff were 

subsequently hired. Another source of delays was the need for lead time to train staff and design 

curriculum at the beginning of the grant period. Schools implementing some specialized themes 

(e.g., whole-school Montessori, International Baccalaureate) required extensive staff training 

during the first year of the grant. Our survey data indicate that some projects anticipated the need 

for planning time by designating 1998–1999 as a planning year. In 1998–1999, about 15 percent 

of the magnet schools either were not yet open or viewed their programs as still in the planning 

stages. 

Time appears to have been especially critical for schools that chose themes requiring new 

curriculum materials, new types of staff, and new equipment and supplies. For example, magnet 

schools were required to locate the needed materials and equipment, decide among alternatives, 

and put them in place. At times, decisions on materials had to wait for the hiring of appropriate 

staff. For example, the technology specialist in a District B magnet school explained that 

although the school had purchased additional technological resources during the initial years of 

the grant, there had been no technology specialist on staff to help coordinate the installation and 

maintain the equipment. Once she was hired during the second grant year, her first responsibility 

was to get all the computers and other technology hooked up and running. The MSAP project 

director in another district recounted the experience of a first-time magnet school that adopted a 

complicated mathematics and science theme. It took a long time to get the staff on board and 

trained, she said. “They had success eventually in meeting their achievement goals, but they did 

not see the effects until the fourth year” (the year after the grant ended). Project directors in 

several other districts commented on the unexpectedly long time required to install and learn to 

use new technology (e.g., computer animation, fiber optic networks, and computer labs). 

In addition, for many, if not most, magnet schools, the theme and program were worked 

out in fairly broad strokes in the MSAP application; much was left to be worked out during 

implementation. Many MSAP project plans described in grant applications included extensive 

curriculum development efforts during the initial years of the grant and included project 

objectives such as “By the end of the first year of the grant, at least 85 percent of the staff will 

have implemented at least one innovative practice in their classrooms.” While themes like the 

International Baccalaureate, Montessori, and Paideia offer relatively detailed designs, many 

magnet themes are less completely specified in advance. For example, a new magnet school in 

District F has a MicroSociety theme that addresses real life experiences in an academic setting 

and incorporates a job-shadowing program with local businesses. At the time of the site visits in 

the second and third year of the MSAP grants, observers found that the school was still trying to 

integrate the program with other school activities, and to develop a more systematic way of 

recruiting businesses whose jobs were aligned with the magnet theme. Much recent work 

suggests that reforms that are well specified in advance are more easily and quickly implemented 

than less specific reforms. (For a summary of the literature on reform specificity, see Desimone, 

2002.) 

Finally, the pool of organizational supports for schools adopting some kinds of new 

themes is relatively limited. While Magnet Schools of America and other organizations offer 

some support to magnet schools, schools adopting themes in the arts, the environment, and other 

areas may have difficulty locating the required supports. Some MSAP-supported districts that 

had contracted with the same professional evaluator were linked into an informal mutual support 
VI. Putting the Results in Context

VI-8

network that allowed less experienced projects to benefit from the expertise developed by more 

mature ones. 

Tension between Creating a High-Quality School Program and Identifying Themes 

that Will Attract Students 

 The time required for implementation may have been magnified by the fact that some 

schools cycled through themes before settling on one or more that “worked.” Our case study data 

suggest that themes sometimes lose their appeal after a few years, and new themes are needed to 

regenerate families’ interest in a school. For example, one of District B’s MSAP elementary 

schools added a new theme of academic excellence and changed the magnet program focus when 

it was not drawing enough students. The school administrator reported that the frequent change 

in themes created a perception in the community that the school lacked stability even though the 

staff viewed the changes as simply an enhancement of previous themes. At the MSAP middle 

school in this district, the new principal added a career exploration program. His intention was 

not to change the themes that had been in place previously but to enhance the existing programs 

and prepare students for high school and college. If this is true, it suggests that magnet schools 

may need continuing external support to sustain attractive programs. 

Teacher Resistance to Change 

Another explanation for implementation difficulties is teacher resistance. Many project 

directors commented on the importance of faculty “buy-in” to the success of a magnet program. 

Although MSAP grants provide additional resources and excitement to schools, implementing 

new programs requires much of a school’s staff—a willingness to invest extra time in planning 

and reflection, to abandon established practices and try unfamiliar ones, and to collaborate with 

colleagues. Our survey data suggest that most magnet schools are staffed primarily by the same 

individuals who worked in them in previous years, not all of whom are necessarily committed to 

becoming part of a magnet school. Under these conditions, it is not surprising that resistance 

sometimes occurs. At a District C magnet school, for example, the amount of professional 

development required for the teachers to learn the program’s philosophy generated resistance 

among the teaching staff and delayed the implementation of the program. Teachers refused to 

attend the professional development activities and did not implement the themes in their 

classrooms until the following year, after school administrators received assistance from the 

district office and the district’s magnet resource center. In District G, integrated arts programs in 

two secondary schools called for teachers to work in cross-disciplinary teams to develop 

curriculum units that combined arts content and activities with standards-aligned content in other 

academic subjects. A relatively small proportion of the teachers participated in these 

collaborative efforts, although other aspects of the school’s arts and academics program appeared 

successful. 
 VI. Putting the Results in Context 

 VI-9

District Context and Desegregation Outcomes 

A fundamental premise underlying magnet schools, from a managed choice perspective, 

is that if schools implement well designed, distinctive, and effective programs, students from 

across the district, and even other districts in some cases, will be attracted to apply, and that 

prevention, reduction, or elimination of minority group isolation will result. The success of most 

of MSAP’s desegregation-targeted schools in reducing MGI depends on the extent to which the 

magnet program assists the school in attracting and retaining nonminority students.4

 Some of the 

challenges to achieving these objectives are the pressures of demographic change, the 

development of adequate recruitment and parental support strategies, the retention of students 

once they enter the program, and features of the federal program itself. The case studies and 

project interviews provide insight into these challenges and ways in which the MSAP projects 

have attempted to address some of them. 

 

Coping with Demographic Change 

As reported in Chapter IV, most of the desegregation-targeted schools are in districts in 

which the proportion of minority students enrolled increased between 1997–1998 and 2000–

2001 by at least 1 percentage point. While the Department of Education takes account of such 

demographic changes in evaluating the success of MSAP programs, these changes reflect the 

shrinking pool of nonminority students in many of the districts supported by MSAP. Most of the 

schools targeted for desegregation are in districts in which the number of nonminority students 

decreased over the years of the MSAP grant. 

One strategy used by a district to increase the pool of candidates is to actively recruit 

from outside the district. District G, for example, created interdistrict magnets that drew 

nonminority students into high-minority inner city schools from surrounding suburban schools 

where nonminority students predominate. For one of the interdistrict magnets, the district 

succeeded in gaining a substantial number of nonminority applicants and reduced the school’s 

minority isolation. However, this strategy also poses new challenges for districts that use it. In 

the recruitment effort, for example, District G did not have the mailing addresses to send 

information to homes of potential students in other districts. Project recruiters had to arrange 

with administrators of suburban schools to publicize their programs at student assemblies, and 

the suburban districts were not always eager to have their students transfer to another district. 

Importantly, District G also had support from the state in providing transportation for students. It 

is important to note that not all of the magnet programs in District G were interdistrict magnets. 

 

4

 In the case of the small proportion of targeted feeder schools, reducing MGI may depend on attracting minority 

students from MGI feeder schools into schools with a lower percentage of minority students. 

Some of the challenges facing the MSAP projects that may explain the modest impact that 

MSAP magnet programs have had on MGI. These factors include decreasing number of 

nonminority students in many districts, a need for more effective recruitment, the need to 

support parents and retain students, limitations on factors that are used in the selection of 

students, and delays in recruitment efforts related to the timing of grant awards. Our case 

study data suggest that districts are experimenting with strategies to address some of these 
VI. Putting the Results in Context

VI-10

The district officials noted that magnet programs need to be sufficiently developed in order to 

attract suburban students, suggesting that interdistrict recruitment may not be practical for newly 

developed magnet programs. The use of interdistrict magnets may be increasingly necessary for 

districts with high proportions of minority students. Nineteen of the 57 MSAP districts have a 

minority enrollment of 75 percent or higher in the grades served by their programs. 

A second strategy used by a district for coping with the demographic challenge is to 

recruit students from private schools. In their survey responses, project directors from onequarter of the MSAP districts report that private schools (religious and non-sectarian) represent 

“a lot” of competition for public schools in their district. While all of the eight case study project 

directors indicated that private schools represented “some” or “a lot” of competition for magnet 

schools in their district, none indicated that they were successful in recruiting students from 

private schools. In fact, District E, which showed some progress toward desegregation in all 

seven of its targeted magnet schools despite a districtwide decline in minority enrollment 

students, specifically reported, “During the three-year project period, no students attending 

private schools applied to attend any magnet programs included in the grant.” The recruitment 

specialist in District F maintained that recruiting from private schools is becoming more successful 

in that district because parents are becoming increasingly concerned about the high costs for private 

schooling and many see magnet schools as having more resources than private schools. However, 

there was no direct evidence of the number of students in the MSAP magnets who had been 

recruited from private schools. Successfully recruiting students from private schools may involve 

some of the same challenges that District G faced with interdistrict recruiting. Informational 

mailings, for example, may be more difficult if the projects do not have lists of students’ 

addresses, and administrators of private schools are unlikely to be receptive to having magnet 

program representatives visit their schools for purposes of recruiting their students to attend a 

public school. 

Recruitment 

The recruitment efforts of MSAP projects are often characterized by “broadcast” methods 

that may reach out to both minority and nonminority students, rather than targeting one group. 

Nearly one-third of all project directors reported that none of their outreach efforts were focused 

on a targeted group of students. Less than one-fifth of all project directors reported that their 

projects focused all of their outreach on a targeted group of students. 

There is some evidence from the case studies and project director interviews that project 

officials have recognized the need for more targeted recruitment over the course of the project 

and have sought creative methods in attempting to overcome these challenges. Alternate 

strategies for improving recruitment have been offered in some of the case studies and project 

director interviews: 

• In District F, the project director reported that in order to increase the nonminority 

applicant pool, the focus of the recruitment effort had shifted in the second year of the 

project to the east side of town where there is a larger nonminority population. 

• In commenting on the struggle to attract enough nonminority applicants, one project 

director indicated that the project was “reaching out to the desired population in 
 VI. Putting the Results in Context 

 VI-11

places that they would be, regardless of how far or close.” That project had focused 

on visiting day care centers throughout the city to find nonminority parents. 

• Targeting recruitment does not always require going great distances. One project 

director reported that to recruit more nonminority candidates the project was shifting 

its advertising to papers with primarily nonminority readership. 

• The project director in another district reported that, based on advice from a 

consultant from an Equity Assistance Center, the program decided to target 

predominantly white churches to recruit more nonminority students. 

Stable and strong school leadership is another important factor in developing a successful 

recruitment strategy. As one project director noted, the principals who are most active in 

recruiting are the ones who are committed to the program and to decreasing minority isolation. 

The principal can also be instrumental in shaping the identity of a school and its program. In this 

regard, several project directors noted problems in some of their schools with principal turnover. 

Developing a positive and distinctive identity for a school is another important element in 

successful recruitment. Building a strong program, producing positive student outcomes, and 

communicating those successes to a wide public are challenging processes that require effort, 

strategic planning, and time. A reputation capable of attracting students from across town may 

take more time to build than the three years of one MSAP funding cycle. 

Supporting Parents 

The analysis in Chapter IV indicated that parental involvement in school events and 

activities is associated with a school’s progress in meeting its desegregation objective. Securing 

parental involvement may begin by supporting parents throughout the application process and is 

likely to continue after students have been selected to attend the magnet school. District B 

maintains a Web site with information on what each magnet school offers and how students can 

apply for admission. In District A, the Director of Student Assignment, who is in charge of 

magnet schools, informs parents about their options, tries to direct them to schools that have 

programs that best match their children’s interests and needs, and provides them with the name 

of a contact person for the school they choose. Her office then facilitates the final steps of the 

enrollment process by alerting the school contact person to expect the arrival of the parent at the 

school, where the application is completed. In District G, the MSAP project staff help parents 

negotiate the application process and continue to help families adjust to the magnet schools to 

which their children have transferred. In District E, the MSAP project employs monitors at each 

of the elementary magnet schools to provide parents with a sense of security. The monitors check 

each day on children from outside the neighborhood and make themselves available to parents by 

cell phone. 

Retaining Students 

The success of a program depends not only on recruiting students but also on retaining 

them. The fact that MSAP funds cannot be used to support student transportation can be a 

serious issue for the success of a program. In District E, the inability of the district to provide 

student transportation was a major barrier to students and their parents. Due to transportation 
VI. Putting the Results in Context

VI-12

difficulties, some students arrived late for school or did not arrive home on time. As a result, 

according to the project director, many parents withdrew their children from the program to 

attend their neighborhood school. The parents praised the program but found the transportation 

problem too much of an impediment to continue. While all the magnet schools in District E 

showed progress in reducing MGI, the comment by the project director indicates that the extent 

of the progress might have been greater if transportation had not become a problem for the 

district. 

Features of the Federal Program 

Some of the features of the federal MSAP program itself may limit the level of success of 

the desegregation efforts of the local MSAP programs. One feature is related to the duration of 

time available to MSAP projects to implement their programs and recruit students. The 1998 

MSAP grants operated on a three-year period of funding from the 1998–1999 school year to the 

2000–2001 school year. Recruitment and selection of students into a magnet program for a given 

year typically occurs during the fall and spring of the preceding year. This means that 

recruitment of students for the first year of the 1998 MSAP projects needed to occur during the 

preceding school year. However, the project awards for districts were not announced until late 

June 1998 and in many instances not until September 1998. 

The timing of the grant awards did not offer a realistic opportunity for schools to conduct 

recruitment activities until the second year of the project. Consequently we examined the 

proportion of targeted schools making progress in reducing, preventing, or eliminating MGI 

between 1998–1999 and 2000–2001. The analysis showed that 55 percent of targeted schools 

were successful in reducing, preventing, or eliminating MGI over the two-year period. While this 

is close to the 57 percent obtained when using 1997–1998 as the baseline year, two years is a 

short timeframe in which to expect school districts to successfully implement programs and 

recruit the mix of minority and nonminority applicants that will reduce MGI. The percentage of 

schools making progress on their desegregation objectives and the extent of that progress might 

be greater if progress were measured for three school years following the date that MSAP 

supported-schools received their funds. 

A second feature of the federal program concerns the selection process used by local 

MSAP programs. For districts with required desegregation plans, selection factors are 

determined by the court or agency that requires the plan over the MSAP grant period. For 

districts with voluntary plans, beginning with the MSAP grants awarded in 1998, selection 

factors are reviewed by the U.S. Department of Education (ED) periodically, and any use of race 

must be narrowly tailored and approved by the Office for Civil Rights (OCR). OCR requires 

voluntary plan districts to try or seriously consider race-neutral alternatives before resorting to 

race-conscious action. Because the use of race or other factors in student selection is specific to 

each school district's unique situation and is permitted only on a case-by-case basis, ED 

determined that we should not gather or report data on the ways such criteria are applied in 

MSAP districts as a part of the study. 

District officials in some of our case studies suggested that limitations on the factors that 

districts use in selecting students may have played a part in the proportion of schools that did not 

succeed in reducing minority group isolation. In District C, for example, the project director 
 VI. Putting the Results in Context 

 VI-13

contended that it is difficult to meet the desegregation objective when school officials are 

prohibited from taking race into account in making school assignments, even though 

administrators did consider eligibility for free or reduced-price lunches and reading scores 

instead. 

Status of MSAP Projects and Schools after the Grant Period Ended 

One way to assess the impact of the MSAP program is to ask whether the magnet schools 

supported with MSAP funds continue their magnet programs after the end of the grant period. As 

we explained in Chapter I, the 57 projects included in this evaluation received initial funding in 

the 1998–1999 school year, and funding continued through the 2000–2001 school year. 

We contacted the project directors of all 57 projects after the end of the 2000–2001 

school year to obtain information on the status of the projects and schools subsequent to their 

initial grant. In more than half of the 57 projects (31, or 53 percent), project directors reported 

that all of the MSAP schools are still operating, with no important changes in their programs. In 

about one-third (18, or 32 percent) of the projects, the schools are operating but with reductions 

in staff. These reductions generally involve school-level resource teachers or specialists (e.g., 

science specialists in a math-science magnet). 

In more than half of the 57 MSAP projects that received funding from 1998 to 2001, project 

directors report that all MSAP schools were still operating in 2001-2002, the year after 

funding ended. In another third of the projects, the schools were still operating in 2001-2002, 

but with reduced staff. 
VI. Putting the Results in Context

VI-14

Exhibit VI-2. 

2001–2002 Status of magnet schools in 57 MSAP projects funded in 1998 

Exhibit reads: In 53 percent of the MSAP projects funded in 1998, all MSAP-supported magnet schools were still 

operating their magnet programs with full staffing after the grant year, while in 32 percent of the projects, all schools 

were still operating, but with reduced staff. 

N = 57 projects 

Source: MSAP Project Director Interview, 2000-2001, Item 1 

Two projects (4 percent of those funded in 1998) are reportedly operating with other 

reductions (fewer after-school programs and less professional development), and six projects (11 

percent) have one or more schools that no longer offer magnet programs. Among these six 

projects is one in which only one of nine MSAP schools is still a magnet; one project with five of 

eight MSAP schools that are still magnets; and four projects with one or two schools that have 

ceased to be magnets because of declining enrollment, loss of critical staff, or underperformance. 

For the most part, the surviving magnet schools are being supported by district and state 

funds; many of them also receive Title I funds. About one-fourth of the projects (12, or 21 

percent) reported that they had obtained funds from other sources such as the National Science 

Foundation and the Comprehensive School Reform Demonstration Program. One project 

director, whose MSAP schools have been successful in obtaining numerous grants, attributed the 

success to the fact that his schools have well-thought-out programs in place that address student 

academic success. 

One issue complicates the interpretation of our data on the continuation of MSAP schools 

during the first year following MSAP funding: The federal MSAP program funded a new set of 

projects beginning in the 2000–2001 school year. Of the 57 districts funded in 1998, 24 (42 

percent) were awarded MSAP grants again in 2001. Districts receiving another round of MSAP 

funds may have been better able to support existing schools. It appears that 34 schools in 10 

school districts received support in both the 1998 and 2001 grant cycles. 

53%

4%

11%

32%

All operating with no

change

Operating with

reduced staff

Operating with other

reductions

One or more MSAP

schools not operating
 VI. Putting the Results in Context 

 VI-15

To examine the possible role of winning a new MSAP award in the continuation of 

existing MSAP schools, we compared results for districts that won subsequent awards and those 

that did not. The evidence suggests that winning a subsequent grant did not make much 

difference to the immediate survival of magnet programs in MSAP schools. In particular, the 31 

projects in which all MSAP schools continued with no change are about evenly divided between 

those that received 2001 grants (15 projects, or 48 percent) and those that did not (16 projects, or 

52 percent). 

The survival of MSAP schools may also be influenced by one other factor, at least in the 

short run. We learned in our interviews that some MSAP projects had funds remaining at the end 

of the 1998–2001 grant period, and they were provided the opportunity to use these funds in 

subsequent years. These no-cost extensions may have softened the impact of the end of MSAP 

funding. It clearly would be desirable to obtain data on the success of the 1998 cohort of MSAP 

schools in retaining their magnet programs over a longer period of time. 

Other Outcomes of MSAP Projects 

Clearly, one important benefit of the program to districts that received MSAP funds was 

the opportunity to initiate magnet schools, and, as shown above, the magnet programs in these 

schools appear to have continued through at least the first year after project funding. Our case 

study data suggest that the MSAP program also brought other benefits to the participating 

districts. 

Positive Publicity for Schools 

Magnet programs can create public interest and support for schools, through positive 

discussions in the newspapers and on TV and radio about new programs and student projects. 

This is particularly helpful for schools whose public identities have historically been based on 

problems and failings reported in local media. For example, successful publicity strategies in 

District B included a flyer to wrap around the weekly grocery ads in the local newspaper, an 

infomercial describing the magnet schools that ran for six weeks on two local TV stations, and 

commercials on four local radio stations. In another district, magnet schools held “grand 

opening” events that brought parents and newspaper reporters into classrooms. As a result, the 

newspapers published some positive articles about the schools whereas in the past they had 

published negative ones. 

District R and D 

Magnet schools can serve as mechanisms for district “R and D”—trying out new 

curricula or methods that are then refined and implemented in other district schools. For 

Data from the case studies suggest that MSAP projects provided a number of benefits for 

participating districts, including positive publicity for schools, a setting in which to conduct 

district “R and D,” improved administration of the district choice system, an improved 

technology infrastructure, and the capacity to hire specialized staff. 
VI. Putting the Results in Context

VI-16

example, District B’s standards initiative began with magnet schools and led to the development 

of district curriculum that has been linked with state and national standards. In addition, a 

professional development program to help the MSAP teachers understand the effects of poverty 

on education was subsequently extended to all the other schools in the district. In District F, the 

concept of converting a junior high to a middle school was pilot tested by the MSAP project and 

due to its success, the district planned to convert all of its junior high schools to middle schools. 

Similarly, District G’s elementary MSAP schools were enlisted by the district to pilot a complex 

curriculum mapping and alignment project that the district office planned to institute later in 

other schools throughout the city. 

Improved Administration of District Choice Systems 

MSAP resources can help support a district school choice system. In many MSAP 

districts, the MSAP schools are part of a larger system of choice, and MSAP helps to sustain the 

system. For example, in District G, MSAP funds have supported the development of 

management structures for coordinating the recruitment and placement of students in its complex 

school choice system. This has simplified the choice process for families, reduced the demands 

on school personnel for managing applications and selections, and allowed final placement 

decisions to be made sooner than was the case when the process was decentralized. Yu, et al. 

(1997), in their analysis of magnet schools in Cincinnati, St. Louis, and Nashville, report that a 

well-administered choice process (including information on dissemination, advising, and 

recruitment) is an especially important ingredient in securing the participation of the 

disadvantaged families in the choice process. (See also Henig, 1996, who makes the same point 

in a study of the Montgomery County, Maryland, magnet schools.) 

Improved Technology Infrastructure 

MSAP resources, in coordination with other federal, state, and local funds, can help build 

a school or district’s infrastructure (e.g., technology). As reported in Chapter III, MSAP schools 

have more technology than non-MSAP schools—for example, the latest generation of 

computers, software, and theme-related equipment such as a digital photography lab and 

aeronautics equipment. Technology is incorporated into daily instruction rather than being 

treated as a stand-alone activity. Furthermore, the arrival of the technology has usually been 

accompanied by staff development to encourage teachers to learn not only how to use the 

computers, but also how to capitalize on their potential as instructional tools—e.g., through the 

development of Internet research projects, simulation programs in mathematics and science, and 

the use of software that is also commonly used in the workplace. 

Capacity to Hire Specialized Staff 

MSAP resources can also help schools hire specialized staff, who may have lasting 

effects on schools. For example, MSAP provides funds for resource teachers and technology 

specialists who have technical expertise to assist magnet schools. In some MSAP schools, 

resource teachers are specialists in the theme and they act as “extra hands” and facilitate 

curriculum alignment activities and the development of interdisciplinary projects. At one school, 

a publishing resource teacher helped students publish a school newspaper and learn to operate 

broadcasting equipment. Technology specialists helped to coordinate installation and 
 VI. Putting the Results in Context 

 VI-17

maintenance of the school’s new computers and other equipment, acted as system or network 

administrators, taught computer classes, and provided staff development technology sessions. 

Summary 

The case study data collected in eight MSAP projects suggest that, at least in part, the 

modest achievement outcomes we observed may have been due to the fact that, in many schools, 

the intended programs were not fully implemented during the three-year period of MSAP 

funding. Several factors may have influenced the progress schools made in implementing MSAP 

programs: turnover in district and school leadership, tension between MSAP projects and other 

reform efforts, the absence of sufficient time to implement new programs, the need to revise 

themes to attract students, and teacher resistance to changes being implemented. 

Some of the challenges facing the MSAP projects that may explain the modest impact 

that MSAP magnet programs have had on MGI include the decreasing number of nonminority 

students in many districts, a need for more effective recruitment, the need to support parents and 

retain students, limitations on factors that are used in the selection of students, and delays in 

recruitment efforts related to the timing of grant awards. Our case study data suggest that 

districts are experimenting with strategies to address some of these challenges. Additional 

research into these and other challenges is needed to better understand the processes that 

influence the success of programs in meeting their desegregation objectives. 

The case study data indicate that most schools that received MSAP funds during the 

period from 1998 to 2001 continued to operate in the year after the grants ended. In particular, in 

more than half of the 57 MSAP projects that received funding, project directors report that all 

MSAP schools were still operating in 2001–2002, the year after funding ended. In another third 

of the projects, the schools were still operating, but with reduced staff. 

Data from the case studies suggest that MSAP projects provided a number of benefits for 

participating districts, in addition to the opportunity to initiate or strengthen magnet schools. 

These “side benefits” include positive publicity for schools, a setting in which to conduct district 

“R and D,” improved administration of the district choice system, an improved technology 

infrastructure, and the capacity to hire specialized staff. 



 VII. Conclusions 

 VII-1

VII. Conclusions 

In this chapter, we summarize our findings concerning the four main questions addressed 

by the evaluation:1

• To what extent do federally funded magnet projects feature innovative educational 

methods and practices that meet identified student needs and interests? 

• To what extent are federally funded magnet projects promoting systemic, standardsbased reform? 

• To what extent are federally funded magnet projects reducing the incidence or degree 

of minority isolation in their programs? 

• To what extent do federally funded magnet projects strengthen students’ knowledge 

of academic subjects and skills needed for successful careers in the future? 

We consider each of these questions in turn. Following the summary of results, we 

consider the strengths and limitations of our data and analyses, and then turn to implications for 

the operation of magnet programs, as well as for research on magnet schools. 

Innovative Educational Methods and Practices 

Magnet schools are expected to adopt distinctive themes and innovative programs, 

designed to promote a positive school climate and professional community among teachers. 

These conditions, in turn, are expected to lead to effective instructional practices and ultimately 

to improved student achievement. (For a more complete discussion of our conceptualization of 

the features of magnet programs and their potential relationship to achievement, see 

Exhibit III-1.) 

To examine features of magnet schools that we hypothesized might be related to 

achievement outcomes, we drew on data from a survey of principals in all MSAP schools, as 

well as survey and observation data in both MSAP and comparison schools in the eight case 

study sites. As reported in Chapter III, our data suggest that MSAP-supported schools differ from 

non-MSAP schools serving comparable student populations in some key aspects of school 

organization and instruction. 

 

1

 As indicated in Chapter I, we also considered three other questions. The first question concerns descriptive 

characteristics of the schools and programs that make up MSAP projects. That question is addressed in the first 

interim report (2001), and was summarized in Chapter II of this report. The second question concerns the basic 

descriptive characteristics and was addressed in the first interim report (2001). The final question, which 

concerns the role of MSAP in contributing to the development and implementation of magnet programs, was also 

addressed in the first interim report (2001). Survey and interview responses revealed that the MSAP funding had 

enabled schools to invest in a variety of professional development opportunities, technology, and other 

specialized materials that they would not otherwise have been able to afford. Many of the districts used the 

grants to fund the initial expensive stages of setting up new programs, but expected to be able to maintain the 

programs from other funds once they were up and running. 
VII. Conclusions

VII-2

• Magnet themes. According to data from the principal survey, MSAP schools have 

adopted a wide variety of themes. More than one-third of MSAP schools included 

technology among their themes, and more than a quarter of MSAP schools included a 

science theme. Arts, communication, and mathematics were also common themes. 

• Adoption of comprehensive school reform models. More than half of the MSAP 

schools have adopted comprehensive school reform models, such as Success for All. 

This is a substantially higher proportion than is observed among the full national 

population of schools. It is also higher than among comparable schools in the MSAP 

districts. 

• School climate and community. Principal data indicate that MSAP schools differ 

from comparable schools in the same district in some organizational and instructional 

features. For example, MSAP schools on average have a somewhat more positive 

sense of professional community than comparable schools. Magnet schools also have 

a somewhat more positive student climate, although there is considerable variation. 

• Instructional practices. Teacher survey data indicate that MSAP schools make 

somewhat more use of technology in instruction than do comparable schools, and 

place more emphasis on instructional methods designed to elicit higher-order thinking 

skills, such as open-ended projects and presentations. 

We surveyed MSAP principals in both 1999–2000 (the second year of program 

implementation) and 2000–2001 (the third year of implementation), making it possible to 

examine the extent to which changes in school climate and instructional practice occurred as the 

magnet schools had more time to implement their programs. We found some evidence of 

improvement. In particular, principals’ responses indicated improvement in professional 

community and parent involvement between the second and third years of implementation. Other 

elements, including the level of challenge as reflected in the number of academic requirements 

for students, showed little improvement over the second and third year of project 

implementation. 

Magnet Schools and Systemic, Standards-Based Reform 

While MSAP schools were intended to adopt innovative themes and practices, they were 

also intended to ensure that these programs were aligned with state and district standards and 

assessments. We hypothesized that magnet schools would be more likely to flourish if their 

themes and programs were aligned with state and district standards and assessments, but we also 

anticipated that innovative magnet themes might at times conflict with the emphases in many 

state and district assessments. 

To examine the role of MSAP projects and schools in supporting systemic, standardsbased reform, we drew on data from surveys administered to the full population of MSAP 

principals, as well as data collected from the eight in-depth case study sites. 

As reported in Chapter III, we found a multifaceted relationship between magnet school 

programs and state and district standards. In their responses to the MSAP surveys, principals 

reported a high degree of familiarity with standards and assessments, and they indicated that the 
 VII. Conclusions 

 VII-3

content of state standards and assessments matched the goals of their magnet programs. Our case 

study data provide a somewhat more complex picture. In particular, while the case study data 

support the conclusion that magnet themes are generally consistent with the content emphasized 

in state standards, staff in some MSAP schools indicated that they feel pressured to learn how to 

teach a new theme or curriculum while simultaneously being mindful of the state content 

standards and assessments. There is some evidence that some MSAP schools altered their initial 

plans to bring the curriculum more in line with standards and assessments, or reduced their 

emphasis on novel programs to increase the time for work more directly related to state standards 

and assessments. 

Minority Group Isolation 

One of the major objectives of the MSAP program is to prevent, eliminate, or reduce 

minority student isolation in MSAP schools. Minority group isolation (MGI) refers to schools in 

which minority group children constitute more than 50 percent of school enrollment. As reported 

in Chapter IV, we found that most of the 294 MSAP schools targeted for desegregation sought to 

reduce minority group isolation: 

• Less than one-tenth (8 percent) of targeted schools aimed to prevent MGI by keeping 

the school’s minority enrollment from exceeding 50 percent. 

• Less than one-sixth (16 percent) aimed to eliminate MGI by reducing their minority 

enrollment to 50 percent or less. 

• More than three-quarters (77 percent) of targeted schools aimed to reduce, rather than

eliminate, MGI. 

The small proportion of schools seeking to prevent or eliminate MGI is understandable in 

that three-quarters of the targeted schools are in districts in which minority students constitute 

more than 50 percent of public school enrollment, and two-thirds of the targeted schools are in 

districts with over 60 percent minority student enrollment in public schools. 

To evaluate the success of MSAP schools in making progress toward their desegregation 

objective, we examined minority student enrollments from 1997–1998 (the year prior to the 

initiation of the three-year MSAP programs in the 57 MSAP districts), through 2000–2001 (the 

final year of implementation) for all schools targeted for desegregation by MSAP programs. The 

analysis indicated a modest impact of the MSAP-supported schools on preventing, eliminating, 

or reducing MGI at desegregation-targeted schools, with some variation by project, program, and 

school features: 

• Adjusting for districtwide demographic trends, 57 percent of the desegregationtargeted schools succeeded in preventing, eliminating, or reducing minority group 

isolation, while 43 percent did not succeed.

• The proportion of targeted schools in districts with voluntary desegregation plans that 

prevented, eliminated, or reduced MGI was slightly higher than among districts with 

desegregation plans required by a court or other agency (60 percent vs. 53 percent).
VII. Conclusions

VII-4

• A larger proportion of elementary schools prevented, eliminated, or reduced MGI 

compared with middle schools or high schools (60 percent vs. 54 percent and 48 

percent, respectively).

• A larger proportion of whole school programs (offered to all students in the school) 

prevented, eliminated, or reduced MGI compared to programs-within-schools 

(offered to only some of the students attending a school) (59 percent vs. 49 percent).

Our analysis also examined the amount of progress desegregation-targeted schools made 

toward reducing MGI. A small proportion of targeted schools prevented or eliminated minority 

group isolation, while most of the successful schools reduced minority group isolation. 

• One in twenty (5 percent) of the targeted schools prevented or eliminated MGI. 

• About one in six (17 percent) of the targeted schools experienced a reduction of 5 or 

more percentage points in MGI relative to the district. About one-quarter (28 percent) 

experienced a reduction of 1 to 5 percentage points, and one in fourteen (7 percent) 

experienced a reduction that was less than 1 percentage point. 

• In the remaining 43 percent of schools that did not make progress, MGI increased or 

remained constant during this period. 

The preceding analysis addresses the question of what progress MSAP-targeted schools 

made in meeting their desegregation objectives. Hierarchical linear modeling was used to 

analyze factors that influenced the ability of targeted elementary schools to reduce minority 

isolation. The model identified several school features that influenced changes in minority 

isolation, but explained only a modest part of the variation across districts and schools in 

reducing MGI. Nevertheless, some of the results of this analysis indicate: 

• Schools are more likely to experience decreasing minority isolation when the school 

has a racially and ethnically mixed group of minority students. 

• Schools are more likely to experience decreasing minority isolation when parents are 

involved in school events and activities. 

• Schools with larger numbers of students per teacher are more likely than those with 

lower student-to-teacher ratios to experience increases in minority group isolation. 

Some of the challenges facing the MSAP projects that may explain the modest impact 

that MSAP magnet programs have had on MGI include such factors as the decreasing number of 

nonminority students in many districts, a need for more effective recruitment, the need to support 

parents and retain students, limitations on factors that are used in the selection of students, and 

delays in recruitment efforts related to the timing of grant awards. Our case study data suggest 

that districts are experimenting with strategies to address some of these challenges. Additional 

research into these and other challenges is needed to better understand the processes that 

influence the success of programs in meeting their desegregation objectives. 
 VII. Conclusions 

 VII-5

Student Achievement Objectives and Outcomes 

MSAP schools are expected to strengthen student knowledge and skills. As reported in 

Chapter V, we examined student achievement in MSAP schools in two ways. First, we analyzed 

the progress schools made toward meeting the achievement goals set for them in their district’s 

MSAP application. Then we examined differences in achievement trends between MSAP 

schools and a set of matched comparison schools, using achievement data from statewide testing 

programs. 

Progress toward Achievement Objectives Set by MSAP Projects 

MSAP projects were required to establish objectives for student achievement, which 

translated into a large number of school-level achievement goals. MSAP projects set a wide 

variety of achievement objectives, but most of them related to student performance on 

standardized tests in English language arts and mathematics. For each grant year, usable schoollevel data were available for about one-third of these objectives. The most common reason given 

for the lack of outcome data was that the state had revised or discontinued the assessments upon 

which the projects had based their objectives. 

Overall, about half of the schools for which outcome data were available met a 

substantial proportion (half or more) of their English language arts and mathematics achievement 

goals. Schools were more successful in meeting goals set for the first year of their MSAP grants 

than they were in meeting second- and third-year goals. Elementary schools showed smaller 

declines in goal attainment than middle and high schools. 

When the standard of success is making progress toward meeting achievement goals 

rather than meeting them outright, the success rates increases. Overall, about 70 percent of the 

schools made progress toward meeting at least half of their first year goals in both English 

language arts and mathematics, and 50 percent and 60 percent, respectively, made progress 

toward meeting at least half of their third year goals. 

Although the MSAP funded almost 300 magnet schools, between one-half and one-third 

of the schools lacked data to determine their success in meeting their achievement goals in 

particular years. Principal reasons for missing data included late implementation of magnet 

school programs, changes in state tests that made it impossible to track goals over the entire 

grant period, and outcome data reported by grantees that did not align with the objectives as 

originally stated. This finding has implications for the utility of requiring federal grantees to set 

and track achievement objectives locally, a topic we consider further in the final section of this 

chapter. 

Achievement Trends Using Statewide Achievement Data 

In addition to tracking MSAP schools’ success in meeting locally imposed achievement 

goals and locally analyzed outcome data, AIR conducted a multilevel regression study of trends 

in student performance on state standardized reading and mathematics tests between 1997–1998 
VII. Conclusions

VII-6

and 2000–2001. The gains in 135 MSAP-supported elementary schools were compared with 

those in 1,350 similar schools without MSAP funding. 

 

As reported in Chapter V, the results indicate that when changes in demographic 

composition were taken into account, MSAP schools’ gains were not significantly different on 

average from those of their comparison schools. 

Although MSAP schools did not outperform their matched comparison schools overall, 

we found considerable variation in gains among MSAP schools. A companion study explored the 

association between achievement gains and magnet school program characteristics reported by 

the principals of the MSAP-supported schools. As we hypothesized in our conceptual framework 

(Exhibit III-1), we found that a number of school characteristics are associated with achievement 

gains: 

• Greater achievement gains in both reading and mathematics ARE associated with 

reports of strong professional community among teachers (e.g., staff sharing a 

common vision, coordinating instruction, and seeking new ideas). 

• Greater gains in reading ARE also associated with professional development related 

to standards-based reform. 

• Greater gains in mathematics ARE associated with greater influence of state 

curriculum frameworks and assessments on curriculum and instruction decisions 

made in the school, the perception that state frameworks and assessments ARE 

aligned with the goals of the school’s program, a positive school climate, and magnet 

programs that were already operating prior to the beginning of the 1998 MSAP 

funding period. 

Although the overall achievement results seem modest, it is important to understand the 

conditions under which one might expect to see dramatic changes in MSAP achievement relative 

to that of other students. First, the improved instructional program would need to be adequately 

implemented and to have functioned long enough for teachers in a school to change their 

methods and for students to respond to the changes with increased performance scores. Second, 

the innovative program would actually have to work. There is, of course, no guarantee that the 

range of innovative programs adopted by MSAP schools will work in all situations. Third, the 

degree of improvement fostered by the magnet school’s program would have to be greater than 

improvements fostered by programs operating in non-MSAP schools. However, in this era of 

high accountability, it is likely that non-magnet schools will also be implementing promising 

programs as well, using non-MSAP funding sources. The first condition alone is problematic for 

new MSAP-supported schools that are given a maximum of three years to design programs, 

acquire materials, train teachers, and have teachers use new methods well and consistently 

enough to affect student performance on an annually administered achievement test. Due to late 

notification or funding, the use of a planning year, or the opening of a school during the second 

or third year of the grant, some schools had substantially less than three years in which to 

produce measurable changes in achievement. Thus, it may not be surprising that the achievement 

gains experienced by the MSAP-supported schools did not differ from those of comparison 

schools. 
 VII. Conclusions 

 VII-7

Strengths and Limitations of the Study 

The results we have obtained must be understood in terms of the strengths and 

weaknesses of the evaluation design. 

One strength of the evaluation design is the availability of data on the full population of 

projects and schools. As outlined in Chapter I, we conducted telephone interviews with MSAP 

project directors during the second and third years of implementation, and in the first year 

following implementation, achieving a 100 percent response rate each year. We conducted 

MSAP principal surveys during the second and third years of implementation, achieving more 

than a 90 percent response rate each year. These survey data for the full population are 

accompanied by data on school-level demographics and achievement, drawn from national 

databases and MSAP annual project reports. 

Another strength of the evaluation is that we also collected detailed case study and 

teacher survey data in a sample of MSAP and matched comparison schools in eight in-depth case 

study districts. This information allowed us to contextualize the survey findings from the full 

population of projects and schools. 

The evaluation data have two main limitations, however. First, our results on school 

practices in the full population of magnet schools are based entirely on principal self-reported 

data. We conducted analyses to compare principal and teacher reports for schools in the case 

study districts, and these data indicate that conclusions based on principal and teacher reports are 

reasonably similar for those questions that we asked of both sets of respondents. Nonetheless, it 

would be preferable to have data from teachers for a larger sample of MSAP schools. 

Second, our data on achievement are restricted to aggregate school data. A more 

powerful study design for gauging the impact of magnet schools on student achievement would 

use linked-longitudinal student-level data, which would allow analyses of student growth in 

magnet and comparison schools that took account of prior achievement and other student 

background characteristics. We had initially intended to conduct such analyses, and selected the 

in-depth case study districts to make such analyses possible. But during the period of the study, 

the Department of Education placed a moratorium on the collection of individual student-level 

data until issues pertaining to the Family Educational Records and Privacy Act (FERPA) could 

be resolved. Consequently, we used the next-best alternative: school-level average scores from 

many schools in the states in which MSAP schools were located. Given the increasing emphasis 

on providing evidence of the achievement effects of school interventions—outcomes that can be 

best measured with individual level data—it would be useful to examine potential ways in which 

individual-level data might be made available for evaluations without compromising important 

privacy concerns. 

Implications 

Overall, our results indicate that MSAP projects and schools achieved mixed results over 

the three years that are the focus of our evaluation. In particular, most MSAP schools appear to 

have adopted innovative themes, and there is some evidence, based on both survey and case 

study data, that MSAP schools on average were able to establish a more positive school climate 
VII. Conclusions

VII-8

and level of professional community than other schools serving similar students in the same 

districts. In addition, our survey and case study data indicate that MSAP schools made efforts to 

align their programs with state and district standards and assessments. 

At the same time, MSAP schools made only modest progress in preventing, reducing, or 

eliminating minority student isolation and improving student achievement. There is some 

indication, however, that MSAP schools with more positive program features (e.g., school 

climate) outperformed comparison schools in student achievement. 

These results suggest a number of approaches that might be taken to enhance the 

prospects that MSAP schools will produce positive desegregation and achievement outcomes. 

• Narrowing the focus. It may be useful to permit districts to prioritize the goals or 

purposes of MSAP projects to achieve one or two focused outcomes rather than 

attempting to pursue multiple, sometimes competing, goals with limited resources. 

Case study data suggest that projects sometimes had difficulty balancing the four 

goals required of them under MSAP’s authorization under IASA. As reauthorized 

under NCLB, MSAP projects are now expected to focus on six goals, potentially 

making it even more difficult to achieve these goals. Narrowing the focus—and 

evaluating the grantees on this narrower focus—might improve the chances for 

positive results. 

• Reexamining the definition of minority group isolation. Federal regulations 

currently define minority group isolated schools as those in which more than 50 

percent of the students are minority group members. In light of the high proportion of 

minority students in urban school districts, federal policymakers might wish to reexamine the meaning and utility of equating minority group isolation with a single 

fixed percentage. Given the high percentages of minorities in the large urban districts 

that MSAP typically serves, it was not surprising to find that the desegregation 

objective of targeted schools was overwhelmingly to reduce, rather than eliminate or 

prevent, MGI. As the proportion of minorities in schools generally continues to rise, 

there would seem to be a diminishing opportunity for schools to prevent or eliminate 

MGI as it is currently defined without adversely impacting other schools in the 

district. Policymakers might wish to examine more broadly the meaning of minority 

group isolation in an increasingly diverse population. 

• Awarding MSAP grants in a more timely manner. The timing of a grant award is 

critical to the first year implementation of recruitment efforts and magnet programs. 

Inability of districts to begin recruitment efforts and delays in first year programming 

are particularly likely when funds are not received until June or later of the year the 

program is expected to begin operation. If funds were awarded by March, or even 

earlier, of the calendar year the project is to begin, districts may be able to implement 

more effective recruitment efforts for the first year of magnet programs and schools 

would have more time to secure materials, training, and personnel for the first year of 

the program. 

• Extending the period of funding. The MSAP program might provide more than 

three years of funding. Three years may not be sufficient to plan, develop, implement 

curriculum and expect to see change in enrollment and student performance around a 
 VII. Conclusions 

 VII-9

new theme. It may take several years for magnet school programs to build a strong 

reputation before they can attract students from outside the immediate neighborhood. 

Whether or not the period of funding is extended, it would be helpful to continue to 

examine school outcomes for a period longer than three years. 

• Improving the use of annual performance benchmarks. Districts receiving MSAP 

funds are required to set annual performance benchmarks for improvements in 

minority student isolation and student achievement, and to report on their success in 

meeting these benchmarks. The benchmarks districts set for their schools varied 

considerably in ambition and plausibility. If districts are required to set benchmarks, 

additional technical assistance may be required to ensure that the benchmarks are 

meaningful and that outcomes are monitored consistently. 

As we have indicated, the data collection for this evaluation was conducted prior to the 

enactment of No Child Left Behind (NCLB). Our results do, however, suggest that the 

provisions of NCLB may have special implications for magnet schools. 

• Providing assistance in the use of disaggregated achievement data. By 2005–

2006, all states must conduct annual testing in grades 3–8, and at least one grade from 

10–12. In addition, achievement scores must be reported disaggregated by ethnicity 

and other subgroups. The availability of these data will enhance the capacity to 

examine achievement outcomes, and, in particular, to determine whether magnet 

schools are effective in closing the achievement gap between minority and 

nonminority students. As such data are increasingly available, it may be useful to 

provide technical assistance to MSAP grantees to encourage appropriate uses of 

disaggregated achievement data in evaluating magnet schools. 

• Supporting district choice systems. Under the provisions of NCLB, districts are 

expected to offer a choice of schools to students enrolled in schools that fail to meet 

adequate yearly progress standards for two consecutive years. Magnet schools may be 

particularly attractive options for families with students in failing schools, and thus 

MSAP projects may have particular strengths in assisting districts to build 

coordinated district choice systems. 

• Providing support for magnet schools that fail to meet adequate yearly progress 

requirements. Under NCLB, schools that fail to make adequate yearly progress for 

two consecutive years are expected to revise their plan for the use of Title I funds and 

engage in other interventions. Magnet schools, confronting a failure to meet adequate 

yearly progress standards may face an additional challenge of maintaining the 

continuity and integrity of the school’s distinctive mission or theme while 

incorporating changes in curriculum and instruction to improve test scores. 



 Glossary 

 Glossary-1

Glossary 

Achievement objective—a stated goal for students to attain a specified level of performance in a 

particular domain of content knowledge, skill, or behavior within a designated time frame. 

MSAP projects must include such objectives among their performance indicators. Example: By 

the end of the school year, all students in grade 4 will score in the 70th percentile or higher on 

the state reading assessment. 

Combined-level school—a school serving a grade range that crosses the common designations 

of elementary, middle, or high school. In this study, the category includes schools with grades 

K–12 and grades 4–12. 

Core content areas—the four principal academic subjects: language arts, mathematics, science, 

and social studies (including history and geography) 

Desegregation objectives—the statute-defined purposes of reducing, eliminating, or preventing 

minority group isolation in schools that underlie MSAP-funded programs 

Desegregation plan—a document that establishes a method for reassigning children or faculty in 

order to remedy the illegal separation of minority group children or faculty in the schools of an 

LEA, or a plan for the reduction, elimination, or prevention of minority group isolation in one or 

more of the schools of an LEA 

Elementary school—a school that most commonly serves grades kindergarten through grade 5. 

In this study, the category includes schools serving grades as high as 8 if they also serve grades 

lower than 4 

Eliminate minority isolation—a desegregation objective for minority-isolated schools that aims 

to reduce minority enrollments to below 50 percent of total enrollment (that is, for the school to 

cease being minority isolated) 

Feeder school—for this evaluation, a school from which students are drawn to attend a magnet 

school; that is, a school that “loses” students to a magnet school that serves the same grade level. 

(Note that this differs from the more common definition of feeder school—a school from which 

students are drawn for the next level of schooling; for example, the K–5 elementary school 

which students attend before moving on to grade 6 in a middle school.) 

Grantee (district)—a school district (or consortium of districts) that received MSAP funding in 

FY 1998 

High school—a school that most commonly serves grades 9–12. In this study, all high schools in 

this category either have grades 9–12 or are adding those grades incrementally 

Innovative practices—instructional and organizational approaches that have been examined in 

the scholarly literature on high-performing schools, and for which some evidence has 
Glossary 

Glossary-2

accumulated on the relationship between the approaches and student achievement; the specific 

practices that MSAP projects consider to be innovative 

Local Education Agency (LEA)—a public school district or education center serving students 

in some or all of grades K–12. MSAP grantees include both individual LEAs and consortia of 

LEAs. 

Magnet program—the curriculum, instructional strategies, and activities that support the theme 

or focus of the magnet school and set it apart from other schools. A magnet school can have one 

or more magnet programs. 

Magnet project—a district-sponsored effort that promotes desegregation by offering one or 

more individual magnet schools or programs within the district. 

Magnet school—a public elementary or secondary school or public elementary or secondary 

education center that offers a special curriculum capable of attracting substantial numbers of 

students of different racial backgrounds. 

Magnet Schools Assistance Program (MSAP)—a federal program that provides financial 

assistance for school districts or local education agencies to develop or expand magnet school 

programs designed to promote the reduction, elimination, or prevention of minority group 

isolation and quality instruction. During the evaluation study, MSAP was authorized by Title V 

of the Improving America’s Schools Act. MSAP has supported magnet programs with grants to 

171 school districts since FY 1985. 

Middle school—a school that most commonly serves grades 6 through 8. In this study, the 

category includes schools with grades no lower than 4 and no higher than 9. 

Minority group isolation (MGI)—a condition in which minority group children constitute more 

than 50 percent of the enrollment of the school, as defined in 34 CFR 280.4(b). 

Minority group students—includes students who are black, Hispanic, Asian or Pacific Islander, 

or American Indian or Alaskan Native, as defined in the MSAP regulations, 34 CFR 280.4(b). 

MSAP grant—federal financial assistance for magnet schools awarded through a competitive 

process under the Magnet Schools Assistance Program. Currently grants are awarded for threeyear periods, with second- and third-year funding contingent on a review of progress. This 

evaluation is based on grants awarded for the period 1998–2001. 

MSAP-supported school—a magnet school that receives MSAP funds for special staff, 

equipment, materials, professional development, and other assistance to facilitate the 

implementation of one or more magnet programs.

Nonminority group—includes students who are white (not Hispanic). 
 Glossary 

 Glossary-3

Performance indicator—a measure of how well a program is meeting its goals or objectives. 

Used in the Government Performance and Results Act (GPRA) to establish a system for all 

federal programs (including the MSAP) to set goals, measure performance, and report results. 

Performance standard—a description of what students must demonstrate, and how well, to be 

considered advanced, proficient, or partially proficient in academic areas 

Prevent minority isolation—a desegregation objective for schools that are in danger of 

becoming minority-isolated that aims to keep minority enrollments from rising to above 50 

percent of enrollment during the three-year project period. These schools’ minority enrollment is 

expected to remain below 50 percent with the magnet program. 

Professional development—special training for teachers and other educators that aims to extend 

their skills and knowledge, improve classroom instruction, and foster increased student 

achievement 

Program-within-a-school (PWS)—a magnet program that is offered to some but not all of the 

students in a school (e.g., an aeronautics program for 200 students in a 1,000-student high 

school) 

Reduce minority isolation—a desegregation objective for minority-isolated schools that aims to 

reduce the percentage of minority students in the school 

Required desegregation plan—a plan specifying procedures for meeting the legal requirements 

of a court, state government, or other government agency that a district desegregate 

School climate—the environment in a school that reflects the attitudes and behaviors of both its 

staff and students 

Special curriculum—a course of study embracing subject matters and teaching methods that are 

not generally available to all students of the same age or grade level in the same local education 

agency (LEA) or consortium of LEAs, as the students to whom the special curriculum is offered 

in the magnet schools. This term does not include a course of study or a part of a course of study 

(1) that is designed solely to provide basic educational services to students with disabilities or to 

students of limited English-speaking ability; (2) in which any student is unable to participate 

because of his or her limited English-speaking ability; (3) in which any student is unable to 

participate because of his or her limited financial resources; or (4) that fails to allow for a 

participating student to meet the requirements of elementary or secondary education in the same 

period as other students enrolled in the LEA’s schools. 

Standards—content standards are statements of what should be included in the curriculum in 

particular subjects at each grade level; performance standards are statements of what students 

should know and be able to do. 

Statutory goals [of MSAP]—the desegregation goals specified in the MSAP statute: to reduce, 

eliminate, or prevent minority group isolation in schools 
Glossary 

Glossary-4

Systemic reform—efforts to create a coherent, coordinated set of state and district policies to 

guide schools and teachers in improving student achievement 

Targeted school—the school (magnet or feeder) in which minority group isolation is to be 

reduced, eliminated, or prevented as a result of the funded magnet program 

Targets—specific minority enrollment goals for schools, usually stated in terms of achieving a 

specific level of minority enrollment (e.g., 40 percent) or one falling within a range of values 

(e.g., 35–45 percent) 

Technical assistance—consultation, training, or other help provided to individuals involved in 

program implementation. For MSAP projects, this assistance generally comes from the U.S. 

Department of Education, state agencies, or national and state organizations such as Magnet 

Schools of America. 

Theme—the focus of a school’s magnet program that is integrated into some or all of the 

curriculum. Examples include science and technology, mass communications, language 

immersion, and global studies. 

Voluntary desegregation plan—a documented strategy that a district is voluntarily 

implementing to reduce, eliminate, or prevent isolation of minority students in its schools. For 

districts operating under voluntary desegregation plans to receive MSAP funds, the Secretary of 

Education must approve the plans as adequate under Title VI, Civil Rights Act of 1964. 

Whole school magnet—a magnet program that is offered to every student in a school 
 References 

 References-1

References 

Adcock, E.P., and Phillips, G.W. Accountability Evaluation of Magnet School Programs: A 

Value-Added Model Approach. Paper presented at the annual meeting of the American 

Educational Research Association. New Orleans, La.: April 2000. 

American Association of University Women and Wellesley College Center for Research on 

Women. How Schools Short Change Girls: A Study of Major Findings on Girls and 

Education. Washington, D.C.: AAUW Educational Foundation, 1992. 

Archbald, D. “Magnet Schools, Voluntary Desegregation, and Public Choice Theory: Limits and 

Possibilities in a Big City School System.” Unpublished doctoral dissertation, University of 

Wisconsin, Madison, 1998. 

Archbald, D. “A Longitudinal Cohort Analysis of Achievement Among Elementary-Magnet 

Students, Neighborhood-School Students, and Transfer Students.” Journal of Research and 

Development in Education, 28 (1995): 161–168. 

Berends, M., Bodilly, S., and Kirby, S. “District and School Leadership for Whole-School 

Reform: The Experience of New American Schools.” In Leadership Lessons from 

Comprehensive School Reforms, edited by J. Murphy and A. Datnow. Thousand Oaks, Calif.: 

Corwin Press, in press. 

Blank, R. “Analyzing Education Effects of Magnet Schools Using Local District Data.” 

Sociological Practice Review, 1 (1990): 40–41. 

Blank, R.K. “Educational Effects of Magnet High Schools.” In Choice and Control in American 

Education, Vol. 2: The Practice of Choice, Decentralization and School Restructuring, edited 

by W. Clune and J. Witte. Bristol, Pa.: Falmer Press, 1990. 

Blank, R., Dentler, R., Baltzell, C., and Chabotar, K. (1983). Survey of Magnet Schools: 

Analyzing a Model for Quality Integrated Education (prepared by James H. Lowry and 

Associates and Abt Associates). Washington, D.C.: U.S. Department of Education, 1983. 

Bloom, H.S., Ham, S., Melton, L., and O’Brien, J, with Doolittle, F.C. and Kagehiro, S. 

Evaluating the Accelerated Schools Approach: A Look at Early Implementation and Impacts 

on Student Achievement in Eight Elementary Schools. New York: Manpower Research 

Development Corporation, 2001. 

Brophy, J.E. and Good, T. “Teacher Behavior and Student Achievement.” In Handbook of 

Research on Teaching (3rd ed.), edited by M. Wittrock. New York: Macmillan, 1986. 

Bryk, A., Lee, V., and Holland, P. Catholic Schools and the Common Good. Cambridge, Mass.: 

Harvard University, 1993. 
References

References-2

Carpenter, T.P., Fennema, E,, and Franke, M.L. “Cognitively Guided Instruction: A Knowledge 

Base for Reform in Primary Mathematics Instruction.” Elementary School Journal, 97 

(1996): 3–20. 

Cohen, D.K., and Hill, H.C. State Policy and Classroom Performance Mathematics Reform in 

California. Washington, D.C.: U.S. Department of Education, Office of Educational 

Research and Improvement, Educational Resources Information Center, 1998. 

Coleman, J.S., and Hoffer, T. Public and Private High Schools: The Impact of the Community. 

New York: Basic Books, 1987. 

Corcoran, T.B., Wang, A., and Foley, E. What Kinds of Professional Development Build 

Capacity Better? An Analysis of the Impact of Professional Development on Teaching 

Practice in the School District of Philadelphia. Paper presented at the annual meeting of the 

American Educational Research Association, Consortium for Policy Research in Education 

University of Pennsylvania, Philadelphia, Pa., 1999. 

Crain, R.L., Allen, A., Thaler, R., Sullivan, D., Zellman, G., Little, J.W. and Quigley, D.D. 

(Eds.). The Effects of Academic Career Magnet Education on High Schools and Their 

Graduates. Berkeley, Calif.: University of California, National Center for Research in 

Vocational Education, 1999. 

Desimone, L. “How can comprehensive school reform models be successfully implemented?” 

Review of Educational Research, 72(3), 2002: 433–479. 

ERIC Clearinghouse on Educational Management. “Trends and Issues: School Choice” 2002, 

< http://eric.uoregon.edu/trends_issues/choice> (August 2003), Introduction. 

Gamoran, A. “Student Achievement in Public Magnet, Public Comprehensive, and Private City 

High Schools.” Educational Evaluation and Policy Analysis, 18(1), 1996: 1–18. 

Gamoran, A. Do Magnet Schools Boost Achievement? Educational Leadership 54, 2 (October 

1996). 

Gamoran, A., Porter, A.C., Smithson, J., and White, P.A. “Upgrading High School Mathematics 

Instruction: Improving Learning Opportunities for Low-Achieving, Low-Income Youth.” 

Educational Evaluation and Policy Analysis, 19(4), 1997: 325–328. 

Goldhaber, D.D. “School Choice: An Examination of the Empirical Evidence on Achievement, 

Parental Decision-making, and Equity.” Educational Researcher, 28(9), 1999: 16–25. 

Haynes, N.M. “Lessons Learned.” Journal of Education for Students Placed at Risk, 3(1), 1998: 

87–1999. 
 References 

 References-3

Heid, C., and Webber, A. School-Level Implementation of Standards-Based Reform: Findings 

from the Follow-up Survey on Education Reform. Washington, D.C.: US Department of 

Education, Planning and Evaluation Service, 1999. 

Hill, P.T., Foster, G.E., and Gendler, T. High Schools with Character. Santa Monica, CA: 

RAND, 1990. 

Henig, J.R. “The Local Dynamics of School Choice: Ethnic Preferences and Institutional 

Responses.” In Who Chooses? Who Loses? Culture, Institutions, and the Unequal Effects of 

School Choice, edited by B. Fuller, R.F. Elmore, and G. Orfield. New York: Teachers 

College Press, 1996. 

Kennedy, M.M. “Education Reform and Subject Matter Knowledge.” Journal of Research in 

Science Teaching, 35(3), 1998: 249–263. 

Knapp, M.S., Shields, P.M., and Turnbull, B.J. Academic Challenge for the Children of Poverty. 

Washington, D.C.: U.S. Department of Education, 1992. 

Lee, V.E., and Smith, J.B. Social Support and Achievement for Young Adolescents in Chicago: 

The Role of School Academic Press. American Educational Research Journal, 36(4), 1999: 

907–945. 

Lee, V.E., Chen, X., and Smerdon, B.A. “Debunking the Myths: Exploring Common 

Explanations for Gender Differences in High School Science Achievement.” University of 

Michigan, Ann Arbor, Mich., unpublished. 

Magnet Schools of America. “Bibliography of Magnet School Education,” 

<http://www.magnet.edu/bibliography.html> (August 2003). 

Marks, H.M., Secada, W.G., and Doane, K.B. Social Support for Achievement: Building 

Intellectual Culture in Restructuring Schools. Washington, D.C.: U.S. Department of 

Education, Office of Educational Research and Improvement, Educational Resources 

Information Center, 1996. 

National Reading Panel. Report of the National Reading Panel Teaching Children to Read: An 

Evidence-Based Assessment of the Scientific Research Literature on Reading and Its 

Implications for Reading Instruction. Bethesda, Md.: National Institutes of Health, 2000. 

Newmann, F.M. and Associates. Authentic Assessment: Restructuring Schools for Intellectual 

Quality. San Francisco: Jossey-Bass, Inc., 1996. 

Newmann, F.M. and Wehlage, G.G. Successful School Restructuring: A Report to the Public and 

Educators by the Center on Organization and Restructuring of Schools. Madison, Wisc.: 

Board of Regents of the University of Wisconsin System, 1995. 
References

References-4

Purkey, S.C. and Smith, M.S. (1983). Effective schools: A review. Elementary School Journal, 

83(4), 1983: 427–452. 

Rossell, C.H. “The Desegregation Efficiency of Magnet Schools.” Urban Affairs Review (38)5, 

May 2003: 697-725. 

Sebring, P.B., and Bryk, A. “School Leadership and the Bottom Line in Chicago.” Phi Delta 

Kappan 8(6), 2000: 440–443. 

Sebring, P.B., Bryk, A.S., Easton, J.Q., Luppescu, S., Thum, Y.M., Lopez, W.A., and Smith, B. 

Charting Reform in Chicago: The Teachers Take Stock—Executive Summary. Chicago: 

Consortium on Chicago School Research, 2000. 

Steel, L., and Eaton, M. Reducing, Eliminating, and Preventing Minority Group Isolation in 

American Schools: The Impact of the Magnet Schools Assistance Program. Washington, 

D.C.: U.S. Department of Education, 1996. 

Steel, L., and Levine, R. Educational Innovation in Multiracial Contexts: The Growth of Magnet 

Schools in American Education. Washington, D.C.: U.S. Department of Education, 1994. 

Taylor, B.M., Pearson, P.D., Clark, K.F., and Walpole, S. Beating the Odds in Teaching All 

Children to Read. Ann Arbor: University of Michigan, Center for the Improvement of Early 

Reading Achievement, 1999. 

Teddlie, C. and Stringfield, S. Schools Making a Difference: Lessons From a 10-year Study of 

School Effects. New York: Teachers College, Columbia University. 1993. 

U.S. Department of Education, Office of the Under Secretary, Planning and Evaluation Service, 

Elementary and Secondary Education Division. The Same Standards for Migrant Students: 

Holding Title I Schools Accountable, Volume I: Title I Schools Serving Migrant Students: 

Recent Evidence from the National Longitudinal Survey of Schools, Washington, D.C.: 

Author, 2002. 

U.S. Department of Education, Office of the Under Secretary, Planning and Evaluation Service, 

Elementary and Secondary Education Division. Schools Identified as in Need of 

Improvement under Title I: Recent Evidence from the National Longitudinal Survey of 

Schools. Washington, D.C.: Author, 2002. 

U.S. Department of Education. Office of the Under Secretary, Planning and Evaluation Service, 

Elementary and Secondary Education Division. Evaluation of the Magnet Schools Assistance 

Program, 1998 Grantees: Year 1 Interim Report Appendices. Washington, D.C.: Author, 

2001. 

Wasley, P.A., Hampel, R.L. and Clark, R.N. “The Puzzle of Whole-School Change.” Phi delta 

Kappan, 78(9), May 1997: 690–697. 
 References 

 References-5

Wenzel, S.A., Smylie, M.A., Sebring, P.B., Allensworth, E., Gutierrez, T., Hallman, S., 

Luppescu, S., and Miller, S.R. Development of Chicago Annenberg Schools: 1996–1999. 

Improving Chicago's Schools. Report of the Chicago Annenberg Research Project. Chicago: 

Consortium on Chicago School Research, 2001. 

Wong, K.K., Hedges, L.V., Borman, G.D., and D’Agostino, J.V. Final Report Prospects: 

Special Analysis. Washington, D.C.: U.S. Department of Education, Office of the 

Undersecretary, 1996. 

Yu, C.M., Taylor, W.L., Goldring, E., Smrekar, C., and Piche, D. Difficult Choices: Do Magnet 

Schools Serve Children in Need? Washington, D.C.: Citizens’ Commission on Civil Rights, 

1997. 
References

References-6
Appendix I— Introduction and Overview of the Magnet 

Schools Assistance Program (MSAP) and Evaluation 

Appendix I

 Appendix-I-1

Methodology 

This section presents information about the data collection, interpretation, and analyses 

that contributed to this evaluation. The first section outlines the four studies that are discussed in 

the final report and the main data collections associated with each of them. Later sections 

provide additional information about the data we used to address evaluation questions about 

desegregation objectives, enrollment trends, staffing characteristics, and student achievement 

outcomes. These sections are intended as updates to the extensive discussions that appeared in 

our first report. 

Overview of the Four Evaluation Studies in this Report 

This report features four interrelated strands of inquiry that are described briefly below 

and summarized in Exhibit A-I-1. 

Exhibit A-I-1. 

Overview of the MSAP evaluation studies, data sources, and schedule 

Study and Activities 1999–2000 2000–2001 2001–2002 

Study 1: Profile of Districts 

Develop database on 57 MSAP projects with information 

from grant applications and performance reports 

√ √ √

 

Interview 57 MSAP Project Directors √ √ √

 Disseminate District Data Request—57 Projects √ √

 Administer Project Survey—57 Projects √ √

Study 2: Profile of Schools 

Develop database on 292 schools with information from 

grant applications and performance reports √ √ √

Administer Principal Survey—292 MSAP Schools √ √

Obtain and analyze school-level standardized test data for 

the national sample of schools, as available 

 √

Study 3: In-Depth Case Studies of 8 MSAP Projects 

Interview District Staff—8 Projects 

√ √

Obtain and analyze districtwide standardized test scores 

and other student achievement data as available (dropped 

in response to federal research policy changes)

 
Appendix I

Appendix-I-2

Exhibit A-I-1. (continued) 

Overview of the MSAP evaluation studies, data sources, and schedule 

Study and Activities 

1999–

2000 

2000–

2001 

2001–

2002 

Study 4: In-Depth Case Studies of MSAP Schools

Interview Principal, Other School Staff √ √

Observe Classrooms—32 MSAP Schools, 15 

Comparison Schools 

√ √

Administer Teacher Surveys—32 MSAP Schools, 15 

Comparison Schools 

√

Administer Principal Surveys—21 Comparison Schools √ √

Conduct Student Focus Groups—Selected Schools √

Study 1. Profile of all 57 projects 

We developed a profile of the full population of 57 MSAP-supported districts with 

descriptive analyses of program context, program characteristics, and enrollment and 

achievement outcomes. The profile is based on data extracted from existing MSAP program 

documents—grant applications and the annual performance reports that grantees submit to ED—

as well as through three data collections conducted by the evaluation that are described below. 

We conducted telephone interviews of approximately one hour with all of the MSAP 

Project Directors during fall 1999 and winter 2000. The open-ended questions in the first 

interview protocol were designed primarily to identify ways in which existing programs differed 

from plans described in the project’s application and to obtain verification of data (e.g., MSAPsupported magnet schools, feeder schools, desegregation goals, and achievement objectives). 

Shorter interviews were conducted in fall 2000 to obtain status reports and identify any program 

changes, and in fall 2001 to determine the status of MSAP projects after federal funding had 

ended. 

We sent self-administered Project Surveys, consisting mostly of close-ended questions, to 

MSAP project directors in late fall 1999 and 2000. These surveys focused on student recruitment 

and outreach, program planning and implementation, accountability, coordination of funding, 

systemic reform, the role of ED in the MSAP project, and the project director’s background and 

role. 

Finally, in fall 1999 we sent each grantee a District Data Request (DDR) that asked for 

information about student, teacher, and administrator characteristics for each school in the 

district that served the same grade level (or levels) as those served by the MSAP-supported 

schools during the 1999–2000 school year. Although most districts tried to respond to this 
Appendix I

 Appendix-I-3

request, the comprehensiveness of the data provided varied widely. We subsequently 

supplemented this information with data obtained from several state departments of education. 

Study 2. Profile of all MSAP-supported schools 

For this study, we focus on the nearly 300 schools1 that received program funds in 

MSAP-funded districts: the school context, program characteristics, and enrollment and 

achievement outcomes. This profile is based on data extracted from MSAP grant applications 

and performance indicator data provided in the annual performance reports for all schools that 

received MSAP support. In addition, it uses responses to Principal Surveys that were 

administered to the principals of the schools that were operating MSAP-supported programs 

during the 1999–2000 and 2000–2001 school years. 

The Principal Surveys were self-administered instruments that were sent to the principals 

in the late fall. (In accordance with the wishes of each MSAP project director, the surveys were 

either sent directly to the principals or were sent to the project director for distribution.) 

Questions in the 1999–2000 survey focused on features of the school’s MSAP program, systemic 

reforms, accountability, professional development, use and coordination of program funds, the 

working environment, parent involvement, and the principal’s background and role. The 

somewhat shorter survey administered in 2000 featured items on systemic reform and classroom 

instruction not covered in the first survey, and repeated some items that had appeared in the first 

survey in order to measure changes in the school’s magnet program. 

Exhibit A-I-2 summarizes the number of schools funded by the MSAP that were 

operating during each year of the grant. (These figures represent the maximum number of 

schools that could contribute data to analyses for a given year; the actual number included in 

analyses is generally somewhat smaller due to survey nonresponse or missing data on the 

particular topics analyzed.) During 1998–1999, some magnet school facilities were still under 

construction, and others devoted their first grant year to planning. Although these “planning 

schools” may have enrolled students, their special instructional programs were incompletely 

developed, and recruiting for the magnet program and consequent changes in the proportions of 

minority and nonminority students enrolled were not expected to occur until 1999–2000. By the 

third grant year, nearly all of the programs had begun operating, but four had been dropped from 

their district’s MSAP project and two had been added. The number of cases included in 

descriptions of magnet schools’ enrollment characteristics depended not only on which schools 

were operating in a given year but also on the number of these schools for which enrollment data 

were available from grantees’ performance reports and from the National Center for Education 

Statistics (NCES) Common Core of Data (CCD) electronic files. Finally, analyses based on 

responses to the Principal Survey were limited to schools whose principals completed the survey 

(267 in 1999–2000 and 266 in 2000–2001). 

 

1

 The 57 MSAP projects comprise 293 programs located in 292 schools. One school contains two small programswithin-a-school (PWSs). All the other schools are either whole school programs or contain only one MSAPsupported PWS. 
Appendix I

Appendix-I-4

Exhibit A-I-2. 

MSAP schools in 1998–1999 through 2000–2001 

 

Category N 

Schools Funded 292 

Schools Operating 1998–1999 262 

Schools Operating 1999–2000 284 

Schools Operating 2000–2001 285 

Schools Operating in 1998–1999 and 2000–2001 257 

Schools Never Fully Operational During the Grant Period 2 

Schools Returning the 1999–2000 Survey 267 

Schools Returning the 2000–2001 Survey 266 

Study 3. In-depth case studies of eight selected MSAP projects 

We have developed Case Studies to illuminate the aggregate results obtained from the 

national data collection (Studies 1 and 2). Although the case study districts and schools were not 

sampled at random from the full population, the Case Studies provide examples for the national 

profiles and permit comparisons of student achievement outcomes in MSAP schools and nonmagnet schools enrolling similar students, within each case study district. During our initial Case 

Study visits in spring 2000, we interviewed the MSAP project director, recruitment specialist, 

district curriculum specialist, and any project-level staff funded by MSAP (e.g., resource 

teachers). During our spring 2001 visits, we conducted an interview with the MSAP Project 

director and other staff as needed to obtain updates on the status of their programs. We had 

originally intended to collect student-level data on achievement outcomes in MSAP-supported 

and comparison schools directly from some of the Case Study districts. However, changes in 

federal policy regarding the collection of individually identifiable student data by federal 

contractors led us to revise this aspect of the study. Rather than conducting detailed analyses of 

student assessment data from a few Case Study sites, we obtained and analyzed school-level 

achievement data from 135 MSAP-supported elementary schools and 1,350 comparison schools 

in 14 states. Further details are presented in a later section of this appendix. 

 

As shown in Exhibit A-I-3, the eight case study projects were selected to reflect the 

characteristics of the 57 projects considered most salient to this evaluation; all eight projects 

agreed to participate. At the request of these project directors, the names of the Case Study 

projects will be identified in this study only as Districts A to H. 

The Case Study projects cannot be considered to represent all 57 projects, but as Exhibit 

A-I-3 indicates, the eight projects include both required and voluntary programs and provide 

variety in student populations, location, and size. To permit in-depth exploration of student-level 

achievement outcomes that were being planned at the time the Case Study sites were selected, 

we intentionally over-sampled states in which such data were likely to be available. 
Appendix I

 Appendix-I-5

Exhibit A-I-3. 

Characteristics of eight case study sites in comparison to all 57 MSAP projects 

 8 Case Study Projects All 57 MSAP Projects 

Characteristic N % N % 

Desegregation Plan 

 Voluntary 

 Required 

5 

3 

 62.5% 

 37.5 

31 

26 

 54.4% 

 45.6 

Average Minority Percentage in District* 

Range of Minority Percentages in Districts* 

Predominant Minority Group(s): 

Predominantly African American 

Predominantly Hispanic 

Predominantly Asian 

8 

6 

2 

0 

 63.0 

32–88% 

 75.0 

 25.0 

 0.0 

57 

38 

17 

2 

 61.1 

25–93% 

 66.7 

 29.8 

 3.5 

Geographic Region** 

Northeast 

Southeast 

Central (Middle) 

West 

2 

3 

0 

3 

 25.0 

 37.5 

 0.0 

 37.5 

17 

19 

6 

15 

 29.8% 

 33.3 

 10.5 

 26.3 

No. of States Represented 8 

 

25 

State Categories*** 

 A States 

 B States 

 C States 

 D States 

 E States 

 F States 

 G States 

4 

1 

1 

1 

1 

0 

0 

 50.0 

 12.5 

 12.5 

 12.5 

 12.5 

 0.0 

 0.0 

18 

8 

6 

6 

15 

0 

4 

 31.6% 

 14.0 

 10.5 

 10.5 

 26.3 

 0.0 

 7.0 

* Based on Common Core of Data Non-Fiscal Survey (CCD) for 1997–1998, National Center for Education 

Statistics 

** Based on definitions used by NAEP, NEA, and the Bureau of Economic Analysis of the U.S. Department of 

Commerce 

*** Based on data categories presented at a meeting on student achievement sponsored by the U.S. Department of 

Education. These categories are based on the availability of longitudinal student-level achievement data on a state 

assessment, with “A” states compiling such data and “B” through “G” states compiling progressively less detailed 

achievement data. 
Appendix I

Appendix-I-6

Study 4. In-depth case studies of MSAP schools 

Our Case Study selection process was designed to obtain a sample of schools that 

together would represent all three of the grade levels and a variety of themes. Within each Case 

Study district, we selected four MSAP-supported schools for study. To help motivate 

participation and ensure fairness, we invited each MSAP project director to choose one school 

for inclusion in the study. To the extent possible, we then selected three other MSAP-supported 

schools at the same level as the project director’s choice. 

We matched the MSAP-supported schools with two comparison schools on the basis of 

enrollment data from the Common Core of Data Non-Fiscal Survey (CCD) maintained by the 

National Center for Education Statistics. The comparison schools identified were schools in the 

district that served students with racial-ethnic backgrounds similar to those in the MSAP schools, 

but which did not operate magnet programs. In most cases, close matches were found, but in 

districts that were small or in which there were numerous magnet schools, the comparison 

schools tended to have fewer minority students than their magnet counterparts. Exhibit A-I-4 

compares the MSAP-supported schools in the Case Studies and the MSAP-schools in the entire 

group of 57 MSAP districts. As the exhibit shows, high schools were slightly overrepresented 

and middle schools slightly underrepresented in the sample. 

Exhibit A-I-4. 

Levels of 32 MSAP-supported schools in case studies and the 292 schools in all 57 

MSAP projects 

32 Case Study Schools 292 MSAP Project Schools 

Grade Level n % n % 

Elementary 

Middle 

High School 

Combined Levels 

18 

6 

7 

1 

56.2% 

18.8 

21.9 

3.1 

175 

71 

40 

6 

59.9% 

24.3 

13.7 

 2.1 

Site visits were made to the eight Case Study districts in April–May 2000 and April–May 

2001. Two site visitors went to each site. They spent one day together in interviewing projectlevel staff, and then each visited two MSAP-supported schools and one comparison school. 

During two-day visits in each school, they interviewed principals, talked with teachers, and 

conducted classroom observations. In comparison schools, principals were asked to complete 

Principal Surveys that paralleled those that MSAP principals had already completed. In four of 

the districts, where schools were not all at the same level, additional comparison schools were 

identified and their principals were asked to complete surveys, to provide additional information 

about the sites. During the second site visit, student focus group discussions were conducted in 

four Case Study districts. Summary information about the first round of visits in spring 2000 was 

presented in our first report. Exhibit A-I-5 summarizes the same information for the spring 2001 

visits. Descriptive information collected during these visits is contained in the Case Study reports 

included in the Case Studies Appendix, and is also integrated into the chapters of the main 

report. 
Appendix I

 Appendix-I-7

Exhibit A-I-5. 

Number of schools visited or surveyed in case study districts 

Case MSAP-supported Schools Visited 

(Principal Surveyed)

Comparisons—Visited 

(Principal Surveyed)

Comparisons—No 

Visit (Principal 

Surveyed)

Site Elem. Middle High Other Elem. Middle High Elem. Middle High 

A 3 0 1 0 1 0 1 1 0 0 

B 3 1 0 0 1 1 0 1 0 0 

C 2 2 0 1 1 1 0 1 1 0 

D 0 0 3 0 0 0 1 0 0 0 

E 4 0 0 0 2 0 0 0 0 0 

F 2 2 0 0 1 1 0 1 1 0 

G 0 1 3 0 0 1 1 0 0 0 

H 4 0 0 0 2 0 0 0 0 0 

Total 18 6 7 1 8 4 3 4 2 0 

Finally, in spring 2001 we surveyed a sample of teachers in the Case Study and 

comparison schools. The surveys contained closed-ended questions about the teachers’ 

perceptions of the professional communities of their schools, the impact of state curriculum 

frameworks and student assessments on instruction, the degree of emphasis on specific teaching 

practices and strategies, the professional development in which the teachers had recently 

participated, and the teachers’ professional backgrounds. Five subject-specific versions of the 

survey were administered to teachers in MSAP schools: elementary school language arts and 

mathematics; and middle or high school language arts, mathematics, and special classes related 

to the magnet theme. Similar language arts and mathematics surveys (minus the questions 

pertaining to magnet program features) were administered to teachers in comparison schools. 

Coding of District and School Characteristics 

Determination of districts’ desegregation plan type 

To be eligible to receive MSAP funds, districts must be implementing a formal 

desegregation plan—either a plan that they are undertaking voluntarily or one that has been 

required by an external authority. We identified each district’s plan type using information 

provided in MSAP applications. One of the documents districts usually submitted in Part V of 

their MSAP application was the Desegregation Plan Information form (Reference §280.20), on 

which the district indicates the nature of its plan. In addition, most application narratives 

included historical and descriptive information about their plans. We verified our classifications 

of the 57 grantees with staff of MSAP and the Department of Education. 

Assignment of grade level categories to schools

The content and structure of magnet programs differ somewhat by grade level. For 

example, high school programs are more likely to focus on vocational preparation than are 

programs for lower grades, and are more likely to be organized as programs within a school 

(PWSs) rather than whole school programs. Consequently, most analyses in this study 

disaggregate results by school grade level. Although most schools in the 57 MSAP districts serve 
Appendix I

Appendix-I-8

conventional grade ranges (kindergarten through grade 5 elementary schools, grade 6 through 8 

middle schools, and grade 9 through 12 high schools), there are many variations on the basic 

pattern. Some schools open with just a grade or two and phase in additional grades over time. 

Others serve wide grade ranges such as K–8 or 6–12. In order to group similar schools together 

for comparative analyses and to minimize the number of schools in the “other” category, we used 

the following rules for assigning schools to grade level categories: 

• Elementary school: low grade is 3 or below; high grade does not exceed 8. 

• Middle school: no grade is lower than 4; high grade does not exceed 9. One school 

that contains a kindergarten as well as grades 6–8 is counted as a middle school. 

• High school: low grade is no lower than 9; high grade is up to 12. 

• Combined levels school: lowest grade is in the elementary or middle school range; 

high grade in the high school range (e.g., 4–12, 6–12, 7–12). 

Identification of desegregation objectives 

A major legislative purpose of the MSAP program is to assist school districts in reducing, 

eliminating, or preventing minority group isolation in their schools through the development of 

attractive instructional programs. Within the context of the MSAP, “minority” includes students 

from the following race-ethnic backgrounds: American Indian or Alaskan Native, Asian or 

Pacific Islander, Hispanic, and black (not of Hispanic origin).2 A schools is defined as minority 

group isolated if minority group students comprise 50 percent or more of the school’s 

enrollment. When they apply for MSAP grants, districts identify one or more schools that will be 

targeted for desegregation impact by their proposed magnet program and specify a particular 

desegregation objective for each one. 

Several of the questions addressed by this evaluation concern the desegregation 

objectives that are set by grantees and the schools’ success in meeting them. During this 

investigation, we collected the data needed to identify the schools that will be included in the 

analysis, as well as the objectives of the schools targeted for desegregation impact by the MSAPsupported project. We then collected and analyzed data on the trends in each school’s enrollment 

composition to determine the extent to which grantees were able to meet—or make progress 

toward meeting—the desegregation objectives of reducing, preventing, or eliminating minority 

group isolation as those are defined under MSAP regulations. 

Districts with voluntary plans must set objectives for their targeted schools that conform 

with the federally defined desegregation goals of reducing, eliminating, or preventing minority 

group isolation. Districts with required desegregation plans may have requirements established 

by a court or other agency that vary from those defined for voluntary districts.3 The specific 

objectives set by required plans do not always conform to the MSAP definitions used. All 

applicants must explain how their programs support the legislated purposes of the MSAP. In 

general, however, the grantees in districts with required desegregation plans have also identified 

goals consistent with the desegregation objectives defined above. Based on discussions with the 

 

2

 34 CFR §280.4 

3

 34 CFR §280.2(a)(1) 
Appendix I

 Appendix-I-9

Department of Education, this evaluation does not attempt to analyze the extent to which the 

schools in districts operating under plans required by an external authority have met the 

requirements set out by the external authority. 

MSAP grant applications were one source of baseline data for this investigation. As 

stated earlier, applicants for MSAP support are required to submit detailed explanations of their 

desegregation plans that contain most of the information we need to establish the framework for 

our analyses. In their narratives, applicants were to identify the schools in which the federally 

supported magnet programs will operate (magnet schools) and the schools from which magnet 

students will be drawn (feeder schools), and indicate which of these schools are targeted for 

desegregation impact.4

 In addition, they were to supply specific information about the 

desegregation objectives of each targeted school, including enrollment statistics for the year prior 

to the initiation of the magnet program and projected or benchmark enrollment statistics to be 

met each year of the grant. 

Although most schools and desegregation objectives were clearly identified in the 

applications, we did encounter ambiguities in these documents. Some applications contained 

varying statements of their schools’ desegregation objectives. Because the questions to be 

addressed by this evaluation are predicated on federal definitions, we devoted considerable effort 

to resolving ambiguities in a manner that is both uniform and faithful to the intent of the MSAP 

statute and regulations. We considered both the wording of the objective and the nature of the 

change indicated by the baseline and projected enrollment statistics provided in the application. 

To resolve ambiguities encountered during our initial coding, as well as to ascertain that the 

desegregation plan described in the application did not change before the MSAP project 

commenced, we also reviewed grantees’ first year performance reports and asked each project 

director to verify the information about their schools and desegregation objectives coded in our 

database. Finally, we consulted with the Department to resolve the most intractable cases. 

Documentation of trends in school enrollment 

The evaluation uses school- and district-level enrollment statistics to describe the context 

within which the magnet projects operate, to compare characteristics of the MSAP-supported 

magnets and other public schools, and to document the degree to which each magnet school 

meets its desegregation objectives. In order to support rigorous comparative analyses, these data 

must be: 

• Collected at the same time each year. 

• Available for all schools in the district that serve students in the grade level or levels 

that are served by the district’s MSAP magnets. 

• Disaggregated by minority status as defined by the regulations governing the MSAP, 

or by the five major racial-ethnic categories used by federal data-collecting agencies. 

 

4

 Most of the schools targeted for desegregation impact are the schools in which the magnet programs are located. 

Sometimes, however, a magnet program targets a feeder school. That is, the magnet is intended to draw minority 

students away from a feeder school and thereby reduce, eliminate, or prevent minority group isolation in the 

feeder school. 
Appendix I

Appendix-I-10

We have obtained school and district enrollment data from four primary sources, each of 

which involves strengths and limitations. 

Data that grantees provide to the MSAP in applications and performance reports

document the minority or nonminority composition of the students in magnet and targeted feeder 

schools as well as districtwide. These data are disaggregated for grade and minority status and 

are reported separately for programs within a school (PWSs) where they exist. Some of the 

limitations in the quality of the data that grantees provide in their applications and annual reports 

included enrollment counts that are not from the same time of year in all districts, counts that are 

from an unspecified time of the year, inconsistent or conflicting counts in the same report, and 

use of rounded percentages rather than precise counts. 

A few items on the 1999–2000 Principal Survey and the 2000–2001 Principal Survey

also elicited information about student and program characteristics. By virtue of their positions in 

the magnet schools, principals are the best source of information about some aspects of their 

magnet programs. However, their information on other topics (e.g., the funding sources for 

special programs in their schools, whether their school operates a Title I Schoolwide Program) 

may be limited and at variance with information derived from other sources. 

CCD school-level files contain data on the racial-ethnic composition of virtually all 

public schools in the United States between 1987 and the present. Thus they are the only source 

of data available to support analyses of historical trends in district and school enrollment 

composition. These data are collected at the same time each year and are reported in the five 

federal ethnic-racial categories. For some grantees, the CCD is also the only source of 

comparable enrollment data for the magnet and non-magnet schools in the district. The primary 

disadvantages of the CCD data are the time lag between collection and dissemination (certified 

data collected in October 1998 became publicly available in September 2000) and the fact that 

they do not allow analyses of enrollment trends for within-school programs, or for schools that 

have not been identified by the National Center for Education Statistics (i.e., several magnet 

schools in New York City), nor do they permit analysis of enrollment trends for single grades 

(e.g., kindergarten, first grade, second grade,...twelfth grade) for the period of this evaluation.5

We have acquired the CCD school-level files for the 1997–1998, 1998–1999, 1999–2000, and 

2000–2001 school years, as well as a number of prior years. 

In view of the strengths and limitations described above, our study uses different data 

sources for different analyses. For analyses of desegregation outcomes over the course of the 

grant period, we are relying primarily on data from the CCD because it reduces the variability in 

reporting standards and provides an independent source of data on which to base evaluation of 

the MSAP programs. When data for a targeted school are not available in the CCD, the 

information provided by the grantees is used to make the evaluation. 

 

5

 NCES introduced reporting of race-ethnicity by single grades (e.g., kindergarten, first grade, second grade, 

...twelfth grade) in the 1998–1999 CCD. Prior to that year race-ethnic counts were provided for each school, but 

not for single grades within schools. While the number of states providing the more detailed enrollment data is 

increasing, the data are not adequate for a more detailed analyses of minority enrollment trends for single grades 

for the period covered by the 1998 grant cycle. 
Appendix I

 Appendix-I-11

School-Level Data Used in the Analysis of Achievement Trends in 

MSAP Schools 

This evaluation addresses the issue of student achievement in MSAP-supported schools 

through two major lines of inquiry. The first focuses on the diverse objectives that local MSAP 

projects set for their magnet schools and the outcome data they used to document the schools’ 

progress toward meeting those objectives. A detailed discussion of data collection and 

interpretation issues arising from the study of achievement objectives was provided in our first 

report. 

The second line of inquiry uses a somewhat narrower range of measures—English 

language arts and mathematics scores on state standardized tests—to conduct better controlled 

studies, with more generalizable findings, of achievement trends in MSAP magnet schools and 

matched comparison schools without federally funded magnet programs. We planned to conduct 

two studies of this type: (1) an analysis of comprehensive, longitudinal student-level data 

(demographic background, program participation, and achievement measures for students in 

MSAP and non-MSAP schools) obtained from a subset of the eight Case Study districts and (2) 

an analysis of school-level achievement data for a larger number of districts if the necessary data 

became available during the course of the evaluation. By spring 2002, federal research policy 

dictated that student-level data should not be collected from Case Study districts, and at the same 

time the National School-Level State Assessment Score Database (developed by Don 

McLaughlin and his team at AIR) was sufficiently complete to go forward with a study of 

school-level achievement outcomes. This section discusses the data assembled for the state 

assessment analysis. 

Characteristics of the national school-level state assessment score database 

As its name suggests, the National School-Level State Assessment Score Database 

contains school-level test scores obtained directly from the testing agencies of most of the 50 

states. Data for most subjects and grades tested are available back to 1997–1998 for many states 

and back to 1998–1999 for most. Scores for spring 2001 became available in summer 2002. 

Reflecting the diversity of states’ reporting practices, the achievement measures included in the 

database vary from state to state and even from year to year for particular states. Common 

reporting metrics are mean scaled scores, mean national percentile ranks and normal score 

equivalents (NCEs), and percentages of students performing at criterion levels set by their states. 

The database also includes the NCES code for each school, which permits the student 

achievement data to be associated with demographic data from the Common Core of Data NonFiscal Survey (CCD). (A small fraction of schools with achievement data but no identifiable 

NCES code are excluded from the database.) 

Availability of data for MSAP achievement analyses 

The database contains at least some data for each of the 25 states in which MSAP-funded 

schools were located during the 1998 funding cycle. However, not all of the states and MSAP 

schools met the criteria necessary for inclusion in our analysis of trends in achievement. For 

inclusion in the analysis, a school required the following data: 
Appendix I

Appendix-I-12

• Mean test scores in mathematics, reading (or both) for at least one grade in the 

baseline year6 and the final MSAP grant year, 2000–2001 (the same grade in each 

year). 

• Ethnic composition of the enrollment in the baseline year and 2000–2001. 

• Percentage of students eligible for free or reduced price lunch in 1999–2000. 

In addition, in order to determine the amount of variation between MSAP schools in each 

state, it was necessary that any state included in the analysis would have data for at least two 

MSAP schools. We initially determined that the greatest number of MSAP schools with 

complete data were elementary schools; consequently, we restricted our analyses to trends in 

student achievement in grades three through five. 

A total of 11 states were excluded from the analysis because they lacked test data for the 

baseline year or for 2000–2001, tested students at grade levels other than those served by MSAP 

programs, contained only one MSAP school with data, or lacked demographic data. Individual 

MSAP schools in the remaining 14 states were excluded from the analysis because data for one 

or both years were unavailable in the National School-Level State Assessment Score Database. 

The most common reasons for missing data were that the school was not open during the 

baseline year or did not have an identifiable NCES code that would allow the necessary linkage 

to demographic data in the CCD. A total of 135 MSAP elementary schools located in 14 states 

met all of the criteria for inclusion in the study. (See the Appendix-Chapter V for technical 

details about the analysis of these data.) 

 

6

 The preferred baseline year was 1997–1998, the year prior to the beginning of the MSAP grants. For a few states 

that did not have test data for 1997–1998, we treated 1998–1999 as the baseline year. 
Appendix II—Characteristics of MSAP-Supported Schools 

and Programs

 Appendix II 

 Appendix-II-1

Exhibit A-II-1. 

Percentage and frequency of MSAP-funded schools by grade level: 2000–2001 

 Percent Frequency 

Elementary 60.4% 172 

Middle School 23.2% 66 

High School 14.0% 40 

Combined Levels 2.5% 7 

 

Total 100.0% 285 

Source: MSAP applications and 2000–2001 performance reports 

Exhibit A-II-2. 

Percentage distribution of magnet schools nationwide by grade level: 1999–2000 

Percent Std. error 

Elementary 63.7% 2.68 

Middle School 15.0% 2.01 

High School 19.6% 1.81 

Combined Levels 1.6% 0.26 

 

Total 100.0% 

Source: 1999–2000 NCES: Schools and Staffing Survey: Public School 

Questionnaire, item 15a and c

 

Exhibit A-II-3. 

Percentage and frequency of MSAP schools by type of program within grade level: 

2000–2001 

Grade Level All Grades 

Type of Program 

Elementary 

(N = 172) 

Middle 

School 

(N = 66) 

High School

(N = 40) 

Combined 

Levels 

(N = 7) 

% 

(N = 285) Frequency

Program within a School 6.4% 15.2% 30.0% 0.0% 11.6% 33 

Whole School 93.6% 84.8% 70.0% 100.0% 88.4% 252 

 

Total 100.0% 100.0% 100.0% 100.0% 100.0% 285 

Source: MSAP applications and 2000–2001 performance reports

Exhibit A-II-4. 

Percentage of whole school and PWS magnets nationwide by grade level: 1999–2000 

 School Wide PWS Total 

Elementary 78.2% 21.8% 100.0% 

Middle School 52.5% 47.5% 100.0% 

High School 44.0% 56.0% 100.0% 

 

All Grade Levels 67.6% 32.4% 100.0% 

Source: 1999–2000 NCES: Schools and Staffing Survey: Public School Questionnaire, 

item 15a, b, and c 
 Appendix II 

Appendix-II-2

Exhibit A-II-5 

Budget means and percentages for 233 MSAP-supported schools, by budget category 

and year, based on total direct funds 

Year 

Staff and 

Fringe 

Travel Equipment Supplies Contractual Construction Other 

Direct 

Training 

Stipends 

Indirect 

Costs Total 

1998–1999 $124,227 $2,162 $79,422 $63,611 $21,511 $603 $12,460 $6,483 $3,431 $313,910 

1999–2000 139,552 1,760 67,795 65,346 18,413 359 8,114 4,769 3,392 309,500 

2000–2001 143,687 1,505 61,698 59,534 16,494 344 7,437 3,105 3,236 297,040 

N = 233 schools in 50 projects 

Source: U.S. Department of Education

Exhibit A-II-6 

Average full time equivalent of MSAP-funded staff and district-funded recruitment 

specialist devoted to recruitment activities 

Type of Desegregation Plan

Voluntary 

(N = 26) 

Required 

(N = 21) 

All Districts 

(N = 46) 

MSAP-Funded Staff 1.41 1.96 1.65 

District-Funded Recruitment Specialist 0.45 0.47 0.45 

Total FTE Devoted to Recruitment 1.86 2.43 2.11 

 Source: Project Survey, 1999–2000, Items 4 and 5

Exhibit A-II-7 

Percentage of districts with specific types of participation of school staff, students, and 

parents in recruitment effort by type of desegregation plan 

Type of Desegregation Plan

Voluntary 

(N = 30) 

Required 

(N = 26) 

All Districts 

(N =56) 

Principals/Assistant Principals Organize School Tours 97% 96% 96% 

Magnet School Teachers Make Presentations to Visitors 93% 88% 91% 

Magnet School Teachers Visit Other Schools to Recruit Students 70% 69% 70% 

Students Help Conduct School Tours 80% 88% 84% 

Students Visit Other District Schools to Recruit 23% 46% 34% 

Parents Make Presentations to Other Parents 53% 46% 50% 

Parents Make Telephone Calls to Provide Information 23% 50% 36% 

Other 43% 27% 36% 

Source: Project Survey, 1999–2000, Item 6 
 Appendix II 

 Appendix-II-3

Exhibit A-II-8 

Percentage of districts using specific types of outreach used to recruit students by type 

of desegregation plan 

Type of Desegregation Plan

Voluntary 

(N = 30) 

Required 

(N = 26) 

All Districts 

(N = 56) 

Brochures 100% 96% 98% 

Information Distributed to Students 100% 92% 96% 

Information Mailed to Parents who Request It 93% 85% 89% 

Information Mailed to All Parents 67% 73% 70% 

Presentations at Other Schools 77% 85% 80% 

Presentations at Churches or Community Organizations 77% 92% 84% 

Tours of School without Transportation Provided 90% 92% 91% 

Tours of School with Transportation Provided 20% 35% 27% 

Babysitters Provided for School Events 20% 46% 32% 

Telephoning Parents to Explain Program 70% 58% 64% 

Home Visits to Parents/Students 13% 23% 18% 

Media Advertisements 87% 88% 88% 

Advertising at Fairs or Other Forums 87% 88% 88% 

Information on an Internet Site 63% 73% 68% 

Other 30% 15% 23% 

Source: Project Survey, 1999–2000, Item 6 

Exhibit A-II-9 

Extent to which district's outreach activities focus on targeted groups of students by 

type of desegregation plan 

Type of Desegregation Plan

Voluntary 

(N = 30) 

Required 

(N = 26) 

All Districts 

(N = 56) 

None-Focus Outreach on All Students 20% 50% 34% 

Less Than Half of the Outreach Effort 20% 15% 18% 

Three-Quarters of the Outreach Effort 43% 19% 32% 

All Outreach Efforts Focused on Targeted Groups of 

Students 

17% 15% 16% 

Total 100% 100% 100% 

Source: Project Survey, 1999–2000, Item 3 

Exhibit A-II-10 

Percentage of MSAP districts in which waiting lists are 

maintained for one or more programs 

 

Type of Desegregation Plan Percent 

Voluntary 52% 

Required 77 

All Districts 63 

Source: Project Director Interview, 1999–2000, Item 7 
 Appendix II 

Appendix-II-4

Exhibit A-II-11. 

Percentage and frequency of students in MSAP-funded programs by grade level: 

2000–2001 

 Percent Frequency 

Elementary 51.9% 95,493 

Middle School 24.6% 44,126 

High School 19.6% 36,141 

Combined Levels 3.9% 8,191 

 

Total 100.0% 183,951 

Source: MSAP 2000–2001 performance reports, Principal Surveys 2000–2001 

Items 24b and 24c, and NCES 2000–2001 Common Core of Data 

Exhibit A-II-12. 

Percentage and number of students in MSAP programs by type of program within grade 

level: 2000–2001 

 Grade Level All Grades 

Type of Program 

Elementary

(N = 172) 

Middle 

School 

(N = 66) 

High School

(N = 40) 

Combined 

Levels 

(N = 7) 

% 

(N = 285) Frequency

Program within a School 2.1% 9.1% 11.6% 0.0% 5.6% 10,325 

Whole school 97.9% 90.9% 88.4% 100.0% 94.4% 173,626 

 

Total 100.0% 100.0% 100.0% 100.0% 100.0% 183,951 

Source: MSAP 2000–2001 performance reports, MSAP Principal Survey 2000–2001, Item 24, and NCES 2000–2001 Common Core of 

Data 

Exhibit A-II-13. 

Percentage of minority students in MSAP-funded programs at each grade level and 

overall: 2000–2001 

% Minority of N students 

Elementary 74.8% 95,493 

Middle School 72.1% 45,184 

High School 77.5% 36,141 

Combined Levels 70.0% 7,133 

 

Overall 74.5% 183,951 

Source: MSAP 2000–2001 performance reports, MSAP Principal Survey 2000–2001, 

Items 24b and 24c, and NCES 2000–2001 Common Core of Data 
 Appendix II 

 Appendix-II-5

 Exhibit A-II-14. 

Percentage minority enrollment in each type of MSAP-funded program within grade level: 

2000–2001 

 Grade Level 

Type of Program Elementary 

Middle 

School 

High 

School 

Combined 

Levels All Grades N students 

Program within a School 65.7% 65.2% 60.6% – 63.4% 10,325 

Whole school 75.0% 72.8% 79.7% 70.0% 75.2% 173,626 

 

 N students 95,493 45,184 36,141 8,191 183,951 

Source: 2000–2001 performance reports, Principal Survey 2000–2001 - Items 24b and 24c, and NCES Common Core of Data 2000–2001 

Exhibit A-II-15 

Mean percentage of minority student enrollment within districts for MSAP and non-MSAP 

schools by race-ethnic group: 1999–2000 

 Grade Level 

MSAP v. Non-MSAP schools Black Hispanic 

Asian/Pacific 

Islander 

Am. Indian/ 

Alaskan 

Native 

All 

Minorities* 

Schools with MSAP-Funded 

Program 

42.1% 24.3% 5.7% 0.5% 72.6% 

Schools without MSAP-Funded 

Program 

34.4% 22.6% 5.5% 0.6% 63.1% 

N = 55 districts 

Includes only schools in district that are at same grade levels as MSAP programs in district 

Source: NCES 1999–2000 Common Core of Data 

Exhibit A-II-16. 

Mean and median percentage of MSAP and non-MSAP students who are eligible for free 

or reduced price lunch: 1999–2000 

 

 MSAP Schools 

Non-MSAP 

Schools 

Mean Percent 60.3% 57.6% 

Median Percent 65.0% 60.3% 

N =55 Districts 

Source: NCES 1999–2000 Common Core of Data 
 Appendix II 

Appendix-II-6

Exhibit A-II-17. 

Mean and median percentage of MSAP and non-MSAP students who are limited English 

proficient (LEP): 1999–2000 

 

 MSAP Schools 

Non-MSAP 

Schools 

Mean Percent 14.4% 11.9% 

Median Percent 8.9 7.1 

N =24 Districts 

Source: various districts and state agencies supplemented with enrollment data from NCES 1999–2000 Common Core of 

Data 

Exhibit A-II-18. 

Mean and median percentage of MSAP and non-MSAP students who have an 

individualized education plan (IEP): 1999–2000 

 

 MSAP Schools Non-MSAP Schools 

Mean Percent 13.2% 12.1% 

Median Percent 12.9 12.5 

N =19 Districts 

Source: various districts and state agencies supplemented with enrollment data from NCES 1999–2000 Common Core of Data 

Exhibit A-II-19. 

Percentage of MSAP-supported schools with targeted and schoolwide Title I programs 

compared with non-MSAP schools in districts and schools nationally: 1999–2000 

 

Type of Title I Program MSAP Schools Non-MSAP Schools Nationwide 

Schoolwide 59.9% 43.1% 26.6% 

Targeted 11.2 7.1 32.4% 

 

Total Percent with Title I Program 71.1% 50.2% 59.0% 

(N of Schools) (152) (1,431) (1,186) 

N = 32 out of 57 MSAP districts 

Source: Data for schools in MSAP districts are from NCES 1999–2000 Common Core of Data. National data are for the 1997-1998 school 

year as reported from the Follow-up Public School Survey on Education Reform by Heid and Webber (1999). 

Exhibit A-II-20. 

Pupil-teacher ratios in MSAP and non-MSAP schools by grade level: 1999–2000 

MSAP Schools Non-MSAP 

Elementary 15.9 17.0 

Middle School 17.8 17.0 

High School 16.2 18.2 

N = 51 districts 

Source NCES 1999–2000 Common Core of Data 
Appendix III—Innovative Educational Practice and Support 

for Systemic Reform in MSAP Schools 

 Appendix III

 Appendix-III-1

Exhibit A-III-1. 

Principals’ familiarity with state’s mathematics curriculum framework, 2000–2001 

 Overall 

(%) 

(N = 257) 

Elementary 

(%) 

(N = 162) 

Middle 

(%) 

(N = 59) 

High 

(%) 

(N = 36) 

Not at All Familiar 0 0 0 0 

Somewhat Familiar 3 1 3 11 

Familiar 25 22 29 31 

Very Familiar 72 77 68 58 

Note: N excludes 6 combined-level schools. 

Source: MSAP Principal Survey, 2000–2001, Item 1 

Exhibit A-III-2. 

Principals’ familiarity with school’s scores on state’s statewide mathematics 

assessment, 2000–2001 

 Overall 

(%) 

(N = 254) 

Elementary 

(%) 

(N = 158) 

Middle 

(%) 

(N = 59) 

High 

(%) 

(N = 37) 

Not at All Familiar 0 0 0 0 

Somewhat Familiar 0 0 2 0 

Familiar 8 5 5 24 

Very Familiar 92 95 93 76 

Note: N excludes 6 combined-level schools. 

Source: MSAP Principal Survey, 2000–2001, Item 3 

Exhibit A-III-3. 

Principals’ familiarity with state’s language arts curriculum framework, 2000–2001 

 Overall 

(%) 

(N = 254) 

Elementary 

(%) 

(N = 161) 

Middle 

(%) 

(N = 57) 

High 

(%) 

(N = 36) 

Not at All Familiar 0 0 0 0 

Somewhat Familiar 3 1 4 11 

Familiar 19 16 23 25 

Very Familiar 78 83 74 64 

Note: N excludes 6 combined-level schools. 

Source: MSAP Principal Survey, 2000–2001, Item 1
Appendix III

Appendix-III-2

Exhibit A-III-4. 

Principals’ familiarity with school’s scores on state’s statewide language arts 

assessment, 2000–2001 

 Overall 

(%) 

(N = 254) 

Elementary 

(%) 

(N = 158) 

Middle 

(%) 

(N = 59) 

High 

(%) 

(N = 37) 

Not at All Familiar 0 0 0 0 

Somewhat Familiar 0 0 2 0 

Familiar 8 5 5 24 

Very Familiar 92 95 93 76 

Note: N excludes 6 combined-level schools. 

Source: MSAP Principal Survey, 2000–2001, Item 3

Exhibit A-III-5. 

Extent to which principals report that state’s mathematics curriculum framework was 

used to guide curriculum and instruction decisions at school, 2000–2001 

 Overall 

(%) 

(N = 255) 

Elementary 

(%) 

(N = 160) 

Middle 

(%) 

(N = 59) 

High 

(%) 

(N = 36) 

Not at All 0 0 0 0 

To Some Extent 11 9 14 14 

To a Great Extent 89 81 86 86 

Note: N excludes 6 combined-level schools. 

Source: MSAP Principal Survey, 2000–2001, Item 2

Exhibit A-III-6. 

Extent to which principals report that state’s mathematics statewide assessments was 

used to guide curriculum and instruction decisions at school, 2000–2001 

 Overall 

(%) 

(N = 252) 

Elementary 

(%) 

(N = 159) 

Middle 

(%) 

(N = 59) 

High 

(%) 

(N = 35) 

Not at All 0 0 0 0 

To Some Extent 12 10 10 20 

To a Great Extent 88 90 90 80 

Note: N excludes 6 combined-level schools. 

Source: MSAP Principal Survey, 2000–2001, Item 4
 Appendix III

 Appendix-III-3

Exhibit A-III-7. 

Extent to which principals report that state’s language arts curriculum framework was 

used to guide curriculum and instruction decisions at school, 2000–2001 

 Overall 

(%) 

(N = 253) 

Elementary 

(%) 

(N = 160) 

Middle 

(%) 

(N = 57) 

High 

(%) 

(N = 36) 

Not at All 0 0 0 0 

To Some Extent 9 7 14 14 

To a Great Extent 91 93 86 86 

Note: N excludes 6 combined-level schools. 

Source: MSAP Principal Survey, 2000–2001, Item 2

Exhibit A-III-8. 

Extent to which principals report that state’s language arts statewide assessments was 

used to guide curriculum and instruction decisions at school, 2000–2001 

 Overall 

(%) 

(N = 252) 

Elementary 

(%) 

(N = 158) 

Middle 

(%) 

(N = 59) 

High 

(%) 

(N = 35) 

Not at All 0 0 0 0 

To Some Extent 12 10 10 20 

To a Great Extent 88 90 90 80 

Note: N excludes 6 combined-level schools. 

Source: MSAP Principal Survey, 2000–2001, Item 4

Exhibit A-III-9. 

Extent to which project directors report that state’s curriculum frameworks, assessments 

and performance standards match MSAP schools’ instructional goals, 1999–2000 

 Mathematics 

(%) 

Language Arts

(%) 

Science 

(%) 

Social Studies 

(%) 

Not at All 2 2 7 9 

Only Slightly 0 0 0 2 

To Some Extent 9 9 16 22 

To a Great Extent 89 89 76 67 

N varies between 55 and 56 projects. 

Source: MSAP Project Survey, 1999–2000, Item 21 

Exhibit A-III-10. 

Extent to which principals report that state’s mathematics statewide assessments reflect 

the goals of the schools’ magnet program, 2000–2001 

 Overall 

(%) 

(N = 249) 

Elementary 

(%) 

(N = 155) 

Middle 

(%) 

(N = 59) 

High 

(%) 

(N = 35) 

Not at All 0 0 2 0 

To Some Extent 30 28 36 31 

To a Great Extent 69 72 63 69 

Note: N excludes 6 combined-level schools. 

Source: MSAP Principal Survey, 2000–2001, Item 5 
Appendix III

Appendix-III-4

Exhibit A-III-11. 

Extent to which principals report that state’s language arts statewide assessments 

reflect the goals of the schools’ magnet program, 2000–2001 

 Overall 

(%) 

(N = 249) 

Elementary 

(%) 

(N = 157) 

Middle 

(%) 

(N = 56) 

High 

(%) 

(N = 36) 

Not at All 1 1 2 0 

To Some Extent 28 24 34 36 

To a Great Extent 71 75 64 64 

Note: N excludes 6 combined-level schools. 

Source: MSAP Principal Survey, 2000–2001, Item 5 

Exhibit A-III-12. 

Type and frequency of technical assistance provided in 1998–1999 to 2000–2001 by 

MSAP project directors and other district-level MSAP staff 

1998–1999 2000–2001 

Once a 

month 

(%) 

Every two 

weeks 

(%) 

Once a 

week 

(%) 

Once a 

month 

(%) 

Every two 

weeks 

(%) 

Once a 

week 

(%) 

Curriculum and Instruction 

Designing Curriculum 34 30 35 40 30 30 

Planning Professional Development 43 24 33 57 27 16 

Developing Theme 40 27 33 55 22 24 

Designing Assessments 72 22 6 84 10 6 

Keeping Teachers Motivated 26 27 47 49 16 35 

Leadership 

Budgeting 16 16 69 31 18 51 

Helping Principals Lead 37 20 43 45 24 31 

Working with Parents 51 18 31 47 20 33 

N varies between 49 and 51 projects. 

Source: MSAP Project Survey, 1999–2000, Item 13; MSAP Project Survey, 2000–2001, Item 4 
 Appendix III

 Appendix-III-5

Exhibit A-III-13. 

Provision of technical assistance focused on curriculum and instruction by MSAP 

project directors and other district-level MSAP staff 

 1998–1999 2000–2001 

Scale score Frequency of provision Percent Frequency Percent Frequency 

1.0–1.25 Never 2 1 0 0 

1.26–1.75 0 0 0 0 

1.76–2.25 Less Than Once a Month 6 3 8 4 

2.26–2.75 10 5 6 3 

2.76–3.25 About Once a Month 16 8 42 21 

3.26–3.75 22 11 10 5 

3.76–4.25 About Once Every 2 Weeks 16 8 20 10 

4.26–4.75 12 6 12 6 

4.76–5.0 Once a Week or More 16 8 2 1 

Mean and Standard Deviation 3.64 0.89 3.37 0.84 

N = 51 projects 

Note: The technical assistance scale was created by averaging five technical assistance variables: 1998–1999=Q12e, Q12f, Q12g, 

Q12h, and Q12k; 2000–2001=Q4d, Q4e, Q4f, Q4g, and Q4j. 

Source: MSAP Project Survey, 1999–2000, Item 13; MSAP Project Survey, 2000–2001 
Appendix III

Appendix-III-6

Exhibit A-III-14. 

Provision of technical assistance focused on leadership by MSAP project directors and 

other district-level MSAP staff 

 1998–1999 2000–2001 

Scale score Frequency of provision Percent Frequency Percent Frequency 

1.0–1.25 Never 0 0 0 0 

1.26–1.75 0 0 0 0 

1.76–2.25 Less Than Once a Month 2 1 0 0 

2.26–2.75 4 2 10 5 

2.76–3.25 About Once a Month 10 5 8 4 

3.26–3.75 24 12 30 15 

3.76–4.25 About Once Every 2 Weeks 12 6 12 6 

4.26–4.75 30 15 34 17 

4.76–5.0 Once a Week or More 18 9 6 3 

Mean and Standard Deviation 4.00 0.75 3.83 0.75 

N = 51 projects 

Note: The technical assistance scale was created by averaging three technical assistance variables: 1998–1999=Q12c, Q12j, and 

Q12l; 2000–2001=Q4b, Q4l, and Q4k. 

Source: MSAP Project Survey, 1999–2000, Item 12; MSAP Project Survey, 2000–2001, Item 4 

Exhibit A-III-15. 

Principal reports of depth of magnet programs, 1999–2000 and 2000–2001 

1999–2000 2000–2001 

Middle 

(%) 

High 

(%) 

Middle 

(%) 

High 

(%) 

Special or Elective Courses Offered as Part of Magnet 

Program 70 81 74 91 

Students Can Pick a Theme or Area of Specialization 44 55 39 70 

(Of These) Students are Required to Select a 

Theme or Area of Specialization 71 67 79 67 

Students Required to Complete Courses in Addition to 

Those Required by Other Schools in the District 49 65 45 68 

N varies between 51 to 54 middle schools and between 32 and 34 high schools. 

Source: MSAP Principal Survey, 1999–2000, Items 48–50; MSAP Principal Survey, 2000–2001, Items 31–33 
 Appendix III

 Appendix-III-7

Exhibit A-III-16. 

Teacher reports of the magnet programs’ influence on instruction, 2000–2001 

 Not at all

(%) 

~ Greater 

extent ~ 

(%) 

~ Greater 

extent ~ 

(%) 

~ Greater 

extent ~ 

(%) 

To a great 

extent 

(%) 

Mean 

(standard 

deviation) 

Added New Topics 

Elementary Teachers 0 8 20 40 32 3.95 (0.93)

Secondary Teachers 14 12 23 42 9 3.21 (1.15)

Reading/Language Arts Teachers 4 9 24 38 24 3.73 (1.04)

Mathematics Teachers 2 14 16 46 23 3.69 (1.08)

Overall 6 10 21 41 22 3.71 (1.06)

Added More Advanced Work 

Elementary Teachers 2 5 27 35 32 3.90 (0.97)

Secondary Teachers 5 9 23 42 21 3.62 (1.15)

Reading/Language Arts Teachers 0 7 27 40 27 3.75 (1.16)

Mathematics Teachers 7 7 20 36 30 3.87 (0.89)

Overall 3 7 25 38 27 3.81 (1.03)

Changed Teaching Methods 

Elementary Teachers 0 8 17 42 33 4.00 (0.92)

Secondary Teachers 12 7 21 40 21 3.50 (1.26)

Reading/Language Arts Teachers 4 9 20 40 27 3.93 (1.03)

Mathematics Teachers 2 9 14 42 33 3.76 (1.09)

Overall 5 8 18 41 28 3.84 (1.06)

Changed Assessment Methods 

Elementary Teachers 5 15 27 30 23 3.52 (1.16)

Secondary Teachers 14 11 30 32 14 3.21 (1.29)

Reading/Language Arts Teachers 11 9 29 33 18 3.45 (1.21)

Mathematics Teachers 7 16 24 30 23 3.38 (1.21)

Overall 9 13 28 31 19 3.42 (1.20)

Increased the Use of Technology 

Elementary Teachers 0 5 15 17 63 4.38 (0.92)

Secondary Teachers 5 9 20 30 36 3.69 (1.14)

Reading/Language Arts Teachers 0 4 27 18 51 4.16 (1.12)

Mathematics Teachers 2 9 14 20 55 4.16 (0.98)

Overall 2 7 17 22 52 4.16 (1.04)

Increased Material that Connects 

(Reading/ Language Arts or Mathematics) 

with Other Subjects 

 

 

Elementary teachers 0 3 15 40 42 4.20 (0.82)

Secondary teachers 7 7 32 32 23 3.38 (1.18)

Reading/language arts teachers 2 2 24 40 31 3.91 (1.12)

Mathematics teachers 5 7 18 34 36 3.96 (0.93)

Overall 3 5 22 37 34 3.93 (1.02) 

N = 60 elementary teachers, 29 secondary teachers, 44 reading/language arts teachers, and 45 mathematics teachers 

Note: Response scale was from 1 to 5. 

Source: Elementary Mathematics Teacher Survey, 2000–2001, Item 33; Elementary Reading Teacher Survey, 2000–2001, Item 

38; Middle and High School Mathematics Teacher Survey, 2000–2001, Item 33; Middle and High School Language Arts Teacher 

Survey, 2000–2001, Item 32
Appendix III

Appendix-III-8

Exhibit A-III-17. 

Aspects of professional community in MSAP elementary schools 

 1999–2000 2000–2001 

 

Strongly 

Disagree

(%) 

Disagree 

(%) 

Agree

(%) 

Strongly 

Agree 

(%) 

Strongly

Disagree 

(%) 

Disagree

(%) 

Agree

(%)

Strongly

Agree 

(%) 

Rules for Student Behavior Consistently 

Enforced by Teachers 2 5 54 39 0 3 44 53 

Teachers Support Rule Enforcement by Principal 1 1 41 56 0 1 30 70 

Teachers Coordinate across Grades 1 13 62 24 0 7 54 39 

School Seems Like Big Family 1 6 49 43 1 5 44 51 

Staff Members Support Each Other 1 2 46 52 0 3 40 58 

Most Teachers Willing to Put in Extra Hours 1 5 35 58 0 4 30 66 

Most Teachers Share Beliefs about Mission 1 2 46 51 1 1 39 59 

Most Teachers Seek New Ideas 1 5 39 54 0 4 33 63 

Most Teachers Care about Students 1 1 24 73 0 0 0 77 

Administrators and Teachers Collaborate 2 1 41 56 1 1 32 66 

Ethnic Differences Among Staff Create Tensions 74 19 6 1 77 20 3 0 

Principal Has Confidence in Expertise of 

Teachers 1 2 52 45 0 2 32 66 

Principal Usually Looks Forward to Working 

Each Day 1 2 28 69 0 2 20 78 

Sometimes Principal Feels It's a Waste of Time 

To Do Best 80 16 2 2 84 12 1 3 

N varies between 147 and 152 schools. 

Source: MSAP Principal Survey, 1999–2000, Item 11; MSAP Principal Survey, 2000–2001, Item 27 
 Appendix III

 Appendix-III-9

Exhibit A-III-18. 

Aspects of professional community in MSAP middle schools 

 1999–2000 2000–2001 

 

Strongly 

Disagree

(%) 

Disagree 

(%) 

Agree

(%) 

Strongly 

Agree 

(%) 

Strongly

Disagree 

(%) 

Disagree

(%) 

Agree

(%)

Strongly

Agree 

(%) 

Rules for Student Behavior Consistently 

Enforced by Teachers 0 11 64 25 0 9 62 29 

Teachers Support Rule Enforcement by Principal 0 2 45 53 0 4 53 44 

Teachers Coordinate across Grades 2 9 62 27 2 4 65 29 

School Seems Like Big Family 0 15 57 28 2 6 55 38 

Staff Members Support Each Other 0 0 1 44 2 4 51 44 

Most Teachers Willing to Put in Extra Hours 2 4 53 42 2 11 40 47 

Most Teachers Share Beliefs about Mission 0 4 60 36 0 5 51 42 

Most Teachers Seek New Ideas 2 7 58 33 0 7 62 31 

Most Teachers Care about Students 2 0 45 53 2 4 42 53 

Administrators and Teachers Collaborate 0 2 51 47 0 4 49 47 

Ethnic Differences Among Staff Create Tensions 60 29 9 2 64 33 2 2 

Principal Has Confidence in Expertise of 

Teachers 0 2 58 40 2 2 45 51 

Principal Usually Looks Forward to Working 

Each Day 0 2 33 65 0 0 33 67 

Sometimes Principal Feels It's a Waste of Time 

To Do Best 84 15 2 0 76 18 4 2 

N varies between 53 and 55 schools. 

Source: MSAP Principal Survey, 1999–2000, Item 11; MSAP Principal Survey, 2000–2001, Item 27 
Appendix III

Appendix-III-10

Exhibit A-III-19. 

Aspects of professional community in MSAP high schools, 1999–2000 and 2000–2001 

 1999–2000 2000–2001 

 

Strongly 

Disagree

(%) 

Disagree 

(%) 

Agree

(%) 

Strongly 

Agree 

(%) 

Strongly

Disagree 

(%) 

Disagree

(%) 

Agree

(%)

Strongly

Agree 

(%) 

Rules for Student Behavior Consistently 

Enforced by Teachers 0 11 74 14 0 17 54 29 

Teachers Support Rule Enforcement by Principal 0 0 69 31 0 9 60 31 

Teachers Coordinate across Grades 0 20 69 11 0 23 63 14 

School Seems Like Big Family 0 14 63 23 0 26 43 31 

Staff Members Support Each Other 0 11 37 51 0 9 49 43 

Most Teachers Willing to Put in Extra Hours 0 9 37 54 0 9 43 49 

Most Teachers Share Beliefs about Mission 0 9 66 26 0 6 54 40 

Most Teachers Seek New Ideas 0 9 69 23 0 9 60 31 

Most Teachers Care about Students 0 6 29 66 0 3 49 49 

Administrators and Teachers Collaborate 0 6 49 46 0 6 57 37 

Ethnic Differences Among Staff Create Tensions 7 17 6 6 63 31 0 6 

Principal Has Confidence in Expertise of 

Teachers 0 3 40 57 0 3 43 54 

Principal Usually Looks Forward to Working 

Each Day 0 3 31 66 0 0 29 71 

Sometimes Principal Feels It's a Waste of Time 

To Do Best 76 21 3 0 71 26 0 3 

N varies between 34 and 35 schools. 

Source: MSAP Principal Survey, 1999–2000, Item 11; MSAP Principal Survey, 2000–2001, Item 27 
 Appendix III

 Appendix-III-11

Exhibit A-III-20. 

Extent of professional community in MSAP elementary, middle, and high schools, 1999–

2000 and 2000–2001 

 1999–2000 2000–2001 

Scale score 

range 

Professional 

community scale 

Elementary 

(%) 

Middle 

(%) 

High 

(%) 

Elementary 

(%) 

Middle 

(%) 

High 

(%) 

1.00–1.24 Strongly Disagree 1 0 0 0 0 0 

1.25–1.74 0 0 0 0 0 0 

1.75–2.24 Disagree 1 0 0 0 2 3 

2.25–2.74 2 5 8 3 7 9 

2.75–3.24 Agree 23 33 24 16 35 30 

3.25–3.74 50 53 54 47 42 41 

3.75–4.00 Strongly Agree 23 8 14 34 15 19 

Mean and Standard Deviation 3.44 (0.42) 3.33 (0.35) 3.32 (0.37) 3.55 (0.35) 3.33 (0.43) 3.31 (0.43) 

N = 161 elementary school, 60 middle school, and 37 high school 

Note: N excludes 6 principals from combined-level schools. 

Note: The professional community scale was created by averaging fourteen technical assistance variables: 1999–2000=Q11_1 

– Q11_3 and Q11_5 – Q11_15; 2000–2001=Q27a – Q27c and Q27e – Q27o. 

Source: MSAP Principal Survey, 1999–2000, Item 11; MSAP Principal Survey, 2000–2001, Item 27 

Exhibit A-III-21. 

Extent of professional community in case study MSAP and matched comparison 

schools, 1999–2000 and 2000–2001 

 1999–2000 2000–2001 

Scale score 

range 

Professional 

community scale 

MSAP 

(%) 

Comparison

(%) 

MSAP 

(%) 

Comparison

(%) 

1.00–1.24 Strongly Disagree 0 0 0 0 

1.25–1.74 0 0 0 0 

1.75–2.24 Disagree 0 0 0 0 

2.25–2.74 0 0 0 6 

2.75–3.24 Agree 14 44 24 63 

3.25–3.74 66 50 41 25 

3.75–4.00 Strongly Agree 21 6 35 6 

Mean and Standard Deviation 3.50 (0.27) 3.32 (0.27) 3.53 (0.35) 3.18 (0.35) 

N = 29 MSAP schools and 16 matched comparison schools 

Note: N excludes 6 principals from combined-level schools. 

Note: The professional community scale was created by averaging fourteen technical assistance variables: MSAP, 

1999–2000=Q11_1 – Q11_3 and Q11_5 – Q11_15; Non-MSAP, 1999–2000=Q8_1 – Q8_3 and Q8_5 – Q8_15; 

MSAP, 2000–2001=Q27a – Q27c and Q27e – Q27o; Non-MSAP, 2000–2001=Q22a – Q22c and Q22e – Q22o. 

Source: MSAP Principal Survey, 1999–2000, Item 11; Principal Survey—Non-MSAP School, 1999–2000, Item 8; 

MSAP Principal Survey, 2000–2001, Item 27; Principal Survey—Non-MSAP School, 2000–2001, Item 22 
Appendix III

Appendix-III-12

Exhibit A-III-22. 

Student behavior problems in MSAP elementary, middle, and high schools, 1999–2000 

and 2000-2001 

 1999–2000 2000–2001 

 

None 

(%) 

Minor 

(%) 

Moderate 

(%) 

Serious 

(%) 

None 

(%) 

Minor 

(%) 

Moderate 

(%) 

Serious 

(%) 

Physical Conflicts among Students 

Elementary Schools 46 44 13 1 46 44 6 3 

Middle Schools 22 56 20 2 24 55 20 2 

High Schools 37 57 6 0 26 54 17 3 

Robbery or Theft 

Elementary Schools 63 34 2 1 72 26 1 1 

Middle Schools 43 46 7 4 48 50 2 0 

High Schools 37 63 0 0 34 54 11 0 

Vandalism 

Elementary Schools 59 36 5 1 68 30 2 0 

Middle Schools 46 45 9 0 50 46 4 0 

High Schools 43 49 6 3 31 57 11 0 

Verbal Abuse of Teachers 

Elementary Schools 65 33 2 0 72 26 2 0 

Middle Schools 39 39 14 7 41 41 16 2 

High Schools 43 40 17 0 40 43 11 6 

Physical Abuse of Teachers 

Elementary Schools 91 7 1 0 92 7 1 0 

Middle Schools 85 9 6 0 82 18 0 0 

High Schools 86 14 0 0 89 9 0 0 

Disrespect for Teachers 

Elementary Schools 41 51 7 1 46 47 7 1 

Middle Schools 25 42 25 7 25 45 24 5 

High Schools 35 47 12 6 38 35 24 3 

N varies between 144 and 152 for elementary schools, between 55 and 56 for middle schools and between 34 and 35 for high 

schools. 

Source: MSAP Principal Survey, 1999–2000, Item 12; MSAP Principal Survey, 2000–2001, Item 28 
 Appendix III

 Appendix-III-13

Exhibit A-III-23. 

Extent of student behavior problems in MSAP elementary, middle, and high schools, 

1999–2000 and 2000–2001 

1999–2000 2000–2001 

Scale Score 

Range School Climate Scale 

Elementary 

(%) 

Middle 

(%) 

High 

(%) 

Elementary 

(%) 

Middle 

(%) 

High 

(%) 

1.00–1.24 Not a Problem 36 25 30 45 23 24 

1.25–1.74 38 25 22 38 33 27 

1.75–2.24 Minor Problem 22 28 43 15 30 35 

2.25–2.74 2 16 5 1 11 8 

2.75–3.24 Moderate Problem 1 7 0 1 3 5 

3.25–3.74 0 0 0 0 0 0 

3.75–4.00 Serious Problem 0 0 0 0 0 0 

Mean and Standard Deviation 1.47 (0.41) 1.78 (0.59) 1.64 (0.44) 1.41 (0.40) 1.70 (0.51) 1.74 (0.55)

N = 161 elementary schools, 61 middle schools, 37 high schools 

Note: N excludes 6 principals from combined-level schools. 

Note: The school climate scale was created by averaging six school climate variables: 1999–2000=Q12_6, Q12_7, Q12_8, 

Q12_13, Q12_14, Q12_15; 2000–2001= Q28e, Q28f, Q28g, Q28m, Q28n, Q28o. 

Source: MSAP Principal Survey, 1999–2000, Item 12; MSAP Principal Survey, 2000–2001, Item 28 

Exhibit A-III-24. 

Extent of student behavior problems in case study MSAP and matched comparison 

schools, 1999–2000 and 2000–2001 

1999–2000 2000–2001 

Scale Score 

Range School Climate Scale

MSAP 

(%) 

Comparison

(%) 

MSAP 

(%) 

Comparison

(%) 

1.00–1.24 Not a Problem 28 25 41 19 

1.25–1.74 38 19 34 31 

1.75–2.24 Minor Problem 24 38 24 31 

2.25–2.74 7 6 0 13 

2.75–3.24 Moderate Problem 3 13 0 6 

3.25–3.74 0 0 0 0 

3.75–4.00 Serious Problem 0 0 0 0 

Mean and Standard Deviation 1.61 (0.52) 1.81 (0.60) 1.44 (0.35) 1.84 (0.57) 

N = 29 elementary schools and 16 matched comparison schools 

Note: The school climate scale was created by averaging six school climate variables: MSAP, 1999–2000=Q12_6, 

Q12_7, Q12_8, Q12_13, Q12_14, Q12_15; Non-MSAP, 1999–2000=Q9_6, Q9_7, Q9_8, Q9_13, Q9_14, Q9_15; 

MSAP, 2000–2001= Q28e, Q28f, Q28g, Q28m, Q28n, Q28o; Non-MSAP, 2000–2001= Q23e, Q23f, Q23g, Q23m, 

Q23n, Q23o. 

Source: MSAP Principal Survey, 1999–2000, Item 12; MSAP Principal Survey, 2000–2001, Item 28; Principal 

Survey—Non-MSAP School, 1999–2000, Item 9; Principal Survey—Non-MSAP School, 2000–2001, Item 23 
Appendix III

Appendix-III-14

Exhibit A-III-25. 

Extent of student disengagement problems in case study MSAP and matched 

comparison schools, 1999–2000 and 2000–2001 

1999–2000 2000–2001 

Scale Score 

Range School Climate Scale

MSAP 

(%) 

Comparison

(%) 

MSAP 

(%) 

Comparison

(%) 

1.00–1.24 Not a Problem 3 0 10 0 

1.25–1.74 21 6 10 6 

1.75–2.24 Minor Problem 17 13 24 13 

2.25–2.74 31 31 38 25 

2.75–3.24 Moderate Problem 14 38 7 31 

3.25–3.74 14 13 10 25 

3.75–4.00 Serious Problem 0 0 0 0 

Mean and Standard Deviation 2.30 (0.71) 2.66 (0.50) 2.23 (0.69) 2.73 (0.66) 

N = 29 MSAP schools and 16 matched comparison schools 

Note: The school climate scale was created by averaging eight school climate variables: MSAP, 1999–2000=Q12_1, 

Q12_2, Q12_3, Q12_17, Q12_18, Q12_19, Q12_20, Q12_21; Non-MSAP, 1999–2000=Q9_1, Q9_2, Q9_3, Q9_17, 

Q9_18, Q9_19, Q9_20, Q9_21; MSAP, 2000–2001= Q28a, Q28b, Q28c, Q28q, Q28r, Q28s, Q28t, Q28u; Non-MSAP, 

2000–2001= Q23a, Q23b, Q23c, Q23q, Q23r, Q23s, Q23t, Q23u. 

Source: MSAP Principal Survey, 1999–2000, Item 12; MSAP Principal Survey, 2000–2001, Item 28; Principal 

Survey—Non-MSAP School, 1999–2000, Item 9; Principal Survey—Non-MSAP School, 2000–2001, Item 23 

Exhibit A-III-26. 

Professional development areas emphasized in MSAP elementary schools, 1999–2000 

and 2000–2001 

 1999–2000 2000–2001 

 

No 

emphasis 

(%) 

Little 

Emphasis 

(%) 

Moderate 

Emphasis 

(%) 

Major 

Emphasis

(%) 

No 

emphasis 

(%) 

Little 

Emphasis 

(%) 

Moderate 

Emphasis 

(%) 

Major 

Emphasis

(%) 

State or District Curriculum and 

Performance Standards 0 2 26 72 0 1 19 81 

Student Performance 

Assessments 0 9 40 51 1 5 28 66 

Addressing the Needs of 

Students with Limited English 

Proficiency or from Diverse 

Cultural Backgrounds 

13 26 35 26 9 18 42 32 

Addressing the Needs of 

Students with Disabilities 3 23 42 32 3 15 44 38 

N = 151 schools 

Source: MSAP Principal Survey, 1999–2000, Item 29; MSAP Principal Survey, 2000–2001, Item 17 
 Appendix III

 Appendix-III-15

Exhibit A-III-27. 

Professional development areas emphasized in MSAP middle schools, 1999–2000 and 

2000–2001 

 1999–2000 2000–2001 

 

No 

emphasis 

(%) 

Little 

Emphasis 

(%) 

Moderate 

Emphasis 

(%) 

Major 

Emphasis

(%) 

No 

emphasis 

(%) 

Little 

Emphasis 

(%) 

Moderate 

Emphasis 

(%) 

Major 

Emphasis

(%) 

State or District Curriculum and 

Performance Standards 0 0 21 79 0 0 33 67 

Student Performance 

Assessments 0 9 44 47 0 4 47 49 

Addressing the Needs of 

Students with Limited English 

Proficiency or from Diverse 

Cultural Backgrounds 

16 27 30 27 7 27 34 32 

Addressing the Needs of 

Students with Disabilities 0 14 49 37 2 14 47 37 

N = 57 schools 

Source: MSAP Principal Survey, 1999–2000, Item 29; MSAP Principal Survey, 2000–2001, Item 17 

Exhibit A-III-28. 

Professional development areas emphasized in MSAP high schools, 1999–2000 and 

2000–2001 

 1999–2000 2000–2001 

 

No 

emphasis 

(%) 

Little 

Emphasis 

(%) 

Moderate 

Emphasis 

(%) 

Major 

Emphasis

(%) 

No 

emphasis 

(%) 

Little 

Emphasis 

(%) 

Moderate 

Emphasis 

(%) 

Major 

Emphasis

(%) 

State or District Curriculum and 

Performance Standards 0 9 26 66 0 3 26 71 

Student Performance 

Assessments 6 3 54 37 0 14 29 57 

Addressing the Needs of 

Students with Limited English 

Proficiency or from Diverse 

Cultural Backgrounds 

9 29 40 23 6 23 31 40 

Addressing the Needs of 

Students with Disabilities 0 9 54 37 0 11 26 63 

N = 35 schools 

Source: MSAP Principal Survey, 1999–2000, Item 29; MSAP Principal Survey, 2000–2001, Item 17 
Appendix III

Appendix-III-16

Exhibit A-III-29. 

Extent of emphasis on professional development focused on high standards for all 

students in MSAP elementary, middle, and high schools, 1999–2000 and 2000–2001 

 1999–2000 2000–2001 

Scale score 

range 

Professional 

development scale 

Elementary

(%) 

Middle 

(%) 

High 

(%) 

Elementary

(%) 

Middle 

(%) 

High 

(%) 

1.00–1.24 No Emphasis 0 0 0 0 0 0 

1.25–1.74 0 0 0 0 0 0 

1.75–2.24 Little Emphasis 4 0 3 2 2 0 

2.25–2.74 11 13 8 6 10 5 

2.75–3.24 Moderate Emphasis 24 27 27 20 23 27 

3.25–3.74 49 47 59 56 55 32 

3.75–4.00 Great Deal of Emphasis 13 13 3 16 11 35 

Mean and Standard Deviation 3.23 (0.54) 3.24 (0.48) 3.23 (0.48) 3.39 (0.48) 3.29 (0.51) 3.45 (0.51)

N = 160 elementary school, 62 middle school, and 37 high school 

Note: N excludes 6 principals from combined-level schools. 

Note: The professional development scale was created by averaging four professional development variables: 1999–

2000=Q29_3, Q29_5, Q29_7, and Q29_9; 2000–2001=Q17c, Q17e, Q17g, and Q17h. 

Source: MSAP Principal Survey, 1999–2000, Item 29; MSAP Principal Survey, 2000–2001, Item 17

Exhibit A-III-30. 

Parental involvement in school activities at MSAP elementary schools, 1999–2000 and 

2000–2001 

 1999–2000 2000–2001 

 

Few 

(%) 

Less 

Than 

Half 

(%) 

More 

Than 

Half 

(%) 

Most 

(%) 

Few 

(%) 

Less 

Than 

Half 

(%) 

More 

Than 

Half 

(%) 

Most 

(%) 

Special Events 

Open House or Back-to-School Night 4 19 40 35 1 15 45 38 

Regularly Scheduled Schoolwide ParentTeacher Conferences 3 15 31 45 1 10 28 56 

Special Subject-Area Events 7 39 34 15 7 30 41 16 

Parent Education Workshops or Courses 37 44 3 1 38 40 7 2 

Parent-Child Learning Activities at School 32 26 12 5 25 36 20 3 

At-Home Learning Activities to Support 

School Objectives 29 30 18 13 21 32 23 15 

School Governance 

Instructional Issues 65 17 4 1 62 23 6 1 

Governance 63 12 2 1 68 17 5 1 

Budget Decisions 62 9 1 3 68 13 3 0 

N varies between 34 and 35 schools. 

Note: where the row does not add up to 100, the difference is the percent of schools that reported not sponsoring that 

activity. 

Source: MSAP Principal Survey, 1999–2000, Item 34; MSAP Principal Survey, 2000–2001, Item 18 
 Appendix III

 Appendix-III-17

Exhibit A-III-31. 

Parental involvement in school activities at MSAP middle schools, 1999–2000 and 

2000–2001 

 1999–2000 2000–2001 

 

Few 

(%) 

Less 

Than 

Half 

(%) 

More 

Than 

Half 

(%) 

Most 

(%) 

Few 

(%) 

Less 

Than 

Half 

(%) 

More 

Than 

Half 

(%) 

Most 

(%) 

Special Events 

Open House or Back-to-School Night 5 22 49 24 4 31 44 22 

Regularly Scheduled Schoolwide ParentTeacher Conferences 8 38 34 19 6 28 38 23 

Special Subject-Area Events 13 38 25 15 10 37 27 21 

Parent Education Workshops or Courses 36 30 4 0 43 25 6 2 

Parent-Child Learning Activities at School 20 22 2 2 35 25 6 4 

At-Home Learning Activities to Support 

School Objectives 11 28 13 6 21 26 15 6 

School Governance 

Instructional Issues 60 11 2 4 57 19 8 4 

Governance 74 13 0 0 66 19 6 2 

Budget Decisions 67 9 0 0 57 11 4 2 

N varies between 34 and 35 schools. 

Note: where the row does not add up to 100, the difference is the percent of schools that reported not sponsoring that 

activity. 

Source: MSAP Principal Survey, 1999–2000, Item 34; MSAP Principal Survey, 2000–2001, Item 18 
Appendix III

Appendix-III-18

Exhibit A-III-32. 

Parental involvement in school activities at MSAP high schools, 1999–2000 and 2000–

2001 

 1999–2000 2000–2001 

 

Few 

(%) 

Less 

Than 

Half 

(%) 

More 

Than 

Half 

(%) 

Most 

(%) 

Few 

(%) 

Less 

Than 

Half 

(%) 

More 

Than 

Half 

(%) 

Most 

(%) 

Special Events 

Open House or Back-to-School Night 6 42 33 15 3 52 30 15 

Regularly Scheduled Schoolwide ParentTeacher Conferences 6 21 21 15 6 30 15 12 

Special Subject-Area Events 9 56 25 3 16 44 22 9 

Parent Education Workshops or Courses 32 35 2 0 35 21 12 0 

Parent-Child Learning Activities at School 9 19 6 0 16 13 9 0 

At-Home Learning Activities to Support 

School Objectives 28 16 3 6 25 22 3 3 

School Governance 

Instructional Issues 52 27 3 0 67 15 0 0 

Governance 62 15 6 0 50 15 6 0 

Budget Decisions 50 6 3 0 47 9 3 0 

N varies between 34 and 35 schools. 

Note: where the row does not add up to 100, the difference is the percent of schools that reported not sponsoring that 

activity. 

Source: MSAP Principal Survey, 1999–2000, Item 34; MSAP Principal Survey, 2000–2001, Item 18 
 Appendix III

 Appendix-III-19

Exhibit A-III-33. 

Extent of parental involvement in special events at MSAP elementary, middle, and high 

schools, 1999–2000 and 2000–2001 

1999–2000 2000–2001 

Scale Score 

Range 

Parental Involvement 

Scale 

Elementary 

(%) 

Middle 

(%) 

High 

(%) 

Elementary 

(%) 

Middle 

(%) 

High 

(%) 

0.00–0.24 None 1 0 0 0 2 0 

0.25–0.75 1 0 17 0 2 19 

0.75–1.24 Few 6 20 19 6 13 25 

1.25–1.74 15 21 25 11 28 25 

1.75–2.24 Less than Half 32 34 28 27 26 14 

2.25–2.74 25 15 8 27 15 6 

2.75–3.24 More than Half 16 5 3 23 8 8 

3.25–3.74 3 5 0 4 7 3 

3.75–4.00 Most 1 0 0 2 0 0 

Mean and Standard Deviation 2.20 (0.67) 1.85 (0.65) 1.51 (0.69) 2.34 (0.63) 1.92 (0.76) 1.50 (0.79)

N = 161 elementary schools, 61 middle schools, 36 high schools 

Note: N excludes 6 principals from combined-level schools. 

Note: This parental involvement scale was created by averaging six parental involvement variables: 1999–2000=Q18a – Q18c, 

Q18f, Q18h, and Q18n; 2000–2001= Q28a – Q28c, Q28f, Q28h, and Q28n. 

Source: MSAP Principal Survey, 1999–2000, Item 18; MSAP Principal Survey, 2000–2001, Item 28 

Exhibit A-III-34. 

Extent of parental involvement in special events at case study MSAP and matched 

comparison schools, 1999–2000 and 2000–2001 

1999–2000 2000–2001 

Scale Score 

Range 

Parental Involvement 

Scale 

MSAP 

(%) 

Comparison

(%) 

MSAP 

(%) 

Comparison

(%) 

0.00–0.24 None 0 0 4 0 

0.25–0.75 0 6 4 0 

0.75–1.24 Few 7 13 0 31 

1.25–1.74 14 25 18 38 

1.75–2.24 Less than Half 21 44 18 13 

2.25–2.74 32 13 14 13 

2.75–3.24 More than Half 21 0 36 6 

3.25–3.74 0 0 7 0 

3.75–4.00 Most 4 0 0 0 

Mean and Standard Deviation 2.28 (0.68) 1.68 (0.55) 2.33 (0.82) 1.64 (0.62) 

N = 28 MSAP schools and 16 matched comparison schools 

Note: This parental involvement scale was created by averaging six parental involvement variables: MSAP, 1999–2000=Q18a – 

Q18c, Q18f, Q18h, and Q18n; Non-MSAP, 1999–2000=Q28a – Q28c, Q28f, Q28h, and Q28n; MSAP, 2000–2001= Q28a – 

Q28c, Q28f, Q28h, and Q28n; Non-MSAP, 2000–2001= Q18a – Q18c, Q18f, Q18h, and Q18n. 

Source: MSAP Principal Survey, 1999–2000, Item 18; Principal Survey—Non-MSAP School, 1999–2000, Item 28; MSAP 

Principal Survey, 2000–2001, Item 28; Principal Survey—Non-MSAP School, 2000–2001, Item 18 
Appendix III

Appendix-III-20

Exhibit A-III-35. 

Extent of parental involvement in school governance at MSAP elementary, middle, and 

high schools, 1999–2000 and 2000–2001 

1999–2000 2000–2001 

Scale Score 

Range 

Parental 

Involvement Scale 

Elementary 

(%) 

Middle 

(%) 

High 

(%) 

Elementary 

(%) 

Middle 

(%) 

High 

(%) 

0.00–0.24 None 8 7 3 4 5 8 

0.25–0.75 26 31 39 18 25 33 

0.75–1.24 Few 46 44 33 46 42 36 

1.25–1.74 9 10 17 18 11 11 

1.75–2.24 Less than Half 8 7 3 8 5 6 

2.25–2.74 3 2 3 4 7 6 

2.75–3.24 More than Half 1 0 3 1 2 0 

3.25–3.74 1 0 0 0 0 0 

3.75–4.00 Most 1 0 0 0 2 0 

Mean and Standard Deviation 1.01 (0.63) 0.93 (0.49) 1.01 (0.64) 1.15 (0.57) 1.12 (0.74) 0.92 (0.62)

N = 160 elementary schools, 59 middle schools, 36 high schools 

Note: N excludes 6 principals from combined-level schools. 

Note: This parental involvement scale was created by averaging three parental involvement variables: 1999–2000=Q18k, 

Q18l, and Q18m; 2000–2001= Q28k, Q28l, and Q28m. 

Source: MSAP Principal Survey, 1999–2000, Item 18; MSAP Principal Survey, 2000–2001, Item 28 
 Appendix III

 Appendix-III-21

Exhibit A-III-36. 

Mathematics teachers’ usage of higher order thinking skills instructional methods: case 

study MSAP and matched comparison schools, 2000–2001 

 Case Study MSAP schools Matched Comparison Schools 

 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Write a Few Sentences about Their 

Mathematics Work 

Elementary Teachers 19 23 46 12 28 28 29 15 

Secondary Teachers 17 44 28 11 26 52 19 4 

Talk to the Class about Their 

Mathematics Projects 

Elementary Teachers 11 15 23 51 10 10 28 51 

Secondary Teachers 17 15 26 43 15 22 37 26 

Write Reports or Do Math Projects 

Elementary Teachers 44 46 7 2 70 24 6 0 

Secondary Teachers 33 57 9 0 67 26 4 4 

Work and Discuss Math Problems 

that Reflect Real-Life Situations 

Elementary Teachers 2 17 38 43 6 7 34 52 

Secondary Teachers 2 19 41 39 7 19 67 7 

Use a Computer 

Elementary Teachers 5 8 36 51 16 6 37 41 

Secondary Teachers 37 33 20 9 48 33 11 7 

Work on Problems for which There 

is No Immediately Obvious Method 

or Solution 

Elementary Teachers 26 33 29 12 35 32 29 3 

Secondary Teachers 22 41 26 11 41 44 11 4 

Work on Interdisciplinary Lessons 

Elementary Teachers 14 24 27 35 15 16 22 47 

Secondary Teachers 39 43 11 7 70 19 11 0 

N varies between 81 and 83 for MSAP elementary teachers and between 67 and 68 for non-MSAP elementary teachers. 

N = 54 MSAP secondary teachers and 27 non-MSAP secondary teachers 

Source: Elementary Mathematics Teacher Survey, 2000–2001, Item 23; Elementary Mathematics Teacher Survey (non-MSAP), 

2000–2001, Item 17; Middle and High School Mathematics Teacher Survey, 2000–2001, Item 23; Middle and High School 

Mathematics Teacher Survey (non-MSAP) Survey, 2000–2001, Item 17 
Appendix III

Appendix-III-22

Exhibit A-III-37. 

Elementary reading/language arts teachers’ usage of higher order thinking skills 

instructional methods: case study MSAP and matched comparison schools, 2000–2001 

 Case Study MSAP schools Matched Comparison Schools 

 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Ask Students to Talk with Each Other 

about What They Have Read 0 7 38 55 5 7 52 36 

Ask Students to Write about 

Something They Have Read 3 7 48 43 5 9 46 40 

Ask Students to Discuss Different 

Interpretations of What They Have 

Read 

11 19 38 32 14 23 42 21 

Ask Students to Explain or Support 

Their Understanding of What They 

Have Read 

4 3 31 63 5 12 30 53 

N varies between 72 and 75 for MSAP teachers and between 56 and 57 for non-MSAP teachers. 

Source: Elementary Reading Teacher Survey, 2000–2001, Item 25; Elementary Reading Teacher Survey (non-MSAP) Survey, 2000–

2001, Item 19 

Exhibit A-III-38. 

Secondary reading/language arts teachers’ usage of higher order thinking skills 

instructional methods: case study MSAP and matched comparison schools, 2000–2001 

 Case Study MSAP schools Matched Comparison Schools 

 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Ask Students to Discuss Different 

Interpretations of What They Have 

Read 

7 11 33 49 3 16 52 29 

Ask Students to Explain or Support 

Their Understanding of What They 

Have Read 

4 2 33 60 0 6 35 58 

N = 45 MSAP teachers and 31 non-MSAP teachers 

Source: Middle and High School Language Arts Teacher Survey, 2000–2001, Item 23; Middle and High School Language Arts 

Teacher Survey (non-MSAP), 2000–2001, Item 17 
 Appendix III

 Appendix-III-23

Exhibit A-III-39. 

Frequency with which mathematics teachers use computers for specific instructional 

purposes: case study MSAP and matched comparison schools, 2000–2001 

 Case Study MSAP schools Matched Comparison Schools 

 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Mastering Recently Learning Skills 

Elementary Teachers 7 33 39 21 12 33 41 14 

Secondary Teachers 51 33 13 3 47 47 6 0 

Remediating Skills not Learned 

Well 

Elementary Teachers 9 24 46 21 9 28 47 16 

Secondary Teachers 59 26 15 0 35 59 6 0 

Expressing Themselves in Writing 

Elementary Teachers 33 20 30 17 42 21 26 12 

Secondary Teachers 51 38 5 5 82 6 12 0 

Communicating Electronically with 

Other People 

Elementary Teachers 79 11 7 4 86 10 5 0 

Secondary Teachers 64 26 5 5 82 6 12 0 

Finding Out about Ideas or 

Information 

Elementary Teachers 18 34 33 15 33 41 24 2 

Secondary Teachers 21 64 8 8 53 35 12 0 

Engaging in Simulations or 

Explorations of Ideas 

Elementary Teachers 23 44 24 9 47 23 26 5 

Secondary Teachers 55 23 18 5 59 35 6 0 

Analyzing Information 

Elementary Teachers 36 38 18 8 44 19 33 5 

Secondary Teachers 44 33 18 5 59 24 12 6 

Presenting Information to an 

Audience 

Elementary Teachers 55 26 12 7 77 12 9 2 

Secondary Teachers 51 38 5 5 76 24 0 0 

Improving Computer skills 

Elementary Teachers 8 22 39 31 21 14 47 19 

Secondary Teachers 38 40 20 2 53 41 6 0 

N varies between 74 and 76 for elementary MSAP teachers, varies between 42 and 43 for elementary non-MSAP teachers, varies 

between 39 and 40 secondary MSAP teachers and equals 17 secondary non-MSAP teachers. 

Source: Elementary Mathematics Teacher Survey, 2000–2001, Item 30; Elementary Mathematics Teacher Survey (non-MSAP), 

2000–2001, Item 24; Middle and High School Mathematics Teacher Survey, 2000–2001, Item 30; Middle and High School 

Mathematics Teacher Survey (non-MSAP) Survey, 2000–2001, Item 24 
Appendix III

Appendix-III-24

Exhibit A-III-40. 

Frequency with which reading or language arts teachers use computers for specific 

instructional purposes: case study MSAP and matched comparison schools, 2000–2001 

 Case Study MSAP schools Matched Comparison Schools 

 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Mastering Recently Learning Skills 

Elementary Teachers 16 22 45 16 26 25 38 21 

Secondary Teachers 51 37 11 0 45 35 15 5 

Remediating Skills not Learned 

Well 

Elementary Teachers 15 19 50 16 28 19 36 17 

Secondary Teachers 66 26 6 3 67 24 5 5 

Expressing Themselves in Writing 

Elementary Teachers 24 28 31 18 37 35 19 10 

Secondary Teachers 14 54 29 3 33 48 19 0 

Communicating Electronically with 

Other People 

Elementary Teachers 70 17 9 5 89 2 2 8 

Secondary Teachers 78 17 6 0 76 19 5 0 

Finding out About Ideas or 

Information 

Elementary Teachers 16 35 34 15 37 31 25 8 

Secondary Teachers 11 56 28 6 29 62 5 5 

Engaging in Simulations or 

Explorations of Ideas 

Elementary Teachers 24 37 31 8 46 25 25 4 

Secondary Teachers 41 53 6 0 52 33 10 5 

Analyzing Information 

Elementary Teachers 38 33 23 6 53 23 19 7 

Secondary Teachers 42 36 17 6 57 29 5 10 

Presenting Information to an 

Audience 

Elementary Teachers 53 25 19 3 71 17 10 2 

Secondary Teachers 47 42 11 0 38 52 10 0 

Improving Computer Skills 

Elementary Teachers 13 15 44 28 13 17 52 17 

Secondary Teachers 47 33 19 0 43 38 19 0 

N varies between 66 and 68 for elementary MSAP teachers, between 52 and 53 for elementary non-MSAP teachers, between 34 and 

36 for secondary MSAP teachers, and between 20 and 21 for secondary non-MSAP teachers. 

Source: Elementary Reading Teacher Survey, 2000–2001, Item 37; Elementary Reading Teacher Survey (non-MSAP), 2000–2001, 

Item 31; Middle and High School Language Arts Teacher Survey, 2000–2001, Item 31; Middle and High School Language Arts 

Teacher Survey (non-MSAP) Survey, 2000–2001, Item 25 
 Appendix III

 Appendix-III-25

Exhibit A-III-41. 

Mathematics teachers’ approaches to assessing student academic progress: case study 

MSAP and matched comparison schools, 2000–2001 

 Case Study MSAP schools Matched Comparison Schools 

 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Multiple-Choice Tests 

Elementary Teachers 39 8 36 17 32 40 41 16 

Secondary Teachers 23 28 46 4 15 22 48 15 

Problem Sets 

Elementary Teachers 16 5 36 43 18 3 33 46 

Secondary Teachers 8 47 45 0 11 67 22 0 

Short Written Responses 

Elementary Teachers 31 16 34 19 38 19 26 16 

Secondary Teachers 17 27 38 17 48 15 37 0 

Individual or Group Projects or 

Presentations 

Elementary Teachers 33 22 27 18 50 13 29 7 

Secondary Teachers 9 43 40 8 56 22 22 0 

Portfolio Collections of Each 

Student’s Work 

Elementary Teachers 30 23 26 21 40 7 25 28 

Secondary Teachers 40 21 23 17 67 19 7 7 

N varies between 80 and 83 for elementary MSAP teachers, varies between 67 and 68 for elementary non-MSAP teachers, varies 

between 51 and 53 secondary MSAP teachers, and equals 27 for secondary non-MSAP teachers. 

Source: Elementary Mathematics Teacher Survey, 2000–2001, Item 30; Elementary Mathematics Teacher Survey (non-MSAP), 

2000–2001, Item 24; Middle and High School Mathematics Teacher Survey, 2000–2001, Item 30; Middle and High School 

Mathematics Teacher Survey (non-MSAP) Survey, 2000–2001, Item 24 
Appendix III

Appendix-III-26

Exhibit A-III-42. 

Reading/language arts teachers’ approaches to assessing student academic progress: 

case study MSAP and matched comparison schools, 2000–2001 

 Case Study MSAP schools Matched Comparison Schools 

 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Never or 

Hardly 

Ever 

(%) 

Once or 

Twice a 

Month 

(%) 

Once or 

Twice a 

Week 

(%) 

Almost 

Every 

Day 

(%) 

Multiple-Choice Tests 

Elementary Teachers 25 4 45 26 18 5 41 36 

Secondary Teachers 16 11 52 20 16 6 47 31 

Short-Answer Tests 

Elementary Teachers 15 4 28 53 21 7 37 35 

Secondary Teachers 2 9 61 27 3 16 59 22 

Paragraph-Level Written Responses 

about What Students Have Read 

Elementary Teachers 25 11 27 37 23 12 28 37 

Long Essays 

Secondary Teachers 18 18 59 5 19 28 50 3 

Individual or Group Projects or 

Presentations 

Elementary Teachers 16 17 60 7 21 21 45 13 

Reading Portfolios 

Elementary Teachers 49 11 20 20 40 18 23 19 

Individual or Group Projects, 

Presentations, or Portfolios 

Secondary Teachers 7 27 57 9 6 31 59 3 

N varies between 71 and 75 elementary MSAP teachers, varies between 55 and 56 elementary non-MSAP teachers, equals 44 

secondary MSAP teachers and equals 32 secondary non-MSAP teachers. 

Source: Elementary Reading Teacher Survey, 2000–2001, Item 26; Elementary Reading Teacher Survey (non-MSAP), 2000–2001, 

Item 20; Middle and High School Language Arts Teacher Survey, 2000–2001, Item 28; Middle and High School Language Arts 

Teacher Survey (non-MSAP) Survey, 2000–2001, Item 22 
 Appendix III

 Appendix-III-27

Exhibit A-III-43. 

Average percentage of MSAP elementary, middle, and high schools that have provided 

information to other schools or other audiences 

 1999–2000 2000–2001 

Elementary

(%) 

Middle 

(%) 

High 

(%) 

Elementary

(%) 

Middle 

(%) 

High 

(%) 

Provided information, hosted visits, or provided 

professional development opportunities to educators 

from other schools in my district, state, or out of state 

66 71 80 72 79 80 

Received requests for information or permission to 

visit from reporters, researchers, or any other 

individuals or groups besides educators or parents 

72 80 79 82 78 79 

N = 146 and 147 elementary schools, 56 and 55 middle schools, and 35 and 34 high schools 

Note: N excludes 6 principals from combined-level schools. 

Source: MSAP Principal Survey, 1999–2000, Items 37–38; MSAP Principal Survey, 2000–2001, Item 21–22 
Appendix III

Appendix-III-28Exhibit A-III-44. List of survey items composing the scales measuring the components of the conceptual framework 

Measure Name Survey Items Included in Measure 

Survey 

Respondents 

Years 

Asked 

(Scale 

Reliability) 

External Supports for Change Alignment with standards and assessments Influence of state standards on MSAP themes and goals 

To what extent have the state frameworks, assessments, and performance standards influenced the themes and goals of your MSAP schools? 

1. Mathematics 2. Language Arts 3. Science 4. Social studies 

Response scale: 1=Not at all, 2=Only slightly, 3=Somewhat, 4=To a great extent 

MSAP Projects 1999–2000 

(0.89); 2000–2001 (0.95) 

Coordination with other reformsCoordination between MSAP project director and other district administrative staff 

For each of the following administrative roles or positions, please indicate whether your district has the position and whether or not you fill the position, in addition to your role as MSAP director. Please check Yes or No for each position. Also, please indicate the extent to which you interact with the person holding each position in planning and implementing the MSAP project. 

1. Assistant Superintendent or Coordinator of Curriculum and Instruction 2. Assistant Superintendent or Coordinator of Professional Development 

3. Assistant Superintendent or Coordinator of Assessment and Testing 4. Title I Coordinator 5. Federal Programs Coordinator 6. Coordinator of district choice or magnet programs (other than MSAP) 

Response scale: 1=Not at all, 2=To some extent, 3=To a great extent 

MSAP Projects 1999–2000 

(0.67) 

Accountability for achievementPossible sanctions for poor student achievement 

In this district, do schools receive any of the following rewards or sanctions for student achievement? 

1. Have the principal reassigned or released 2. Be taken over by a higher level governing body 3. Have the teaching staff reconstituted 

Response scale: 0=No, 1=Yes 

MSAP Principals; Non-MSAP Comparison Principals 

1999–2000 (0.73) 

a A repeated measures MANOVA was used to test for differences in measure between 1999–2000 and 2000–2001. b Analysis of variance was used to test for difference in the measure between MSAP and matched comparison non-MSAP schools. 
 

Measure Name Survey Items Included in Measure 

Survey 

Respondents 

Years 

Asked 

(Scale 

Reliability) 

District MSAP support for schools Technical assistance provided by MSAP project officers on curriculum and instruction 

On average, how frequently did you or other MSAP-supported district staff provide technical assistance or guidance in the following areas to each MSAP school? Include visits you or other MSAP district staff made to the school, as well as visits the principals or other school staff made to the district office. 

1. Designing curriculum or selecting curriculum materials 2. Planning professional development 3. Developing theme-related activities 

4. Designing assessments 5. Keeping teachers motivated to implement the magnet program 

Response scale: 1=Never, 2=Less than once a month, 3=About once a month, 

4=About once every 2 weeks, 5=Once a week or more 

MSAP Projects 1999–2000 

(0.87); 2000–2001 (0.83) 

Technical assistance provided by MSAP project officers on leadership 

On average, how frequently did you or other MSAP-supported district staff provide technical assistance or guidance in the following areas to each MSAP school? Include visits you or other MSAP district staff made to the school, as well as visits the principals or other school staff made to the district office. 

1. Budgeting and resource allocation 2. Helping principals provide leadership for magnet program 3. Working with parents 

Response scale: 1=Never, 2=Less than once a month, 3=About once a month, 

4=About once every 2 weeks, 5=Once a week or more 

MSAP Projects 1999–2000 

(0.59); 2000–2001 (0.45) 

Supports for Teaching and Learning Ongoing professional development for teachers Professional development opportunities related to standards-based reform 

Over the past 12 months, how much emphasis has this school placed on providing professional development for teachers in the following areas? 

1. State or district curriculum and performance standards 2. Student performance assessment (e.g., methods of testing, interpreting 

assessment data, applying results to modify instruction) 

3. Addressing the needs of students with limited English proficiency or 

from diverse cultural backgrounds 

4. Addressing the needs of students with disabilities 

Response scale: 1=No emphasis, 2=Little emphasis, 3=Moderate emphasis, 4=A 

great deal of emphasis 

MSAP Principals; Non-MSAP Comparison Principals 

1999–2000 (0.65); 2000–2001 (0.66) 
Appendix III

Appendix-III-30

Measure Name Survey Items Included in Measure 

Survey 

Respondents 

Years 

Asked 

(Scale 

Reliability) 

Professional community among teachersStrength of professional community 

To what extent do you agree or disagree with each of the following statements 

1. I have confidence in the expertise of teachers on my staff. 2. Rules for student behavior are consistently enforced by teachers in this 

school, even for students who are not in their classes. 

3. I have the support of teachers in enforcing school rules. 4. The level of student misbehavior interferes with teaching. (reverse 

coded) 

5. Most teachers at this school make a conscious effort to coordinate their 

teaching with instruction at other grade levels. 

6. This school seems like a big family. Everyone is close and cordial. 7. Most staff members support and encourage each other at this school. 8. Most teachers are willing to put in extra hours to help this school be 

successful. 

9. Most teachers share my beliefs and values about what the central 

mission of this school should be. 

10. Most teachers at this school are continually learning and seeking new 

ideas. 

11. Most teachers at this school really care about their students. 12. I usually look forward to working each day at this school. 13. Sometimes I feel it is a waste of time to try to do my best. (reverse 

coded) 

14. The administration and teaching staff collaborate to make school run 

effectively. 

15. Racial and ethnic differences among staff members create tensions in 

the school. (reverse coded) 

Response scale: 1=Strongly disagree, 2=Disagree, 3=Agree, 4=Strongly agree 

MSAP Principals; Non-MSAP Comparison Principals 

1999–2000 (0.91); 2000–2001 (0.91) 
 

Measure Name Survey Items Included in Measure 

Survey 

Respondents 

Years 

Asked 

(Scale 

Reliability) 

Positive school climate Student disengagement problems 

To what extent is each of the following matters a problem in this school? Indicate whether it is a SERIOUS problem, a MODERATE problem, a MINOR problem, or NOT a problem in this school. 

1. Student tardiness 2. Student absenteeism 3. Student transience 4. Student apathy 5. Lack of parental involvement 6. Poverty 7. Students coming to school unprepared to learn 8. Poor student health 

Response scale: 1=Not a problem in this school, 2=Minor problem in this 

school, 3=Moderate problem in this school, 4=Serious problem in this 

MSAP Principals; Non-MSAP Comparison Principals 

1999–2000 (0.88); 2000–2001 (0.90) 

Student behavior problems To what extent is each of the following matters a problem in this school? Indicate 

whether it is a SERIOUS problem, a MODERATE problem, a MINOR problem, or NOT a problem in this school. 

1. Physical conflicts among students 2. Robbery or theft 3. Vandalism of school property 4. Verbal abuse of teachers 5. Physical abuse of teachers 6. Student disrespect for teachers 

Response scale: 1=Not a problem in this school, 2=Minor problem in this 

school, 3=Moderate problem in this school, 4=Serious problem in this 

MSAP Principals; Non-MSAP Comparison Principals 

1999–2000 (0.82); 2000–2001 (0.85) 

Other severe student problems 

To what extent is each of the following matters a problem in this school? Indicate whether it is a SERIOUS problem, a MODERATE problem, a MINOR problem, or NOT a problem in this school. 

1. Students cutting classes 2. Student pregnancy 3. Student use of alcohol 4. Student drug abuse 5. Student possession of weapons 6. Students dropping out 

Response scale: 1=Not a problem in this school, 2=Minor problem in this 

school, 3=Moderate problem in this school, 4=Serious problem in this 

MSAP Principals; Non-MSAP Comparison Principals 

1999–2000 (0.90); 2000–2001 (0.91) 
Appendix III

Appendix-III-32

Measure Name Survey Items Included in Measure 

Survey 

Respondents 

Years 

Asked 

(Scale 

Reliability) 

Mechanisms for parental involvementParental involvement in special events 

During the last school year, were the following means of facilitating parent participation in place at this school? (“Parents” includes parents and other family 

members.) If yes, what proportion of parents participated? 

1. Open house or back-to-school night 2. Regularly scheduled schoolwide parent-teacher conferences 3. Special subject-area events (e.g., science fair, concert) 4. Parent education workshops or courses 5. Parent-child learning activities at school 

Response scale: 0=None, 1=Few, 2=Less than half, 3=More than half, 4=Most 

MSAP Principals; Non-MSAP Comparison Principals 

1999–2000 (0.74); 2000–2001 (0.76) 

Parental involvement in school governance 

During the last school year, were the following means of facilitating parent participation in place at this school? (“Parents” includes parents and other family 

members.) If yes, what proportion of parents participated? 

1. Parents involved in instructional issues 2. Parents involved in governance 3. Parents involved in budget decisions 

Response scale: 0=None, 1=Few, 2=Less than half, 3=More than half, 4=Most 

MSAP Principals; Non-MSAP Comparison Principals 

1999–2000 (0.75); 2000–2001 (0.76) 

High standards for all studentsRequired academic activities for students in grades K–5 

Does the program require any of the following practices for ALL magnet students enrolled in any of grades K–5? 

1. At least one hour of instructional time devoted to reading each day 2. At least one hour of instructional time devoted to math each day 3. At least three hours of instructional time devoted to science each week 4. A required number of books or pages to be read each week (in and out 

of class) 

5. A required amount of writing to be completed each week (in and out of 

class) 

6. A required oral presentation at regular intervals (i.e., weekly or 

monthly) 

7. A required amount of homework to be completed each day 

MSAP Principals; Non-MSAP Comparison Principals 

1999–2000 (0.75); 2000–2001 (0.78) 
 

Measure Name Survey Items Included in Measure 

Survey 

Respondents 

Years 

Asked 

(Scale 

Reliability) 

8. Completion of a portfolio of work in one or more academic subjects 

during the year 

Response scale: 0=No, 1=Yes 

Required academic activities for students in grade 8 

Does the program require any of the following for ALL 8th grade magnet students? 

1. A required year-long or semester-long project 2. A required number of books to be read each month or grading period 

(in and out of class) 

3. A required amount of writing to be completed each week (in and out of 

class) 

4. A required oral presentation at regular intervals (i.e., weekly or 

monthly) 

5. A required minimum amount of homework to be completed each day 6. Completion of a portfolio of work in one or more academic subjects 

during the year 

7. An individual or small group project to be completed during the year 

Response scale: 0=No, 1=Yes 

MSAP Principals; Non-MSAP Comparison Principals 

1999–2000 (0.78); 2000–2001 (0.78) 

Required academic activities for students in grade 10 

Does the program require any of the following for ALL 10th grade magnet students? 

1. A required year-long or semester-long project 2. A required number of books to be read each month or grading period 

(in and out of class) 

3. A required amount of writing to be completed each week (in and out of 

class) 

4. A required oral presentation at regular intervals (i.e., weekly or 

monthly) 

5. A required minimum amount of homework to be completed each day 6. Completion of a portfolio of work in one or more academic subjects 

during the year 

7. An individual or small group project to be completed during the year 

Response scale: 0=No, 1=Yes 

MSAP Principals; Non-MSAP Comparison Principals 

1999–2000 (0.73); 2000–2001 (0.85) 
Appendix III

Appendix-III-34

Measure Name Survey Items Included in Measure 

Survey 

Respondents 

Years 

Asked 

(Scale 

Reliability) 

Classroom Instructional Practices Appropriate instructional methods Elementary mathematics teachers’ use of instructional methods emphasizing higher order thinking skills 

How often do the students in this class do each of the following? 

1. Write a few sentences about their mathematics work 2. Talk to the class about their mathematics work 3. Write reports or do mathematics projects 4. Work and discuss mathematics problems that reflect real-life situations 5. Use a computer 6. Work on problems for which there is no immediately obvious method 

or solution 

7. Work on interdisciplinary lessons (e.g., writing journals in class) 

Response scale: 1=Never or hardly ever, 2=Once or twice a month, 3=Once or 

twice a week, 4=Almost every day 

MSAP Mathematics Teachers; NonMSAP Comparison Mathematics Teachers 

2000–2001 (0.66) 

Secondary mathematics teachers’ use of instructional methods emphasizing higher order thinking skills1

How often do the students in this class do each of the following? 

1. Write a few sentences about their mathematics work 2. Talk to the class about their mathematics work 3. Write reports or do mathematics projects 4. Work and discuss mathematics problems that reflect real-life situations 5. Use a computer 6. Work on problems for which there is no immediately obvious method 

or solution 

7. Work on interdisciplinary lessons (e.g., writing journals in class) 

Response scale: 1=Never or hardly ever, 2=Once or twice a month, 3=Once or 

twice a week, 4=Almost every day 

MSAP Mathematics Teachers; NonMSAP Comparison Mathematics Teachers 

2000–2001 (0.78) 

Elementary reading/language arts teachers’ use of instructional methods emphasizing higher order thinking skills 

How often do the students in this class do each of the following? 

1. Ask students to talk with each other about what they have read 2. Ask students to write about something they have read 3. Ask students to discuss different interpretations of what they have read 4. Ask students to explain or support their understanding of what they 

have read 

Response scale: 1=Never or hardly ever, 2=Once or twice a month, 3=Once or 

twice a week, 4=Almost every day 

MSAP Reading/ Language Arts Teachers; NonMSAP Comparison Reading/ Language Arts Teachers 

2000–2001 (0.81) 

1 Elementary 
 

Measure Name Survey Items Included in Measure 

Survey 

Respondents 

Years 

Asked 

(Scale 

Reliability) 

Secondary reading/language arts teachers’ use of instructional methods emphasizing higher order thinking skills 

How often do the students in this class do each of the following? 

1. Ask students to discuss different interpretations of what they have read 2. Ask students to explain or support their understanding of what they 

have read 

Response scale: 1=Never or hardly ever, 2=Once or twice a month, 3=Once or 

twice a week, 4=Almost every day 

MSAP Reading/ Language Arts Teachers; NonMSAP Comparison Reading/ Language Arts Teachers 

2000–2001 (0.79) 

Appendix IV—MSAP-Defined Desegregation Objectives and 

Outcomes in Targeted Schools 

 Appendix IV 

 Appendix-IV-1

Exhibit A-IV-1. 

Desegregation objectives of targeted schools by type of desegregation 

plan within grade levels 

Objectives Desegregation Plan 

Elementary Voluntary Required Total 

N Schools 91 84 175 

Prevent 5.5% 10.7% 8.0% 

Eliminate 22.0% 17.9% 20.0% 

Reduce 72.5% 71.4% 72.0% 

 

Total 100.0% 100.0% 100.0% 

Middle School 

N Schools 43 29 72 

Prevent 4.7% 17.2% 9.7% 

Eliminate 11.6% 0.0% 6.9% 

Reduce 83.7% 82.8% 83.3% 

 

Total 100.0% 100.0% 100.0% 

High School 

N Schools 17 23 40 

Prevent 0.0% 4.3% 2.5% 

Eliminate 23.5% 8.7% 15.0% 

Reduce 76.5% 87.0% 82.5% 

 

Total 100.0% 100.0% 100.0% 

Combined Levels 

N Schools 3 4 7 

Prevent 0.0% 0.0% 0.0% 

Eliminate 0.0% 0.0% 0.0% 

Reduce 100.0% 100.0% 100.0% 

 

Total 100.0% 100.0% 100.0% 

Total (all grade levels) 

N Schools 154 140 294 

Prevent 4.5% 10.7% 7.5% 

Eliminate 18.8% 12.1% 15.6% 

Reduce 76.6% 77.1% 76.9% 

 

Total 100.0% 100.0% 100.0% 

Sources: Grantee applications, performance reports, NCES 1997–1998 and 

1998–1999 Common Core of Data, and U.S. Department of Education personnel 
Appendix IV 

Appendix-IV-2

Exhibit A-IV-2. 

Proportion of MSAP’s desegregation targeted schools in districts with 50%+, 60%+, and 

70%+ minority enrollment by grade level: 1997–1998 

Proportion of MSAP’s targeted schools in district 

in which 1997–1998 minority enrollment is: 

Grade Level of Targeted School: 50% or More 60% or More 70% or More 

N of Targeted 

Schools 

Elementary 71% 63% 43% 175 

Middle School 85% 69% 57% 72 

High School 75% 73% 43% 40 

Combined Levels 71% 71% 71% 7 

Overall 75% 66% 47% 294 

Source: Grantee applications, performance reports, NCES 1997–1998 and 1998–1999 Common Core of Data 

Exhibit A-IV-3. 

Extent of district change in percentage of minorities at grade level of targeted school by 

type of desegregation plan in district: 1997–1998 to 2000–2001 

Type of Desegregation Plan

Percent Minority in District at Grade Level of 

Targeted School: 

Voluntary 

(N = 154) 

Required 

(N = 140) 

All Districts 

(N = 294) 

Increased by 5% or More 11.0% 8.6% 9.9% 

Increased by 1% to <5% 61.7% 75.7% 68.4% 

Increased by <1% 9.7% 11.4% 10.5% 

No Change 1.3% 0.0% 0.7% 

Decreased by < 1% 3.2% 0.0% 1.7% 

Decreased by 1% to 5% 11.0% 3.6% 7.5% 

Decreased by 5% or More 1.9% 0.7% 1.4% 

Total 100.0% 100.0% 100% 

Source: Grantee applications, performance reports, NCES 1997–1998 and 2000–2001 Common Core of Data, and U.S. 

Department of Education personnel 

Exhibit A-IV-4. 

Percentage of desegregation targeted schools making progress on desegregation 

objective by objective and type of desegregation plan, adjusting for district change in 

percent minority: 1997–1998 to 2000–2001 

Type of Desegregation Plan

Voluntary Required All Schools Type of Desegregation Objective % of N % of N % of N 

Prevent 42.9 7 53.3 15 50.0 22 

Eliminate 48.3 29 64.7 17 54.3 46 

Reduce 64.4 118 50.9 108 58.4 226 

Overall 60.4 154 52.9 140 56.8 294 

Source: Grantee applications, performance reports, NCES 1997–1998 and 2000–2001 Common Core of Data, and U.S. 

Department of Education personnel 
 Appendix IV 

 Appendix-IV-3

Exhibit A-IV-5. 

Percentage of desegregation targeted schools making progress on desegregation 

objective by grade level and type of desegregation plan, adjusting for district change in 

percent minority: 1997–1998 to 2000–2001 

Type of Desegregation Plan

Voluntary Required All Schools Grade Level % of N % of N % of N 

Elementary 62.6 91 57.1 84 60.0 175 

Middle School 60.4 43 44.8 29 54.2 72 

High School 47.1 17 47.8 23 47.5 40 

Combined Levels 66.7 3 50.0 4 57.1 7 

Overall 60.4 154 52.9 140 56.8 294 

Source: Grantee applications, performance reports, NCES 1997–1998 and 2000–2001 Common Core of Data, and U.S. 

Department of Education personnel 

Exhibit A-IV-6. 

Percentage of desegregation targeted schools making progress in achieving goal by type 

of program and type of desegregation plan, adjusting for district change in percent 

minority: 1997–1998 to 2000–2001 

Type of Desegregation Plan

Voluntary Required All Schools Type of Program % of N % of N % of N 

Program-within-School (PWS) 75.0 8 40.0 25 48.5 33 

Whole School 61.0 136 56.8 95 59.3 231 

Targeted Feeder 40.0 10 50.0 20 46.7 30 

Overall 60.4 154 52.9 140 56.8 294 

 Source: Grantee applications, performance reports, NCES 1997–1998 and 2000–2001 Common Core of Data 

Exhibit A-IV-7. 

Summary of extent of progress in meeting desegregation objective by type of 

desegregation plan, adjusting for district change in percent minority: 1997–1998 to 2000–

2001 

Type of Desegregation Plan

Extent of Progress 

Voluntary 

(N = 154) 

Required 

(N = 140) 

All Districts 

(N = 294) 

Prevented or Eliminated MGI 2.6% 7.9% 5.1% 

Reduced Proportion Minority by 5% or More 18.8% 15.7% 17.3% 

Reduced Proportion Minority by 1% to < 5% 31.8% 23.6% 27.9% 

Reduced Proportion Minority by < 1% 7.1% 5.7% 6.5% 

Increase or No Change in Percent Minority 39.6% 47.1% 43.2% 

Total 100.0% 100.0% 100% 

Source: Grantee applications, performance reports, NCES 1997–1998 and 2000–2001 Common Core of Data 
Appendix IV 

Appendix-IV-4

Technical Note: Hierarchical Linear Modeling of Trends in Percent 

Minority Enrollment 

Factors influencing changes in minority enrollment are analyzed using a three-level 

hierarchical linear regression model estimated using the SAS PROC MIXED procedure. The 

three levels included in the model are the MSAP districts, schools in the district targeted for 

desegregation by an MSAP-funded program, and school years from 1997–1998 to 2000–2001. 

The baseline unconditional change model is represented by the equations below, where Yijt is the 

percent minority enrollment in school j of district i in year t. 

Level-1: Year 

Yijt = π 0ij + π 1ij (Year) ij + rijt

Level-2: School 

π 0ij = γ00j + r0ij 

π 1ij = γ1+j + r1ij

Level-3: District 

γ00j = β000 + e00j 

γ10j = β100 + e10j

The model includes two random effects: 

r00kl = random school-level effect that varies over time 

e0jkl = random district-level effect that varies over time 

In the models developed for this analysis, the fixed effects appear as covariates in either 

the school-level equations or the district-level equations, depending on whether the measure is a 

school or district characteristic. The exhibits that follow present the results of separate analyses 

on elementary and secondary schools. 
 Appendix IV 

 Appendix-IV-5

Exhibit A-IV-8. 

Parameter estimates for unconditional and conditional models of change in percent 

minority enrollment of MSAP’s desegregation targeted elementary schools 

between 1997–1998 and 2000–2001* 

 Est. SE P-value Est. SE P-value 

Main Effect Parameters 

Baseline 

Unconditional Change Model Conditional Change Model 

Intercept 76.263 2.106 <.0001 75.585 7.529 <.0001 

Year 0.342 0.280 0.223 -1.654 1.309 0.207 

 

Effects on Intercept 

District Level 

- % Minority 1997 0.249 0.076 0.002 

- ∆ in % Minority -0.922 0.707 0.199 

School-Level 

- Attend Zone school 3.424 1.459 0.019 

- Pupil Teacher Ratio -0.298 0.315 0.344 

- Mixed Minority -4.989 2.702 0.065 

- Parental Involvement in 

Special Events 

 

-1.815 1.027 0.078 

- PWS 13.597 2.872 <.0001 

- Prevent MGI -25.835 2.711 <.0001 

- Eliminate MGI -15.536 2.573 <.0001 

 

Effects on Slope 

District Level 

- ∆ in % Minority 0.414 0.120 0.001 

School-Level 

- Attend Zone School -1.169 0.354 0.001 

- Pupil Teacher Ratio 0.187 0.072 0.009 

- Mixed Minority -1.979 0.723 0.006 

- Parental Involvement in 

Special Events 

 

-0.672 0.269 0.013 

- PWS -1.337 0.712 0.061 

 

Unexplained Variance 

District Intercept 170.350 44.053 <.0001 63.183 16.994 0.000 

District Slope 2.253 0.815 0.003 1.484 0.586 0.006 

School Intercept 96.826 12.733 <.0001 44.016 6.024 <.0001 

School Slope 3.291 0.533 <.0001 2.683 0.451 <.0001 

Residual 3.822 0.296 <.0001 3.817 0.295 <.0001 

Loglikelihood Ratio 4019.5 3819.6 

# Fixed Parameters 1 16 

* Sample consists of 172 elementary schools in 47 districts. 

 

 

Appendix V—Student Achievement Outcomes in 

MSAP-Supported Schools 

 Appendix V 

 Appendix-V-1

Technical Note on Determining Goal Attainment for Static and 

Cumulative Achievement Objectives 

School-level goals fall into two major categories based on whether each year’s 

benchmarks build on the previous year’s benchmarks or are independent of them. “Static” goals 

set a benchmark for each year that does not depend on the previous year’s benchmark or 

performance level. Examples of static goals are: “By the end of the first year of the grant, 75 

percent of students will score at proficiency level on a test; by the end of the second year, 80 

percent of the students will be proficient, and by the end of the third year, 85 percent will be 

proficient,” or “the average score of magnet students will meet or exceed the district average 

each year.” “Cumulative” goals, by contrast, call for incremental improvements of a specified 

amount over a baseline level, usually the school’s performance level in the baseline year 

(generally the year prior to the awarding of the grant). For example, a goal may call for schools 

to improve their average mathematics scores by 2 percentile points each year. A school with a 

baseline year score of 65 would be expected to attain a score of 67 points by the end of the first 

grant year, but a school with a baseline score of 71 would be expected to reach 73 percentile 

points. It was generally unclear how districts intended to evaluate attainment of second year 

goals when schools failed to reach their first-year benchmark—for instance, if a school advanced 

by only one percentile point in the first year, was it expected to advance 2 percentile points in the 

second year or to “make up the difference” and advance 3 percentile points? We dealt with this 

ambiguity by tracking goals in two ways: one-year goals for each of the three grant years as well 

as end-of-project goals. For cumulative goals, we determined that the one-year benchmarks had 

been met if the school grew by the amount called for in a particular year, regardless of its 

performance in the previous year. (Thus, the school that increased by only one point in the first 

year would be counted as meeting the second year’s goal if it increased its score by 2 percentile 

points in the second year.) However, for to meet end-of-project goals, the school’s improvement 

had to equal or exceed the cumulative improvement called for in the objective (a full six points 

of improvement by the end of the third year.) For static goals, the benchmark for the end of the 

project was the same as for the third year’s benchmark. 

Most school magnet programs were in operation for the full three years of the MSAP 

grant. Their baseline year was the 1997–1998 school year. When programs got underway later, 

we treated the year prior to the first implementation year as the baseline year. 
Appendix V 

Appendix-V-2

Appendix Exhibits for Student Achievement Chapter 

Exhibit A-V-1. 

Project-level objectives by subject domain 

Subject % N 

Math 19 106

Language Arts 30 169 

Science 6 31 

Social Studies 5 28 

Computer / Vocational Skills 5 27 

Career Awareness 2 14 

Arts and Performances 2 10 

Other 6 33 

Not Subject-Specific 26 146 

Total 564 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-2. 

Project-level objectives by type of measure 

Measure % N 

Standardized Test 74 418 

Other Assessments 14 80 

Course-Taking and Grades 6 34 

Educational Trajectory Benchmark 2 11 

Preparation for Postsecondary Education 2 9 

Advanced Vocation-Related Outcomes 1 7 

Other 1 5 

Total 564 

Source: MSAP grant application and annual performance reports 
 Appendix V 

 Appendix-V-3

 

Exhibit A-V-3. 

School-level goals by subject domain 

Subject % of Total Goals # of Goals 

Language Arts 46 1345 

Math 36 1048

Science 6 185 

Social Studies 2 51 

Computer/ Vocational Skills 1 16 

Not Subject-Specific 1 16 

Arts and Performances 0.07 2 

Career Awareness 0.03 1 

Other 7 210 

Missing 1 20 

Total 2894 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-4. 

School-level goals by type of measure 

Measurement % of Total Goals # of Goals 

Standardized Test 88 2541 

Course-Taking and Grades 3 93 

Alternative Assessments 2 44 

Prep. for Postsecondary Ed. 1 43 

Ed. Trajectory Benchmark 0.4 11 

Other 0.1 4 

Missing 5 158 

Total 2894 

Source: MSAP grant application and annual performance reports 
Appendix V 

Appendix-V-4

Exhibit A-V-5. 

Number of English language arts goals per school 

 1998–1999 goals 1999–2000 goals 2000–2001 goals Final goals 

Number of 

goals # of Schools

% of 

Schools # of Schools

% of 

Schools # of Schools

% of 

Schools 

# of 

Schools 

% of 

Schools 

1 16 11% 43 23% 27 15% 29 16% 

2 34 23% 35 19% 56 31% 54 29% 

3 37 25% 18 10% 17 9% 17 9% 

4 27 18% 29 16% 25 14% 27 15% 

5 15 10% 14 8% 11 6% 11 6% 

6–10 11 7% 23 12% 33 18% 33 18% 

More than 10 7 5% 23 12% 13 7% 13 7% 

Total Schools 147 100% 185 100% 182 100% 184 100% 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-6. 

Number of mathematics goals per school 

 1998–1999 goals 1999–2000 goals 2000–2001 goals Final goals 

Number of 

goals per 

school 

# of 

Schools 

% of 

Schools 

# of 

Schools 

% of 

Schools 

# of 

Schools 

% of 

Schools 

# of 

Schools 

% of 

Schools 

1 33 25% 56 31% 67 35% 68 36% 

2 32 24% 36 20% 30 16% 28 15% 

3 32 24% 28 15% 24 13% 26 14% 

4 12 9% 17 9% 24 13% 24 13% 

5 4 3% 5 3% 8 4% 8 4% 

6–10 5 4% 24 13% 20 11% 20 10% 

More than 10 16 12% 17 9% 17 9% 17 9% 

Total Schools 134 100% 183 100% 190 100% 191 100% 

Source: MSAP grant application and annual performance reports 
 Appendix V 

 Appendix-V-5

Exhibit A-V-7. 

Number of all schools that met goals for English language arts 

English 

Language Arts 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 42 28.6 65 35.1 74 40.7 67 36.4 

Less than half 21 14.3 28 15.1 31 17.0 24 13.0 

Half or more 43 29.2 52 28.1 46 25.3 54 29.4 

All 41 27.9 40 21.6 31 17.0 39 21.2 

Total 147 100.0 185 100.0 182 100.0 184 100.0 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-8. 

Number of all schools that met goals for mathematics 

Math 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools -

Final 

None 48 35.8 83 45.4 87 45.8 89 46.6 

Less than half 10 7.5 21 11.5 40 21.0 34 17.8 

Half or more 37 27.6 39 21.3 29 15.3 32 16.8 

All 39 29.1 40 21.9 34 17.9 36 18.8 

Total 134 100.0 183 100.0 190 100.0 191 100.0 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-9. 

Number of elementary schools that met goals for English language arts 

English 

Language Arts 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools -

Final 

None 24 26.4 36 31.9 37 33.3 35 31.5 

Less than half 14 15.4 16 14.2 19 17.1 14 12.6 

Half or more 26 28.6 36 31.9 31 27.9 34 30.6 

All 27 29.7 25 22.1 24 21.6 28 25.2 

Total 91 100.0 113 100.0 111 100.0 111 100.0 

Source: MSAP grant application and annual performance reports 
Appendix V 

Appendix-V-6

Exhibit A-V-10. 

Number of elementary schools that met goals for mathematics 

Math 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 23 28.0 43 38.7 45 38.8 46 39.7 

Less than half 8 9.8 9 8.1 24 20.7 20 17.2 

Half or more 25 30.5 29 26.1 22 19.0 23 19.8 

All 26 31.7 30 27.0 25 21.6 27 23.3 

Total 82 100.0 111 100.0 116 100.0 116 100.0 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-11. 

Number of middle schools that met goals for English language arts 

English 

Language Arts 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 15 42.9 18 40.9 24 55.8 23 52.3 

Less than half 4 11.4 10 22.7 7 16.3 6 13.6 

Half or more 10 28.6 7 15.9 10 23.36 11 25.0 

All 6 17.1 9 20.4 2 4.6 4 9.1 

Total 35 100.0 44 100.0 43 100.0 44 100.0 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-12. 

Number of middle schools that met goals for mathematics 

Math 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 19 59.4 27 61.4 23 48.9 26 54.2 

Less than half 1 3.1 8 18.2 13 27.7 11 22.9 

Half or more 7 21.9 4 9.1 5 10.6 5 10.4 

All 5 15.6 5 11.4 6 12.8 6 12.5 

Total 32 100.0 44 100.0 47 100.0 48 100.0 

Source: MSAP grant application and annual performance reports 
 Appendix V 

 Appendix-V-7

Exhibit A-V-13. 

Number of high schools that met goals for English language arts 

English 

Language 

Arts 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 3 17.6 10 45.4 11 52.4 7 31.8 

Less than half 2 11.8 0 0.0 3 14.3 3 13.6 

Half or more 5 29.4 7 31.8 4 19.0 8 36.4 

All 7 41.2 5 22.7 3 14.3 4 18.2 

Total 17 100.0 22 100.0 21 100.0 22 100.0 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-14. 

Number of high schools that met goals for mathematics 

Math 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 5 29.4 11 50.0 16 76.2 14 66.7 

Less than half 0 0.0 2 9.1 3 14.3 3 14.3 

Half or more 5 29.4 5 22.7 0 0.0 2 9.5 

All 7 41.2 4 18.2 2 9.5 2 9.5 

Total 17 100.0 22 100.0 21 100.0 21 100.0 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-15. 

Number of all schools that made progress toward goals for English language arts 

English 

Language 

Arts 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 28 19.0 43 23.2 52 28.6 39 21.2 

Less than half 16 10.9 24 13.0 24 13.2 22 12.0 

Half or more 49 33.3 56 30.3 58 31.9 52 28.3 

All 54 36.7 62 33.5 48 26.4 71 38.6 

Total 147 100.0 185 100.0 182 100.0 184 100.0 

Source: MSAP grant application and annual performance reports 
Appendix V 

Appendix-V-8

Exhibit A-V-16. 

Number of all schools that made progress toward goals for mathematics 

Math 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 29 21.6 55 30.0 62 32.6 52 27.2 

Less than half 9 6.7 21 11.5 28 14.7 26 13.6 

Half or more 34 25.4 46 25.1 43 22.6 37 19.4 

All 62 46.3 61 33.3 57 30.0 76 39.8 

Total 134 100.0 183 100.0 190 100.0 191 100.0 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-17. 

Number of elementary schools that made progress toward goals for English language 

arts 

English 

Language 

Arts 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 17 18.7 23 20.4 29 26.1 19 17.1 

Less than half 11 12.1 14 12.4 14 12.6 14 12.6 

Half or more 30 33.0 37 32.7 37 33.3 33 29.7 

All 33 36.3 39 34.5 31 27.9 45 40.5 

Total 91 100.0 113 100.0 111 100.0 111 100.0 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-18. 

Number of elementary schools that made progress toward goals for mathematics 

Math 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 13 15.8 28 25.2 37 31.9 31 26.7 

Less than half 6 7.3 9 8.1 17 14.7 14 12.1 

Half or more 23 28.0 32 28.8 25 21.6 24 20.7 

All 40 48.8 42 37.8 37 31.9 47 40.5 

Total 82 100.0 111 100.0 116 100.0 116 100.0 

Source: MSAP grant application and annual performance reports 
 Appendix V 

 Appendix-V-9

Exhibit A-V-19. 

Number of middle schools that made progress toward goals for English language arts 

English 

Language 

Arts 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 11 31.4 12 27.3 17 39.5 15 34.1 

Less than half 4 11.4 8 18.2 7 16.3 6 13.6 

Half or more 12 34.3 8 18.2 14 32.6 12 27.3 

All 8 22.9 16 36.4 5 11.6 11 25.0 

Total 35 100.0 44 100.0 43 100.0 44 100.0 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-20. 

Number of middle schools that made progress toward goals for mathematics 

Math 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 14 43.8 17 38.6 15 31.9 14 29.2 

Less than half 2 6.2 8 18.2 8 17.0 9 18.8 

Half or more 8 25.0 8 18.2 13 27.7 11 22.9 

All 8 25.0 11 25.0 11 23.4 14 29.2 

Total 32 100.0 44 100.0 47 100.0 48 100.0 

Source: MSAP grant application and annual performance reports 

Exhibit A-V-21. 

Number of high schools that made progress toward goals for English language arts 

English 

Language 

Arts 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 0 0.0 7 31.8 6 28.6 5 22.7 

Less than half 1 5.9 1 4.6 2 9.5 2 9.1 

Half or more 4 23.5 8 36.4 5 23.8 5 22.7 

All 12 70.6 6 27.3 8 38.1 10 45.4 

Total 17 100.0 22 100.0 21 100.0 22 100.0 

Source: MSAP grant application and annual performance reports 
Appendix V 

Appendix-V-10

Exhibit A-V-22. 

Number of high schools that made progress toward goals for mathematics 

Math 

# of 

Schools - 

Year 1 

% of 

Schools - 

Year 1 

# of 

Schools - 

Year 2 

% of 

Schools - 

Year 2 

# of 

Schools - 

Year 3 

% of 

Schools - 

Year 3 

# of 

Schools - 

Final 

% of 

Schools - 

Final 

None 1 5.9 8 36.4 9 42.9 5 23.8 

Less than half 0 0.0 2 9.1 3 14.3 3 14.3 

Half or more 3 17.6 5 22.7 1 4.8 0 0.0 

All 13 76.5 7 31.8 8 38.1 13 61.9 

Total 17 100.0 22 100.0 21 100.0 21 100.0 

Source: MSAP grant application and annual performance reports 
 Appendix V 

 Appendix-V-11

Technical Note: Exclusions from Achievement Analysis 

Exhibit A-V-23. 

MSAP-supported elementary schools excluded from trend analyses 

 Number of Schools 

Reason for Exclusion 19 

Grades in School not Covered by State Test 5 

Magnet School Not in Operation in 1998–1999 11 

Magnet School Eliminated before 2000–2001 2 

School Lacked an NCES Identification Code 11 

Reason Yet to be Determined 2 

Technical Note: Standardization of Outcome Data 

States not only administer different tests, but they also differ in which grades they test 

and the way they report their results. Some states report average scores (scale scores or national 

percentile scores) while others report the percentage of students who scored at various levels of 

mastery (or proficiency). Between 1997–1998 and 2000–2001, several states also changed their 

assessments so that baseline and final scores were not directly comparable. In order to combine 

these diverse measures in an overall analysis of MSAP student achievement, we translated all of 

the scores into a common metric: a standardized score that indicated the amount (in standard 

deviation units) that each school’s score differed from the average score for students in its state. 

In states whose assessments did not change over the study period, both the baseline and final 

year’s test scores were standardized in terms of baseline year standard deviation units, allowing 

us to measure actual gains in standard deviation units over time. In states whose assessments 

changed, we standardized the scores for each year separately. In these cases, changes in 

standardized scores over time showed the degree to which each school’s performance moved 

toward or away from the state average between the baseline year and 2000–2001. 

Technical Note: Characteristics of MSAP Elementary and Matched 

Non-MSAP Elementary Schools 

The analysis of student achievement trends is based state assessment results of 135 

MSAP-funded elementary schools and a comparison sample of 1350 non-MSAP elementary 

schools. Each MSAP school was matched with 10 non-MSAP schools from the same state that 

resembled it most closely on student demographic characteristics and prior achievement. 

Exhibit A-V-24 summarizes the mean demographic characteristics of the MSAP elementary 

schools and the Non-MSAP schools used in the analysis. The exhibit does not display average 

initial achievement levels of MSAP and non-MSAP schools because the metric of the 

achievement measures varied from state to state. 
Appendix V 

Appendix-V-12

Exhibit A-V-24. 

Demographic characteristics of MSAP and matched non-MSAP elementary school 

samples used in analysis of achievement trends. 

Demographic 

Characteristic 

MSAP 

(N = 135) 

Non-MSAP 

(N = 1,350) 

Urbanicity 

 Percent Large City 43 43 

 Percent Medium City 37 37 

 Percent Urban Fringe 20 20 

Average Number of Students 622 649 

Average Percent Students 

Eligible for Free or Reducedprice Lunches 74 70 

Average Percent Minority 78 72 

Average Percent: 

 Black 39 33 

 Hispanic 31 31 

 Asian or Pacific Islander 7 6 

 Native American or Alaskan 

Native 0.5 0.6 

Source: 1997-1998 Common Core Data, except for percentages of students eligible for free or 

reduced-price lunches which are based on data from the 1999-2000 Common Core Data. 

Technical Note: Calculation of the “Residual Achievement Measure” 

The analysis of factors associated with greater student achievement in MSAP-supported 

schools uses an academic improvement measure that sets each school’s achievement growth in 

the context of the growth observed for ten comparison schools matched on demographic 

composition and prior academic performance. The measure is the calculated difference between 

the amount of growth predicted by a regression equation that includes variables for demographic 

composition and prior test scores and the actual growth achieved by the school. A large positive 

difference indicates that the school’s growth is higher than would be expected, given the school’s 

demographics and prior achievement levels. A small difference indicates that the school’s 

growth is close to that predicted by the regression equation. 
 Appendix V 

 Appendix-V-13

Technical Note: Hierarchical Linear Modeling of Reading and 

Mathematics Trends 

The trends in achievement for MSAP and non-MSAP schools are analyzed in a four-level 

hierarchical linear regression model. The four levels included in the model are the states (schools 

from 14 states are included in the analysis), comparison blocks of schools within the states (each 

of 135 blocks consists of one MSAP magnet school and 10 comparison schools from the same 

state, for a total of 1,485 schools), schools within the blocks, and grades within the schools (test 

scores are reported separately for each of the grades 3 through 5, and states vary in which grades 

they test). The full (four-level) hierarchical linear model is given by the following, where Yijkl is 

the change in performance for grade i in school j in block k in state l, between the baseline year 

and 2000–2001, the final year of the MSAP grants. 

Level-1: Grade 

Yijkl = α0jkl + α1jklGrade4ijkl + α2jklGrade5ijkl + sijkl

Level-2: School 

α0jkl = π00kl + π01kl(PctMin01-PctMin97)jkl + π02klPctFRLjkl + π03klMagnetjkl + e0jkl

Level-3: Comparison Block 

π00kl = β000l + r00kl 

Level-4: State 

β000l = γ0000 + u000l 

Random Effects: 

sijkl = random grade-level effect; varies between grades within schools across blocks across states 

e0jkl = random school-level effect; varies between schools across blocks across states 

r00kl = random block-level effect; varies across blocks across states 

u000l = random state-level effect; varies across states 

The results of the HLM analysis for elementary reading and mathematics are shown in Exhibit 

A-V-25. 
Appendix V 

Appendix-V-14

Exhibit A-V-25. 

Parameter estimates for MSAP and comparison elementary schools: multilevel model for 

change in standardized test sores between 1997–1998 and 2000–2001* 

 Est. SE P-value Est. SE P-value 

Grade-level variables Reading Mathematics 

Intercept 0.485 0.118 0.001 0.455 0.122 0.002 

Grade 4 -0.039 0.035 0.263 0.045 0.039 0.244 

Grade 5 -0.080 0.034 0.018 0.030 0.038 0.431 

School-level variables 

MSAP 0.020 0.052 0.697 -0.055 0.054 0.311 

Percent Change in 

Minority 

-0.017 0.003 <.0001 -0.015 0.003 <.0001 

Percent Free/Reduced 

Lunch (1999) 

-0.001 0.001 0.192 <0.001 <0.001 0.530 

 

Unexplained variance 

Between States 0.102 0.118 

Between Comparison 

Blocks 

0.082 0.047 

Between Schools 0.133 0.115 

Between Grades 0.276 0.346 

Number of schools 1485** 1485** 

*Change is measured in baseline year standard deviation units. For most states, the baseline year is 1997–1998, but in a 

few it is 1998–1999. In all states, the terminal year is 2000–2001. 

** Sample consists of 135 blocks, each containing one MSAP elementary school and ten matched comparison schools 

from the same state. 

Source: National School Level Assessment Score Database and responses to 2000 and 2001 Principal Surveys