The NAEP 1996

Technical Report Technical Report

U.S. Department of Education

Office of Educational Research and Improvement NCES 1999-452

NATIONAL CENTER FOR EDUCATION STATISTICS
What is The Nation’s Report Card?

THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is the only nationally

representative and continuing assessment of what America’s students know and can do in various subject areas. Since 1969,

assessments have been conducted periodically in reading, mathematics, science, writing, history, geography, and other

fields. By making objective information on student performance available to policymakers at the national, state, and local

levels, NAEP is an integral part of our nation’s evaluation of the condition and progress of education. Only information

related to academic achievement is collected under this program. NAEP guarantees the privacy of individual students and

their families.

NAEP is a congressionally mandated project of the National Center for Education Statistics, the U.S. Department of

Education. The Commissioner of Education Statistics is responsible, by law, for carrying out the NAEP project through

competitive awards to qualified organizations. NAEP reports directly to the Commissioner, who is also responsible

for providing continuing reviews, including validation studies and solicitation of public comment, on NAEP’s conduct

and usefulness.

In 1988, Congress established the National Assessment Governing Board (NAGB) to formulate policy guidelines for

NAEP. The Board is responsible for selecting the subject areas to be assessed from among those included in the National

Education Goals; for setting appropriate student performance levels; for developing assessment objectives and test

specifications through a national consensus approach; for designing the assessment methodology; for developing guidelines

for reporting and disseminating NAEP results; for developing standards and procedures for interstate, regional, and national

comparisons; for determining the appropriateness of test items and ensuring they are free from bias; and for taking actions

to improve the form and use of the National Assessment.

The National Assessment Governing Board

Mark D. Musick, Chair

President

Southern Regional Education Board

Atlanta, Georgia

Michael T. Nettles, Vice Chair

Professor of Education & Public Policy

University of Michigan

Ann Arbor, Michigan

and Director

Frederick D. Patterson Research Institute

United Negro College Fund

Moses Barnes

Secondary School Principal

Fort Lauderdale, Florida

Melanie A. Campbell

Fourth-Grade Teacher

Topeka, Kansas

Honorable Wilmer S. Cody

Commissioner of Education

State of Kentucky

Frankfort, Kentucky

Edward Donley

Former Chairman

Air Products & Chemicals, Inc.

Allentown, Pennsylvania

Honorable John M. Engler

Governor of Michigan

Lansing, Michigan

Thomas H. Fisher

Director, Student Assessment Services

Florida Department of Education

Tallahassee, Florida

Michael J. Guerra

Executive Director

National Catholic Education Association

Secondary School Department

Washington, DC

Edward H. Haertel

Professor, School of Education

Stanford University

Stanford, California

Juanita Haugen

Local School Board President

Pleasanton, California

Carole Kennedy

Elementary School Principal

Columbia, Missouri

Honorable Nancy Kopp

Maryland House of Delegates

Bethesda, Maryland

Honorable William J. Moloney

Commissioner of Education

State of Colorado

Denver, Colorado

Mitsugi Nakashima

First Vice-Chairperson

Hawaii State Board of Education

Honolulu, Hawaii

Debra Paulson

Eighth-Grade Mathematics Teacher

El Paso, Texas

Honorable Norma Paulus

Former Superintendent

of Public Instruction

Oregon State Department of Education

Salem, Oregon

Honorable Jo Ann Pottorff

Kansas House of Representatives

Wichita, Kansas

Diane Ravitch

Senior Research Scholar

New York University

New York, New York

Honorable Roy Romer

(Member Designate)

Former Governor of Colorado

Denver, Colorado

John H. Stevens

Executive Director

Texas Business and Education Coalition

Austin, Texas

Adam Urbanski

President

Rochester Teachers Association

Rochester, New York

Deborah Voltz

Assistant Professor

Department of Special Education

University of Louisville

Louisville, Kentucky

Marilyn A. Whirry

Twelfth-Grade English Teacher

Manhattan Beach, California

Dennie Palmer Wolf

Senior Research Associate

Harvard Graduate School of Education

Cambridge, Massachusetts

C. Kent McGuire (Ex-Officio)

Assistant Secretary of Education

Office of Educational Research

and Improvement

U.S. Department of Education

Washington, DC

Roy Truby

Executive Director, NAGB

Washington, DC
The NAEP 1996

Technical Report

	







	


	



	





 !"#

!$%"
&%
$	'#("#)&#


*+,)+
%
,%#(#

)-(*#'./$
#0

+
)	#1(
 .#*2 #+
" #


1%
,#%,
%%
11(#

#.#
#

(3



	









		

  !"

The NAEP 1996

Technical Report

NATIONAL CENTER FOR EDUCATION STATISTICS
U.S. Department of Education

Richard W. Riley

Secretary

Office of Educational Research and Improvement

C. Kent McGuire

Assistant Secretary

National Center for Education Statistics

Pascal D. Forgione, Jr.

Commissioner

Assessment Division

Peggy G. Carr

Associate Commissioner

July 1999

This Internet edition of The NAEP 1996 Technical Report includes the changes to the printed report

that are being distributed as an errata sheet.

SUGGESTED CITATION

U.S. Department of Education. Office of Educational Research and Improvement. National Center for

Education Statistics. The NAEP 1996 Technical Report, NCES 1999–452, by Allen, N.L., Carlson, J.E., &

Zelenak, C.A. (1999). Washington, DC: National Center for Education Statistics.

FOR MORE INFORMATION

Content contact:

Sheida White

202–219–1675

To obtain single copies of this report, while supplies last, or ordering information on other U.S. Department

of Education products, call toll free 1–877– 4ED PUBS (877–433–7827), or write:

Education Publications Center (ED Pubs)

U.S. Department of Education

P.O. Box 1398

Jessup, MD 20794–1398

TTY/TDD 1–877–576–7734

FAX 301–470–1244

Online ordering via the Internet: http://www.ed.gov/pubs/edpubs.html

Copies also are available in alternate formats upon request.

This report also is available on the World Wide Web: http://nces.ed.gov/naep

The work upon which this publication is based was performed for the National Center for Education

Statistics, Office of Educational Research and Improvement, by Educational Testing Service.
iii

THE NAEP 1996 TECHNICAL REPORT

 Table of Contents 

Introduction The NAEP 1996 Technical Report

James E. Carlson, Educational Testing Service........................................................................1

Chapter 1 Overview of Part I: The Design and Implementation of the 1996 NAEP

Nancy L. Allen and Eugene G. Johnson, Educational Testing Service.....................................5

1.1 Introduction ...............................................................................................................................5

1.2 The 1996 NAEP Design ............................................................................................................6

1.3 Development of Assessment Objectives, Items, and Background Questions .........................15

1.4 The 1996 Sample Design.........................................................................................................16

1.5 Assessment Instruments ..........................................................................................................19

1.6 Field Operations and Data Collection .....................................................................................22

1.7 Materials and Data Processing ................................................................................................23

1.8 Professional Scoring................................................................................................................23

1.9 Creation of the Database .........................................................................................................24

Chapter 2 Developing the NAEP Objectives, Items, and Background Questions for the

1996 Assessments of Mathematics and Science

Stephen Lazer, Educational Testing Service ...........................................................................25

2.1 Introduction .............................................................................................................................25

2.2 General Overview of the 1996 Assessment Frameworks........................................................27

2.3 General Overview of Procedures for Developing the Items....................................................28

2.4 Developing the Mathematics Assessment ...............................................................................29

2.5 Developing the Science Assessment .......................................................................................43

Chapter 3 Sample Design

Leslie Wallace and Keith F. Rust, Westat, Inc. .......................................................................55

3.1 Introduction .............................................................................................................................55

3.2 Primary Sampling Units ..........................................................................................................57

3.3 Selection of Schools ................................................................................................................59

3.4 Assignment of Sample Type to Schools..................................................................................63

3.5 Assignment of Sessions to Schools .........................................................................................64

3.6 Sampling Students ...................................................................................................................65

3.7 Oversampling of SD/LEP Students for Main Sample Mathematics and Science

Assessments.............................................................................................................................67

3.8 Excluded Students ...................................................................................................................69

3.9 Student Participation Rates .....................................................................................................70

3.10 Overall Student Participation Rates ........................................................................................71

3.11 Sampling Teachers ..................................................................................................................72
iv

Chapter 4 Assessment Instruments

Stephen Lazer, Educational Testing Service ......................................................................... 75

4.1 Introduction............................................................................................................................ 75

4.2 Student Booklets—Main Assessments.................................................................................. 75

4.3 Student Booklets—Long-Term Trend Assessments.............................................................. 88

4.4 Student, Teacher, and School Questionnaires ....................................................................... 93

Chapter 5 Field Operations and Data Collection

Lucy M. Gray, Mark M. Waksberg, and Nancy W. Caldwell, Westat, Inc. ........................... 97

5.1 Introduction............................................................................................................................ 97

5.2 Preparing for the Assessments............................................................................................... 98

5.3 Selecting the Student Sample for Main NAEP.................................................................... 103

5.4 Conducting the Assessment Sessions .................................................................................. 106

5.5 Results of the Main NAEP Assessment............................................................................... 107

5.6 Long-Term Trend Assessments........................................................................................... 109

5.7 Field Management ............................................................................................................... 112

Chapter 6 Processing Assessment Materials

Patrick B. Bourgeacq, Bradley Thayer, and Timothy Robinson,

National Computer Systems................................................................................................. 115

6.1 Introduction.......................................................................................................................... 115

6.2 Printing ................................................................................................................................ 127

6.3 Packaging and Shipping....................................................................................................... 127

6.4 Processing............................................................................................................................ 128

6.5 Data Delivery....................................................................................................................... 137

6.6 Miscellaneous ...................................................................................................................... 138

Chapter 7 Professional Scoring

Patrick B. Bourgeacq, Bradley Thayer, and Timothy Robinson

National Computer Systems................................................................................................. 141

7.1 Introduction.......................................................................................................................... 141

7.2 Long-Term Trend Assessments........................................................................................... 144

7.3 Main NAEP Assessment...................................................................................................... 155

Chapter 8 Creation of the Database, Quality Control of the Data Entry, and Creation

of the Database Products

John J. Ferris, Katharine E. Pashley, David S. Freund, and Alfred M. Rogers,

Educational Testing Service ................................................................................................ 167

8.1 Introduction.......................................................................................................................... 167

8.2 Creation of the Database...................................................................................................... 167

8.3 Quality Control of NAEP Data Entry for 1996 ................................................................... 169

8.4 NAEP Database Products .................................................................................................... 172
v

Chapter 9 Overview of Part II: The Analysis of 1996 NAEP Data

Nancy L. Allen and James E. Carlson, Educational Testing Service.................................. 179

9.1 Introduction ......................................................................................................................... 179

9.2 Summary of the NAEP Design............................................................................................ 180

9.3 Analysis Steps ..................................................................................................................... 182

9.4 Overview of Chapters 10 through 19 .................................................................................. 194

Chapter 10 Weighting Procedures and Estimation of Sampling Variance

Eugene G. Johnson and Jiahe Qian, Educational Testing Service; Leslie Wallace and

Keith F. Rust, Westat, Inc.................................................................................................... 197

10.1 Introduction ......................................................................................................................... 197

10.2 Derivation of the Sample Weights ...................................................................................... 199

10.3 Procedures Used by NAEP to Estimate Sampling Variability............................................ 222

10.4 Approximating the Sampling Variance Using Design Effects............................................ 226

Chapter 11 Scaling Procedures

Nancy L. Allen, Eugene G. Johnson, Robert J. Mislevy, and Neal Thomas,

Educational Testing Service................................................................................................ 235

11.1 Introduction ......................................................................................................................... 235

11.2 Background.......................................................................................................................... 235

11.3 Scaling Methodology........................................................................................................... 237

11.4 Inferences about Proficiencies............................................................................................. 246

11.5 Describing Student Performance......................................................................................... 250

11.6 Overview of the 1996 NAEP Scales.................................................................................... 253

Chapter 12 Data Analysis for the Mathematics Assessment

Frank Jenkins, Hua-Hua Chang, and Edward Kulick, Educational Testing Service ......... 255

12.1 Introduction ......................................................................................................................... 255

12.2 Description of Student Sample, Items, and Assessment Booklets ...................................... 256

12.3 Special Mathematics Assessments ...................................................................................... 261

12.4 Assessing the Effect of Changing Inclusion Criterion and Accommodations for

SD/LEP Students................................................................................................................. 262

12.5 Item Analysis....................................................................................................................... 263

Chapter 13 Data Analysis for the Science Assessment

John R. Donoghue, Jinming Zhang, Steven P. Isham, Lois H. Worthington,

and Ingeborg U. Novatkoski, Educational Testing Service ................................................ 291

13.1 Overview ............................................................................................................................. 291

13.2 Description of Items and Assessment Booklets .................................................................. 292

13.3 Special Science Assessment................................................................................................ 297

13.4 Item Analysis....................................................................................................................... 298
vi

Chapter 13 (continued)

13.5 Item Response Theory (IRT) Scaling ....................................................................................305

13.6 Generation of Plausible Values .............................................................................................318

13.7 The Final Proficiency Scales .................................................................................................319

13.8 Partitioning of the Estimation Error Variance.......................................................................321

13.9 Science Teacher Questionnaires...........................................................................................322

 13.10 The Weight Files ...................................................................................................................322

Chapter 14 Data Analysis for the Long-Term Trend Reading Assessment

Jo-Lin Liang and Lois H. Worthington, Educational Testing Service ..................................323

14.1 Introduction............................................................................................................................323

14.2 Item Analysis for the Reading Long-Term Trend Assessment .............................................326

14.3 Treatment of Constructed-Response Items............................................................................328

14.4 IRT Scaling for the Reading Long-Term Trend Assessment ................................................328

14.5 Generation of Plausible Values .............................................................................................332

14.6 The Final Reading Long-Term Trend Scale..........................................................................333

14.7 Partitioning of the Estimation Error Variance.......................................................................333

Chapter 15 Data Analysis for the Long-Term Trend Mathematics Assessment

Jiahe Qian and Norma A. Norris, Educational Testing Service ...........................................335

15.1 Introduction............................................................................................................................335

15.2 Item Analysis for the Mathematics Long-Term Trend Assessment ......................................338

15.3 IRT Scaling for the Mathematics Long-Term Trend Assessment.........................................340

15.4 Derived Background Variables..............................................................................................340

15.5 Generation of Plausible Values .............................................................................................342

15.6 The Final Mathematics Long-Term Trend Scale...................................................................343

15.7 Partitioning of the Estimation Error Variance.......................................................................344

Chapter 16 Data Analysis for the Long-Term Trend Science Assessment

Jinming Zhang and Norma A. Norris, Educational Testing Service .....................................345

16.1 Introduction............................................................................................................................345

16.2 Item Analysis for the Science Long-Term Trend Assessment ..............................................348

16.3 IRT Scaling for the Science Long-Term Trend Assessment .................................................349

16.4 Generation of Plausible Values .............................................................................................351

16.5 The Final Science Long-Term Trend Scale...........................................................................351

16.6 Partitioning of the Estimation Error Variance.......................................................................353

Chapter 17 Data Analysis for the Long-Term Trend Writing Assessment

Eiji Muraki and Bruce A. Kaplan, Educational Testing Service...........................................355

17.1 Introduction............................................................................................................................355

17.2 Long-Term Trend Writing Data Analysis .............................................................................356

17.3 Item Analysis for the Writing Long-Term Trend Assessment ..............................................358
vii

Chapter 17 (continued)

17.4 IRT Scaling for the Writing Long-Term Trend Assessment.................................................362

17.5 Generation of Plausible Values .............................................................................................364

17.6 The Final Writing Long-Term Trend Scale...........................................................................365

17.7 Partitioning of the Estimation Error Variance.......................................................................366

Chapter 18 Conventions Used in Hypothesis Testing and Reporting NAEP Results

Spencer S. Swinton, David S. Freund, and Nancy L. Allen, Educational Testing Service....367

18.1 Overview ...............................................................................................................................367

18.2 Minimum School and Student Sample Sizes for Reporting Subgroup Results.....................369

18.3 Identifying Estimates of Standard Errors with Large Mean Squared Errors ........................370

18.4 Treatment of Missing Data from the Student, Teacher, and School Questionnaires............370

18.5 Hypothesis Testing Conventions...........................................................................................371

Chapter 19 Statistical Summary of the 1996 NAEP Samples

Bruce A. Kaplan, Educational Testing Service .....................................................................377

19.1 Introduction ...........................................................................................................................377

19.2 Measurement Instruments......................................................................................................377

19.3 Sample Characteristics ..........................................................................................................378

19.4 Population Estimates .............................................................................................................379

Appendix A Participants in the Objectives and Item Development Process .............................................423

Appendix B Reporting Subgroups for the 1996 NAEP Assessment .........................................................431

Appendix C Conditioning Variables and Contrast Codings......................................................................449

Appendix D IRT Parameters......................................................................................................................671

Appendix E Estimation Error Variance by Gender and Race/Ethnicity ...................................................725

Appendix F Setting the NAEP Achievement Levels for the 1996 Mathematics Assessment ..................739

Appendix G Report on Developing Achievement Level Descriptions for the 1996 NAEP

Science Assessment...............................................................................................................759

Appendix H The Information Weighting Error .........................................................................................769

Appendix I Constructed-Response Item Score Statistics .........................................................................785

Appendix J Differential Item Functioning (DIF) Results.........................................................................811

Appendix K Correction of the NAEP Program Documentation Error in the 1992 State

Mathematics Results..............................................................................................................813

References ...............................................................................................................................................827
viii
ix

THE NAEP 1996 TECHNICAL REPORT

 List of Tables 

Table 1-1 NAEP 1996 Student Samples............................................................................................................ 9

 1-2 National Assessment of Educational Progress Subject Areas, Grades, and Ages

Assessed: 1969-1996...................................................................................................................... 13

Table 2-1 Percentage Distribution of Items by Grade and Content Strand as Specified in the

NAEP Mathematics Framework...................................................................................................... 35

2-2 Percentage Distribution of Items by Grade and Mathematical Ability as Specified in

the NAEP Mathematics Framework................................................................................................ 35

2-3 Distribution of Assessment Items for the Mathematics Assessment, Grade 4................................ 39

2-4 Distribution of Assessment Items for the Mathematics Assessment, Grade 8................................ 39

2-5 Distribution of Assessment Items for the Mathematics Assessment, Grade 12.............................. 40

2-6 Percentage Distribution of Items by Grade and Field of Science as Specified in the

NAEP Science Framework.............................................................................................................. 47

2-7 Percentage Distribution of Items by Grade and Knowing and Doing Science as

Specified in the NAEP Science Framework.................................................................................... 48

2-8 Percentage Distribution of Items Devoted to Nature of Science and Themes as

Specified in the NAEP Science Framework.................................................................................... 48

2-9 Distribution of Assessment Time by Field Science ........................................................................ 51

2-10 Distribution of Assessment Time Across Cognitive Domains (Knowing and Doing) ................... 52

2-11 Percentage of Assessment Time Devoted to Themes ..................................................................... 52

2-12 Percentage of Assessment Time Devoted to Nature of Science ..................................................... 52

Table 3-1 Assessment Type by Age Class and Assessment Period................................................................. 56

3-2 Geographic Regions Used for Stratification ................................................................................... 57

3-3 The Sampling Major Strata and the Number of Noncertainty Strata in Each................................. 58

3-4 Grade Definition of School Eligibility for Sampling Frame Inclusion and Frame Sizes,

Main and Long-Term Trend Samples ............................................................................................. 60

3-5 School Sample Sizes, Refusals, and Substitutes for the Main Samples.......................................... 63

3-6 School Sample Sizes, Refusals, and Substitutes for the Long-Term Trend Samples ..................... 63

3-7 Number of Students Per School for Each Session Type ................................................................. 66

3-8 Percentage of Sampled Students Who Were Specified as SD/LEP in the 1996 Main

Samples - Mathematics and Science ............................................................................................... 68

3-9 Student Exclusion Rates by Age Class and School Type and Sample Type, Weighted................. 69

3-10 Student Participation Rates by Age Class and School Type, Unweighted ..................................... 70

3-11 Overall Participation Rates (School and Student Combined) by Age Class, Unweighted ............. 71

3-12 Weighted Participation Rates by Age Class and Session Type, 1996 Main NAEP

Reporting Samples .......................................................................................................................... 72

3-13 Weighted Participation Rates by Age Class and Session Type, 1996 Long-Term Trend

Samples ........................................................................................................................................... 72
x

Table 4-1 Main Sample Booklet Configuration Grade 4, Mathematics ................................................... 77

 4-2 Main Sample Booklet Configuration Grade 8, Mathematics ................................................... 78

 4-3 Main Sample Booklet Configuration Grade 12, Mathematics ................................................. 79

4-4 1996 Mathematics Assessment, Main BIB............................................................................... 80

4-5 1996 Mathematics Assessment, Estimation and Targeted Assessment ................................... 81

4-6 Background Sections of Student Mathematics Booklets.......................................................... 81

4-7 Main Sample Booklet Configuration Grade 4, Science ........................................................... 83

4-8 Main Sample Booklet Configuration Grade 8, Science ........................................................... 84

4-9 Main Sample Booklet Configuration Grade 12, Science ......................................................... 85

4-10 Main Sample Booklet Configuration Grade 12, Advanced Science ........................................ 86

4-11 1996 Science Assessment, Main BIB....................................................................................... 87

4-12 1996 Science Assessment, Advanced Assessment................................................................... 88

4-13 Background Sections of 1996 Student Science Booklets......................................................... 88

4-14 Long-Term Trend Sample Booklet Contents and Number of Booklets Administered,

Age Class 9............................................................................................................................... 89

4-15 Long-Term Trend Sample Booklet Contents and Number of Booklets Administered,

Age Class 13............................................................................................................................. 90

4-16 Long-Term Trend Sample Booklet Contents and Number of Booklets Administered,

Age Class 17............................................................................................................................. 90

4-17 Long-Term Trend Sample Block Information, Age Class 9 .................................................... 91

4-18 Long-Term Trend Sample Block Information, Age Class 13 .................................................. 92

4-19 Long-Term Trend Sample Block Information, Age Class 17 .................................................. 93

Table 5-1 Comparison of Student and School Response Rates for Main NAEP, 1990-1996 ................ 107

5-2 Background Questionnaires Received for Schools, Teachers, and SD/LEP Students in

the 1996 Main Assessment..................................................................................................... 109

5-3 Comparison of Student and School Response Rates for Long-Term Trend NAEP,

1990-1996............................................................................................................................... 111

Table 6-1 Processing and Scoring Totals for the 1996 NAEP Assessment............................................ 116

6-2 Student Participation and Session Information for the 1996 NAEP Assessment .................. 118

6-3 Questionnaire Totals for the 1996 NAEP Assessment........................................................... 119

6-4 NCS Schedule for the 1996 NAEP Assessment..................................................................... 120

6-5 Alerts for the 1996 National and State Assessments.............................................................. 139

Table 7-1 Number of Constructed-Response Items by Score-Point Levels............................................ 145

7-2 Professional Scoring - Readers and Dates.............................................................................. 146

7-3 Interreader Reliability Ranges................................................................................................ 147

7-4 Holistic Writing Scoring Reliability Figures.......................................................................... 152

7-5 Mathematics Constructed Responses Scored ......................................................................... 159

7-6 Science Constructed Responses Scored ................................................................................. 165

Table 8-1 Summary of Quality Control Error Analysis for NAEP 1996 Data Entry ............................. 171
xi

Table 9-1 1996 Mathematics and Science State and National Assessments Ranges of Percentage

Exact Agreement Among Readers .................................................................................................. 182

Table 10-1 Reporting Samples for the 1996 Main NAEP Samples .................................................................. 200

10-2 Distribution of Populations of Eligible Students Based on Trimmed Weights of Assessed

Students in Participating Schools, 1996 Main NAEP Science Samples, Sample Type 2 .............. 206

10-3 Reporting Factors for Assessed and Excluded Students................................................................. 207

10-4 Major Subgroups for Poststratification in 1996.............................................................................. 208

10-5 Distribution of Final Student Reporting Weights, 1996 Main Samples......................................... 210

10-6 Distribution of Populations of Eligible Students Based on Full Weighted Sample of

Eligible Schools, Before and After School Nonresponse Adjustments, 1996 Main NAEP

Science Samples, Sample Type 2 ................................................................................................... 212

10-7 Distribution of Populations of Eligible Students Before and After Student Nonresponse

Adjustments, 1996 Main NAEP Science Samples, Sample Type 2 ............................................... 213

10-8 Distribution of Populations of Eligible Students Before School and Student

Nonresponse Adjustments, 1996 Main NAEP Science Samples, Sample Type 2 ......................... 214

10-9 Weighted Distribution of Absent Students by Nature of Absenteeism for All Grades

1996 Science Assessment, Sample Type 2 ..................................................................................... 215

10-10 Distribution of Modular Weights, Main Samples........................................................................... 217

10-11 Distribution of Final Student Weights, Long-Term Trend Samples............................................... 219

10-12 First and Second Categorical Variables Used for Raking .............................................................. 220

10-13 Third Categorical Variable Used for Raking.................................................................................. 221

10-14 Percentiles of Raking Adjustments................................................................................................. 221

10-15 Design Effects by Demographic Subgroup and Grade for Mean Mathematics

Proficiencies.................................................................................................................................... 229

10-16 Design Effects of Subsample A1 by Demographic Subgroup and Grade for Mean

Mathematics Proficiencies.............................................................................................................. 229

10-17 Design Effects of Subsample A2 by Demographic Subgroup and Grade for Mean

Mathematics Proficiencies.............................................................................................................. 230

10-18 Design Effects of Subsample B1 by Demographic Subgroup and Grade for Mean

Mathematics Proficiencies.............................................................................................................. 230

10-19 Design Effects by Demographic Subgroup and Grade for Mean Science Proficiencies................ 231

10-20 Design Effects by Demographic Subgroup and Age Averaged Over Mean Reading and

Writing Proficiencies for the Reading and Writing Long-Term Trend Samples............................ 231

10-21 Design Effects by Demographic Subgroup and Age Averaged Over Mean Mathematics

and Science Proficiencies for the Mathematics and Science Long-Term Trend Samples ............. 232

10-22 Average Design Effects by Demographic Subgroup for Mean State Mathematics

Proficiencies Averaged Across State Samples ............................................................................... 232

10-23 Average Design Effects by Demographic Subgroup for Mean State Science

Proficiencies Averaged Across State Samples ............................................................................... 233

10-24 Within-Grade Mean, Median, and Upper Quartile of the Distribution of Design Effects

for National Main Assessments by Subject Area and Across Subject Areas. ................................ 233

10-25 Mean, Median, and Upper Quartile of the Across-State Average Design Effects for

Mean State Mathematics Proficiency (Distribution Across Demographic Subgroups) ................. 234

10-26 Mean, Median, and Upper Quartile of the Across-State Average Design Effects for

Mean State Science Proficiency (Distribution Across Demographic Subgroups) ......................... 234

10-27 Mean, Median, and Upper Quartile of the Across-State Average Design Effects for Mean

State Proficiency (Distribution Across Demographic Subgroups and Across Subjects)................ 234

10-28 Mean, Median, and Upper Quartile of the Distribution of Design Effects for the LongTerm Trend Samples....................................................................................................................... 234
xii

Table 11-1 Estimation Error Variance and Related Coefficients for the 1992 Grade 4 Reading

Composite (Based on Five Plausible Values) ............................................................................ 250

Table 12-1 NAEP 1996 Mathematics Student Samples............................................................................... 256

12-2 National Main BIB: Numbers of Scaled Mathematics Items Common Across Grade

Levels, by Content Strand Scale ................................................................................................ 257

12-3 1996 NAEP Mathematics Block Composition by Content Strand and Item Type, Grade

4, As Defined Before Scaling..................................................................................................... 258

12-4 1996 NAEP Mathematics Block Composition by Content Strand and Item Type, Grade

4, As Defined After Scaling....................................................................................................... 259

12-5 1996 NAEP Mathematics Block Composition by Content Strand and Item Type, Grade

8, As Defined Before Scaling..................................................................................................... 259

12-6 1996 NAEP Mathematics Block Composition by Content Strand and Item Type, Grade

8, As Defined After Scaling. ...................................................................................................... 260

12-7 1996 NAEP Mathematics Block Composition by Content Strand and Item Type, Grade

12, As Defined Before Scaling................................................................................................... 260

12-8 1996 NAEP Mathematics Block Composition by Content Strand and Item Type, Grade

12, As Defined After Scaling..................................................................................................... 261

12-9 Block Structure of the Special Study Booklets.......................................................................... 261

12-10 Number of Cognitive Items in the 1996 Special Mathematics Assessment Blocks .................. 262

12-11 Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the Mathematics Main Sample, Grade 4, As Defined After Scaling .............. 264

12-12 Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the Mathematics Main Sample, Grade 8, As Defined After Scaling .............. 265

12-13 Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the Mathematics Main Sample, Grade 12, As Defined After Scaling ............ 266

12-14 DIF Category by Grade for Dichotomous Items ........................................................................ 269

12-15 DIF Category by Grade for Polytomous Items........................................................................... 270

12-16 Labels for Score Levels of Polytomous Items............................................................................ 271

12-17 Items from the 1996 Assessment in Mathematics Receiving Special Treatment ...................... 272

12-18 Proportion of Scale Score Variance Accounted for by the Conditioning Model for the

Mathematics Main Assessment.................................................................................................. 284

12-19 Coefficients of the Linear Transformations that Transform the Five Content Strand

Scales from the 1996 Calibration Metric to the 1992 Reporting Metric ................................... 286

12-20 Weights for the Mathematics Composite by Grade ................................................................... 286

12-21 Means and Standard Deviations on the Mathematics Composite Scale .................................... 286

12-22 Estimation Error Variance and Related Coefficients for the Mathematics Main Assessment .. 287

12-23 Conditional Correlations from Conditioning (CGROUP) ......................................................... 288

12-24 Marginal Correlations of Science Scales ................................................................................... 289
xiii

Table 13-1 NAEP 1996 Science Student Samples ....................................................................................... 291

13-2 Number of Items in Scales in the Science Main Assessment by Field of Science .................... 293

13-3 1996 NAEP Science Block Composition by Item Type, Grade 4, As Defined Before

Scaling........................................................................................................................................ 294

13-4 1996 NAEP Science Block Composition by Item Type, Grade 8, As Defined Before

Scaling........................................................................................................................................ 294

13-5 1996 NAEP Science Block Composition by Item Type, Grade 12, As Defined Before

Scaling........................................................................................................................................ 295

13-6 1996 NAEP Science Block Composition by Item Type, Grade 4, As Defined After

Scaling........................................................................................................................................ 296

13-7 1996 NAEP Science Block Composition by Item Type, Grade 8, As Defined After

Scaling........................................................................................................................................ 296

13-8 1996 NAEP Science Block Composition by Item Type, Grade 12, As Defined After

Scaling........................................................................................................................................ 297

13-9 Block Structure of the 12 [Sci-Advanced] Special Study Booklets........................................... 298

13-10 Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the Science Main Sample, Grade 4, As Defined After Scaling....................... 299

13-11 Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the Science Main Sample, Grade 8, As Defined After Scaling....................... 300

13-12 Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the Science Main Sample, Grade 12, As Defined After Scaling..................... 301

13-13 DIF Category by Grade for Dichotomous Items ........................................................................ 303

13-14 DIF Category by Grade for Polytomous Items........................................................................... 304

13-15 Labels for Score Levels of Polytomous Items ........................................................................... 306

13-16 Items from the 1996 Science Assessment Receiving Special Attention.................................... 314

13-17 Proportion of Proficiency Variance Accounted for by the Conditioning Model for the

Science Main Assessment.......................................................................................................... 318

13-18 Conditional Correlations from Conditioning (CGROUP) ......................................................... 319

13-19 Marginal Correlations of Science Scales ................................................................................... 319

13-20 Coefficients of Linear Transformations of the Fields of Science Scales from the

Calibrating Scale Units to the Units of Reporting Proficiency.................................................. 320

13-21 Weights Used for Each Field of Science Scale to Form the Science Composite ...................... 321

13-22 Estimation Error Variance and Related Coefficients for the Science Main

Assessment................................................................................................................................. 321

Table 14-1 NAEP 1996 Long-Term Trend Reading Student Samples ........................................................ 324

 14-2 NAEP Reading Samples Contributing to 1996 Long-Term Trend Results, 1971-1996............ 325

 14-3 Numbers of Scaled Reading Long-Term Trend Items Common Across Ages .......................... 326

 14-4 Numbers of Scaled Reading Long-Term Trend Items Common Across Assessments.............. 326

 14-5 Descriptive Statistics for Item Blocks in the Reading Long-Term Trend Samples................... 327

 14-6 Items Deleted from the Reading Long-Term Analysis .............................................................. 328

 14-7 Item Calibrated Separately by Assessment Year in the Reading Long-Term Trend

Analysis...................................................................................................................................... 332

 14-8 Proportion of Proficiency Variance Accounted for by the Conditioning Model for the

Reading Long-Term Trend Assessment..................................................................................... 332

 14-9 Means and Standard Deviations on the Reading Long-Term Trend Scale................................ 334

 14-10 Estimation Error Variance and Related Coefficients for the Reading Long-Term

Trend Assessment ...................................................................................................................... 334
xiv

Table 15-1 NAEP 1996 Mathematics Long-Term Trend Student Samples ................................................... 335

15-2 NAEP Mathematics Samples Contributing to 1996 Long-Term Trend Results,1973-1996 ....... 337

15-3 Numbers of Scaled Mathematics Long-Term Trend Items Common Across Ages .................... 338

15-4 Numbers of Scaled Mathematics Long-Term Trend Items Common Across Assessments ........ 338

15-5 Descriptive Statistics for Item Blocks in the Mathematics Long-Term Trend Samples (1996).. 339

15-6 Items Deleted from the Age 9 Mathematics Long-Term Trend Analysis ................................... 341

15-7 Items Deleted from the Age 13 Mathematics Long-Term Trend Analysis ................................. 341

15-8 Items Deleted from the Age 17 Mathematics Long-Term Trend Analysis ................................. 342

15-9 Proportion of Proficiency Variance Accounted for by the Conditioning Model for the

Mathematics Long-Term Trend Assessment............................................................................... 343

15-10 Means and Standard Deviations on the Mathematics Long-Term Trend Proficiency Scale....... 344

15-11 Estimation Error Variance and Related Coefficients for the Mathematics Long-Term Trend

Assessments ................................................................................................................................. 344

Table 16-1 NAEP 1996 Long-Term Trend Science Student Samples ........................................................... 346

16-2 NAEP Science Samples Contributing to 1996 Long-Term Trend Results, 1970-1996 .............. 347

16-3 Numbers of Scaled Science Long-Term Trend Items Common Across Ages............................. 348

16-4 Numbers of Scaled Science Long-Term Trend Items Common Across Assessments ................ 348

16-5 Descriptive Statistics for Item Blocks in the Science Long-Term Trend Samples (1996).......... 349

16-6 Items Deleted from the Paced-Tape Science Long-Term Trend Analysis .................................. 350

16-7 Proportion of Proficiency Variance Accounted for by the Conditioning Model

for the Science Long-Term Trend Assessment............................................................................ 351

16-8 Means and Standard Deviations on the Science Long-Term Trend Scale .................................. 352

16-9 Estimation Error Variance and Related Coefficients for the Science Long-Term Trend

Assessment................................................................................................................................... 353

Table 17-1 NAEP 1996 Long-Term Trend Writing Student Samples ........................................................... 355

17-2 NAEP Writing Samples Contributing to 1996 Long-Term Trend Results, 1984-1996 .............. 356

17-3 Assignment of 1984-96 Writing Long-Term Trend Items in 1984, 1988, 1990, 1992, 1994,

and 1996....................................................................................................................................... 357

17-4 Percentages of Exact Score Agreement and Interrater Reliability for the Primary Trait

Scoring of the Writing Long-Term Trend Assessment Items...................................................... 359

17-5 Descriptive Statistics for Writing Prompts, 1996 Writing Long-Term Trend Samples ............. 361

17-6 Age/Grade Sample Sizes for Scaling Items Used in Long-Term Trends in Writing

Performance ................................................................................................................................. 363

17-7 Nonoverlapping Sets of Items for the Writing Long-Term Trend Assessment........................... 364

17-8 Proportion of Proficiency Variance Accounted for by the Conditioning Model for the

Writing Long-Term Trend Assessment ....................................................................................... 365

17-9 Means and Standard Deviations on the Writing Long-Term Trend Scale .................................. 366

17-10 Estimation Error Variance and Related Coefficients for the Writing Long-Term Trend

Assessment................................................................................................................................... 366
xv

Table 19-1 Measurement Instruments Developed for 1996 NAEP .......................................................... 380

19-2 Number of Items Administered, by Sample and Age Class ........................................................ 381

19-3 School Characteristics in Main Samples (All Samples).............................................................. 382

19-4 School Characteristics in Long-Term Trend Samples................................................................. 384

19-5 Numbers of Responses to Teacher Questionnaires and Students Matched with

Teacher Data................................................................................................................................ 385

19-6 Number of Students Assessed and Excluded by Sample and Age Class..................................... 386

19-7 Number of Students in the Mathematics Main and Estimation Samples by Subgroup

Classification, Grades 4, 8, and 12 .............................................................................................. 387

19-8 Number of Students in the Mathematics Theme and Advanced Samples by Subgroup

Classification, Grades 4, 18, and 12 ............................................................................................ 388

19-9 Number of Students in the Science Main and Advanced Sample by Subgroup

Classification, Grades 4, 8, and 12 .............................................................................................. 389

19-10 Number of Excluded Students in the Mathematics Main and Estimation Samples by

Subgroup Classification, Grades 4, 8, and 12.............................................................................. 390

19-11 Number of Excluded Students in the Mathematics Theme and Advanced Samples by

Subgroup Classification, Grades 4, 8, and 12.............................................................................. 391

19-12 Number of Excluded Students in the Science Main and Advanced Samples by

Subgroup Classification, Grades 4, 8, and 12.............................................................................. 392

19-13 Number of Students in the Reading and Writing Long-Term Trend Sample by Type

of Eligibility and Subgroup Classification, Age 9/Grade 4......................................................... 393

19-14 Number of Students in the Reading and Writing Long-Term Trend Sample by Type

of Eligibility and Subgroup Classification, Age 13/Grade 8....................................................... 394

19-15 Number of Students in the Reading and Writing Long-Term Trend Sample by Type

of Eligibility and Subgroup Classification, Age 17/Grade 11..................................................... 395

19-16 Number of Students in the Mathematics and Science Long-Term Trend Sample by

Type of Eligibility and Subgroup Classification, Age 9.............................................................. 396

19-17 Number of Students in the Mathematics and Science Long-Term Trend Sample by

Type of Eligibility and Subgroup Classification, Age 13............................................................ 397

19-18 Number of Students in the Mathematics and Science Long-Term Trend Sample by

Type of Eligibility and Subgroup Classification, Age 17............................................................ 398

19-19 Number of Excluded Students in the Reading and Writing Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 9/Grade 4........................................... 399

19-20 Number of Excluded Students in the Reading and Writing Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 13/Grade 8......................................... 400

19-21 Number of Excluded Students in the Reading and Writing Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 17/Grade 11....................................... 401

19-22 Number of Excluded Students in the Mathematics and Science Long-Term Trend

Sample by Type of Eligibility and Subgroup Classification, Age 9/Grade 4.............................. 402

19-23 Number of Excluded Students in the Mathematics and Science Long-Term Trend

Sample by Type of Eligibility and Subgroup Classification, Age 13/Grade 8............................ 403

19-24 Number of Excluded Students in the Mathematics and Science Long-Term Trend

Sample by Type of Eligibility and Subgroup Classification, Age 17/Grade 11.......................... 404

19-25 Weighted Percentage of Students in the Mathematics Main and Estimation Samples

by Subgroup Classification, Grades 4, 8, and 12......................................................................... 405

19-26 Weighted Percentage of Students in the Mathematics Theme and Advanced

Mathematics Samples by Subgroup Classification, Grades 4, 8, and 12 .................................... 406

19-27 Weighted Percentage of Students in the Science Main and Advanced Samples by

Subgroup Classification, Grades 4, 8, and 12.............................................................................. 407

19-28 Weighted Percentage of Excluded Students in the Mathematics Main and Estimation

Samples by Subgroup Classification, Grades 4, 8, and 12 .......................................................... 408
xvi

19-29 Weighted Percentage of Excluded Students in the Mathematics Theme and Advanced

Samples by Subgroup Classification, Grades 4, 8, and 12 .......................................................... 409

19-30 Weighted Percentage of Excluded Students in the Science Main and Advanced Samples

by Subgroup Classification, Grades 4, 8, and 12......................................................................... 410

19-31 Weighted Percentage of Students in the Reading and Writing Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 9/Grades 4 ......................................... 411

19-32 Weighted Percentage of Students in the Reading and Writing Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 13/Grades 8 ....................................... 412

19-33 Weighted Percentage of Students in the Reading and Writing Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 17/Grades 11 ..................................... 413

19-34 Weighted Percentage of Students in the Mathematics and Science Long-Term Trend

Sample by Type of Eligibility and Subgroup Classification, Age 9............................................ 414

19-35 Weighted Percentage of Students in the Mathematics and Science Long-Term Trend

Sample by Type of Eligibility and Subgroup Classification, Age 13.......................................... 415

19-36 Weighted Percentage of Students in the Mathematics and Science Long-Term Trend

Sample by Type of Eligibility and Subgroup Classification, Age 17.......................................... 416

19-37 Weighted Percentage of Excluded Students in the Reading and Writing Long-Term

Trend Sample by Type of Eligibility and Subgroup Classification, Age 9/Grades 4.................. 417

19-38 Weighted Percentage of Excluded Students in the Reading and Writing Long-Term

Trend Sample by Type of Eligibility and Subgroup Classification, Age 13/Grades 8................ 418

19-39 Weighted Percentage of Excluded Students in the Reading and Writing Long-Term

Trend Sample by Type of Eligibility and Subgroup Classification, Age 17/Grades 11.............. 419

19-40 Weighted Percentage of Excluded Students in the Mathematics and Science Long-Term

Trend Sample by Type of Eligibility and Subgroup Classification, Age 9 ................................. 420

19-41 Weighted Percentage of Excluded Students in the Mathematics and Science Long-Term

Trend Sample by Type of Eligibility and Subgroup Classification, Age 13 ............................... 421

19-42 Weighted Percentage of Excluded Students in the Mathematics and Science Long-Term

Trend Sample by Type of Eligibility and Subgroup Classification, Age 17 ............................... 422

Table B-1 NAEP Geographic Regions .................................................................................................... 445

B-2 Scaling Variables for the 1996 National Assessment Samples .............................................. 452
xvii

Table C-1 Description of Specifications Provided for Each Conditioning Variable ................................451

C-2 Summary Table of the 1996 Mathematics Assessment Conditioning Variable

Specifications............................................................................................................................452

C-3 Summary Table of the 1996 Science Conditioning Variable Specifications ...........................463

C-4 Conditioning Variables Specific to the 1996 Mathematics Assessment ..................................475

C-5 Conditioning Variables Specific to the 1996 Science Assessment ..........................................548

C-6 Conditioning Variables for the Long-Term Trend Reading Assessment in 1996....................624

C-7 Conditioning Variables for the Long-Term Trend Mathematics Assessment in 1996.............627

C-8 Conditioning Variables for the Long-Term Trend Science Assessment in 1996.....................632

C-9 Conditioning Variables for the Long-Term Trend Writing Assessment in 1996.....................635

C-10 Proportion of Variance of the Conditioning Variable Contrasts Accounted for by the

Principal Components Used in the Conditioning Model for Mathematics Main

Conditioning Variables, Grade 4 ..............................................................................................638

C-11 Proportion of Variance of the Conditioning Variable Contrasts Accounted for by the

Principal Components Used in the Conditioning Model for Mathematics Main

Conditioning Variables, Grade 8 ..............................................................................................643

C-12 Proportion of Variance of the Conditioning Variable Contrasts Accounted for by the

Principal Components Used in the Conditioning Model for Mathematics Main

Conditioning Variables, Grade 12 ............................................................................................649

C-13 Proportion of Variance of the Conditioning Variable Contrasts Accounted for by the

Principal Components Used in the Conditioning Model for Science Main

Conditioning Variables, Grade 4 ..............................................................................................654

C-14 Proportion of Variance of the Conditioning Variable Contrasts Accounted for by the

Principal Components Used in the Conditioning Model for Science Main

Conditioning Variables, Grade 8 ..............................................................................................660

C-15 Proportion of Variance of the Conditioning Variable Contrasts Accounted for by the

Principal Components Used in the Conditioning Model for Science Main

Conditioning Variables, Grade 12 ............................................................................................666
xviii

Table D-1 IRT Parameters for the 1996 Mathematics Main Samples Number Sense, Properties, and

Operations, Grade 4 ................................................................................................................... 673

D-2 IRT Parameters for the 1996 Mathematics Main Samples Measurement, Grade 4................... 675

D-3 IRT Parameters for the 1996 Mathematics Main Samples Geometry and Spatial Sense,

Grade 4 ....................................................................................................................................... 676

D-4 IRT Parameters for the 1996 Mathematics Main Samples Data Analysis, Statistics, and

Probabilities, Grade 4................................................................................................................. 677

D-5 IRT Parameters for the 1996 Mathematics Main Samples, Algebra and Functions,

Grade 4 ....................................................................................................................................... 678

D-6 IRT Parameters for the 1996 Mathematics Main Samples-Number Sense, Properties, and

Operations, Grade 8 ................................................................................................................... 679

D-7 IRT Parameters for the Mathematics Main Samples Measurement, Grade 8............................ 681

D-8 IRT Parameters for the 1996 Mathematics Main Samples Geometry and Spatial Sense,

Grade 8 ....................................................................................................................................... 682

D-9 IRT Parameters for the 1996 Mathematics Main Samples Data Analysis, Statistics, and

Probability, Grade 8 ................................................................................................................... 683

D-10 IRT Parameters for the 1996 Mathematics Main Samples Algebra and Functions, Grade 8 .... 684

D-11 ÌRT Parameters for the 1996 Mathematics Main Samples-Number Sense, Properties, and

Operations, Grade 12 ................................................................................................................. 685

D-12 IRT Parameters for the 1996 Mathematics Main Samples Measurement, Grade 12................. 686

D-13 IRT Parameters for the 1996 Mathematics Main Samples Geometry and Spatial Sense,

Grade 12 ..................................................................................................................................... 687

D-14 IRT Parameters for the 1996 Mathematics Main Samples Data Analysis, Statistics, and

Probability, Grade 12 ................................................................................................................. 688

D-15 IRT Parameters for the 1996 Mathematics Main Samples, Algebra and Functions,

Grade 12 ..................................................................................................................................... 689

D-16 IRT Parameters for the 1996 Science Main Samples, Physical Science, Grade 4..................... 690

D-17 IRT Parameters for the 1996 Science Main Samples, Earth Science, Grade 4.......................... 691

D-18 IRT Parameters for the 1996 Science Main Samples, Life Science, Grade 4............................ 693

D-19 IRT Parameters for 1996 Science Items, Physical Science, Grade 8......................................... 694

D-20 IRT Parameters for the 1996 Science Main Samples, Earth Science, Grade 8.......................... 696

D-21 IRT Parameters for the 1996 Science Main Samples, Life Science, Grade 8............................ 698

D-22 IRT Parameters for 1996 Science Items, Physical Science, Grade 12....................................... 700

D-23 IRT Parameters for the 1996 Science Main Samples, Earth Science, Grade 12........................ 702

D-24 IRT Parameters for the 1996 Science Main Samples, Life Science, Grade 12.......................... 704

D-25 IRT Parameters for the Reading Long-Term Trend Samples, Age 9......................................... 706

D-26 IRT Parameters for the Reading Long-Term Trend Samples, Age 13....................................... 708

D-27 IRT Parameters for the Reading Long-Term Trend Samples, Age 17....................................... 710

D-28 IRT Parameters for the Mathematics Long-Term Trend Samples, Age 9 ................................. 712

D-29 IRT Parameters for the Mathematics Long-Term Trend Samples, Age 13 ............................... 714

D-30 IRT Parameters for the Mathematics Long-Term Trend Samples, Age 17 ............................... 716

D-31 IRT Parameters for the Science Long-Term Trend Samples, Age 9.......................................... 718

D-32 IRT Parameters for the Science Long-Term Trend Samples, Age 13........................................ 720

D-33 IRT Parameters for the Science Long-Term Trend Samples, Age 17........................................ 722

D-34 IRT Parameters for the Writing Long-Term Trend Samples, Age 9.......................................... 724

D-35 IRT Parameters for the Writing Long-Term Trend Samples, Age 13........................................ 724

D-36 IRT Parameters for the Writing Long-Term Trend Samples, Age 17........................................ 724
xix

Table E-1 Estimation Error Variance for the Main Mathematics Number Sense, Properties, and

Operations Scale ........................................................................................................................ 725

E-2 Estimation Error Variance for the Main Mathematics Measurement Scale ................................... 726

E-3 Estimation Error Variance for the Main Mathematics Geometry and Spatial Sense Scale............ 727

E-4 Estimation Error Variance for the Main Mathematics Data Analysis, Statistics, and

Probability Scale ............................................................................................................................. 728

E-5 Estimation Error Variance for the Main Mathematics Algebra and Functions Scale .................... 729

E-6 Estimation Error Variance for the Main Mathematics Composite Scale........................................ 730

E-7 Estimation Error Variance for the Main Science Earth Science Scale........................................... 731

E-8 Estimation Error Variance for the Main Science Physical Science Scale...................................... 732

E-9 Estimation Error Variance for the Main Science Life Science Scale............................................. 733

E-10 Estimation Error Variance for the Main Science Composite Scale................................................ 734

E-11 Estimation Error Variance for the Reading Long-Term Trend Scale............................................. 735

E-12 Estimation Error Variance for the Mathematics Long-Term Trend Scale ..................................... 736

E-13 Estimation Error Variance for the Science Long-Term Trend Scale.............................................. 737

E-14 Estimation Error Variance for the Writing Long-Term Trend Scale.............................................. 738

Table G-1 Percentage of Panelists Approving the Edited Versions of the Descriptions (Narrative

Format)............................................................................................................................................ 766

Table H-1 Mathematics Cutpoints and Percents At or Above as Reported..................................................... 770

H-2 Corrected Mathematics Cutpoints and Percents At or Above as Reported .................................... 770

H-3 Composite NAEP Scale Cutpoint Differences in Mathematics Due to Errors............................... 771

H-4 NAEP Mathematics Achievement Levels: Cutpoints and 1992 Distribution Data ....................... 772

H-5 Comparison Between the Incorrect and Correct Information Weights for 1992

Mathematics Achievement Levels Based on Corrected Item Parameters ...................................... 774
xx

Table I-1 Score Range, Percent Agreement, and Cohen’s Kappa for the Dichotomously Scored

Constructed-Response Mathematics Items Used in Main Assessment Scaling, Grade 4............786

I-2 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously Scored

Constructed-Response Mathematics Items Used in Main Assessment Scaling, Grade 4............787

I-3 Score Range, Percent Agreement, and Cohen’s Kappa for the Dichotomously Scored

Constructed-Response Mathematics Items Used in Main Assessment Scaling, Grade 8............788

I-4 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously Scored

Constructed-Response Mathematics Items Used in Main Assessment Scaling, Grade 8............789

I-5 Score Range, Percent Agreement, and Cohen’s Kappa for the Dichotomously Scored

Constructed-Response Mathematics Items Used in Main Assessment Scaling, Grade 12..........790

I-6 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously Scored

Constructed-Response Mathematics Items Used in Main Assessment Scaling, Grade 12..........792

I-7 Score Range, Percent Agreement, and Cohen’s Kappa for the Dichotomously Scored

Constructed-Response Science Items Used in Main Assessment Scaling, Grade 4....................793

I-8 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously Scored

Constructed-Response Science Items Used in Main Assessment Scaling, Grade 4....................793

I-9 Score Range, Percent Agreement, and Cohen’s Kappa for the Dichotomously Scored

Constructed-Response Science Items Used in Main Assessment Scaling, Grade 8....................796

I-10 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously Scored

Constructed-Response Science Items Used in Main Assessment Scaling, Grade 8....................796

I-11 Score Range, Percent Agreement, and Cohen’s Kappa for the Dichotomously Scored

Constructed-Response Science Items Used in Main Assessment Scaling, Grade 12..................800

I-12 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously Scored

Constructed-Response Science Items Used in Main Assessment Scaling, Grade 12..................800

I-13 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously

Scored Constructed-Response Reading Items Used in Long-Term Assessment Scaling,

Age Class 9 ..................................................................................................................................803

I-14 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously

Scored Constructed-Response Reading Items Used in Long-Term Assessment Scaling,

Age Class 13 ................................................................................................................................804

I-15 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously

Scored Constructed-Response Reading Items Used in Long-Term Assessment Scaling,

Age Class 17 ................................................................................................................................804

I-16 Score Range, Percent Agreement, and Cohen’s Kappa for the Dichotomously Scored

Constructed-Response Mathematics Items Used in Long-Term Assessment Scaling,

Age Class 9 ..................................................................................................................................805

I-17 Score Range, Percent Agreement, and Cohen’s Kappa for the Dichotomously Scored

Constructed-Response Mathematics Items Used in Long-Term Assessment Scaling,

Age Class 13 ................................................................................................................................806

I-18 Score Range, Percent Agreement, and Cohen’s Kappa for the Dichotomously Scored

Constructed-Response Mathematics Items Used in Long-Term Assessment Scaling,

Age Class 17 ................................................................................................................................807

I-19 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously

Scored Constructed-Response Writing Items Used in Long-Term Assessment Scaling,

Age Class 9 ..................................................................................................................................808

I-20 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously

Scored Constructed-Response Writing Items Used in Long-Term Assessment Scaling,

Age Class 13 ................................................................................................................................808

I-21 Score Range, Percent Agreement, and Intraclass Correlation for the Polytomously

Scored Constructed-Response Reading Items Used in Long-Term Assessment Scaling,

Age Class 17 ................................................................................................................................809
xxi

Table J-1 1996 Mathematics Items Identified as “C” or “CC” Items in At Least One Comparison ......... 811

J-2 1996 Science Items Identified as “C” or “CC” Items in At Least One Comparison ................ 811

Table K-1 Transformation Constants for the 1992 Trial State Assessment in Mathematics...................... 814

K-2 Original and Revised ACT Achievement Level Cut Scores for 1992 Mathematics ................. 815

K-3 NAEP 1992 Trial State Assessment in Mathematics Grade 4 Weighted Percentages

and Composite Scale Means Weighted Means, Standard Deviations, and Percentiles

Original Results ......................................................................................................................... 816

K-4 NAEP 1992 Trial State Assessment in Mathematics Grade 4 Weighted Percentages

and Composite Scale Means Weighted Means, Standard Deviations, and Percentiles

Revised Results.......................................................................................................................... 817

K-5 NAEP 1992 Trial State Assessment in Mathematics Grade 8 Weighted Percentages

and Composite Scale Means Weighted Means, Standard Deviations, and Percentiles

Original Results ......................................................................................................................... 818

K-6 NAEP 1992 Trial State Assessment in Mathematics Grade 8 Weighted Percentages

and Composite Scale Means Weighted Means, Standard Deviations, and Percentiles

Revised Results.......................................................................................................................... 819

K-7 NAEP 1992 Trial State Assessment in Mathematics Grade 8 Weighted Percentages

and Composite Scale Means Percent of Students At or Above the Achievement Levels

Original Results ......................................................................................................................... 820

K-8 NAEP 1992 Trial State Assessment in Mathematics Grade 4 Weighted Percentages

and Composite Scale Means Percent of Students At or Above the Achievement Levels

Revised Results ......................................................................................................................... 821

K-9 NAEP 1992 Trial State Assessment in Mathematics Grade 8 Weighted Percentages

and Composite Scale Means Percent of Students At or Above the Achievement Levels

Original Results ........................................................................................................................ 822

K-10 NAEP 1992 Trial State Assessment in Mathematics Grade 8 Weighted Percentages

and Composite Scale Means Percent of Students At or Above the Achievement Levels

Revised Results.......................................................................................................................... 824

xxiii

THE NAEP 1996 TECHNICAL REPORT

 List of Figures 

Figure 1-1 Subsamples of the Mathematics Assessment: 1996 .....................................................................8

Figure 2-1 Descriptions of Content Strands in Mathematics .......................................................................32

2-2 Descriptions of Mathematical Abilities......................................................................................33

2-3 Total Number of Items for the 1996 Mathematics Main Assessment ........................................34

2-4 Descriptions of the Three Fields of Science...............................................................................45

2-5 Descriptions of Knowing and Doing Science.............................................................................46

2-6 Descriptions of Overarching Domains .......................................................................................46

2-7 Total Number of Items for the 1996 Science Main Assessment ................................................51

Figure 7-1 Image Scoring Flow Chart........................................................................................................142

7-2 Paper Scoring Flow Chart.........................................................................................................143

Figure 11-1 Example Cross-Sectional Dichotomous Item (R016102, Age 13/Grade 8) Exhibiting

Good Model Fit ........................................................................................................................242

11-2 Example Polytomous Item (HC00004/Grade 8) Exhibiting Good Model Fit ..........................243

Figure 12-1 Dichotomous Item Exhibiting Good Model Fit........................................................................276

12-2 Polytomous Item Exhibiting Good Model Fit ..........................................................................277

12-3 Polytomous Item Exhibiting Bad Model Fit That Was Deleted from the Assessment ............278

12-4 Dichotomous Item Exhibiting Different Empirical Item Functions for Different

Assessment Years .....................................................................................................................279

12-5 Dichotomous Item Fit Separately to the 1992 Data and Exhibiting Good Model Fit ..............280

12-6 Dichotomous Item Fit Separately to the 1996 Data and Exhibiting Good Model Fit ..............281

12-7 Polytomous Item Exhibiting Poor Model Fit in the Upper Category.......................................282

12-8 Polytomous Item With the Upper Two Categories Collapsed, Now Exhibiting Good

Model Fit ..................................................................................................................................283

Figure 13-1 Plot Comparing Empirical and Model-Based Estimates of Item Response Functions

for Binary Scored (Multiple-Choice) Items Exhibiting Good Model Fit.................................308

13-2 Plot Comparing Empirical and Model-Based Estimates of Item Category

Characteristic Curves for a Polytomously Scored Item Exhibiting Good Model Fit ...............309

13-3 Plot Comparing Empirical and Model-Based Estimates of Item Response Functions

for Binary-Scored (Multiple-Choice) Item Exhibiting Some Model Misfit.............................310

13-4 Plot Comparing Empirical and Model-Based Estimates of Item Category

Characteristic Curves for a Polytomously Scored Item Exhibiting Some Model Misfit .........311

13-5 Plot Comparing Empirical and Model-Based Estimates of Item Category

Characteristic Curves for a Polytomously Scored Item (K044101) Exhibiting Poor

Model Fit ..................................................................................................................................312

13-6 Plot Comparing Empirical and Model-Based Estimates of Item Category

Characteristic Curves for a Polytomously Scored Item (K044101) After Collapsing

Categories 2 and 3 ....................................................................................................................313
xxiv

Figure 14-1 Example Long-Term Trend Item (N014502, Age 9) Demonstrating Differential Item

Functioning Across Assessment Years 1994 and 1996 .......................................................... 330

14-2 Example Long-Term Trend Item (N014502, Age 9) Fitting Separate Item Response

Functions for Each Assessment Year ..................................................................................... 331

Figure F-1 Final Description of 1992 Mathematics Achievement Levels................................................ 747

F-2 Draft Descriptions of the Achievement Levels Prepared by the Original LevelSetting Panel ........................................................................................................................... 750

F-3 Revised Draft Descriptions of the Achievement Levels Recommended by the

Follow-up Validation Panel .................................................................................................... 753

F-4 Meeting Participants, NAEP Mathematics Achievement Level-Setting Original

Meeting, St. Louis, Missouri, March 20-24, 1992.................................................................. 755

F-5 Meeting Participants, NAEP Mathematics Achievement Level-Setting Follow-up

Validation Meeting, Nantucket, Massachusetts, July 17-19, 1992......................................... 758

Figure G-1 Participating Jurisdictions in Developing Achievement Level Descriptions for the

1996 NAEP Science Assessment............................................................................................ 760

G-2 1996 NAEP Science Achievement Level Descriptions, Grade 4 ........................................... 763

G-3 1996 NAEP Science Achievement Level Descriptions, Grade 8 ........................................... 764

G-4 1996 NAEP Science Achievement Level Descriptions, Grade 12 ......................................... 765

Figure H-1 Comparison Between the Incorrect and Correct Information Functions Using

Hypothetical Item Parameters a=0.2, b=0, d0=0, d1=2, d2=0, and d3=-2, in Case of

Four Response Categories ...................................................................................................... 775

H-2 Comparison Between the Incorrect and Correct Information Functions Using

Hypothetical Item Parameters a=0.5, b=0, d0=0, d1=2, d2=0, and d3=-2, in Case of

Four Response Categories....................................................................................................... 776

H-3 Comparison Between the Incorrect and Correct Information Functions Using

Hypothetical Item Parameters a=1, b=0, d0=0, d1=2, d2=0, and d3=-2, in Case of

Four Response Categories....................................................................................................... 777

H-4 Comparison Between the Incorrect and Correct Information Functions Using

Hypothetical Item Parameters a=1.5, b=0, d0=0, d1=2, d2=0, and d3=-2, in Case of

Four Response Categories....................................................................................................... 779

H-5 Comparison Between the Incorrect and Correct Information Functions Using

Hypothetical Item Parameters a=1.5, b=-1, d0=0, d1=2, d2=0, and d3=-2, in Case of

Four Response Categories....................................................................................................... 780

H-6 Comparison Between the Incorrect and Correct Information Functions Using

Hypothetical Item Parameters a=1.5, b=1, d0=0, d1=2, d2=0, and d3=-2, in Case of

Four Response Categories....................................................................................................... 781

H-7 Comparison Between the Incorrect and Correct Information Functions Using

Hypothetical Item Parameters a=1.5, b=0, d0=0, d1=1, d2=0, and d3=-1, in Case of

Four Response Categories....................................................................................................... 782

H-8 Comparison Between the Incorrect and Correct Information Functions Using

Hypothetical Item Parameters a=1.5, b=0, d0=0, d1=0.52, d2=0, and d3=-0.52, in

Case of Four Response Categories ......................................................................................... 783
xxv

ACKNOWLEDGMENTS

The design, development, administration, analysis, and reporting of the 1996 National

Assessment of Educational Progress (NAEP) program was a collaborative effort among staff from the

National Center for Education Statistics (NCES), the National Assessment Governing Board (NAGB),

the Council of Chief State School Officers (CCSSO), Educational Testing Service (ETS), Westat, Inc.,

and National Computer Systems (NCS). This report documents NAEP design, administration, and data

analysis procedures, indicating what technical decisions were made and the rationale behind those

decisions. The development of this report and of the national assessment program is the result of the

considerable knowledge, experience, creativity, and dedication of many individuals. I would like to

acknowledge these individuals for their contribution to NAEP.

The 1996 NAEP assessment was funded through NCES, in the Office of Educational Research

and Improvement of the U.S. Department of Education. The NCES staff, played a crucial role in all

aspects of the program. We are grateful for the reviews of this report contributed by Janis Brown, Arnold

Goldstein Andrew Kolstad, Ralph Lee, Larry Ogle, Andrew Malizio, Marilyn McMillen, Holly Spurlock,

Michael Ross, Sheida White, and Shi-Chang Wu.

I would also like to gratefully acknowledge Huynh Huynh of the University of South Carolina,

Edward Kifer of the University of Kentucky, and Gary Marco of ETS for conducting a technical review

of this report.

Special thanks also go to the members of the National Assessment Governing Board (NAGB)

and the NAGB staff who provided advice and guidance in the preparation of this report, particularly

Mary Lyn Bourque, who provided information for some of the report appendices.

ETS management and the NAEP program area within ETS’s School and College Services

Division have been very supportive of NAEP’s technical work. Special thanks go to Nancy Cole as well

as to Paul Ramsey, Henry Braun, Charles Davis, Juliet Shaffer (formerly of ETS), and James Carlson of

ETS research management, and Paul Williams (formerly of ETS), Steve Lazer, John Mazzeo, Beth

Durkin, and Kim Whittington of the NAEP program within ETS’s School and College Services Division.

Significant contributions to the project were also received from Nada Ballator, Jay Campbell, Patricia

Donahue, Christine O’Sullivan, and Hilary Persky.

The guidance of the NAEP Design and Analysis Committee on technical aspects of NAEP has

been outstanding. During the period of analysis of the 1996 data, the members were Sylvia Johnson

(chair), Albert Beaton, Jeri Benson, Jeremy Finn, Paul Holland, Huynh Huynh, Gaea Leinhardt, David

Lohman, Anthony Nitko, Ingram Olkin, Tej Pandey, Juliet Shaffer, and Hariharan Swaminathan.

The design and data analysis of the 1996 National Assessment was primarily the responsibility of

the NAEP research and data analysis staff at ETS with significant contributions from NAEP

management, Westat, and NCS staffs. In addition to managing day-to-day data analytic operations, these

staff members have made many innovative statistical and psychometric contributions. Major

contributions from the Division of Statistics and Psychometric Research were made by James Carlson,

John Donoghue, Frank Jenkins, Hua-Hua Chang, Jiahe Qian, Spencer Swinton, and Jinming Zhang. Jolin Liang served as research associate. Eugene Johnson, Robert Mislevy, and Juliet Shaffer provided

valuable statistical and psychometric advice.
xxvi

The Division of Data Analysis and Technology Research at ETS, under the leadership of John

Barone, was responsible for developing the operating systems and carrying out the data analyses. David

Freund and Alfred Rogers developed and maintained the large and complex NAEP data management

systems, and Katharine Pashley managed database activities. Alfred Rogers developed the production

versions of key analysis and scaling systems. Special recognition goes to Phillip Leung and Patricia

O’Reilly for their expertise in the design and development of the NAEP Web site. Thanks also go to

David Freund, Steven Isham, Bruce Kaplan, Debbie Kline, and Edward Kulick for their continuing roles

as leaders and developers of innovative solutions to NAEP data analysis challenges. Many other

members of this division made important contributions of their time and talent to NAEP data analyses

and analysis software and data products, including John Ferris, Shuyi Hua, Gerry Kokolis, Laura Jenkins,

Mike Narcowich, Norma Norris, Ingeborg Novatkoski, Craig Pizzuti, Steve Szyszkiewicz, Xiaohui

Wang, and Lois Worthington.

The staff at Westat contributed their talents and efforts in all areas of the sample design and data

collection. Particular recognition is due to Leslie Wallace, Lucy Gray, Renee Slobasky, Nancy Caldwell,

Keith Rust, Mark Waksberg, Debby Vivari, and Dianne Walsh for directing the sampling and data

collection activities. Thanks are also due to John Burke, James Green, and Penny James.

Critical to the program was the contribution of NCS, which has been responsible for the printing,

distribution, and processing of the assessment materials, as well as an increased role in professional

scoring of constructed-response items. The leadership roles of Patrick Bourgeacq, Charles Brungardt,

Bradley Thayer, and Timothy Robinson are especially acknowledged. Thanks also go to Brent Studer.

Joan Stoeckel, Sharon Stewart, and Donna Lembeck are acknowledged for their administrative

assistance during the preparation of this report.

Carol Errickson also deserves special thanks for coordinating the design of the report’s cover.

Special recognition and appreciation go to Christine Zelenak, Terry Schoeps, and Debbie Kline,

editors of this report. They have been responsible for organizing, scheduling, editing, motivating, and

ensuring the cohesiveness and correctness of the final report.

Finally, NAEP is grateful to the students and school staff members who participated in the

assessment. Without their efforts, there would be no assessment.

Nancy L. Allen

Director, NAEP Analysis: Planning and National Analyses

NAEP Research, ETS

July 1999
1

THE NAEP 1996 TECHNICAL REPORT

Introduction1

James E. Carlson

Educational Testing Service

The 1996 National Assessment of Educational Progress (NAEP) monitored the performance of

students in American schools in the subject areas of reading, mathematics, science, and writing. The

national sample involved nearly 124,000 public and nonpublic-school students who were 9-, 13-, or 17

years old or in grades 4, 8, or 12.

The purpose of this technical report is to provide details on the instrument development, sample

design, data collection, and data analysis procedures of the 1996 assessment. Detailed substantive results

are not presented here but can be found in a series of NAEP reports on the status of and trends in student

performance; several additional reports provide information on how the assessment was designed and

implemented. The reader is directed to the following reports for 1996 results and supporting

documentation:

 NAEP 1996 Mathematics Report Card for the Nation and the States: Findings from

the National Assessment of Educational Progress (Reese, Miller, Mazzeo, & Dossey,

1997)

 NAEP 1996 Science Report Card for the Nation and the States: Findings from the

National Assessment of Educational Progress (O’Sullivan, Reese, & Mazzeo, 1997)

 The NAEP Guide: A Description of the Content and Methods of the 1994 and 1996

Assessments (NAEP, 1996)

 NAEP 1996 Trends in Academic Progress: Achievement of U.S. Students in Science,

1969 to 1996; Mathematics, 1973 to 1996; Reading, 1971 to 1996; and Writing,

1984 to 1996 (Campbell, Voelkl, & Donahue, 1997)

 NAEP 1996 Mathematics Cross-State Data Compendium for the Grade 4 and Grade

8 Assessment (Shaughnessy, Nelson, & Norris, 1997)

 NAEP 1996 Science Cross-State Data Compendium for the Grade 8 Assessment

(Keiser, Nelson, Norris, & Szyszkiewicz, 1998)

 Mathematics Framework for the 1996 National Assessment of Educational Progress

(National Assessment Governing Board, 1994)

 Science Framework for the 1996 National Assessment of Educational Progress

(National Assessment Governing Board, 1993)

 

1

 James E. Carlson is responsible for psychometric and statistical analyses of NAEP.
2

 Technical Report of the NAEP 1996 State Assessment Program in Mathematics

(Allen, Jenkins, Kulick, & Zelenak, 1997)

 Technical Report of the NAEP 1996 State Assessment Program in Science (Allen,

Swinton, Isham, & Zelenak, 1998)

 NAEP 1996 National Assessment Secondary-Use Data Files User Guide (Rogers,

Kline, & Schoeps, 1999)

 NAEP 1996 State Assessment Program in Mathematics Secondary-Use Data Files

User Guide (O’Reilly, Zelenak, Rogers, & Kline, 1999)

 NAEP 1996 State Assessment Program in Science Secondary-Use Data Files User

Guide (O’Reilly, Zelenak, Rogers, & Kline, 1999)

 NAEP 1996 Science Performance Standards: Achievement Results for the Nation

and the States (Bourque, Champagne, & Crissman, 1997)

 School Policies Affecting Instruction in Mathematics: Findings from the National

Assessment of Educational Progress (Hawkins, Stancavage, & Dossey, 1998)

 Student Work and Teacher Practices in Mathematics (Mitchell, Hawkins, Jakwerth,

Stancavage, & Dossey, 1999)

 Estimation Skills, Mathematics-in-context, and Advanced Skills in Mathematics

(Hawkins, Mitchell, Stancavage, & Dossey, 1999)

 Students Learning Science: A Report on Policies and Practices in U.S. Schools

(O’Sullivan, Weiss, & Askew, 1998)

 Student Work and Teacher Practices in Science: A Report on What Students Know

and Can Do (O’Sullivan & Weiss, 1999)

 The 1996 NAEP Sampling and Weighting Report (Wallace & Rust, 1999)

 Report on Data Collection Activities for the 1996 National Assessment of

Educational Progress (Westat, Inc., 1996)

 Report of Processing and Professional Activities (National Computer Systems, 1996)

The Report Card publications highlight results for the nation, states, and selected subgroups.

Reports on student work and teacher practices focus on instructional variables related to mathematics and

science education and are designed to meet the information needs of teachers and curriculum specialists.

The aim of the reports on school policies, which focus on instruction-relevant variables from the school

or community level, is to meet the information needs of principals, school boards, and interested citizens.

Technical and other reports listed above provide more detailed information on the NAEP data and

analysis procedures. Many of the NAEP reports, including the almanacs (summary data tables), are also

available on the Internet at http://nces.ed.gov/naep. For ordering information on printed copies of these
3

reports, go to the Department of Education web page http://www.ed.gov/pubs/edpubs.html, call toll free

1–877– 4ED PUBS (877–433–7827), or write to:

Education Publications Center (ED Pubs)

U.S. Department of Education

P.O. Box 1398

Jessup, MD 20794–1398

The Frameworks are designed to assess the outcomes of students’ education in mathematics

and science in grade 4, 8, and 12 as part of NAEP. For ordering information on these reports, write:

National Assessment Governing Board

800 North Capitol Street NW

Suite 825

Washington, DC 20002

The Frameworks and other NAGB documents are also available through the web at

http://www.nagb.org.

Additional samples of approximately 125,000 fourth- and 125,000 eighth-graders in 48

jurisdictions were assessed in the 1996 state assessment in mathematics. Also a sample of

approximately 125,000 fourth-graders in 47 states and jurisdictions was assessed as part of the 1996

state assessment in science. A representative sample of about 2,500 students was selected in each

jurisdiction for each subject at each grade level. The state-level sampling plan allowed for cross-state

comparisons and comparisons with the nation in fourth-grade science and fourth- and eighth-grade

mathematics achievement. Technical details of the state assessments are not presented in this technical

report but can be found in the state technical reports.

AN OVERVIEW OF NAEP IN 1996

For the 1996 assessment, NAEP researchers continued to build on the original design

technology outlined in A New Design for a New Era (Messick, Beaton, & Lord, 1983). In order to

maintain its links to the past and still implement innovations in measurement technology, NAEP

continued its multistage sampling approach. Long-term trend and short-term trend samples use the

same methodology and population definitions as in previous assessments. Main assessment samples

use innovations associated with new NAEP technology and address current educational issues. Longterm trend data are used to estimate changes in performance from previous assessments; main

assessment sample data are used primarily for analyses involving the current student population, but

also to estimate short-term trends for a small number of recent assessments. In continuing to use this

two-tiered approach, NAEP reaffirms its commitment to maintaining long-term trends while at the

same time implementing the latest in measurement technology.

A major new design feature was introduced for 1996 to permit the introduction of new inclusion

rules for students with disabilities (SD) and limited English proficient (LEP) students, and the

introduction of testing accommodations for those students. The 1996 national NAEP incorporated a

multiple sampling plan that allowed for the study of changes in NAEP inclusion and accommodation

procedures. In order to provide for studies of the effects of these changes, students from different

samples were administered the NAEP instruments using different sets of inclusion rules and

accommodation procedures. Testing accommodations were provided for SD and LEP students in certain

samples who could be assessed, but not with standard instruments or administration procedures.
4

In the 1996 assessment, many of the innovations that were implemented for the first time in 1988

were continued and enhanced. For example, a variant of the focused balanced incomplete block (focusedBIB) booklet design that was used in 1988 and has continued to be used in other assessment years, was

used in the 1996 main assessment samples in mathematics and science. In the focused-BIB design, an

individual receives blocks of cognitive items in the same subject area. The focused-BIB design allows for

improved estimation within a particular subject area, and estimation continues to be optimized for groups

rather than individuals.

In 1996, NAEP continued to apply the plausible values approach to estimating means for

demographic as well as curriculum-related subgroups. Proficiency estimates were based on draws from a

posterior distribution that was based on an optimum weighting of two sets of information: the student’s

responses to cognitive items, and his or her demographic and associated educational process variables.

This Bayesian procedure was developed by Mislevy (see Chapter 11 or Mislevy, 1991). The 1996

procedures continued to use an improvement that was implemented first in 1988 and refined for the 1994

assessment. This is a multivariate procedure that uses information from all scales within a given subject

area in the estimation of the proficiency distribution on any one scale in that subject area.

A major improvement used in the 1992 and 1994 assessments, and continued in 1996, was the

use of the generalized partial credit model for item response theory (IRT) scaling. This allowed the

incorporation of constructed-response questions that are scored on a multipoint rating scale into the

NAEP scale in a way that utilizes the information available in each response category.

One important innovation in reporting the 1990 assessment data that was continued through 1996

was the use of simultaneous comparison procedures in carrying out significance tests for the differences

across assessment years. Methods such as the Bonferroni allow one to control for the type I error rate for

a fixed number of comparisons. In 1996, a new procedure that provided more powerful procedures that

control for the false discovery rate were implemented for some comparisons. Tests for linear and

quadratic trends were also applied to the national trend data in reading, mathematics, science, and

writing.

ORGANIZATION OF THE TECHNICAL REPORT

Part I of this report presents the details of the design of the 1996 National Assessment,

summarized in Chapter 1. Chapters 2 through 8 describe the development of the objectives and the items

used in the assessment, the sample selection procedures, the assessment booklets and questionnaires, the

administration of the assessment in the field, the processing of the data from the assessment instruments

into computer-readable form, the professional scoring of constructed-response items, and the methods

used to create a complete NAEP database.

The 1996 NAEP data analysis procedures are described in Part II of the report. Chapter 9

provides a summary of the analysis steps. Subsequent chapters provide a general discussion of the

weighting and variance estimation procedures used in NAEP, an overview of NAEP scaling

methodology, and details of the trend and main assessment analyses performed for each subject area in

the 1996 assessment.

Chapter 19 presents basic data from the 1996 assessment, including the properties of the

measuring instruments and characteristics of the sample.
5

Chapter 1

OVERVIEW OF PART I: THE DESIGN AND IMPLEMENTATION

OF THE 1996 NAEP1

Nancy L. Allen and Eugene G. Johnson

Educational Testing Service

1.1 INTRODUCTION

The 1996 National Assessment collected information on the knowledge, skills, and attitudes of

young Americans in mathematics, science, reading, and writing. The three components of the National

Assessment were the main assessments of mathematics and science, the long-term trend assessments of

mathematics, science, reading, and writing, and special assessments of aspects of mathematics and

science. The basis for the information collected for the National Assessment was a complex sample

survey involving nearly 124,000 students, consisting of national samples of public- and nonpublic-school

students who were in grades 4, 8, and 12 or were 9-, 13-, or 17-year olds. Additional NAEP data came

from the State Assessment program, which in 1996 assessed mathematics at grades 4 and 8 in

representative samples of public- and nonpublic-school students in 44 states, the District of Columbia,

Department of Defense Domestic Dependent Elementary and Secondary Schools (DDESS), Department

of Defense Dependents Schools (DoDDS), and Guam. The 1996 State Assessment program also assessed

science at grade 8 in representative samples of public- and nonpublic-school students in 43 states, the

District of Columbia, DDESS, DoDDS, and Guam; DDESS and DoDDS fourth-grade students were

assessed as part of a separate special science assessment.

This chapter describes the design for the 1996 assessment and gives an overview of the steps

involved in its implementation, from the planning stage through the creation of edited data files. The

major components of the implementation are presented here with references to other chapters in Part I

that provide greater detail on each aspect of the assessment. The procedures used for the analysis of the

data are summarized in the overview to Part II and discussed in detail in the remaining chapters in that

part of the report. Excluded from this technical report are the details of the design and analysis of the

1996 State Assessments, which instead appear in the Technical Report of the NAEP 1996 State

Assessment Program in Mathematics (Allen, Jenkins, Kulick, & Zelenak, 1997), and in the Technical

Report of the NAEP 1996 State Assessment Program in Science (Allen, Swinton, Isham, & Zelenak,

1998). Also excluded are the details of the analyses of special studies of advanced mathematics and

science students and of students receiving special theme-related and estimation mathematics items. The

analyses will be described in the appendices of the reports containing the results of these special studies.

 

1

 Nancy L. Allen is responsible for the psychometric and statistical analysis of national and state NAEP data. Eugene G. Johnson

is a senior psychometrician, contributing to the design of NAEP and to discussions of sampling issues. Previously, he was

responsible for the psychometric and statistical analysis of NAEP data. The authors are indebted to the authors of Chapters 2

through 8 for portions of this chapter.
6

The organization of this chapter, and of Part I, is as follows:

• Section 1.2 provides an overview of the NAEP design for 1996 and includes a

description of the constituent samples. To provide background information, the

section also includes the assessment schedule from the inception of NAEP in 1969

through the 1996 assessment.

• Section 1.3 provides a summary of the development of the objectives for each

subject area in the assessment and a description of the development and review of

the items written to fit those objectives. Details of the objective and item

development processes appear in Chapter 2.

• Section 1.4 provides a summary of the sampling design used for the 1996 assessment

with a fuller description provided in Chapter 3.

• Section 1.5 includes a discussion of the assignment of the cognitive and background

questions to assessment booklets and a description of the complex block designs that

were the basis for assigning cognitive items to assessment booklets and assessment

booklets to individuals. Chapter 4 provides a detailed description of the assessment

booklets.

• Section 1.6 provides a summary of the field administration procedures, including the

processes of training field administrators, attaining school cooperation,

administering the assessment, and conducting quality control. Further details appear

in Chapter 5.

• Section 1.7 includes a description of the flow of data from the receipt of the

assessment materials through data entry, validation, and resolution to the creation of

edited data files. Chapter 6 provides a detailed description of the process.

• Section 1.8 contains a discussion of the professional scoring of students’ responses

to the constructed-response items in the assessment. Details of the process are given

in Chapter 7.

• Section 1.9 provides a summary of the creation of the database, the quality control of

data entry, and lists the 1996 database products. This section also includes a

description of the use of the World Wide Web for dissemination of NAEP

information. Further details appear in Chapter 8.

1.2 THE 1996 NAEP DESIGN

A major purpose of NAEP is the reliable measurement of trends in educational achievement over

time. To do this well, confounding effects due to changes from one assessment to the next in assessment

instrumentation or in assessment procedures must be minimized. This implies a stability in the

measurement process over time. At the same time, the assessment must remain current by allowing the

introduction of new curriculum concepts and changes in educational priorities and by permitting the use

of new measurement technology. The objectives for an assessment are determined through a consensus

process in which committees of subject matter experts, scholars, and citizens representing many diverse
7

constituencies and points of view are assembled to determine the educational goals that students should

achieve. Satisfying these objectives often requires changes in assessment instrumentation and

methodology.

In order to meet the goals of measuring trends reliably and responding to changes in the current

thinking about subject areas, NAEP has instituted a multicomponent assessment system where each

component is itself a set of assessments designed to accomplish a specific goal. There are three

components in the 1996 National Assessment design: (1) main assessments; (2) assessments for longterm trend; and (3) special assessments. In particular, the main assessments respond to changes in

curriculum on a regular basis, while the long-term trend assessments are not changed and measure

longer-term trends in a valid way. These are discussed in detail in this chapter.

Several improvements were made in the design of NAEP in the 1984 and succeeding

assessments. Until the 1984 assessment, NAEP was administered using matrix sampling and tape

recorders; that is, by administering booklets of exercises using an aurally presented stimulus that paced

groups of students through the individual assessment exercises in a common booklet. In the 1984

assessment, BIB spiraling, which does not include aural pacing, was introduced in place of taped matrix

sampling. The NAEP design now includes sampling grade populations as well as the age populations that

NAEP originally assessed. The definitions of student age and the time of year in which the assessment

takes place have been made uniform so that students in the fourth-, eighth-, and twelfth-grades are

assessed. To shorten the timetable for reporting results, the period for national data collection was

decreased in 1992, 1994, and 1996 from the five-month period used in 1990 to a three-month period in

the winter (corresponding to the period used for the winter half-sample of the 1990 National

Assessment). To enhance the coverage of the subject areas assessed, the number of items measuring

knowledge and skills was increased for the 1992, 1994, and 1996 assessments.

A special feature of the 1996 main and state assessments was the introduction of new rules for

inclusion of students with disabilities (SD) and limited English proficient (LEP) students in NAEP

assessments (presented in Chapter 5). A subsample of the schools selected for participation in the 1996

assessments used the old inclusion rules (sample type 1; S1) to determine whether students should be

included in the assessment, and another subsample used the new inclusion rules (sample type 2; S2). In

addition to the two groups of schools using the old and new inclusion rules without offering students

special testing accommodations, the 1996 main assessments included a third group of schools that used

the new inclusion rules and offered students within those schools accommodations to the standard NAEP

administration procedures (sample type 3; S3). Figure 1-1 contains the layout of the pieces of the sample

collected for each grade of the main assessment of mathematics.

The accommodations provided by NAEP in the main assessments were meant to match those

specified in the student’s individualized education program (IEP) or those ordinarily provided in the

classroom for testing situations. The most common accommodation was extended time. The samples of

students from the third group of schools that used the new inclusion rules and offered students

accommodations were not included for most analysis and reporting purposes, although the results for

these samples were studied in follow-up analyses. In the State Assessments, no special accommodations

were offered. The information in Chapters 3 and 5 applies to schools and students in all of the sample

types, while the data analysis chapters reflect schools and students in reporting samples only.

The new inclusion rules are applied only when a student has been categorized in his or her IEP as

a student with disabilities (SD) or as a limited English proficient (LEP) student; all other students were

asked to participate in the assessment. For this reason, the sample of students that was selected for most

analysis and reporting purposes for the main mathematics assessment consisted of students from two
8

Figure 1-1

Subsamples of the Mathematics Assessment:1996

GROUPS OF GROUPS OF SCHOOLS

STUDENTS Sample Type 1

OLD INCLUSION RULES

- NO ACCOMMODATIONS -

Sample Type 2

NEW INCLUSION RULES

- NO ACCOMMODATIONS -

Sample Type 3

NEW INCLUSION RULES

- ACCOMMODATIONS -

NOT SD/LEP1 A1 A2 A3

2

INCLUDED SD/LEP1 B1 B2

2 B32

EXCLUDED SD/LEP1 C1

3 C23 C33

1 Students with Disabilities/Limited English Proficient

2 Results for students in subsamples B2, A3, and B3 were not reported in NAEP 1996 Mathematics: Report Card for the Nation and

the States. 3 Students in subsamples C1, C2, and C3 were not included in the assessment.

groups: those who were not categorized as SD or LEP students and who were from schools providing no

accommodations and used either set of inclusion rules (A1 and A2 in Figure 1-1); and those who were

categorized as SD or LEP students and who were from schools providing no accommodations and using

only the old inclusion rules (B1 in Figure 1-1). The advantage of this reporting sample is that it preserves

trend with previous assessments and it makes use of most of the data from the assessment. The main

science assessment sample did not include students from schools using the old inclusion rules. The

sample of students that was selected for most analysis and reporting purposes for the main science

assessment consisted of students from the schools using new inclusion rules, but not providing any

accommodations (A2 and B2 in Figure 1-1). The advantage of this reporting sample is that it makes use of

the most up-to-date inclusion rules and begins a science trend line.

Special analyses that used the national science and mathematics assessment data to compare the

old and new inclusion rules and examine the effect of offering testing accommodations indicated little

difference in proportions of students included in the assessment. More students were included in the

assessment when they were offered accommodations; however, a portion of students who would have

participated in the assessment under standard conditions was assessed with accommodations when they

were offered. A result of this is that fewer students were assessed under standard conditions when

accommodations were offered. The students from the schools offering accommodations were not

included in the analyses or results contributing to the Mathematics or Science Report Cards, so they did

not affect the measurement of the 1990, 1992, and 1996 trend for mathematics. The results from the

science assessment were not compared to those from previous assessments.

NAEP’s design for 1996 required collecting 24 different samples in order to conduct the

assessments. The various samples collected and reported for the 1996 assessment are summarized in

Table 1-1.
9

Table 1-1

NAEP 1996 Student Samples

Sample

Booklet

IDs Mode

Cohort

Assessed

Time of

Testing1

Age

Definition

Modal

Grade

Reporting

Sample

Size2

9[Math-MainP]

13[Math-MainP]

17[Math-MainP]

101-129, 921

101-130, 921

101-130

Print

Print

Print

Grade 4

Grade 8

Grade 12

1/3/96 - 3/29/96

1/3/96 - 3/29/96

1/3/96 - 3/29/96

6,627

7,146

6,904

4 [Math-Estimation]

8 [Math-Estimation]

12 [Math-Estimation]

127

127

127

Tape

Tape

Tape

Grade 4

Grade 8

Grade 12

1/3/96 - 3/29/96

1/3/96 - 3/29/96

1/3/96 - 3/29/96

2,023

2,183

1,849

4 [Math-Theme]

8 [Math-Theme]

12 [Math-Theme]

128, 129

128, 129

128, 129

Print

Print

Print

Grade 4

Grade 8

Grade 12

1/3/96 - 3/29/96

1/3/96 - 3/29/96

1/3/96 - 3/29/96

3,790

4,027

3,735

8 [Math-Advanced]

12 [Math-Advanced]

130

130

Print

Print

Grade 8

Grade 12

1/3/96 - 3/29/96

1/3/96 - 3/29/96

2,337

2,965

9[Sci-MainP]

13[Sci-MainP]

17[Sci-MainP]

201-237

201-237

201-240

Print

Print

Print

Grade 4

Grade 8

Grade 12

1/3/96 - 3/29/96

1/3/96 - 3/29/96

1/3/96 - 3/29/96

7,305

7,774

7,537

12 [Sci-Advanced] 238-240 Print Grade 12 1/3/96 - 3/29/96 2,431

9[Math-State]

13[Math-State]

101-126

101-126

Print

Print

Grade 4

Grade 8

1/29/96 - 3/4/96

1/29/96 - 3/4/96

*3

*3

13[Sci-State] 201-237 Print Grade 8 1/29/96 - 3/4/96 *3

9[RW-LTTrend]

13[RW-LTTrend]

17[RW-LTTrend]

 51 - 56

 51 - 56

 51 - 56

Print

Print

Print

Age 9/Grade 4

Age 13/Grade 8

Age 17/Grade 11

1/3/96 - 3/8/96

10/9/95 - 12/22/95

3/11/96 - 5/10/96

CY

CY

Not CY

 4

 8

 11

5,019

5,493

4,669

9[MS-LTTrend]

13[MS-LTTrend]

17[MS-LTTrend]

 91 - 93

 91 – 93

 84 – 85

Tape

Tape

Tape

Age 9

Age 13

Age 17

1/3/96 - 3/8/96

10/9/95 - 12/22/95

3/11/96 - 5/10/96

CY

CY

Not CY

 4

 8

 11

5,414

5,658

3,539

1 Final makeup sessions for the winter session (January 3-March 29, 1996) were held April 1-5, 1996.

2 The total number of students assessed in the reporting sample of the national assessment was 43,293 for the main assessment, 29,792 for

the long-term trend assessment, and 25,340 for the special studies.

3 Note: consists of distinct samples in 48 jurisdictions

LEGEND: Math Mathematics Main Main assessment, print administration

Sci Science LTTrend Long-term trend assessment

RW Reading and writing CY Calendar year: birthdates in 1986 and 1982

MS Mathematics and science respectively for ages 9 and 13

Print Print administration Not CY Age 17 only: birthdates between October 1, 1978

Tape Audiotape administration and September 30, 1979

Each row of Table 1-1 corresponds to a particular sample and each column of the table indicates the

following major features of that sample:

1. Sample is the sample identifier. The first part of the sample code is a number (the age

class) representing the student cohort included in the sample (note that this part of the

code does not indicate whether an age or grade sample was selected); the second part,

in brackets, denotes the specific sample type. For example, 9[Math Main] is a main
10

assessment mathematics sample for grade 4, assessed in print mode. A full description

of the purposes for the various sample types is given below.

2. Booklets gives the identifier numbers for the booklets used for the assessment of the

particular sample.

3. Mode indicates the mode of assessment, which may be print or tape. NAEP originally

assessed students using a tape recorder in addition to booklets, thus pacing the

students through exercises at a fixed rate. In 1996, NAEP used a paced audiotape for

its mathematics and science long-term trend assessments. However, most other

assessments in 1996 used printed instructions with the student expected to read the

exercises. The only other exception was the 1996 assessment of mathematics

estimation skills.

4. The cohort assessed denotes the age, grade, or age/grade of the population being

sampled. For example, grade 4 represents students who are in the fourth grade; an age

17 cohort consists of students (in any grade) who are 17 years old. Samples for the

1996 main assessments were selected on the basis of grade only. The traditional

NAEP samples used in long-term trend estimation were defined by age only.

However, the 1996 reading and writing long-term trend assessments were defined by

being either of a particular age or of the modal grade for students of that age. For

reading and writing, results are reported for grade and age samples, respectively. The

definitions of age, and thus the corresponding grade, have changed in ways that are

described in Section 1.2.1.

5. Time of testing indicates the time of year in which the assessment is performed.

NAEP traditionally assessed 9-year-olds in the winter, 13-year-olds in the fall, and 17-

year-olds in the spring; like the 1994 main assessment, in 1996, all grades were

assessed in the winter (between January 3-March 29, 1996; final makeup sessions

were held April 1-5, 1996).

6. Age definition is denoted as calendar year (CY) or not calendar year (Not CY). NAEP

originally defined age by birth within a calendar year at ages 9 and 13 but defined age

17 as being born between October 1 of one year and September 30 of the next. In the

1996 main assessments, no students were selected on the basis of their age.

7. The modal grade is the grade attended by most of the students of the sampled

age. For example, if an age 17 sample is listed as having a modal grade of 11,

then most of the 17-year-old students, as defined, are in the eleventh grade. The

definition of age affects the modal grade of the sample. All students sampled for

the 1996 main assessments were in the grade defined by the cohort assessed.

8. The reporting sample size is the number of students in the sample who were

actually administered the assessment and whose results were used in the NAEP

subject-area reports.

1.2.1 The 1996 NAEP Samples

The NAEP samples in 1996 consisted of four types: the samples from the National

Assessment—the main NAEP samples, the long-term trend samples, and the special studies samples—

and the State Assessment samples.
11

The Main NAEP Samples. The main NAEP samples are labeled in Table 1-1 as [Math Main]

and [Science Main]. The samples used complex spiraling procedures (defined in Section 1.5), and were

intended to form the basis for future assessments. Each sample was assessed in the winter period

(January 3 through March 29, 1996). In these samples, only grade populations were sampled, although

age/grade populations were assessed in previous assessment years. The main NAEP samples, and their

purposes, are as follows:

[Math Main] are grades 4, 8, and 12 mathematics assessment samples used for measuring

mathematics achievement in 1996. The fourth- and eighth-grade samples also provided the

comparison groups for the 1996 State Assessment of mathematics in grades 4 and 8. These

samples used print administration.

[Science Main] are grades 4, 8, and 12 science assessment samples used for measuring science

achievement in 1996. The eighth-grade samples also provided the comparison groups for the

1996 State Assessment in science in grade 8. These samples used print administration.

The Long-Term Trend Samples. The long-term trend samples are labeled as [RW-LTTrend]

and [MS-LTTrend] in Table 1-1. Each sample was defined in the same way as equivalent samples in

previous assessments and used the same assessment technology as was used in those assessments.

Therefore, the long-term trend samples are directly comparable to those from previous assessments and

so can be used for continuing the NAEP long-term trend lines. Because these samples were designed to

link the 1996 data with data from previous assessments, they are also referred to as bridge samples. The

long-term trend samples and their purposes are as follows:

[RW-LTTrend] are age/grade samples used for estimating long-term trends in reading and

writing. These samples used assessment booklets identical to those initially used in 1984

and subsequently used in 1988, 1990, 1992, and 1994 (many of the items were also used in

pre-1984 assessments). As in 1984, 1988, 1990, 1992, and 1994, print administration was

used. These samples used the age definitions and time of testing originally used by NAEP

in the 1970s and the early 1980s. The estimates of reading achievement from these samples

link to eight previous reading assessments (1971, 1975, 1980, 1984, 1988, 1990, 1992, and

1994); the estimates of writing achievement link to five previous writing assessments

(1984, 1988, 1990, 1992, and 1994). [Please note that a review of the 1996 long-term trend

writing assessment data analyses has been undertaken by NCES. Additional analyses may be

required before revised results are released.]

[MS-LTTrend] are age-only samples used for estimating long-term trends in mathematics

and science achievement. These samples used the same age definitions and time of

testing as were used since 1969 and used the same assessment instruments as were used

in the 1986, 1990, 1992, and 1994 long-term trend assessments of mathematics and

science. As in previous assessments, the administration of the mathematics and science

questions was paced with an audiotape. The estimates of science achievement from these

samples link to eight previous science assessments (1970, 1973, 1977, 1982, 1986, 1990,

1992, and 1994); the estimates of mathematics achievement link to seven previous

assessments (1973, 1978, 1982, 1986, 1990, 1992, and 1994).
12

The Special Studies Samples. Three sets of samples were collected as part of special NAEP

studies. The samples used special innovative procedures to allow the study of specific aspects of

mathematics and science. Each sample was assessed in the winter period (January 3 through March 29,

1996). In these samples, only grade populations were sampled. The special studies samples, and their

purposes, are as follows:

[Math-Estimation] are samples of specially selected students in grades 4, 8, and 12 who

were administered mathematics estimation booklets in separate paced-tape sessions. The

students are representative of the fourth-, eighth-, and twelfth-grade students in the nation.

[Math-Theme] are samples of specially selected students in grades 4, 8, and 12 who were

administered mathematics theme booklets. These samples were assessed in print

administrations. The students were selected to represent the national populations of

fourth-, eighth-, and twelfth-grade students. The students in these samples were assessed

in separate sessions.

[Math-Advanced] and [Sci-Advanced] are samples of specially selected students in grade

8 (for mathematics only) and grade 12 (for mathematics and science) who received

advanced mathematics and science booklets. They were assessed in separate sessions.

The students were selected from students who were taking advanced courses in

mathematics or science.

The State Assessment Samples. In Table 1-1, 9[Math-State], 13[Math-State], and 13[Sci-State]

are samples of fourth- and eighth-grade public- and nonpublic-school students from each of the states and

jurisdictions participating in the 1996 State Assessment of mathematics, and eighth-grade public- and

nonpublic-school students from each of the states and jurisdictions participating in the 1996 State

Assessments of science.2

 The assessment booklets were the same print-administered booklets as those

used for the matching samples 9[Math Main], 13[Math Main], and 13[Science Main] but the

administrative procedures varied from that of the national assessment in that state personnel collected the

data.

1.2.2 NAEP Assessments Since 1969

Table 1-2 shows the subject areas, grades, and ages assessed since the NAEP project began in

1969. As can be seen, in addition to the 1996 subject areas of mathematics, science, reading, and writing,

several other subject areas have been assessed over the years—social studies, U.S. history, civics,

citizenship, geography, literature, music, career development, art, and computer competence. Many

subject areas are reassessed periodically to measure trends over time.

Assessments were conducted annually through 1980, but budget restrictions since then have

reduced data collection to a biennial basis. Since its inception, NAEP has assessed 9-year-olds, 13-yearolds, and in-school 17-year-olds, although the age definitions changed in 1986 and again in 1988.

Because of budget restrictions, NAEP no longer routinely assesses out-of-school 17-year-olds or young

adults. (A separate assessment of young adults of ages 21 to 25 was conducted in 1985 under a separate

grant.)

 

2

 Fourth-grade students in DDESS and DoDDS schools were assessed as a separate special science assessment.
13

Table 1-2

National Assessment of Educational Progress

Subject Areas, Grades, and Ages Assessed: 1969-1996

Grades/Ages Assessed

Assessment

Year Subject Area(s)

Grade

3

Grade

4

Age

9

Grade

7

Grade

8

Age

13

Grade

11

Grade

12

Age

17

Age

17OS1 Adult

1969-70 Science

Writing

Citizenship

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

1970-71 Reading

Literature

X

X

X

X

X

X

X

X

X

X

1971-72 Music

Social Studies

X

X

X

X

X

X

X

X

X

X

1972-73 Science

Mathematics

X

X

X

X

X

X

X

X

X

X

1973-74 Career and Occupational

 Development

Writing

X

X

X

X

X

X

X

X

X

1974-75 Reading

Art

Index of Basic Skills

X

X

X

X

X

X

X

X

X

X

1975-76 Citizenship/Social Studies X X X X

1976-77 Science

Basic Life Skills2

Sci/ RD / Energy / Health2

XXX

X

X

1977-78 Mathematics

Consumer Skills2

XXX

X

1978-79 Writing, Art, and Music X X X

1979-80 Reading/Literature

Art

X X

X

X X

1981-82 Science2

Mathematics and

Citizenship/Social Studies

X

X

X

X

X

X

1984 Reading

Writing

Writing (long-term trend)

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

1985 Adult Literacy2 X

1986 Reading

Mathematics

Science

Computer Competence

U.S. History2

Literature2

Reading (long-term trend)

Mathematics (long-term trend)

Science (long-term trend)

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

1 Age 17 students who had dropped out of school or had graduated prior to assessment.

2 Small, special-interest assessment conducted on limited samples at specific grades or ages.

(continued)
14

Table 1-2 (continued)

National Assessment of Educational Progress

Subject Areas, Grades, and Ages Assessed: 1969-1996

Grades/Ages Assessed

Assessment

Year3 Subject Area(s)

Grade

3

Grade

4

Age

9

Grade

7

Grade

8

Age

13

Grade

11

Grade

12

Age

17

Age

17OS1 Adult

1988 Reading

Writing

Civics

U.S. History

Document Literacy2

Geography2

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

Reading (long-term trend)

Writing (long-term trend)

Mathematics (long-term trend)

Science (long-term trend)

Civics (long-term trend)

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

1990 Reading

Mathematics

Science

Reading (long-term trend)

Writing (long-term trend)

Mathematics (long-term trend)

Science (long-term trend)

Trial State Mathematics

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

1992 Reading

Writing

Mathematics

Reading (long-term trend)

Writing (long-term trend)

Mathematics (long-term trend)

Science (long-term trend)

Trial State Mathematics

Trial State Reading

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

1994 Reading

U.S. History

Geography

Reading (long-term trend)

Writing (long-term trend)

Mathematics (long-term trend)

Science (long-term trend)

Trial State Reading

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

1 Age 17 students who had dropped out of school or had graduated prior to assessment.

2 Small, special-interest assessment conducted on limited samples at specific grades or ages.

3 It should be noted that somewhat different age definitions were used in the 1984, 1986, and 1988 assessments. In the 1984 assessment, the two

younger ages were defined on a calendar-year basis, while the 17-year-olds were defined on an October 1 to September 30 basis. This resulted in

modal grades of 4, 8, and 11. To allow for age cohorts that were exactly four years apart, in the 1986 main assessment all ages were defined on

an October 1 to September 30 basis, resulting in modal grades of 3, 7, and 11. Special studies (Kaplan, Beaton, Johnson, & Johnson, 1988) were

conducted to measure the effect of the changes in age definition. Because of problems encountered in assessing third graders, in 1988 the ages

were redefined on a calendar-year basis, with the modal grades being 4, 8, and 12. These were the age definitions used in the 1990, 1992, 1994,

and 1996 main assessments.

(continued)
15

Table 1-2 (continued)

National Assessment of Educational Progress

Subject Areas, Grades, and Ages Assessed: 1969-1996

Grades/Ages Assessed

Assessment

Year3 Subject Area(s)

Grade

3

Grade

4

Age

9

Grade

7

Grade

8

Age

13

Grade

11

Grade

12

Age

17

Age

17OS1 Adult

1996 Mathematics

Science

Reading (long-term trend)

Writing (long-term trend)

Mathematics (long-term trend)

Science (long-term trend)

State Mathematics

State Science

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

1 Age 17 students who had dropped out of school or had graduated prior to assessment.

3 It should be noted that somewhat different age definitions were used in the 1984, 1986, and 1988 assessments. In the 1984 assessment, the two

younger ages were defined on a calendar-year basis, while the 17-year-olds were defined on an October 1 to September 30 basis. This resulted in

modal grades of 4, 8, and 11. To allow for age cohorts that were exactly four years apart, in the 1986 main assessment all ages were defined on

an October 1 to September 30 basis, resulting in modal grades of 3, 7, and 11. Special studies (Kaplan, Beaton, Johnson, & Johnson, 1988) were

conducted to measure the effect of the changes in age definition. Because of problems encountered in assessing third graders, in 1988 the ages

were redefined on a calendar-year basis, with the modal grades being 4, 8, and 12. These were the age definitions used in the 1990, 1992, 1994,

and 1996 main assessments.

The table also indicates that in 1984, NAEP began gathering data by grade as well as by age, a

practice that had been continued in assessments up to 1994; the 1996 national assessment included

gathered data by grade only. It should be noted that somewhat different age definitions were used in the

1984, 1986, and 1988 assessments. In the 1984 assessment, the two younger ages were defined on a

calendar-year basis, while the 17-year-olds were defined on an October 1 to September 30 basis. This

resulted in modal grades of 4, 8, and 11. To allow for age cohorts that were exactly four years apart, in

the 1986 main assessment all ages were defined on an October 1 to September 30 basis, resulting in

modal grades of 3, 7, and 11. Special studies (Kaplan, Beaton, Johnson, & Johnson, 1988) were

conducted to measure the effect of the changes in age definition. Because of problems encountered in

assessing third graders, in 1988 the ages were redefined on a calendar-year basis, with the modal grades

being 4, 8, and 12. These were the age definitions used in the 1990, 1992, and 1994 main assessments.

1.3 DEVELOPMENT OF ASSESSMENT OBJECTIVES, ITEMS, AND

BACKGROUND QUESTIONS

In 1996, NAEP conducted main assessments of students at all three grade levels in mathematics

and science. These assessments entailed the generation of a large number of cognitive items—

items measuring knowledge and skills. In addition, a large number of background and attitude questions

were asked of students, and school, teacher and instructional questions were asked of principals and

teachers. Details on the item development procedures for the 1996 main assessment are given in Chapter

2; this section provides an overview. (In addition to the main assessments and the special assessments,

long-term trend studies were conducted in reading, mathematics, science, and writing. Since the

instruments used for these studies consisted entirely of items used in previous assessments, no

developmental tasks were required for their use in the 1996 assessment.)
16

In addition to the cognitive items, several questionnaires were developed: a common student

Background Questionnaire given to all assessed students of a given grade, a School Characteristics and

Policies Questionnaire, Teacher Questionnaires for teachers of fourth- and eighth-grade students in

mathematics and science and for teachers of twelfth-grade students who were assessed in advanced

mathematics, and an SD/LEP Student Questionnaire. Each of these questionnaires was developed

through a broad-based consensus process.

All items in the assessment underwent extensive reviews by subject area and measurement

specialists, as well as careful scrutiny to eliminate any potential bias or lack of sensitivity to any group.

Further, the items were field tested on a representative group of students. Based on the results of the field

test, items were revised or modified as necessary and then again reviewed for bias. With the help of staff

and outside reviewers, the Instrument Development Panels selected the items to include in the

assessment. After the items were selected and formed into the final groupings or blocks of items, they

were carefully reviewed by the National Center for Education Statistics (NCES), the Office of

Management and Budget (OMB), and the National Assessment Governing Board (NAGB).

The assessment instruments included multiple-choice items, constructed-response items scored

dichotomously, constructed-response items scored polytomously, and cluster items in both mathematics

and science. The constructed-response items were professionally scored as described in Chapter 7.

1.4 THE 1996 SAMPLE DESIGN

The sample for the 1996 NAEP assessment was selected using a complex three-stage sample

design. The three-stage sample design includes (1) the sampling of students from (2) selected schools

within (3) 94 selected geographic areas, called primary sampling units (PSUs), across the United States.

The 1996 sample design differed from previous years due to oversampling of SD/LEP students. With the

inclusion of the different inclusion rules and the availability of accommodations, the sample design was

similar to that used in 1986, 1988, 1990, 1992, and 1994 and is described in detail by Westat, Inc., the

firm contracted by NCES to select the sample, in The 1996 NAEP Sampling and Weighting Report

(Wallace & Rust, 1999). The following sections provide an overview of the five steps used to draw

NAEP samples using the three-stage sample design. Further details are given in Chapter 3. Steps 3 and 4

describe the assignment of sample types and assessment sessions to the second sampling unit schools.

Steps 5a through 5c contain procedures for the collection of data for SD/LEP students, teachers, and

schools.

 Step 1: Primary Sampling Units

In the first stage of sampling, the United States (the 50 states and the District of Columbia) was

divided into geographic primary sampling units (PSUs). Each PSU met a minimum size requirement and

generally comprised either a consolidated metropolitan statistical area (CMSA), a metropolitan statistical

area (MSA), a single county, or a group of contiguous counties. The PSUs were classified into four

regions (Northeast, Southeast, Central, West), each containing about one-fourth of the U.S. population.

In each region, PSUs were additionally classified as MSA or nonMSA. This resulted in eight

subuniverses of PSUs.

Ninety-four of the PSUs were selected for the 1996 main assessment. Twenty-two PSUs were

designated as certainty units because of their size, and were included in the sample with certainty. The

remaining smaller PSUs were not guaranteed to be selected and were accordingly designated as

noncertainty PSUs. Within each major stratum (subuniverse), further stratification was achieved by

ordering the noncertainty PSUs according to several additional socioeconomic characteristics. Seventy-
17

two PSUs were selected, one per stratum from each of the noncertainty strata, with probability

proportional to size (total population from the 1990 census). To enlarge the samples of Black and

Hispanic students, thereby enhancing the reliability of estimates for these groups, PSUs from the highminority subuniverses were sampled at twice the rate of those from the other subuniverses. This was

achieved by creating smaller strata within the high-minority subuniverses.

For the long-term trend samples, 52 PSUs were selected: 10 PSUs were selected with certainty;

six additional PSUs were selected from the 12 remaining main sample certainty PSUs; and 36 PSUs were

selected from the 72 noncertainty strata independently of PSU selections for the main samples.

 Step 2: Selection of Schools

In the second stage of sampling for the main assessments, the public schools (including Bureau

of Indian Affairs (BIA) schools and Department of Defense Education Activity (DoDEA) schools) and

nonpublic schools (including Catholic schools) within each of the selected PSUs were listed according to

the grade ranges associated with the three age classes. An independent sample of schools was selected

separately for each of the grades so that some schools were selected for assessment of two grades, and a

few were selected for all three. Schools within each PSU were selected (without replacement) with

probabilities proportional to assigned measures of size with oversampling of nonpublic schools and of

schools with high minority enrollment. Overall probabilities of selection for high-minority schools were

twice those for other schools while the probabilities of selection of nonpublic schools were triple those of

low-minority public schools of the same size. The increased probabilities of selection enlarged the

samples of Black and Hispanic students and the samples of students from nonpublic schools, thereby

enhancing the reliability of estimates for these groups. Details of the probabilities used for school

selection appear in Chapter 3.

The samples of schools for the long-term trend assessments were drawn in a manner very similar

to that used for the main assessments. The chief difference in the two samples was that nonpublic schools

and schools with high minority enrollment were not oversampled for the long-term trend assessments.

Schools were not selected for both main and long-term trend assessments at the same age/grade.

For the main samples, the overall school cooperation rate was 86 percent for grade 4, 83 percent

for grade 8, and 79 percent for grade 12. For the long-term trend samples, the overall school cooperation

rate was 85 percent for age class 9, 84 percent for age class 13, and 81 percent for age class 17. In certain

instances, refusing schools were replaced by substitutes according to the rules indicated in Chapter 3.

 Step 3: Assigning Sample Type to Schools

In order to determine the effect of using different criteria for excluding students from the

assessment, three different sample types were assigned to the schools selected for the main assessment.

In sample type 1 schools, the inclusion criteria for the main samples were identical to those used in 1990

and 1992. In sample type 2 schools, new 1996 inclusion criteria were used. In sample type 3 schools, the

new 1996 inclusion criteria were used and accommodations were offered to SD/LEP students. More

detailed information on assigning sample type to schools is provided in Chapter 3.

 Step 4: Assigning Assessment Sessions to Schools

Sessions were assigned to the selected schools found to be in-scope at the time of session

assignment, as described in Chapter 3. Sessions were assigned to schools with three aims in mind. The

first was to distribute students to the different session types across the whole sample for each age class so

that the target numbers of assessed students would be achieved (in each sample type separately in the
18

main assessments). The second was to maximize the number of different session types that were

administered within a given selected school, without creating unduly small sessions. The third was to

give each student an equal chance of being selected for a given session type regardless of the number of

sessions conducted in the school.

 Step 5: Sampling Students

In the third stage of sampling, a consolidated list was prepared for each school of all grade- and

age-eligible students (for long-term trend) or all grade-eligible students (for the main assessments) for the

age class for which the school was selected. To provide the target sample size, a systematic selection of

eligible students was made from this list, if necessary. In small- and medium-sized schools all eligible

students were in the sample. For schools assigned to more than a single session type, students were

assigned by Westat district supervisors to one of the various session types (audiotape or print

administration) using specified procedures. No student was assigned to more than one session.

Step 5a: Excluded Students

Despite NAEP’s goal to assess all selected students, certain selected students

were judged by school authorities as being incapable of participating meaningfully in the

assessment. For each student who was excluded, school staff who had knowledge of the

student’s capabilities completed an SD/LEP student questionnaire, listing the reason for

exclusion and providing some background information. For each SD/LEP student who

was included in the assessment, school staff also completed an SD/LEP student

questionnaire.

Specific guidelines for exclusion were provided for all samples in the 1996

assessment. However, somewhat different criteria were used for the long-term trend

samples than for the main assessment samples. In addition, the inclusion criteria for the

main samples differed by sample type.

The exclusion guidelines for the long-term trend samples were the same as those

used in previous assessments. Three types of students could be excluded under these

guidelines—non-English speaking students, students with mental retardation who are

educable but who were judged incapable of meaningfully responding to exercises

appropriate to their age level, and students so functionally disabled that they could not

perform in the NAEP assessment situation.

As stated previously, for the main samples, the procedures for assessing students

with disabilities (SD) and students of limited English proficiency (LEP) varied by

sample type. The exclusion procedure used in sample type 1 differed somewhat from that

used in sample types 2 and 3. In sample type 1 schools, the inclusion criteria for the main

samples were identical to those used in 1990 and 1992. These criteria were intended to

be somewhat more rigorously defined than those used in the long-term trend samples. In

sample type 2 schools, new 1996 inclusion criteria were used. In sample type 3 schools,

the new 1996 inclusion criteria were used and accommodations were offered to SD/LEP

students. The new inclusion criteria was developed to more closely match the procedures

used by many states and school districts in testing situations. Both sets of the inclusion

rules are presented in Chapter 5.
19

Step 5b: Sampling Teachers

Teachers of fourth- and eighth-grade students assessed in mathematics and

science and twelfth-grade students assessed in advanced mathematics were identified and

asked to complete a questionnaire (described in Chapter 2) about their background and

experiences and about instructional practices, by class, for any classes containing

assessed students.

Step 5c: The School Characteristics and Policies Questionnaires

A School Characteristics and Policies Questionnaire was mailed to every

sampled school by Westat before the assessment for completion by the principal or

school administrator. The Westat supervisor then collected the questionnaires and

returned them to ETS. The School Characteristics and Policies Questionnaire is

described in Chapter 2.

1.5 ASSESSMENT INSTRUMENTS

Four types of instruments were used in the 1996 assessment: student assessment booklets (which

included the student common Background Questionnaire as well as cognitive items), SD/LEP Student

Questionnaires, Teacher Questionnaires, and a School Characteristics and Policies Questionnaire. This

section provides an overview of these instruments; more detailed information can be found in Chapter 4.

The student common Background Questionnaires were completed by the students participating in

the 1996 assessment. These questionnaires included questions about the students’ race/ethnicity, parental

education levels, and other background variables specified by NCES and a committee of survey, content,

and education experts. These questionnaires appeared at the beginning of some student assessment

booklets and at the end of others. The student assessment booklets also included subject-related

background questions about instructional opportunity, and interest in and attitudes towards the subject

area.

1.5.1 Student Assessment Booklets—Main Assessment

1.5.1.1 Student Assessment Booklets—Main Assessment—Mathematics

Each student assessed in mathematics received a booklet containing a set of general background

questions, content questions, subject-specific background questions, and questions about his or her

motivation and familiarity with the assessment materials. The content questions were assembled into

sections or blocks. Students in the main assessment were given three 15-minute blocks. Those sampled

for the theme assessment completed one 15-minute block and one 30-minute block. Those sampled for

the advanced study at grade 8 completed three 20-minute blocks; at grade 12 advanced sample students

completed three 30-minute blocks. Students in the estimation sample completed one 15-minute block

from the main assessment and two paced-tape sections. The overall assessment time for each student was

approximately 63 minutes.

The assembly of blocks into booklets for the main assessment and their subsequent assignment to

sampled students was determined by a balanced incomplete block (BIB) design with spiraled
20

administration. The student booklets contained two five-minute background sections, a one-minute

background section, and three 15-minute blocks of items according to a BIB design.

The BIB design for the 1996 national mathematics assessment was focused by subject area, so

that students received booklets containing only blocks of mathematics questions (not science). The BIB

design also balances the order of presentation of the 15-minute blocks of items—every 15-minute block

appears as the first cognitive block in two booklets, as the second cognitive block in two other booklets,

and as the third cognitive block in another two booklets.

The design used in 1996 required that 13 blocks of mathematics items at each grade be assembled

into 26 booklets. Theme blocks were placed in two other booklets, and estimation blocks in one other

booklet. At grades 8 and 12, the advanced study was placed in one additional booklet. Once assembled,

the main assessment booklets were then spiraled and bundled. Spiraling involves interweaving the

booklets in a systematic sequence so that each booklet appears an appropriate number of times in the

sample. The bundles were designed so that each booklet would appear equally often in each position in a

bundle.

The final step in the BIB-spiraling procedure was the assigning of the booklets to the assessed

students. The students within an assessment session were assigned booklets in the order in which the

booklets were bundled. Thus, most students in an assessment session received different booklets. In the

assessment design, representative and randomly equivalent samples of students responded to each item at

a given grade level.

Chapter 4 provides more detail on the contents of the mathematics instruments.

1.5.1.2 Student Assessment Booklets—Main Assessment—Science

Each student assessed in science received a booklet containing general background questions,

content questions, subject-specific background questions, and questions about his or her motivation and

familiarity with the assessment materials. The content questions were assembled into sections or blocks.

Students in the main assessment were given three 20-minute blocks at grade 4, and three 30-minute

blocks at grades 8 and 12. The last block in every book was a hands-on block. Those sampled for the

advanced study at grade 12 completed four 30-minute blocks. The overall assessment time for each

student was, on average, 120 minutes.

The assembly of blocks into booklets for the main assessment and their subsequent assignment to

sampled students was determined by a complex design with spiraled administration. The student booklets

contained two five-minute background sections, a one-minute background section, and three blocks of

items.

The design for the 1996 national assessment was focused by subject area, so that students

received booklets containing only blocks of science questions (not mathematics). The design also

balances the order of presentation of the blocks of items, except for the hands-on blocks, which always

appear in position three of a booklet. All other blocks appear an equal number of times in position one

and position two. Further, the design was set up to ensure that no student answered more than one themebased block (though some students did not receive any). This design allows for some balancing of the

impact of context and fatigue effects to be measured and reported, but makes allowance for the

difficulties and disruption of administering hands-on blocks. It also takes into account the limited breadth

of content coverage included in the theme blocks.
21

The design used in 1996 required that fifteen blocks of mathematics items at each grade be

assembled into 37 booklets. At grade 12, the advanced study was composed of three additional booklets.

Once assembled, the main assessment booklets were then spiraled and bundled. Spiraling involves

interweaving the booklets in a systematic sequence so that each booklet appears an appropriate number

of times in the sample. The bundles were designed so that each booklet would appear equally often in

each position in a bundle.

The final step in the spiraling procedure was the assigning of the booklets to the assessed

students. The students within an assessment session were assigned booklets in the order in which the

booklets were bundled. Thus, most students in an assessment session received different booklets. In the

assessment design, representative and randomly equivalent samples of about 2,000 students responded to each

item at a given grade level.

Chapter 4 provides more detail on the contents of the science instruments.

1.5.2 Student Assessment Booklets—Long-Term Trend Samples

There were two distinct long-term trend samples in the 1996 assessment, each of which required

reprinting booklets used in previous assessments:

Reading-Writing Long-Term Trend: Six booklets were used at each of the three age/grades for

the purposes of measuring long-term trends in reading and writing. These booklets were identical to

booklets used in the 1984 main assessments of reading and writing and in the 1988, 1990, 1992, and 1994

long-term trend assessments of those subjects. Each booklet consisted of a common background block in

the beginning of each booklet and three cognitive blocks, either two reading and one writing or one

reading and two writing. All cognitive blocks also contained subject-related background questions. The

booklets were administered without audiotape and were spiraled together for administration.

Mathematics-Science Long-Term Trend: These instruments were used for the measurement of

mathematics and science and were identical to booklets administered in 1990, 1992 and 1994. These

booklets contained 15-minute blocks of mathematics and science items; each mathematics block and each

science block was administered using audiotape pacing. (At the younger two ages, the booklets also

contain a block of reading items, which was print-administered.) There were three booklets each at age 9

and age 13 and two booklets at age 17. The common background questions appeared at the beginning of

each booklet. Combined, the booklets at an age contain three blocks of mathematics items and three

blocks of science items. Because of the audiotape pacing, each booklet was administered in a separate

session.

1.5.3 Other Instruments

Besides the student assessment booklets, other instruments provided data relating to the

assessment:

The SD/LEP Student Questionnaires were completed by the teachers of those students who

were selected to participate in the assessment sample who had disabilities (SD) or were classified as

Limited English Proficient (LEP). The questionnaires were completed for all SD or LEP students,

whether or not they actually participated in the assessment. The questionnaires asked about the nature of

the student’s disability and the special programs in which the student participated. The response rates for
22

this questionnaire ranged from 92 to 95 percent for the different student samples. The criteria used for

excluding students are described in Chapter 5.

Teacher Questionnaires were administered to the teachers of fourth- and eighth-grade students

assessed in mathematics and science and teachers of twelfth-grade students who were assessed in

advanced mathematics. The Teacher Questionnaire included a general section that contained questions

about the teacher’s background and experience. The rest of the questionnaire contained questions about

instructional practices, by class, for any classes containing assessed students. The response rates ranged

from 95 to 100 percent for the different student samples.

School Characteristics and Policies Questionnaires were completed by school principals or

their representatives, who provided information about school administration, staffing patterns, special

programs, subject requirements, and school resources. The response rates for the different student

samples ranged from 92 to 95 percent.

1.6 FIELD OPERATIONS AND DATA COLLECTION

Field operations and data collection for the 1996 assessment were the responsibility of Westat,

Inc., and are documented in Chapter 5. The field operation was conducted by a staff at Westat’s home

office and a larger staff in the field. The Westat home-office staff coordinated all activities related to

field operations and managed materials distribution and home-office receipt of assessment reporting

forms. The field staff consisted of area supervisors, assessment supervisors, and exercise administrators.

The assessment supervisors, who were trained by Westat, were each responsible for the assessment

activities in one or more PSUs. Although ETS made initial contact with participating school districts,

each assessment supervisor was primarily responsible for making follow-up contacts with these districts,

recruiting and training exercise administrators to work with them in administering the assessment

sessions, arranging the assessment sessions, and selecting the sample of students to be assessed within

each school. The assessment supervisors and the exercise administrators administered the assessments,

filled out the necessary forms, performed process control, and shipped the assessment booklets and forms

to National Computer Systems (NCS), the subcontractor responsible for processing NAEP materials and

data.

Gaining school cooperation was the joint responsibility of Westat and ETS. ETS made the

preliminary contacts preparatory to obtaining school cooperation by first contacting the Chief State

School Officers, informing them that schools within their states had been selected for the assessment and,

in a later letter, listing the selected schools and districts. Later mailings were sent to superintendents of

public schools and parochial schools and principals of other nonpublic schools for all schools selected in

the assessment. These materials provided an explanation of NAEP, a list of the selected schools in the

official’s jurisdiction, and a cover letter explaining that a Westat district supervisor would contact them

to set up an introductory meeting. Westat district supervisors then scheduled and conducted introductory

meetings (both by telephone and in person), worked with the schools to schedule the assessments, and,

with the exercise administrators, conducted the assessments. The overall participation rate of schools

originally selected in the 1996 assessments was 83 percent for the main samples and 84 percent for the

long-term trend samples. Further detail on school participation rates is given in Chapter 3.

The main assessment sessions were conducted between January 3 and March 29, 1996, at all

three grade levels. The age 9/grade 4 long-term trend assessments were carried out between January 3

and March 8, 1996; the age 17/grade 11 long-term trend samples were conducted between March 11 and

May 10, 1996. The age 13/grade 8 long-term trend assessments were carried out between October 9 and

December 22, 1995. When the main assessments of the long-term trend subjects were first collected in
23

1987 and 1986, studies were completed to take into account the difference in assessment time across the

samples (Kaplan, Beaton, Johnson, & Johnson, 1988).

Two special studies that required additional steps in the sampling process were included in the

1996 main assessment. One of these special studies involved students who were eligible for advanced

mathematics or science sessions. Advanced sessions were only available to designated students at grade 8

in mathematics and at grade 12 in mathematics and science. Further details on the advanced sessions is

provided in Chapters 3 and 5.

The other special study involved applying two versions of the SD/LEP “inclusion” criteria for

NAEP assessments and, in some schools, offering accommodations for testing students designated as

SD/LEP. In the study, the school sample was divided into three subsamples: S1 (sample type 1), S2

(sample type 2), and S3 (sample type 3). The purpose of these subsamples was to collect data under the

same conditions as previous assessments in order to maintain trend in mathematics within NAEP;

evaluate the impact of a revised, more specific set of inclusion criteria; and evaluate the combined effect

of the new criteria and the use of accommodations for testing students. Further details on this special

study are provided in Chapters 3 and 5. Results for the study appear in the SD/LEP report along with a

technical procedural appendix describing the special analyses completed for the study.

An automated management system tracked and recorded the progress of field work throughout

the 1996 assessment period. In addition, progress was constantly monitored through telephone reports

held between the area supervisors and the assessment supervisors and between the area supervisors and

the home office staff.

Both Westat and ETS participated in the quality control of the field administration, which

involved on-site visits by Westat and ETS staff to verify the sampling of the students and to observe the

conduct of the assessment by the supervisors and the exercise administrators.

1.7 MATERIALS AND DATA PROCESSING

After completing an assessment session, Westat field supervisors and exercise administrators

shipped the assessment booklets and forms from the field to National Computer Systems for entry into

computer files, professional scoring, and creating the data files for transmittal to ETS. Careful checking

assured that all data from the field were received. More than 134,000 booklets and questionnaires were

received and processed for the national portion of the 1996 assessment. The extensive processing of these

data is detailed in Chapter 6.

The student data were transcribed into machine-readable form by scanning the student

instruments with an optical scanning machine. An intelligent data entry system was used for resolution of

the scanned data, the entry of documents rejected by the scanning machine, and the entry of information

from the questionnaires. Additionally, each piece of input data was checked to verify that it was of an

acceptable type, that it was within a specified range or ranges of values, and that it was consistent with

other data values. The entry and editing of materials is discussed in Chapter 6.

1.8 PROFESSIONAL SCORING

Items requiring a written response from the student (constructed-response items) were included

in the main and state assessments in mathematics and science and in the long-term trend assessments in
24

reading, mathematics, and writing. More than nine million constructed responses were read and marked

by the professional scoring staff for the national and state portions of the 1996 assessment.

Image processing and scoring were again used in 1996. Images of students’ responses to the

constructed-response items were scanned into computerized form, then scored online by professional

raters.

Chapter 7 describes the professional scoring operation, including an overview of the scoring

guides, the training procedures, and the scoring process for each subject area.

1.9 CREATION OF THE DATABASE

Before analyses could begin, the student response data, school, teacher, and SD/LEP student

questionnaire data, and all sampling weights had to be integrated into a coherent and comprehensive

database. This database, which was used for all analyses, was also the source for the creation of two

NAEP database products—the item information database and the secondary-use data files. Secondary-use

data files include sample control statement files for SAS and SPSS statistical systems and the NAEP

Data on Disk product suite. The Data on Disk products, including a complete set of secondary-use data

files on CD-ROM, PC-based NAEP data extraction software, and NAEP analysis modules, make

secondary use of NAEP data much easier than it has been in the past. The quality of the data resulting

from the complete data entry system, from the actual instruments collected in the field to the final

machine-readable database used in analysis, was verified by selecting field instruments at random and

performing a character-by-character comparison of these instruments with their representations in the

final database. Chapter 8 provides details on the database, quality control activities, and database

products.
25

Chapter 2

DEVELOPING THE NAEP OBJECTIVES, ITEMS,

AND BACKGROUND QUESTIONS FOR THE

1996 ASSESSMENTS OF MATHEMATICS AND SCIENCE1

Stephen Lazer

Educational Testing Service

2.1 INTRODUCTION

In 1996, the main NAEP assessments were conducted in mathematics and science.2 Long-term

trend assessments were also conducted in reading, writing, mathematics, and science; these assessments

are composed of instruments identical to those used in previous years. Additional data were gathered

under the auspices of the State Assessment program, which in 1996 assessed mathematics at grades 4 and

8 using representative samples of public- and nonpublic-school students in 44 states3

, the District of

Columbia, Department of Defense Domestic Dependent Elementary and Secondary Schools (DDESS),

Department of Defense Dependents Schools (DoDDS), and Guam. The 1996 State Assessment program

also assessed science at grade 8 using representative samples of public- and nonpublic-school students in

43 states, the District of Columbia, DDESS, DoDDS, and Guam; DDESS and DoDDS students were also

assessed in science at grade 4. A summary of the main assessment subject areas follows:

Mathematics: A mathematics assessment was administered to national samples at grades

4, 8, and 12. This assessment was designed around the measurement of five mathematics

content areas, and continued a trend line begun in 1990. In other words, many of the

assessment questions were used in the 1990 and 1992 NAEP assessments; others were

newly developed for 1996. A mixture of multiple-choice, short constructed-response, and

extended constructed-response questions made up the assessment; in aggregate, well

over half of the student assessment time was spent answering constructed-response

questions. On some portion of the assessment, students were required to use calculators

and other hands-on materials. In addition to the instruments used to generate the main

reporting scales, three supplemental mathematics surveys were conducted at the national

level. At each grade, special instruments were administered to representative national

samples that were designed to measure the estimation skills of students. Second, separate

samples at all grades were given “thematic” instruments constructed to measure the

ability of students to solve in-depth mathematics problems. Finally, at grades eight and

twelve a special study was conducted. In this study, students with advanced mathematics

training were administered special assessment booklets whose contents were more

advanced than those of the main assessment.

Science: A science assessment was administered to national samples at grades 4, 8, and

12. The assessment measured a broad range of science-education outcomes. Because the

1996 science assessment was based on a new framework, it represents the beginning of a

 

1

 Stephen Lazer manages assessment development activities for the NAEP program at ETS.

2

 Copies of the frameworks for these assessments are available from the National Assessment Governing Board (NAGB)

http://www.nagb.org. 3

 Not all participating jurisdictions gathered both public- and nonpublic-school samples.
26

new trend line. The assessment involved three different types of testing. Some portions

covered general scientific knowledge and skills. Others sections of the survey tested

students abilities to answer questions in an in-depth thematic or topical area. Finally,

each sampled student completed a component that involved conducting a hands-on

science experiment. A combination of multiple-choice, short constructed-response, and

extended constructed-response questions made up the assessment; in aggregate, well

over half of the student assessment time was spent answering constructed-response

questions. In addition to the main assessment, a special study of students with advanced

scientific training was conducted at grade 12.

From its inception, NAEP has developed assessments through a consensus process and the 1996

instruments were no exception. Under the direction of the National Assessment Governing Board

(NAGB), educators, scholars, and citizens representative of many diverse constituencies and points of

view designed assessment frameworks for both subject areas. Staff at Educational Testing Service (ETS)

who were subject-area experts in their respective fields worked with subject-area consultants well versed

in assessment methodology to develop assessment questions appropriate to the objectives. All questions

underwent extensive reviews by subject-matter specialists, measurement specialists, and ETS employees.

Questions were assembled and printed into booklets suitable for matrix sampling and then administered

either by a trained field staff (for the national program) or by state or local school district staff (for the

State Assessment program) to stratified, multistage probability samples of students.

All 1996 assessment development efforts were governed by four major considerations:

1. The primary goal of the development process was to craft instruments that matched

the content definitions included in the assessment frameworks, which was developed

through consensus processes conducted under the auspices of the National

Assessment Governing Board (NAGB).

2. As outlined in the ETS proposal for the administration of the NAEP cooperative

agreement (ETS, 1992), the development of the items was guided by an Instrument

Development Committee and further reviewed by state representatives and

classroom teachers from across the country. In addition, the items had to be carefully

reviewed for potential bias.

3. As described in the ETS Standards of Quality and Fairness (ETS, 1987), all

materials developed at ETS were in compliance with specified procedures. In

particular, all questions were carefully reviewed for content accuracy,

testworthiness, and potential bias.

4. As per federal regulations, all NAEP cognitive and background items were submitted

to a federal clearance process. This process involved review of all cognitive items by

NCES and NAGB, and review of all background questions by the Office of

Management and Budget (OMB), the Information Management Team (IMT) of the

Department of Education, and NCES.

The development effort for the 1996 assessment included questionnaires4

 for students, teachers,

and school administrators, in addition to a substantial number of cognitive items for both subject areas.

 

4

 These questionnaires can be obtained from the National Center for Education Statistics (NCES).
27

The following sections include general overviews about setting objectives and developing items

and specific details about developing subject-specific objectives and assessments. A list of the

consultants who participated in the 1996 development process is included in Appendix A.

2.2 GENERAL OVERVIEW OF THE 1996 ASSESSMENT FRAMEWORKS

The subject-area objectives for each NAEP assessment are determined through a legislatively

mandated consensus process. These objectives typically take the form of frameworks, or matrices,

delineating the important content and process areas to be assessed. In addition to these broad

frameworks, the Council of Chief State School Officers (CCSSO), and therefore NAGB, provided

detailed descriptions of item types and the numbers of items to be selected for each category. The various

frameworks for the 1996 assessments are described below.

The frameworks for the main 1996 NAEP assessments were developed through consensus

processes conducted by the CCSSO working under contract to NAGB. The projects involved

participation and review by many groups, including teachers, content-area scholars, educational

policymakers, and members of the general public. In addition to people directly involved in the

framework development processes, the documents were reviewed by state education and testing officials,

by representatives of professional associations, and by researchers. In addition, the frameworks were the

subject of testimony at public hearings arranged to allow the widest possible participation in the

consensus process. The objectives resulting from these processes reflect neither a narrowly defined

theoretical framework nor the view of every participant. They do, however, represent the thinking of a

broad cross-section of individuals who are expert in the various content areas and who are deeply

committed to the improvement of American education.

The framework that governs the 1996 NAEP mathematics assessment is an enhanced version of

the framework that governed the development of the 1990 and 1992 assessments. This framework was

originally developed under the auspices of the Council of Chief State School Officers (CCSSO); it was

revised in 1991-1992 through a consensus project managed by the College Board. It was this enhanced

framework that governed the instrument development activities related to the 1996 assessment. The

revisions to the framework were minor, and allow for continued measurement of educational progress

through comparisons with 1990 and 1992 results. The mathematics framework is organized according to

a five-by-three matrix of content strands by mathematical abilities. The content strands, which make up

the main reporting subscales, are:

• Number Sense, Properties, and Operations;

• Measurement;

• Geometry and Spatial Sense;

• Data Analysis, Statistics, and Probability; and

• Algebra and Functions.

In addition, the assessment was designed to measure the three mathematical abilities of:

• Conceptual Understanding,

• Procedural Knowledge, and

• Problem Solving.
28

All exercises in the assessment were classified as measuring one of the content strands and one

of the abilities. Additional specifications in the framework are related to an assessment dimension

referred to as mathematical power. Mathematical power is conceived as consisting of mathematical

abilities within a broader context of reasoning and with connections across the broad scope of

mathematical content and thinking. Communication is viewed as both a unifying thread and a way for

students to provide meaningful responses to tasks.

The framework for the 1996 science assessment is structured according to a matrix organized

according to two major dimensions: fields of science and knowing and doing science. The fields of

science, which make up the subscales on which assessment results are analyzed and reported, are the

earth, physical, and life sciences. The cognitive dimension, knowing and doing science, is organized into

three categories, conceptual understanding, scientific investigation, and practical reasoning. In addition,

the framework includes requirements for measurement of content that crosses other categorical

boundaries. Specifically, the nature of science and themes are categories that should integrate the three

fields of science, rather than represent separate content.

2.3 GENERAL OVERVIEW OF PROCEDURES FOR DEVELOPING THE ITEMS

A carefully developed and tested series of steps, similar to those used for past NAEP

assessments, was utilized to create assessment items that reflected mathematics and science objectives

and that measured achievement related to them (see Sections 2.4 through 2.6 for more detail). The steps

were as follows:

1. NAGB provided item specifications and frameworks in each subject area.

2. The Instrument Development Committees in both subject areas provided guidance to

NAEP staff about how the objectives could be measured given the realistic

constraints of resources and the feasibility of measurement technology. The

committees made recommendations about priorities for the assessment (within the

context of the assessment framework) and the types of items to be developed.

3. Items were chosen for the assessment through an extensive selection process that

involved the input of practitioners from across the country as well as from members

of the Instrument Development Committees.

4. Specialists with subject-matter expertise, skills, and experience in creating items

according to specifications were identified from inside and outside ETS to develop

and review the assessment questions.

5. The items were reviewed and revised by NAEP/ETS staff and external test

specialists.

6. Representatives from the State Education Agencies met and reviewed all items and

background questionnaires that were scheduled to be part of the state assessment.

7. Language editing and sensitivity reviews were conducted as required by the ETS

Standards for Quality and Fairness.

8. Field test materials were prepared, including those necessary to secure clearance by

the Office of Management and Budget.
29

9. A field test was conducted in many states, the District of Columbia, and three

territories.5

10. Representatives from State Education Agencies met and reviewed the field test

results for all exercises selected for the state assessment.

11. Based on the field test analyses, new items for the 1996 assessment were revised,

modified, and re-edited, where necessary. The items once again underwent the full

range of ETS reviews.

12. The Instrument Development Committees approved the selection of items to include

in the 1996 assessment.

13. After a final review and check to ensure that each assessment booklet and each block

met the overall guidelines for the assessment, the booklets were typeset and printed.

The following sections describe the development of the mathematics and science assessments in

more detail.

2.4 DEVELOPING THE MATHEMATICS ASSESSMENT

2.4.1 Overview

The framework that governs the 1996 NAEP mathematics assessment is an enhanced version of

the framework used on the 1990 and 1992 assessment. Similar to other NAEP assessments, the 1990

mathematics framework was developed through a broad-based consensus process managed by the

CCSSO. In 1991-1992, the National Assessment Governing Board (NAGB) contracted with The College

Board to review and revise the framework in preparation for the assessment originally planned for 1994

and administered, in fact, in 1996. The development process involved a committee of mathematicians and

mathematics education specialists. Educators, scholars, and citizens, representative of many diverse

constituencies and points of view, participated in the national consensus process to review and revise

objectives for the assessment.

The instrument used in the 1996 mathematics assessment was composed of a combination of new

items developed for administration in 19966

 and items from the 1992 and 1990 assessments. Those items

that were carried over from the 1992 and 1990 instruments comprised approximately 60 percent of the

1996 instrument. The remaining portion was made up of new items developed according to the

recommendations included in the enhanced assessment specifications. Maintaining approximately 60

percent of the instrument across the two assessment years (1992 and 1996) allowed for the reporting of

 

5

 In this case, two field tests were conducted. The mathematics and science assessments were originally scheduled for 1994; thus

a 1993 field test was conducted. This field test was designed to provide replacement items for those released in mathematics and

all exercises needed in the new science assessment. Because the assessments were delayed until 1996, a supplementary field test

was conducted in 1995. In mathematics, this field test was used for the development of theme and advanced blocks, and in the

development of assessment accommodations. In science, the 1995 field test was used to develop more general science exercises

for the main assessment.

6

 As was noted above, many of the new items were originally scheduled for use in 1994; however, the mathematics assessment

was deferred until 1996. Other items were developed and field tested in 1995. For purposes of this report, we will refer to all

exercises that were used in 1996 but that were not part of earlier NAEP surveys as having been “newly developed for 1996.”
30

trends in mathematics performance. At the same time, developing a new set of items made it possible to

release approximately 40 percent of the 1992 assessment for public use.

In developing the new portion of the 1996 NAEP mathematics assessment, the same procedures

used in 1992 were followed; however, new items were constructed to meet the demands of the revised

framework. All items underwent extensive reviews by specialists in mathematics, measurement, and

bias/sensitivity; items developed for grades four and eight were also reviewed by state representatives.

The core goals of the ETS assessment development process and procedures used to realize these goals

are outlined in the introduction to this chapter and in Section 2.3.

The following sections include a detailed description of the development of the framework,

objectives, and items for the 1996 NAEP mathematics assessment. Section 2.4.8 describes the student

background questionnaires and the reading teacher questionnaire. Additional information on the structure

and content of assessment booklets can be found in Chapter 4. Various committees worked on the

development of the framework, objectives, and items for the mathematics assessment. The list of

committee members and consultants who participated in the 1996 development process is provided in

Appendix A.

2.4.2 Development of the Assessment Framework

NAGB is responsible for developing assessment objectives and test specifications for NAEP

surveys. Appointed by the Secretary of Education from lists of nominees proposed by the board itself in

various statutory categories, the 26-member board is composed of state, local, and federal officials, as

well as educators and members of the public.

Under contract with NAGB, The College Board convened a committee during 1991 and 1992 to

develop an enhanced version of the framework that had been used during the development of the 1990

and 1992 assessments (the 1992 mathematics assessment had already been developed at the time the

development of this enhanced framework was begun). The enhanced version was needed to better reflect

the rapid evolution of mathematics instruction that was underway in the early 1990s as a result of the

emergence of the National Council of Teachers of Mathematics (NCTM) Curriculum and Evaluation

Standards for School Mathematics. The Standards were rapidly becoming one of the acknowledged

barometers for measuring achievement. The development process for the enhanced framework was based

on consensus building, and included the committee listed in Appendix A.

During this development process, input and reactions were continually sought from a wide range

of members of the mathematics field, experts in assessment, school administrators, and state staff. In

particular, the process was informed by recommendations of leading professional organizations in

mathematics.

2.4.3 Framework and Assessment Design Principles

The planning committee involved in the review and revision of the NAEP mathematics

framework was given a number of working guidelines and goals by the National Assessment Governing

Board and by the steering committee that oversaw the process. These guidelines directed the planning

committee to develop assessment specifications that called for performance-oriented exercises that focus

on problem-solving and provide students with opportunities to communicate their understandings in

mathematics. The framework should, according to these guidelines, embody a broad view of mathematics
31

that addressed the high levels of mathematical literacy needed for employability, personal development,

and citizenship. Also, the framework should take into account findings of contemporary research on

mathematics and mathematics education, and would expand the range of assessment tools to include

formats that more closely resembled classroom activities.

The development was further guided by the consideration that the assessment should reflect

many of the states’ curricular emphases and objectives in addition to what various scholars, practitioners,

and interested citizens believed should be included in the curriculum. Accordingly, the committee

focused on several general principles in revising the NAEP mathematics assessment. These principles

are:

• The purpose of the NAEP mathematics assessment is to provide information about

the progress and achievement of students in general rather than to test individual

students’ ability. NAEP is designed to inform policymakers and the public about

mathematics ability in the United States. Furthermore, NAEP state data can be used

to inform states of their students’ relative strengths and weaknesses.

• The term “mathematical literacy” encompasses such broad skills and abilities as

being able to reason numerically, algebraically, geometrically, spatially, and with

data; identify and apply problem-solving strategies appropriately in situations; and

use the language of mathematics to construct clear and coherent responses to

problems or tasks.

• The mathematics assessment should use authentic problems and tasks that address

important mathematics concepts and skills so that the assessment tool will

demonstrate a close link to desired classroom instruction and students’ mathematics

experiences.

• Every effort should be made to make the best use of available methodology and

resources in driving assessment capabilities forward.

• Every effort must be made in developing the assessment to represent a variety of

opinions, perspectives, and emphases among professionals in universities, as well as

in state and local school districts.

2.4.4 Framework for the 1996 Assessment

The framework for the 1996 mathematics assessment is organized according to a five-by-three

matrix of content strands by mathematical abilities. The content strands are:

• Number Sense, Properties, and Operations;

• Measurement;

• Geometry and Spatial Sense;

• Data Analysis, Statistics, and Probability; and

• Algebra and Functions.
32

These content strands were assessed across the three mathematical abilities of:

• Conceptual Understanding,

• Procedural Knowledge, and

• Problem Solving.

Figures 2-1 and 2-2 describe the five content strands and three mathematical abilities that guided

the development of the 1996 mathematics assessment.

Figure 2-1

Descriptions of Content Strands in Mathematics

Number Sense, Properties, and Operations

This strand focuses on students’ understanding of numbers (whole numbers, fractions, decimals,

integers, real numbers, and complex numbers), operations, and estimation, and their application to realworld situations. Students will be expected to demonstrate an understanding of numerical relationships as

expressed in ratios, proportions, and percents. Students also will be expected to understand properties of

numbers and operations, generalize from number patterns, and verify results.

Measurement

The measurement strand focuses on understanding of the process of measurement and on the use

of numbers and measures to describe and compare mathematical and real-world objects. Students will be

asked to identify attributes, select appropriate units and tools, apply measurement concepts, and

communicate measurement-related ideas.

Geometry and Spatial Sense

As described in the NCTM Standards, spatial sense must be an integral component of the study

and assessment of geometry. Understanding spatial relationships allows students to use the dynamic

nature of geometry to connect mathematics to their world.

This content strand is designed to extend well beyond low-level identification of geometric

shapes into transformations and combinations of those shapes. Informal constructions and

demonstrations (including drawing representations), along with their justifications, take precedence over

more traditional types of compass-and-straightedge constructions and proofs. While reasoning is

addressed throughout all of the content strands, this strand continues to lend itself to the demonstration of

reasoning within both formal and informal settings. The extension of proportional thinking to similar

figures and indirect measurement is an important connection here.

(continued)
33

Figure 2-1 (continued)

Descriptions of Content Strands in Mathematics

Data Analysis, Statistics, and Probability

The important skills of collecting, organizing, reading, representing, and interpreting data will be

assessed in a variety of contexts to reflect the pervasive use of these skills in dealing with information.

Statistics and statistical concepts extend these basic skills to include analyzing and communicating

increasingly sophisticated interpretations of data. Dealing with uncertainty and making predictions about

outcomes require an understanding not only of the meaning of basic probability concepts but also the

application of those concepts in problem-solving and decision-making situations.

Questions will emphasize appropriate methods for gathering data, the visual exploration of data,

a variety of ways of representing data, and the development and evaluation of arguments based on data

analysis. Students will be expected to apply these ideas in increasingly sophisticated situations that

require increasingly comprehensive analysis and decision making.

Algebra and Functions

This strand extends from work with simple patterns at grade 4, to basic algebra concepts at grade

8, to sophisticated analysis at grade 12, and involves not only algebra but also precalculus and some

topics from discrete mathematics. As described in the NCTM Standards, these algebraic concepts are

developed throughout the grades with informal modeling done at the elementary level and with increased

emphasis on functions at the secondary level. The nature of the algebraic concepts and procedures

included in the assessment at all levels reflects the NCTM Standards. Students will be expected to use

algebraic notation and thinking in meaningful contexts to solve mathematical and real-world problems,

specifically addressing an increasing understanding of the use of functions (including algebraic and

geometric) as a representational tool.
34

Figure 2-2

Descriptions of Mathematical Abilities

Conceptual Understanding

Students demonstrate conceptual understanding in mathematics when they provide evidence that

they can recognize, label, and generate examples and nonexamples of concepts; use and interrelate

models, diagrams, manipulatives, and varied representations of concepts; identify and apply principles

(i.e., valid statements generalizing relationships among concepts in conditional form); know and apply

facts and definitions; compare, contrast, and integrate related concepts and principles to extend the nature

of concepts and principles; recognize, interpret, and apply the signs, symbols, and terms used to represent

concepts; or interpret the assumptions and relations involving concepts in mathematical settings.

Conceptual understanding reflects a student’s ability to reason in settings involving the careful

application of concept definitions, relations, or representations of either. Such an ability is reflected by

student performance that indicates the production of examples, common or unique representations, or

communications indicating the ability to manipulate central ideas about the understanding of a concept in

a variety of ways.

Procedural Knowledge

Students demonstrate procedural knowledge in mathematics when they select and apply

appropriate procedures correctly; verify or justify the correctness of a procedure using concrete models

or symbolic methods; or extend or modify procedures to deal with factors inherent in problem settings.

Procedural knowledge includes the various numerical algorithms in mathematics that have been

created as tools to meet specific needs efficiently. Procedural knowledge also encompasses the abilities

to read and produce graphs and tables, execute geometric constructions, and perform noncomputational

skills such as rounding and ordering. These latter activities can be differentiated from conceptual

understanding by the task context or presumed student background—that is, an assumption that the

student has the conceptual understanding of a representation and can apply it as a tool to create a product

or to achieve a numerical result. In these settings, the assessment question is how well the student

executed a procedure or how well the student selected the appropriate procedure to effect a given task.

Procedural knowledge is often reflected in a student’s ability to connect an algorithmic process

with a given problem situation, to employ that algorithm correctly, and to communicate the results of the

algorithm in the context of the problem setting. Procedural understanding also encompasses a student’s

ability to reason through a situation, describing why a particular procedure will give the correct answer

for a problem in the context described.

Problem Solving

In problem solving, students are required to use their accumulated knowledge of mathematics in

new situations. Problem solving requires students to recognize and formulate problems; determine the

sufficiency and consistency of data; use strategies, data, models, and relevant mathematics; generate,

extend, and modify procedures; use reasoning (i.e., spatial, inductive, deductive, statistical, or

proportional) in new settings; and judge the reasonableness and correctness of solutions. Problem solving

situations require students to connect all of their mathematical knowledge of concepts, procedures,

reasoning, and communication/ representational skills in confronting new situations. As such, these

situations are, perhaps, the most accurate measures of students’ proficiency in mathematics. 
35

Tables 2-1 and 2-2 show the percentages of assessment time that the framework indicates should

be devoted to each content strand and mathematical ability.

Table 2-1

Percentage Distribution of Items by Grade and Content Strand

as Specified in the NAEP Mathematics Framework

Content Strand Grade 4 Grade 8 Grade 12

Number Sense, Properties, and Operations1 40%-70% 25%-60% 20%-50%

Measurement 20% 15% 15%

Geometry and Spatial Sense 15% 20% 20%

Data Analysis, Statistics, and Probability 10% 15% 20%

Algebra and Functions 15% 25% 25%

1 For this category, these percentages are the minimum and maximum that are acceptable, respectively.

Table 2-2

Percentage Distribution of Items by Grade and Mathematical Ability

as Specified in the NAEP Mathematics Framework

Mathematical Ability Grade 4 Grade 8 Grade 12

Conceptual Understanding 33% 33% 33%

Procedural Knowledge 33% 33% 33%

Problem Solving 33% 33% 33%

Note: Some items carry multiple classifications.

2.4.5 Developing the Cognitive Items

The 1996 assessment was designed to serve a dual purpose: to meet the content specifications

elaborated in the revised NAEP mathematics framework and to allow for the measurement of changes in

student mathematics achievement (that is, to permit linking to the 1990 and 1992 NAEP mathematics

assessment). Because of these objectives, the following strategies were adopted in developing the 1996

mathematics assessment. First, at each grade, 8 of the 13 blocks used in the 1992 assessment were carried

forward and used in 1996. Items were developed and field tested in 1993 to replace the five blocks from

the 1992 assessment that had been released for public use. In addition, other portions of the 1993 field

test were used to develop special components of the assessment. To this end, a new block of “estimation”

exercises was developed, as were exercises for a special study of students at grades eight and twelve who

had received advanced mathematics training. Given the fact that the mathematics assessment was delayed

from 1994 until 1996, a supplemental field test was held in 1995 (exercises were developed for this field

test in 1994). This round of exercise development was used to build extra exercises for the estimation and

advanced components of the assessment. It was also used to build “theme blocks,” which were intended

to measure in-depth mathematics problem-solving ability.

The development of cognitive items involved careful field testing, both locally and nationally, of

grade-appropriate questions and tasks for the assessment. Items were selected from a pool of questions

that were written by teachers from across the country as well as by mathematics assessment specialists on
36

staff at ETS. The framework stated that the assessment should include some performance-based

questions and tasks that require students to reason and make connections within and across different

content strands of mathematics. Final selections of questions used in the 1996 assessment were approved

by the Mathematics Instrument Development Committee.

The assessment included constructed-response (short and extended) and multiple-choice items.

The decision to use a specific item type was based on a consideration of the most appropriate format for

assessing the particular objective. Both types of constructed-response items were designed to provide an

in-depth view of students’ ability to communicate their understanding of important concepts in

mathematics. Short constructed-response questions (scored with either a 2- or 3-level scoring rubric)

were used when students needed to respond briefly in order to demonstrate full comprehension. Extended

constructed-response questions (scored with a 4- or 5-level scoring rubric) were used when the task

required more thoughtful consideration of the problem and engagement in more complex reasoning

processes. Multiple-choice items were used when a straightforward, single correct answer was required.

A carefully developed and proven series of steps was used to create the assessment items. These

steps are described earlier in the chapter under Section 2.3.

As was mentioned above, the assessment was designed to allow for measurement of trends.

Therefore, eight 15-minute blocks at each grade were included in the 1996 assessment that had been a

part of the 1992 assessment (and in some cases the 1990 assessment). These blocks were used in

precisely the same form as they were in 1992. In addition, one of the paced-tape estimation blocks was

carried forward from 1992. The remainder of the exercises used in 1996 were newly developed during

either the 1992-1993 or 1994-1995 development cycles.

2.4.6 Development of the Operational Forms

The field tests of new items for the 1996 assessment were conducted in February and March,

1993 and February and March, 1995. The field test involved a convenience sample in which roughly 500

responses were obtained to each item.

The field test data were collected, scored, and analyzed in preparation for meetings with the

Mathematics Instrument Development Panel. The objectives that guided the review of these items were:

• to determine which items were most suitable for assessing understanding in

mathematics in accordance with the framework;

• to select items that displayed appropriate statistical attributes;

• to determine the need for revisions of items that lacked clarity, or had ineffective

item formats;

• to prioritize items to be included in the assessment; and,

• to determine appropriate timing for assessment items.

Committee members, ETS assessment staff, and NAEP/ETS staff reviewed the materials. Item

analyses (which provided the mean percentage of correct responses, the r-biserial correlations, and the

difficulty level for each item) were used as a guide in identifying and flagging for further review those
37

test questions that were not measuring the intended objective well. In addition, another meeting of

representatives from state education agencies was convened to review the field test results for exercises

included in the grade four or eight assessments.

Once the committees had selected the items, all items were rechecked for content, measurement,

and sensitivity concerns. The federal clearance process was initiated in June 1993 with the submission of

draft materials to NCES. The package containing the set of cognitive items assembled into blocks and

questionnaires was submitted in August 1993. A revised package with the new thematic blocks and the

adjusted advanced and estimation blocks was submitted in July, 1995. Throughout the clearance process,

revisions were made in accordance with changes required by the government. After approval, the blocks

(assembled into booklets) and questionnaires were readied for printing in preparation for the assessment.

2.4.7 Distribution of Assessment Items

The mathematics assessment developed for use in 1996 was organized according to a series of

blocks, each containing a set of questions. Some of the blocks were unique to a particular grade level.

Other blocks were designed to be given to students in two grades (either 4 and 8 or 8 and 12) and a few

blocks (a small percentage of the blocks used to measure trend) overlap all three grades.

At each grade, the main component of the assessment used for the creation of reporting scales

included thirteen different 15-minute blocks of multiple-choice and constructed-response questions

(some with both regular and extended constructed-response questions). Two additional books containing

30-minute theme blocks of multiple-choice and constructed-response questions were also used. Two

estimation blocks at each grade were part of a special study and were presented to students by a pacedaudiotape to assess students’ estimation skills. Two 20-minute grade 8 special study blocks and two 30-

minute grade 12 special study blocks targeted students in advanced mathematics courses. Of the 13

blocks at each grade that were used for the main assessment:

• three to five blocks at each grade level included items designed to be answered using

a calculator. For the grade 4 calculator blocks, students were provided with a

4-function calculator, while at grades 8 and 12 the students were provided with a

scientific calculator (students were also provided calculators for the theme blocks

and allowed to use their own calculators on the advanced blocks)

• one block at all grade levels contained questions requiring the use of a

protractor/ruler (ruler only for grade 4)

• two blocks at each grade involved the use of manipulatives; several of the theme

blocks involved the use of manipulatives as well

• seven blocks at grade 4 and eight blocks at each of grades 8 and 12 included

extended constructed-response items. The extended constructed-response

mathematics items call for the student to work through a complex problem, require

about five minutes to complete, and were scored on a 0-5 point scale. The theme and

advanced blocks also included extended constructed-response items.
38

The sections that follow discuss the distribution of exercises on the mathematics assessment. For

purposes of this discussion, we will limit the calculations to exercises that appeared as part of the main

assessment, or BIB. Special study, theme, and estimation blocks will be discussed below.

Figure 2-3 lists the total number of items at each grade level in the main portion of the 1996

assessment. Of the total of 360 items, there were 194 unique multiple-choice items and 166 unique

constructed-response items. Some of these items were used at more than one grade level. As a result, the

sum of the items that appear at each grade level is greater than the total number of unique items.

Figure 2-3

Total Number of Items for the 1996 Mathematics Main Assessment

Grade 4

79 Multiple-Choice1

58 Short Constructed-Response2

7 Extended Constructed-Response

Grade 8

92 Multiple-Choice1

65 Short Constructed-Response2

8 Extended Constructed-Response

Grade 12

93 Multiple-Choice1

66 Short Constructed-Response2

8 Extended Constructed-Response

50 Multiple-Choice

33 Short Constructed-Response

7 Extended Constructed-Response

37 Multiple-Choice

26 Short Constructed-Response

8 Extended Constructed-Response

52 Multiple-Choice

45 Short Constructed-Response

8 Extended Constructed-Response

14 Multiple-Choice

18 Short Constructed-Response

0 Extended Constructed-Response

26 Multiple-Choice

14 Short Constructed-Response

0 Extended Constructed-Response

1

 These item counts include 15 multiple-choice questions that were presented at all three grades.

2

 These item counts include 7 short constructed-response questions that were presented at all three grades.

In the development process, every effort was made to meet the content and process targets

specified in the assessment framework. Tables 2-3, 2-4, and 2-5 show the approximate percentage of

aggregate assessment items devoted to each content strand at each grade level. Percentages are based on

the classifications agreed upon by NAEP’s 1996 Mathematics Instrument Development Committee, and

confirmed by independent reviewers.
39

Table 2-3

Distribution of Assessment Items

for the Mathematics Assessment, Grade 4

1990 1992 1996

Number

of items

Percentage

Of items

Number

of items

Percentage

of items

Number

of items

Percentage

of items

Content Strand

Number Sense, Properties, and Operations 52 48% 63 40% 59 41%

Measurement 21 19% 31 20% 25 17%

Geometry and Spatial Sense 14 13% 27 17% 25 17%

Data Analysis, Statistics, and Probability 8 7% 20 12% 17 12%

Algebra and Functions 14 13% 17 11% 18 13%

Total 109 100% 158 100% 144 100%

Mathematical Ability

Conceptual Understanding 42 39% 64 40% 61 42%

Procedural Knowledge 31 28% 31 20% 32 22%

Problem Solving 36 33% 63 40% 51 35%

Total 109 100% 158 100% 144 99%

Table 2-4

Distribution of Assessment Items

for the Mathematics Assessment, Grade 8

1990 1992 1996

Number

of items

Percentage

Of items

Number

of items

Percentage

of items

Number

of items

Percentage

of items

Content Strand

Number Sense, Properties, and Operations 45 33% 58 32% 48 29%

Measurement 21 15% 32 17% 27 16%

Geometry and Spatial Sense 26 19% 36 20% 31 19%

Data Analysis, Statistics, and Probability 19 14% 28 15% 25 15%

Algebra and Functions 26 19% 29 16% 34 21%

Total 137 100% 183 100% 165 100%

Mathematical Ability

Conceptual Understanding 59 43% 67 37% 57 35%

Procedural Knowledge 41 30% 45 24% 46 28%

Problem Solving 37 27% 71 39% 62 38%

Total 137 100% 183 100% 165 101%
40

Table 2-5

Distribution of Assessment Items

for the Mathematics Assessment, Grade 12

1990 1992 1996

Number

of items

Percentage

Of items

Number

of items

Percentage

of items

Number

of items

Percentage

of items

Content Strand

Number Sense, Properties, and Operations 37 26% 43 24% 41 25%

Measurement 23 16% 29 16% 23 14%

Geometry and Spatial Sense 24 17% 31 18% 27 16%

Data Analysis, Statistics, and Probability 22 15% 29 16% 34 20%

Algebra and Functions 38 26% 47 26% 42 25%

Total 144 100% 179 100% 167 101%

Mathematical Ability

Conceptual Understanding 53 37% 70 39% 66 40%

Procedural Knowledge 48 33% 52 29% 51 31%

Problem Solving 43 30% 57 32% 50 30%

Total 144 100% 179 100% 167 101%

Before proceeding, it is worth saying a few words about the other assessment components. The

estimation blocks were given to nationally representative samples of students at each grade. Estimation

was not given at the state level. Students in the estimation sample took one block from the main BIB. In

addition, they took two estimation blocks. Both estimation blocks were administered by a paced-tape

method.7

 The first estimation block was all multiple-choice, and was carried forward intact from 1992. It

contained 20 items at grade 4, and 22 items at grades 8 and 12. The second estimation block was newly

developed for 1996. It contained 13 items at grade 4 (10 multiple choice and 3 constructed-response), 15

at grade 8 (9 multiple-choice), and 16 at grade 12 (all multiple-choice). Trend was measured on this

assessment component.

Theme blocks were also given to nationally-representative samples at all grades. Students in the

theme sample first completed one block from the main assessment BIB. Then they were administered one

30-minute theme block. These blocks called for in-depth examination of mathematical problems, and for

the use of a variety of mathematics skills. At grade 4, the two theme blocks included a total of one

multiple-choice question and 13 constructed-response items. At grade 8, the total was 8 multiple-choice

items and 13 constructed-response items. At grade 12, the total was 8 multiple-choice items and 10

constructed-response exercises.

Finally, the advanced special studies were administered to samples of students meeting certain

course-taking criteria at grades 8 and 12 (algebra or higher at grade 8, and calculus, precalculus, or

algebra three at grade 12). Students took a linking block composed of exercises from the main assessment

and then two 20-minute blocks (at grade 8) or two 30-minute blocks (grade 12) of advanced exercises.

These blocks contained 9 multiple-choice and 13 constructed-response items at grade 8, and 7 multiplechoice and 15 constructed-response items at grade 12.

 

7

 The blocks were paced so students had to estimate rather than calculate the answer.
41

2.4.8 Questionnaires

As part of the national assessment (as well as the State Assessment), a series of questionnaires

was administered to students, teachers, and school administrators. Similar to the development of the

cognitive items, the development of the policy issues and questionnaire items was a consensual process

that involved staff work, field testing, and review by external advisory groups. A Background

Questionnaire Panel drafted a set of policy issues and made recommendations regarding the design of the

items. They were particularly interested in capitalizing on the unique properties of NAEP and not

duplicating other surveys (e.g., the National Survey of Public and Private School Teachers and

Administrators, the School and Staffing Survey, and the National Educational Longitudinal Study).

The Panel recommended a focused study that addressed the relationship between student

achievement and instructional practices. The policy issues, items, and field test results were reviewed by

the group of external consultants who identified specific items to be included in the final questionnaires.

In addition, the Mathematics Instrument Development Panel and state representatives were consulted on

the appropriateness of issues addressed in the questionnaires as they relate to mathematics instruction

and achievement. The items underwent internal ETS review procedures to ensure fairness and quality and

were then assembled into questionnaires.

⇒ Student Questionnaires

In addition to the cognitive items, the 1996 assessment included three student questionnaires; two

sets of general and mathematics background items designed to gather contextual information about

students, their instructional experiences in mathematics, and their attitudes toward mathematics, and one

set of background items, given to students at the end of each booklet to determine their motivation in

completing the assessment and their familiarity with assessment tasks. In order to ensure that all fourthgrade students understood the items and had every opportunity to respond to them, the three

questionnaires were read aloud by administrators as fourth-grade students read along and responded in

their booklets. Background questionnaires were not read aloud to eighth- and twelfth-grade students.

The Student Demographics (common background) Questionnaire included items about

race/ethnicity, language spoken in the home, mother’s and father’s level of education, reading materials

in the home, homework, attendance, which parents live at home, and which parents work. This

questionnaire was the first section in every booklet. In many cases the items used were continued from

prior assessments, so as to document changes in contextual factors that occur over time.

The second section of background items was the Mathematics Background Questionnaire.

Categories of information represented in this section include:

Time Spent Studying Mathematics: Students were asked to describe both the amount of

instruction they received in mathematics and the time spent on mathematics homework.

Instructional Practices: Students were asked to report their instructional experiences

related to mathematics in the classroom, including group work, special projects, and writing

in response to mathematics. In addition, they were asked about the instructional practices of

their mathematics teachers and the extent to which the students themselves discussed what

they did in class and demonstrated use of skills and strategies.
42

Attitudes Towards Mathematics: Students were asked a series of questions about their

attitudes and perceptions about mathematics.

The Student Motivation Questionnaire asked students to describe how hard they tried on the

NAEP mathematics assessment, how difficult they found the assessment, how many items they thought

they got right, how important it was for them to do well, and how familiar they were with the assessment

format.

⇒ Teacher, School, and SD/LEP Student Questionnaires

To supplement the information on instruction reported by students, the mathematics teachers of

the students participating in the mathematics assessment were asked to complete a questionnaire about

their instructional practices, teaching backgrounds, and characteristics. The teacher questionnaire

contained two parts. The first part pertained to the teachers’ background and general training. The second

part pertained to specific training in teaching mathematics and the procedures the teacher uses for each

class containing an assessed student, as well as collecting information on teachers’ awareness and

knowledge of the NCTM Standards.

The Teacher Questionnaire, Part I: Background and General Training included questions

pertaining to gender, race/ethnicity, years of teaching experience, certification, degrees, major and minor

fields of study, course work in education, course work in specific subject areas, amount of in-service

training, extent of control over instructional issues, and availability of resources for their classroom.

The Teacher Questionnaire, Part II: Training in Mathematics and Classroom Instructional

Information included questions on the teacher’s exposure to various issues related to mathematics and

teaching mathematics through pre- and in-service training, ability level of students in the class, whether

students were assigned to the class by ability level, time on task, homework assignments, frequency of

instructional activities used in class, methods of assessing student progress in mathematics, instructional

emphasis given to the mathematics abilities covered in the assessment, and use of particular resources.

A School Characteristics and Policies Questionnaire was given to the principal or other

administrator of each school that participated in the assessment. This information provided an even

broader picture of the instructional context for students’ mathematics achievement. This questionnaire

included questions about background and characteristics of school principals, length of school day and

year, school enrollment, absenteeism, dropout rates, size and composition of teaching staff, policies

about grouping students, curriculum, testing practices and uses, special priorities and school-wide

programs, availability of resources, special services, community services, policies for parental

involvement, and school-wide problems.

The SD/LEP Student Questionnaire was completed by the teachers of students who were

selected to participate in the assessment sample who were also identified as students with a disability

(SD) or categorized as being of limited English proficiency (LEP). Some of these students were

determined by the school to be ineligible to be assessed. In order to be excluded from the assessment, a

student must have been identified as SD and must not have been mainstreamed at least 50 percent of the

time, or was categorized as LEP. In addition, the school staff would have needed to determine that it was

inappropriate to include the student in the assessment. This questionnaire asked about the nature of the

student’s disability or about the student’s native language, and the special programs in which the student

participated.
43

2.5 DEVELOPING THE SCIENCE ASSESSMENT

2.5.1 Overview

The science framework for the 1996 National Assessment of Educational Progress (NAEP) was

produced under the auspices of the National Assessment Governing Board (NAGB). The consensus

process was managed by the Council of Chief State School Officers (CCSSO) who worked with the

National Center for Improving Science Education and the American Institutes for Research. Items were

developed that were aligned with the specifications described in the framework and were extensively

reviewed by specialists in science, measurement, and bias/sensitivity, as well as by government officials

and state representatives.

The following sections include a detailed description of the development of the framework,

objectives, and items for the 1996 NAEP science assessment. Section 2.5.8 describes the student

background questionnaires and the science teacher questionnaire. Additional information on the structure

and content of assessment booklets can be found in Chapter 4. Various committees worked on the

development of the framework, objectives, and items for the mathematics assessment. The list of

committee members and consultants who participated in the 1996 development process is provided in

Appendix A.

2.5.2 Development of the Assessment Framework

NAGB is responsible for setting policy for NAEP; this policymaking role includes the

development of assessment frameworks and test specifications. Appointed by the Secretary of Education

from lists of nominees proposed by the Board itself in various statutory categories, the 26-member board

is composed of state, local, and federal officials, as well as educators and members of the public.

The science framework for the NAEP 1996 assessment was developed over a 10-month period

between October 1990 and August 1991. The following sections discuss how the specifications and items

for science assessment were developed. The assessment instrument, the student assessment booklets, and

the student, teacher, school, and SD/LEP questionnaires are also described.

A consensus process run by CCSSO was used to produce the science framework. This process

involved two committees: a Planning Committee that conducted much of the actual framework

development and a steering committee that provided policy and general guidance for the project. As

general guidelines for the Planning Committee, the Steering Committee that recommended that the

framework and ensuing science assessment have the following five characteristics:

• The framework should reflect the best thinking about the knowledge, skills, and

competencies needed for a high degree of scientific understanding among all

students in the United States. Accordingly it should encompass knowledge and use

of organized factual information, relationships among concepts, major ideas unifying

the sciences, and thinking and laboratory skills. In addition, the framework should be

based on current understandings from research of teaching, learning, and students’

performance in science.

• The framework and the assessment should address the nature and practices of

knowing in science, as different from other ways of knowing; reflect the quantitative

aspects of science as well as the concepts of life, earth, and physical sciences; deal
44

with issues raised by the role of science and technology in society; include practical

problem solving in science; take into account the developmental levels of students;

and ensure that students with diverse backgrounds are assessed in ways that provide

them with equal and fair opportunities to reflect their knowledge and performance.

• Assessment formats should be used that are consistent with the objectives being

assessed. A variety of strategies for assessing student performance are advocated,

including performance tasks that allow students to manipulate physical objects and

draw scientific understandings from the materials before them; constructed-response

items that provide insights into students’ levels of understanding and ability to

communicate in the sciences, as well as their ability to generate, rather than simply

recognize information related to scientific concepts and their interconnections; and

multiple-choice items that probe students’ conceptual understanding and ability to

connect ideas in a scientifically sound way.

• The assessment should contain a broad enough range of items at different levels of

proficiency for identifying three achievement levels for each grade.

• Information on students’ demographic and other background characteristics should

be collected. Additional information should be collected from students, teachers and

administrators about instructional programs and delivery systems, so that their

relationships with student achievement can be ascertained and used to inform

program and policy decisions.

A Planning Committee was established to identify goals and objectives and to produce the

framework. This Planning Committee met monthly from November 1990 through April 1991 and was

joined in the first meeting and final meeting by the Steering Committee, which reviewed and reacted to

all framework drafts. During this development process, input and reactions were continually sought from

a wide range of committee members both within the field of science and external to it. A list of

committee members who participated in the developmental process is provided in Appendix A.

2.5.3 Framework for the 1996 Assessment

The framework for the 1996 science assessment is represented as a matrix with two dimensions

represented by three fields of science (earth science, physical science, and life science) and three

elements of knowing and doing science ( conceptual understanding, scientific investigation, and

practical reasoning). In addition, there are two overarching domains that describe science and nature of

science and themes. Figures 2-4 to 2-6, respectively, describe the three fields of science, the elements of

knowing and doing science, and the overarching domains.
45

Figure 2-4

Descriptions of the Three Fields of Science

Earth Science

The earth science component assessed centers on objects and events that are relatively accessible

or visible. The concepts and topics covered are solid earth (lithosphere), water (hydrosphere), air

(atmosphere), and the earth in space. The solid earth consists of composition; forces that alter its surface;

the formation, characteristics, and uses of rocks; the changes and uses of soil; natural resources used by

humankind; and natural forces within the earth. Concepts and topics related to water consist of the water

cycle; the nature of oceans and their effects on water and climate; and the location of water, its

distribution, characteristics, and effect of and influence on human activity. The air is broken down into

composition and structure of the atmosphere (including energy transfer); the nature of weather; common

weather hazards; and air quality and climate. The earth in space consists of setting of the earth in the

solar system; the setting and evolution of the solar system in the universe; tools and technology that are

used to gather information about space; apparent daily motions of the sun, the moon, the planets and the

stars; rotation of the earth about its axis, the earth’s revolution around the sun; and tilt of the earth’s axis

that produces seasonal variations in the climate.

Physical Science

The physical science component relates to basic knowledge and understanding concerning the

structure of the universe as well as the physical principles that operate within it. The major sub-topics

probed are matter and its transformations, energy and its transformations, and the motion of things.

Matter and its transformations are described by diversity of materials (classification and types and the

particulate nature of matter); temperature and states of matter; properties and uses of material (modifying

properties, synthesis of materials with new properties); and resource management. Energy and its

transformations involve different forms of energy; energy transformations in living systems, natural

physical systems, and artificial systems constructed by humans; and energy sources and use, including

distribution, energy conversion, and energy costs and depletion. Motion is broken down into an

understanding of frames of reference; forces and changes in position and motion; action and reaction;

vibrations and waves as motion; general wave behavior; electromagnetic radiation; and the interactions

of electromagnetic radiation with matter.

Life Science

The fundamental goal of life science is to attempt to understand and explain the nature and

function of living things. The major concepts assessed in life science are change and evolution, cells and

their functions, organisms, and ecology. Change and evolution includes diversity of life on earth; genetic

variation within a species; theories of adaptation and natural selection; and changes in diversity over

time. Cells and their functions consists of information transfer; energy transfer for the construction of

proteins; and communication among cells. Organism are described by reproduction, growth and

development; life cycles; and functions and interactions of systems within organisms. The topic of

ecology centers on the interdependence of life—populations, communities, and ecosystems.
46

Figure 2-5

Descriptions of Knowing and Doing Science

Conceptual Understanding

Conceptual understanding includes the body of scientific knowledge that students draw upon

when conducting a scientific investigation or engaging in practical reasoning. Essential scientific

concepts involve a variety of information including facts and events the student learns from science

instruction and experiences with the natural environment and scientific concepts, principles, laws, and

theories that scientists use to explain and predict observations of the natural world.

Scientific Investigation

Scientific investigation probes students’ abilities to use the tools of science, including both

cognitive and laboratory tools. Students should be able to acquire new information, plan appropriate

investigations, use a variety of scientific tools, and communicate the results of their investigations.

Practical Reasoning

Practical reasoning probes students’ ability to use and apply science understanding in new, realword applications.

Figure 2-6

Descriptions of Overarching Domains

The Nature of Science

The nature of science incorporates the historical development of science and technology, the

habits of mind that characterize these fields, and methods of inquiry and problem-solving. It also

encompasses the nature of technology that includes issues of design, application of science to real-world

problems, and trade-offs or compromises that need to be made.

(continued)
47

Figure 2-6 (continued)

Descriptions of Overarching Domains

Themes

Themes are the “big ideas” of science that transcend the various scientific disciplines and enable

students to consider problems with global implications. The NAEP science assessment focuses on three

themes: systems, models, and patterns of change.

• Systems are complete, predictable cycles, structures or processes occurring in natural phenomena.

Students should understand that a system is an artificial construction created to represent, or explain

a natural occurrence. Students should be able to identify and define the system boundaries, identify

the components and their interrelationships and note the inputs and outputs to the system.

• Models of objects and events in nature are ways to understand complex or abstract phenomena. As

such they have limits and involve simplifying assumptions but also possess generalizability and often

predictive power. Students need to be able to distinguish the idealized model from the phenomenon

itself and to understand the limitations and simplified assumptions that underlie scientific models.

• Patterns of change involve students’ recognition of patterns of similarity and differences, and

recognition of how these patterns change over time. In addition, students should have a store of

common types of patterns and transfer their understanding of a familiar pattern of change to a new

and unfamiliar one.

Table 2-6 summarizes the distribution of assessment time across the three fields of science—

earth, physical, and life. These fields provide the basis for the content area scales.

Table 2-6

Percentage Distribution of Items by Grade and Field of Science

as Specified in the NAEP Science Framework

Field of Science Grade 4 Grade 8 Grade 12

Earth Science 33% 30% 33%

Physical Science 33% 30% 33%

Life Science 33% 40% 33%
48

Table 2-7 shows the distribution of assessment time by knowing and doing science.

Table 2-7

Percentage Distribution of Items by Grade and Knowing and Doing Science

as Specified in the NAEP Science Framework

Knowing and Doing Science Elements Grade 4 Grade 8 Grade 12

Conceptual Understanding 45% 45% 45%

Scientific Investigation 45% 30% 30%

Practical Reasoning 10% 25% 25%

A number of items that assess each of the fields of science and each of the ways of knowing and

doing science also probe nature of science and themes (systems, models, and patterns of change). Table

2-8 shows the recommended and actual percentages of assessment time for these two overarching

domains.

Table 2-8

Percentage Distribution of Items Devoted to Nature of Science and Themes

as Specified in the NAEP Science Framework

Overarching Domains Grade 4 Grade 8 Grade 12

Nature of Science ≥15% ≥15% ≥15%

Themes 33% 50% 50%

 In addition to calling for coverage of the content and cognitive domains described above, the

framework instructed that all students participating in NAEP take part in a scientific investigation or

hands-on experiment. In addition, it indicated that at least 30 percent of students should complete

portions of the assessment involving in-depth examination of certain themes or topics in science.

2.5.4 Developing the Cognitive Items

The 1996 assessment was designed to meet the content specifications elaborated in the

framework. Because of the broad content and skills definitions included in the framework, and the need

to assess hand-on and theme-based science skills, the exercise development effort was extensive. At each

grade, enough blocks were field tested to support 13 operational blocks; each block was 20 minutes at

grade 4 and 30 minutes at the older two levels.8

 In addition, other portions of the 1993 field test were

used to develop exercises for a special study of students at grade twelve who had received advanced

science training. Given the fact that the science assessment was delayed from 1994 until 1996, a

supplemental field test was held in 1995 (exercises were developed for this field test in 1994). This round

of exercise development was used to build extra exercises for the general science and advanced

components of the assessment.

The development of cognitive items involved careful field testing, both locally and nationally, of

grade-appropriate questions and tasks for the assessment. Items were selected from a pool of questions

 

8

 Most of the development and field testing were conducted during 1992 and 1993; supplemental development and field testing

of general science blocks was conducted during 1994 and 1995.
49

that were written by teachers from across the country as well as by science assessment specialists on staff

at ETS. The framework stated that the assessment should include some performance-based questions and

tasks that require students to reason and solve problems representing real-life applications of science.

Final selections of items used in the 1996 assessment were approved by the Science Instrument

Development Committee.

The assessment included constructed-response (short and extended) and multiple-choice items.

The decision to use a specific item type was based on a consideration of the most appropriate format for

assessing the particular objective. Both types of constructed-response items were designed to provide an

in-depth view of students’ ability to communicate their understanding of important concepts in science.

Short constructed-response items (scored with either a 2- or 3-level scoring rubric) were used when

students needed to respond briefly in order to demonstrate full comprehension. Extended constructedresponse items (scored with a 4- or 5-level scoring rubric) were used when the task required more

thoughtful consideration of the problem and engagement in more complex reasoning processes. Some

items also required diagrams, graphs, or calculations. It was expected that students could adequately

answer the short constructed-response items in about two to three minutes and the extended constructedresponse items in about five minutes. In addition, blocks of items were developed that required the

manipulation of equipment (hands-on tasks) and others were developed that assessed each of the three

themes: systems, models, and patterns of change. In the case of some of the hands-on blocks, compound

items were created in which student responses to a variety of items were scored as a single item.

Multiple-choice items were used when a straightforward, single correct answer was required.

A carefully developed and proven series of steps was used to create the assessment items. These

steps are described earlier in the chapter under Section 2.3.

2.5.5 Development of the Operational Forms

Most of the items for the 1996 science assessment were field tested in February and March 1993;

however, since the assessment was delayed from 1994 to 1996 an opportunity was afforded for further

items to be field tested in February and March 1995. Each of these field tests involved students in many

states and were intended to try out the cognitive items and hands-on-tasks and to give jurisdictions and

contractors practice and experience with the proposed materials and tasks. Approximately 500 responses

were obtained for each item in each field test.

The field test data were collected, scored, and analyzed in preparation for meetings with the

Science Instrument Development Committee. The objectives that guided the review of these items were:

• to determine which items were most suitable for assessing understanding in science

in accordance with the framework;

• to select items that displayed appropriate statistical attributes;

• to determine the need for revisions of items that lacked clarity, or had ineffective

item formats;

• to determine appropriate timing for assessment items.

Committee members, ETS assessment staff, and NAEP staff reviewed the materials. Item

analyses (which provided the percentage of correct responses, the biserial correlations for multiple-
50

choice and items with a two-level scoring guides, and percentages of responses in each category or at

each level of the scoring guide and the polyserial correlations for other constructed-response items) were

used as a guide in identifying and flagging for further review those test items that were not measuring the

intended objective well.

Once the committees had selected the items, they were rechecked for content, measurement, and

sensitivity concerns. In addition, a meeting of representatives from state education agencies was

convened to review the items chosen for the components of the operational assessment that were to be

administered at the state level (that is, the grade 8 assessment). The federal clearance package containing

13 blocks of cognitive items was submitted to NCES in August 1993. A further clearance package

containing two blocks of items was submitted to NCES in 1995. Throughout the clearance process,

revisions were made in accordance with changes required by the government. Upon approval, the 15

blocks (assembled into booklets) and questionnaires were ready for printing in preparation for the

assessment.

2.5.6 Distribution of Assessment Items

The science assessment developed for use in 1996 was organized according to a series of blocks,

each containing a set of items. Some of the blocks were unique to a particular grade level. Other blocks

were designed to be given to students in two grades (either 4 and 8 or 8 and 12).

At each grade, the main component of the assessment used for the creation of reporting scales

included fifteen different blocks of multiple-choice and constructed-response items (some with both

regular and extended constructed-response items). These blocks each were 20-minutes long at grade four

and thirty minutes at the other levels. Three 30-minute special study blocks targeted students in advanced

mathematics courses. Of the 15 blocks at each grade that were used for the main assessment:

• four blocks at each grade level required students to complete a hands-on science task

• three blocks at each grade required students to work in-depth in a single thematic or

topical area

• eight blocks at each grade covered general knowledge and concepts in the fields of

science

Figure 2-7 lists the total number of items at each grade level in the 1996 assessment. Of the total

of 438 items, there were 167 unique multiple-choice items and 271 unique constructed-response items

that made up the 1996 science assessment. Some of these items were used at more than one grade level.

As a result, the sum of the items that appear at each grade level is greater than the total number of unique

items.
51

Figure 2-7

Total Number of Items for the 1996 Science Main Assessment

Grade 4

50 Multiple-Choice1

69 Short Constructed-Response

18 Extended Constructed-Response

Grade 8

77 Multiple-Choice

93 Short Constructed-Response

25 Extended Constructed-Response

Grade 12

71 Multiple-Choice

90 Short Constructed-Response

27 Extended Constructed-Response

40 Multiple-Choice

50 Short Constructed-Response

14 Extended Constructed-Response

46 Multiple-Choice

50 Short Constructed-Response

17 Extended Constructed-Response

50 Multiple-Choice

66 Short Constructed-Response

23 Extended Constructed-Response

10 Multiple-Choice

19 Short Constructed-Response

4 Extended Constructed-Response

21 Multiple-Choice

24 Short Constructed-Response

4 Extended Constructed-Response

1 The percentage of time for multiple-choice items is low because the hands-on tasks and theme blocks contain very few

multiple-choice items and take up 47% of the assessment time at each grade level.

Table 2-9 summarizes the distribution of assessment time across the three fields of science. Since

these fields make up the core of the Science Framework, care has been taken to ensure the greatest possible

congruence between the proportions used in the assessment and those indicated in the assessment

specifications.

Table 2-9

Distribution of Assessment Time by Field of Science

Earth Science Physical Science Life Science

Grade 4 33% 34% 33%

Grade 8 30% 30% 40%

Grade 12 33% 33% 34%
52

Table 2-10 summarizes the assessment in terms of percentage of time devoted to different cognitive

domains. The classification of items into these domains was conducted by both ETS staff and members of

the instrument development committee. Every effort was made to meet the specified targets.

Table 2-10

Distribution of Assessment Time Across Cognitive Domains (Knowing and Doing)

Conceptual

Understanding

Scientific

Investigation

Practical

Reasoning

Grade 4 45% 38% 17%

Grade 8 45% 29% 26%

Grade 12 44% 28% 28%

Exercises assessing each of the fields of science and each of the cognitive domains also probe

three themes, (models, systems, and patterns of change), and the students’ knowledge of the nature of

science. Table 2-11 shows the distribution of assessment time devoted to themes, and Table 2-12

shows the distribution of assessment time devoted to the nature of science.

Table 2-11

Percentage of Assessment Time Devoted to Themes

Actual Recommended

Grade 4 53%1 30%

Grade 8 49% 50%

Grade 12 55% 50%

1

Several of the hands-on tasks were classified as themes.

Table 2-12

Percentage of Assessment Time Devoted to Nature of Science

Actual Recommended

Grade 4 19% ≥15%

Grade 8 21% ≥15%

Grade 12 31% ≥15%

In addition to the main components of the assessment, an advanced study was conducted at grade

12. Students who were in their fourth year of high-school science were sampled. Each participant took a

linking block made up of 17 exercises from the main assessment, and three special advanced blocks: one

in biology, one in chemistry, and one in physics. Each of the advanced blocks contained seven multiplechoice and nine constructed-response items.
53

2.5.7 Questionnaires

As part of the national assessment (as well as the State Assessment), a series of questionnaires

was administered to students, teachers, and school administrators. Similar to the development of the

cognitive items, the development of the policy issues and questionnaire items was a consensual process

that involved staff work, field testing, and review by external advisory groups. A Background

Questionnaire Committee drafted a set of policy issues and made recommendations regarding the design

of the items. They were particularly interested in capitalizing on the unique properties of NAEP and not

duplicating other surveys (e.g., the National Survey of Public and Private School Teachers and

Administrators, the School and Staffing Study, and the National Educational Longitudinal Study). The

policy issues, items, and field test results were reviewed by the group of external consultants who

identified specific items to be included in the final questionnaires. In addition, the Science Instrument

Development Committee and state representatives were consulted on the appropriateness of issues

addressed in the questionnaires as they relate to science instruction and performance. The items

underwent internal ETS review procedures to ensure fairness and quality and were then assembled into

questionnaires. The questionnaires were then submitted to the Office of Management and Budget (OMB)

for approval.

⇒ Student Questionnaires

In addition to three blocks of cognitive items, each booklet in the assessment included three

student questionnaires. Two of these were sets of general and science background questionnaires

designed to gather contextual information about students, their instructional experiences in science, and

their attitudes toward science. The third questionnaire was given to students at the end of each booklet to

determine students’ motivation in completing the assessment and their familiarity with assessment tasks.

The Student Demographics (common background) Questionnaire included questions about

race/ethnicity, mother’s and father’s level of education, types of reading materials in the home, and

school attendance.

The Science Background Questionnaire included questions that addressed the following.

Attitudes Towards Sciences: Students were asked a series of questions about their attitudes

and perceptions about science.

Time Spent Studying Science: Students were asked to describe both the amount of

instruction they received in science and the time spent on science homework.

Instructional Practices: Students were asked to report their instructional experiences

related to science in the classroom, including group work, special projects, and writing in

response to science. In addition, they were asked about the instructional practices of their

science teachers.

The Student Motivation Questionnaire asked students how many questions they thought they

got right on the NAEP science assessment, how difficult they found it, how hard they tried, how

important it was for them to do well, and how often they wrote long answers on tests or assignments for

science.
54

⇒ Teacher, School, and SD/LEP Student Questionnaires

To supplement the information on instruction reported by students, the science teachers of the

students participating in the assessment were asked to complete a questionnaire that addressed teachers’

background and general training as well as their science preparation and information concerning science

instruction.

The Teacher Questionnaire, Part I: Background and General Training included questions

about gender, race/ethnicity, years of teaching experience, certification, degrees, major and minor fields

of study, course work in education, course work in specific subject areas, amount of in-service training,

professional development activities, and availability of resources for their classroom.

The Teacher Questionnaire, Part II: Science Preparation and Science Instructional

Information included questions on the number and types of science courses taken over the past two

years, membership in science organizations, frequency of instructional activities such as asking students

to prepare a written science report or an oral science report, emphasis on objectives such as developing

science problem-solving skills, methods used to assess student progress in science, and ability level of

students in class.

A School Characteristics and Policies Questionnaire was given to the principal of each school

that participated in the assessment program. This questionnaire asked about background and

characteristics of school principals, length of school day and year, school enrollment, absenteeism, dropout rates, size and composition of teaching staff, policies about grouping students, curriculum, testing

practices and uses, special priorities and school-wide programs, availability of resources, special

services, community services, policies for parental involvement, and school-wide problems.

The SD/LEP Student Questionnaire was completed by the teachers of those students who were

selected to participate in the assessment sample and were identified as students with a disability (SD) or

were categorized as being of limited English proficiency (LEP). Some of these students were determined

by the school to be ineligible to be assessed. In order to be excluded from the assessment, a student must

have been identified as SD and must not have been mainstreamed at least 50 percent of the time, or was

categorized as LEP. In addition, the school staff would have needed to determine that it was

inappropriate to include these students in the assessment. This questionnaire asked about the nature of

the student’s disability or about the students’ native language, and the special programs in which the

student participated.
55

Chapter 3

SAMPLE DESIGN1

Leslie Wallace and Keith F. Rust

Westat, Inc.

3.1 INTRODUCTION

The samples for the 1996 NAEP assessment were selected using a complex multistage sample

design involving the sampling of students from selected schools within 94 selected geographic areas,

called primary sampling units (PSUs), across the United States.

The long-term trend sample design had four steps in the selection process and the main sample

design had five steps in the selection process:

1. selection of geographic PSUs (counties or groups of counties),

2. selection of schools within PSUs,

3. assignment of sample type to schools (main samples only),

4. assignment of session types to schools, and

5. selection of students for session types within schools.

The samples were drawn for the three different age classes,2

 and for each age class the samples

were of two distinct types. The first type consisted of the cross-sectional or “main” samples, while the

second type consisted of the long-term trend samples. The populations surveyed with each of these

sample types are defined in Table 1-1 in Chapter 1. Separate samples of schools were required for the

long-term trend samples and main samples, because of various differences in the calendar period for test

administration, the format of the administration, the fact that the trend samples include age-based

samples, whereas main samples do not and, in the case of age class 17, the grade definition of the

population of interest. (See the description of Table 1-1 in Chapter 1.)

In addition to representing the respective populations as a whole, for the main samples there was

oversampling of nonpublic schools, and of public schools with moderate or high enrollment of Black or

Hispanic students (see Section 3.3). This oversampling was undertaken to increase the sample sizes of

nonpublic-school students and minority students, so as to increase the reliability of estimates for these

groups of students. These oversampling rates have been used in the past several rounds of NAEP. The

oversampling rates were based on experience, after attempting to report results for these groups in

assessments where no oversampling was used.

 

1 Ralph DiGaetano, Keith F. Rust, and Leslie Wallace were responsible for the design and implementation of the sampling

process for the 1996 NAEP assessments.

2

 The term “age class” is used in this report when it is appropriate to discuss one of the three student cohorts in a general way

(not necessarily in reference to a specific sample). For the 1996 assessment, age class 9 refers to age 9 or grade 4 long-term trend,

or grade 4 main sample students; age class 13 refers to age 13 or grade 8 long-term trend, or grade 8 main sample students; and

age class 17 refers to age 17 or grade 11 long-term trend, or grade 12 main sample students.
56

The overall assessment period fell into three time periods—fall, winter, and spring. Not all

assessment components were conducted in each time period. Table 3-1 shows the relationship between

the various sample components and the assessment periods. The sizes of the PSU and school samples and

the procedures for their selection were determined by the assessment period, as well as by the population

to be surveyed and the method of administration in each case.

Table 3-1

Assessment Type by Age Class and Assessment Period

Age Class/Assessment Fall Winter Spring

9

Main

Long-Term Trend

—

—

1/3/96 - 3/29/96

1/3/96 - 3/8/96

—

—

13

Main

Long-Term Trend

—

10/9/95 - 12/22/95

1/3/96 - 3/29/96

—

—

—

17

Main

Long-Term Trend

—

—

1/3/96 - 3/29/96

—

—

3/11/96 - 5/10/96

Special trend samples were required because:

• The long-term trend samples had different school and student eligibility

requirements than the main samples. Both grade- and age-eligible students were

targeted in long-term trend, and only grade-eligible students were targeted in the

main samples. This meant that schools with any of several grades were eligible for

long-term trend (grades 2 to 6 for age class 9, grades 6 to 9 for age class 13, and

grades 9 to 12 for age class 17), while only schools with grades 4, 8, or 12 were

eligible for the main samples.

• The conditions for administration of the assessment varied considerably between the

main sample and long-term trend sample sessions.

• The need in the long-term trend samples for four distinct session types for age class

9 and 13 and three for age class 17, together with the need for up to six distinct

session types for the main samples, made it not feasible to conduct both main sample

sessions and long-term trend sessions in a given school. For long-term trend, the

session types were spiral booklets 51-56 and tape booklets 91-93 for ages 9 and 13,

and spiral booklets 51-56 and tape booklets 84-85 for age 17. For the main samples,

the session types were mathematics, science, mathematics estimation, and

mathematics theme at all grades (4, 8, and 12), advanced mathematics at grades 8

and 12, and advanced science at grade 12.

• For age classes 13 and 17, the main sample administrations were conducted in the

winter; while the long-term trend sample administrations were conducted in the fall
57

and spring respectively. The fall and spring administration periods match

administration periods used in NAEP as far back as 1969-71.

This chapter gives details of the sample selection procedure, and information on the results of the

sampling process. Further details are given in the report The 1996 NAEP Sampling and Weighting Report

(Wallace & Rust, 1999).

3.2 PRIMARY SAMPLING UNITS

In the first stage of sampling, the United States (the 50 states and the District of Columbia) was

divided into geographic primary sampling units (PSUs). The PSUs are those that were used beginning in

1994 and incorporate 1990 U.S. Census information. With a few exceptions, each PSU met a minimum

size requirement (a 1990 U.S. Census population of at least 60,000 in the Northeast and Southeast and

45,000 in the Central and West regions) and comprised either a consolidated metropolitan statistical area

(CMSA), a metropolitan statistical area (MSA), a single county, or (more likely in the case of nonMSA

PSUs) a group of contiguous counties. In the case of New England MSAs, which are not formed from

whole counties, the corresponding New England County Metropolitan Areas (NECMAs), which are

defined in terms of whole counties, were designated as PSUs. The PSUs were designed to serve as the

PSUs for NAEP samples from 1994 until 2002. Thus 1990 total population was used as a size measure,

rather than 1990 school age population, as this was considered likely to correlate more highly with school

age population over this period. Each PSU was contained entirely within one of the four NAEP regions

defined in Table 3-2. These NAEP regions were used to stratify the PSUs, ensuring that each region was

adequately represented in the various assessment samples.

Table 3-2

Geographic Regions Used for Stratification

Northeast Southeast Central West

Connecticut

Delaware

District of Columbia

Maine

Maryland

Massachusetts

New Hampshire

New Jersey

New York

Pennsylvania

Rhode Island

Vermont

Virginia1

Alabama

Arkansas

Florida

Georgia

Kentucky

Louisiana

Mississippi

North Carolina

South Carolina

Tennessee

Virginia1

West Virginia

Illinois

Indiana

Iowa

Kansas

Michigan

Minnesota

Missouri

Nebraska

North Dakota

Ohio

South Dakota

Wisconsin

Alaska

Arizona

California

Colorado

Hawaii

Idaho

Montana

Nevada

New Mexico

Oklahoma

Oregon

Texas

Utah

Washington

Wyoming

1 That part of Virginia that is part of the Washington, DC-MD-VA metropolitan statistical area at

the time of the 1990 Census, is included in the Northeast region; the remainder of the state is

included in the Southeast region.
58

In a few cases an MSA crossed region boundaries. Such MSAs were split into two or more PSUs

as necessary (e.g., the Cincinnati OH-KY-IN MSA was split into the Cincinnati OH-IN PSU in the

Central region and the Cincinnati KY PSU in the Southeast). Ninety-four PSUs were selected for the

main samples and 52 PSUs were selected for the long-term trend samples, as described below.

For the main samples, the 22 largest PSUs were included with certainty. The inclusion of these

PSUs in the sample with certainty provided an approximately optimum, cost-efficient sample of schools

and students when samples were drawn within them at the required national sampling rate. The 22 largest

PSUs by region were:

22 Largest PSUs by Region

Northeast:

 Baltimore, MD MSA

 Boston-Lawrence-Salem-LowellBrockton, MA NECMA

 New York-Northern New Jersey-Long

Island, NY-NJ CMSA (excluding that

part in CT)

 Philadelphia-Wilmington-Trenton,

PA-NJ-DE-MD CMSA

 Pittsburgh-Beaver Valley, PA CMSA

 Washington, DC-MD-VA MSA

Southeast:

 Atlanta, GA MSA

 Miami-Fort Lauderdale, FL CMSA

 Tampa-St. Petersburg-Clearwater, FL

MSA

Central:

 Chicago-Gary-Lake County, IL-IN-WI CMSA

 Cleveland-Akron, OH CMSA

 Detroit-Ann Arbor, MI CMSA

 Minneapolis-St. Paul, MN-WI MSA

 St. Louis, MO-IL MSA

West:

 Dallas-Fort Worth, TX CMSA

 Denver-Boulder, CO CMSA

 Houston-Galveston-Brazoria, TX CMSA

 Los Angeles-Anaheim-Riverside, CA CMSA

 Phoenix, AZ MSA

 San Diego, CA MSA

 San Francisco-Oakland-San Jose, CA CMSA

 Seattle-Tacoma, WA CMSA

The remaining smaller PSUs were not guaranteed to be selected for the sample. These were

grouped into a number of noncertainty strata (so called because the PSUs in these strata were not

included in the sample with certainty), and one PSU was selected from each stratum. The PSUs were

classified into four regions, each containing about one-fourth of the U.S. population. These regions were

defined primarily by state (Table 3-2). In each region, noncertainty PSUs were classified as MSA

(metropolitan) or nonMSA (nonmetropolitan) according to 1990 definitions. The resulting major strata

are shown in Table 3-3.

Table 3-3

The Sampling Major Strata

and the Number of Noncertainty Strata in Each

Region

Number of Strata

for MSA PSUs

Number of Strata

for NonMSA PSUs

Total

Strata

Northeast 6 4 10

Southeast 12 12 24

Central 8 12 20

West 10 8 18

Total 36 36 72
59

 Within each major stratum, further stratification was achieved by ordering the noncertainty

PSUs according to several additional socioeconomic characteristics, yielding 72 strata. The number of

such strata formed within each major stratum is shown in Table 3-3. The strata were defined so that the

aggregate of the measures of size of the PSUs in a stratum was approximately equal for each stratum. The

size measure used was the population from the 1990 Census. The characteristics used to define strata

were the percent minority population, the percentage change in total population since 1980, the per capita

income, the percent of persons age 25 or over with college degrees, the percent of persons age 25 or over

who completed high school, and the civilian unemployment rate. Up to four of these characteristics were

used in any one major stratum. For each major stratum, the characteristics used were chosen by modeling

PSU-level mean reading proficiency scores for 1988, 1990, and 1992. The characteristics chosen were

the best predictors of PSU-level mean reading proficiency scores in these models. One PSU was selected

with probability proportional to size from each of the 72 noncertainty strata. That is, within each stratum,

a PSU’s probability of being selected was proportional to its population. The PSUs were selected with

probability proportional to size (PPS) with the twin aims of obtaining approximately self-weighting

samples of students, and having approximately equal workloads in each PSU.

Samples of 94 PSUs each were drawn for the 1994, 1996, 1998, 2000, and 2002 main samples

simultaneously. They were drawn to minimize overlap of the PSUs from one assessment to the next,

except that certainty PSUs were retained in each assessment year, and some of the larger noncertainty

PSUs are in the sample for more than one of these assessment years. Each main sample of 94 PSUs was

drawn from a population of about 1,000 PSUs. Primarily because of the use of MSAs as PSUs, PSUs

varied considerably as to their probability of selection, since they varied greatly in size. In 1996, the 36

selected noncertainty MSA PSUs had probabilities of selection ranging from 0.04 to 0.58, while the 36

selected nonMSA PSUs had probabilities ranging from 0.03 to 0.08. Parts of 43 states were included in

the main sample PSUs.

For the long-term trend samples, 52 PSUs were selected. The long-term trend samples were much

smaller than the main samples and used separate field staff. Fewer PSUs were used for the long-term

trend samples to avoid having the sample spread too thinly across PSUs. The long-term trend PSUs were

drawn for 1994, 1996, 1998, 2000, and 2002 to minimize overlap of the trend PSUs from one assessment

to the next, and to minimize overlap between trend and main samples within the same assessment.

The 10 largest main sample certainty PSUs were also included with certainty in the long-term

trend samples. Six additional PSUs were selected systematically and with probability proportional to the

1990 population from the 12 remaining main sample certainties. Finally, 36 PSUs were selected from the

72 noncertainty strata so that the overall procedure was equivalent to systematic sampling with

probabilities proportional to the 1990 population. The 72 noncertainty strata from the main sample

design were paired, and one PSU per pair was selected for the trend samples. Note that the noncertainty

long-term trend PSUs are not a subsample of the noncertainty main sample PSUs, in order to minimize

the burden on a given school district in any one year.

3.3 SELECTION OF SCHOOLS

In the second stage of sampling, the public schools (including Bureau of Indian Affairs (BIA)

schools and Department of Defense Education Activity (DoDEA) schools) and nonpublic schools

(including Catholic schools) were listed according to the grade ranges associated with the three age

classes. The lists of schools were obtained from two sources. Regular public, BIA, and DoDEA schools

were obtained from the 1994 list of schools maintained by Quality Education Data, Inc. (QED). Regular

public schools are schools with students who are classified as being in a specific grade (as opposed to
60

schools having only “ungraded” classrooms). This includes statewide magnet schools and charter

schools. Catholic and other nonpublic schools were obtained from both QED and the Private School

Survey (PSS) developed for the National Center for Education Statistics’ 1993-1994 School and Staffing

Survey. The majority of the PSS list comes from complete enumeration of schools, but a small portion of

the PSS list was restricted to a sample of counties selected for the survey. Certain PSS counties,

generally large in population, were also included, independently by chance, in the NAEP sample PSUs.

The schools from such counties were added to the NAEP sampling frame after steps were taken to

eliminate duplicates with the QED list of nonpublic schools. In previous years, nonpublic schools were

also obtained from telephone directories. This process was not repeated in 1996 because the use of the

PSS files supplanted the need for this supplement.

Table 3-4 shows the numbers of schools included in the various sampling frame components.

The population of eligible schools for each age class was restricted to the selected PSUs. Main sample

schools were selected from the 94 main sample PSUs and long-term trend schools were selected from the

52 long-term trend PSUs. Note that there are relatively large numbers of nonpublic schools that are listed

in the QED or PSS only. The discrepancy between the schools contained in the PSS dataset versus those

in the QED dataset is primarily due to two factors: (1) the relative outdatedness of the two school lists,

and (2) PSS’s inclusion of a special area supplement designed to find schools not normally available on

lists.

Table 3-4

Grade Definition of School Eligibility for Sampling Frame Inclusion

and Frame Sizes, Main and Long-Term Trend Samples1

Sample2

Sampling Frame

Included Schools

With Any

of Grades Public3

Nonpublic

from QED

Only4

Nonpublic

from PSS

Only4

Nonpublic

from QED

and PSS4

Main Samples

Grade 4 4 18,046 1,308 1,716 7,300

Grade 8 8 6,093 1,029 1,322 6,409

Grade 12 12 4,357 578 869 2,474

Long-Term Trend Samples

Age Class 9 2 - 5 15,873 1,147 1,600 6,019

Age Class 13 6 - 9 13,667 1,089 1,474 6,592

Age Class 17 9 - 12 3,495 491 829 2,106

1 The numbers in this table reflect the full samples, including all sample types (see Section 3.4).

2 “

Age Class” is a term that refers to either an age or a grade definition of the samples. For the 1996 main assessments, unlike

for previous main assessments, only grade samples were drawn. Long-term trend age class definitions vary by subject area.

They are explained more fully in Chapters 14 through 17.

3 Public, BIA, and DoDEA schools

4 Catholic and other nonpublic schools

Any school having one or more of the eligible grades, and located within an appropriate PSU,

was included in the sampling frame of schools (the list of schools from which the samples of schools

were drawn) for a given sample. For each age class in the long-term trend samples, only a fraction of one

percent of age-eligible students was enrolled in ineligible schools. An independent sample of schools was

selected for each of the age classes, separately for main and long-term trend. Thus, some schools were

selected for assessment of two age classes, and a few were selected for all three. For all three age classes,

a sample of schools was first drawn for the long-term trend assessments. The schools selected for
61

long-term trend at a particular age class were excluded from the sampling frame when the samples of

schools for the corresponding grade were drawn for the main assessments. In addition, the schools

selected for the 1996 NAEP State Assessment program at grade 8 were excluded from the sampling

frame for the main samples at that grade. In regard to both of these situations, adjustments were made to

the sampling weights to reflect the appropriate probabilities of selection to yield unbiased estimates for

both long-term trend and main samples.

For each NAEP sample, schools were selected (without replacement) across all PSUs with

probabilities proportional to assigned measures of size. In those certainty PSUs included in both main

and trend samples, the probability of selection for long-term trend for any school in a given age class was

capped at 0.5, to ensure that adequate schools remained to be selected for the main sample. For long-term

trend samples, the measure of size used for each school was the estimated number of age eligible students

in the school, since for each age class the large majority of students selected were assigned to sessions

for which only students of the appropriate age were eligible. In most schools having the modal grade,

some additional students were selected who were in the modal grade but not age-eligible, so that the

maximum sample size of students within a school was about 80 grade- and age-eligible students. Equal

measures of size were assigned to schools containing estimates of age-eligible students ranging from 20

to 60 for each age class. Schools with more than 60 age-eligible students were selected with probabilities

proportional to the measure of size. Schools with fewer than 20 estimated age-eligible students were

assigned somewhat lower measures of size, and thus lower probabilities of selection, since assessment in

these schools involved substantially higher per-student administrative costs.

For the main samples, equal measures of size were assigned to schools containing estimates of

grade-eligible students ranging from 20 to 120 (for grade 4), 20 to 150 (for grade 8), or 20 to 180

(for grade 12). Schools larger than the indicated maximum size were selected with probabilities

proportional to the measure of size. This procedure was used so as to obtain approximately selfweighting samples of students (i.e., students selected with approximately equal overall probabilities) at

each grade. Three variations to the overall goal of self-weighting samples were implemented. Schools

with fewer than 20 estimated grade-eligible students were assigned somewhat lower measures of size,

and thus lower probabilities of selection. This was designed to increase cost efficiency. Each public

school designated as high-minority (with over 10 percent Black and/or Hispanic enrollment for grades 4

and 8, or over 15 percent Black and/or Hispanic enrollment for grade 12) was given double the

probability of selection of a public school, not designated high-minority, of similar size in the same PSU.

Such high-minority schools were oversampled in order to enlarge the sample of Black and Hispanic

students, thereby enhancing the reliability of estimates for these groups. For a given overall size of

sample, this procedure reduces somewhat the reliability of estimates for all students as a whole and for

those not Black or Hispanic. Each nonpublic school was given triple the probability of selection of a

public school not designated high-minority of similar size in the same PSU. These greater probabilities of

selection were used to ensure adequate samples of nonpublic-school students in order to allow the

derivation of reliable estimates for such students. No subgroups (high minority schools or nonpublic

schools) were oversampled in the long-term trend samples.

The total number of schools selected for each age class in both the long-term trend and main

samples was such that the predesignated student sample sizes would be achieved by selecting all eligible

students in a selected school, up to the maximum sizes indicated above. The target sample size also

allowed for losses due to nonparticipation of selected schools and students and the exclusion of students

from the assessment. This design, with the important exceptions described above, had the goal of

yielding a sample of students in a given age class or grade with approximately uniform probabilities of

selection. The efforts to oversample nonpublic-school students and minority students in the main samples

and the practical constraints on the sample size within each school resulted in some substantial violations
62

of this general goal. The distributions of selection probabilities of the selected students, as reflected in

their sampling weights, are mentioned in Chapter 10.

The QED files do not contain schools that opened between 1994 and the time of the assessments.

Therefore, special procedures were implemented to be sure that the NAEP assessment represented

students in new public schools. Small school districts, which generally contained only one eligible school

for a given age class, were handled differently from large school districts, which generally contained

more than one eligible school for a given age class. In small school districts, the schools selected for a

given age class were thought to contain all students in the district that were eligible for the assessment.

Districts containing such schools in the school sample were asked if other schools with the appropriate

grades for the assessment existed, and if so, they were automatically included in the assessment. For

large school districts, a district-level sampling frame was constructed from the schools on the QED file

that were eligible for one of the national assessments. Then districts were sampled systematically with

probabilities proportional to a measure of size. In most cases, the measure of size was total district

enrollment, but a minimum measure of size was used in districts below a certain cutoff. Each sampled

district was asked to update lists of eligible schools according to information on the QED files. Sampling

frames of eligible new schools for these large districts were then constructed separately for both main

and long-term trend samples at each age class, and separate samples of new schools were selected

systematically with probability proportional to eligible enrollment using the same sampling rates as for

the original sample. Seven new schools were added to the main samples: two at grade 4, three at grade 8,

and two at grade 12. Four new schools were added to the long-term trend samples: one at age class 9,

three at age class 13, and none at age class 17. Although new school sampling procedures were applied at

age class 17, no new schools were selected since schools with the necessary characteristics were not

available. All new schools added to the sample were obtained from large districts.

In a few PSUs where school refusals were relatively heavy for a particular sample, substitute

school selections were made, replacing the refusals (to the extent feasible) with schools from within the

same PSU and similar in size, affiliation (public, Catholic, or other nonpublic), grade span, and minority

composition. The goal of this procedure was to maintain the student sample sizes needed, while keeping

variance and nonresponse bias at acceptable levels. For the main samples, 31 substitute schools were

selected using this procedure (10 at grades 4 and 8, and 11 at grade 12), and 28 substitute schools were

selected for the long-term trend samples (15 at age class 9, 7 at age class 13, and 6 at age class 17).

Tables 3-5 and 3-6 show the number of in-scope schools selected, cooperating, and substituted, in the

main and long-term trend samples, respectively. The participation rates given are based on the original

sample of schools (excluding substitutes). School participation rates for grade 8 and nonpublic schools in

the main samples appear lower compared to those achieved in 1994, while the rates for public schools

appear higher. The other response rates are comparable for the two years. Note that since the response

rates quoted do not include the substitute selections, the potential for nonresponse bias is likely to be a

little less than these rates would indicate. This is because the substitute selections were chosen based on

their similarity to the initially refusing selections.

For the main samples, the schools that were participating with no eligible students left for testing

had all of their eligible students tested for State NAEP, so that no students were left for the main

samples. These schools were accounted for by treating them as nonrespondents for weighting. For longterm trend at age class 13, the considerable numbers of schools selected with no eligible students

enrolled resulted primarily from the fact that some schools with grades 6, 7, or 9, but no grade 8, were

sampled. Such schools had a reasonable chance of containing some age 13 students. Often they did have

a number of eligible students, but sometimes they had none. Because of the grade structure of schools,

this occurred most often for age class 13.
63

Table 3-5

School Sample Sizes, Refusals, and Substitutes for the Main Samples1

Status Grade 4 Grade 8 Grade 12 Total Public2 Nonpublic3

Selected, in scope 723 761 779 2,263 1,392 871

Refusals 99 127 160 386 212 174

Participation rate of

originally selected schools 86% 83% 79% 83% 85% 80%

1994 participation rate 86% 86% 79% 83% 82% 85%

Participating, no eligible

students enrolled 0 0 1 1 0 1

Participating, no eligible

students left for testing4 20 42 27 89 30 59

Substitutes participating 1 1 2 4 4 0

Final assessed sample 605 593 593 1,791 1,154 637

1 The numbers in this table reflect the full samples, consisting of schools in the S1, S2, and S3 samples. These sample types are

described in Section 3.4.

2 Public, BIA, and DoDEA schools

3 Catholic and other nonpublic schools

4 No students were left for testing because they had been tested in the State Assessment

Table 3-6

School Sample Sizes, Refusals, and Substitutes for the Long-Term Trend Samples1

Status Age Class 9 Age Class 13 Age Class 17 Total

Selected, in scope 291 316 237 844

Refusals 43 51 44 138

Participation rate of originally selected

schools 85% 84% 81% 84%

1994 participation rate 87% 82% 81% 83%

Participating, no eligible students enrolled 8 27 2 37

Substitutes participating 8 4 0 12

Final assessed sample 248 242 191 681

1 The numbers in this table reflect the full samples, consisting of schools in the S1, S2, and S3 samples. These sample types are

described in Section 3.4.

3.4 ASSIGNMENT OF SAMPLE TYPE TO SCHOOLS

In order to determine the effect of using different criteria for including students with disabilities

and limited English proficient students in the assessment, three different sample types were assigned to

the schools selected for the main assessment. In sample type 1 (S1) schools, the inclusion criteria for the

main samples were identical to those used in 1990 and 1992 and were intended to be somewhat more

rigorously defined than those used in the long-term trend samples. In sample type 2 (S2) schools, new

1996 inclusion criteria were used. In sample type 3 (S3) schools, the new 1996 inclusion criteria were

used and accommodations were offered to students with disabilities (SD) and students of limited English

proficiency (LEP). For more details of the inclusion criteria and their implementation, and the

accommodations offered students, see Chapter 5. The information in this chapter and in Chapter 5

applies to all three sample types or subsamples.
64

Sample type was assigned to schools separately for each grade. For schools that were not also

selected for the State Assessment program, sample type was assigned as follows. At grade 4, 20 percent

of the schools were assigned sample type 1, 45 percent were assigned sample type 2, and 35 percent were

assigned sample type 3. At grade 8, one-sixth of the schools were assigned sample type 1, and fivetwelfths each were assigned sample types 2 and 3. At grade 12, two-thirteenths of the schools were

assigned sample type 1, six-thirteenths were assigned sample type 2, and five-thirteenths were assigned

sample type 3. Sample type was assigned so that a variety of schools with respect to region, school type,

urbanization, and size were in each sample type at each grade.

For schools selected for both the main samples and State Assessment program, sample type was

initially assigned as described above, and then reassigned for the main samples as follows. Schools

retained their initial sample type assignment for the State Assessment. For the national assessment,

schools were ultimately assigned the same sample type as for the State Assessment, with one exception.

Schools that were initially assigned to sample type 3 for the national assessment and sample type 2 for

the State Assessment, retained these different respective sample types for each assessment. For all other

schools, the sample type for the main samples was switched to match the state sample type. The effect of

this procedure was to assign sample type 1 to somewhat more schools and sample types 2 and 3 to

somewhat fewer schools at grades 4 and 8 than initially assigned.

3.5 ASSIGNMENT OF SESSIONS TO SCHOOLS

Sessions were assigned to the selected schools found to be in-scope at the time of session

assignment in the following manner. First, the number of sessions per school was established (three 

sessions per school were specified for the long-term trend samples and for sample type 1 for the main

samples. Five sessions per school were specified for sample types 2 and 3 for the main samples at grades

4 and 8, and six sessions per school were specified for sample types 2 and 3 for the main samples at

grade 12). This was the maximum number of sessions that could be administered without creating unduly

small session sizes with few eligible students. Thus, in most long-term trend schools, for example, three

sessions were conducted. However, schools with fewer than 20 eligible students were asked to conduct

only a single session.

Session types associated with each sample are listed in Table 3-7. In the main samples, four to

six different session types were conducted at each grade (mathematics, science, mathematics estimation,

and mathematics theme at all grades; advanced mathematics at grades 8 and 12; and advanced science at

grade 12). All of the session types were not offered for all samples. For long-term trend, four session

types were conducted at age classes 9 and 13 (spiral plus three tape sessions), and three session types

were conducted at age class 17 (spiral plus two tape sessions). Schools could be assigned multiple

sessions of the same type (for example, two spiral and one tape sessions in long-term trend, or three

mathematics and three science sessions in the main samples).

Sessions were assigned to schools with three aims in mind. The first was to distribute students to

the different session types across the whole sample for each age class so that the target numbers of

assessed students would be achieved (in each sample type separately in the main assessments). The

second was to maximize the number of different session types that were administered within a given

selected school, without violating the minimum session sizes discussed above. The third was to give each

student an equal chance of being selected for a given session type regardless of the number of sessions

conducted in the school.
65

3.6 SAMPLING STUDENTS

To facilitate the sampling of students, a consolidated list was prepared for each school of all

grade-eligible and age-eligible students (for long-term trend) or all grade-eligible students (for the main

assessments) for the age class for which the school was selected. A systematic selection of eligible

students was made from this list (unless all students were to be assessed) to provide the target sample

size. For schools assigned more than a single session type (the vast majority), students were assigned by

Westat district supervisors to one of the various session types using specified procedures.

For each age class, separately for the long-term trend and main samples, maxima were

established as to the number of students who would be selected for a given school. In those schools that,

according to information on the sampling frame, had fewer eligible students than the established maxima,

each eligible student enrolled at the school was selected in the sample for one of the sessions assigned to

the school. In other schools, a sample of students was drawn, and then students were assigned to sessions

as appropriate. For the main samples, the maximum sample sizes were established by sample type in

terms of the number of grade-eligible students: 72 at grades 4 and 8, and 90 at grade 12 for sample type

1; 120 at grade 4, 160 at grade 8, and 180 at grade 12 for sample types 2 and 3. For the long-term trend

samples, the maximum at each age class was 60 age-eligible students (about 80 grade- plus age-eligible

students in most schools). Note that the number of students actually selected for assessment in a longterm trend sample school generally fell somewhat below 80, because students who were selected for one

of the long-term trend tape-administered sessions and were in the modal grade but not age-eligible were

subsequently dropped from the sample. Similarly, in the main assessments, at grades 8 and 12 in sample

types 2 and 3, students selected for the advanced mathematics and advanced science assessments, who

were subsequently found to be ineligible on the basis of their courses taken, were dropped from the

samples. This reduced the sample size somewhat in these schools.

The sample of students to be selected in each school was derived in the following manner, both

for main and for long-term trend samples. On the basis of data obtained from the school characteristics

and policies questionnaire (or the sample frame when the questionnaire data were not obtained in time)

an estimate of the number of eligible students was established for each school. For the main samples, the

estimated number of grade-eligible students was used; for the long-term trend samples, the number of

age-eligible students was used. A Session Assignment Form was generated for each school, showing the

line numbers (described below) of the students to be selected, indicating the type of session to be taken

by each such student. These line numbers were generated using a sampling interval designed to give the

appropriate sample size for each school. Thus, the overall sampling interval was 1.0 for schools in which

all eligible students were to be assessed. The appropriate sampling interval was specified for schools

with larger numbers of eligible students, such as to give the appropriate maximum sample size (described

above for each age class) in the case that the school had an enrollment of eligible students exactly equal

to that predicted.

If the Westat supervisor found that, when applied to the numbered list of eligible students

assembled in the field for each school, the line numbers generated gave rise to a sample in excess of 120

percent of the appropriate maximum sample size limit specified above, he or she called Westat’s central

office. By use of a personal computer, new line numbers based on the actual number of eligible students

were generated and relayed to the supervisor. A similar revision to the line numbers was made in the case

of a school with a sampling interval in excess of 1.0, and eligible enrollment less than 80 percent of that

initially estimated. In this latter case, the sample size was increased to the appropriate level. This

procedure gave a suitable compromise between control over the sampling rate within each school and

operational autonomy and flexibility for Westat field supervisors. Note that in all cases, sampling
66

intervals were generated in Westat’s central office, and stored for use in sample weighting. Supervisors

were not required to derive or record within-school sampling rates.

Table 3-7 shows the number of students per school who were assessed for each assessment. Note

that, for the various print samples, the number of students assessed per item per school is quite low, even

though typically dozens of students were assessed in total in a particular school. Thus, the extent of

clustering of the sample is in general quite modest, because most sampled schools conducted a few

different assessments with a moderate number of students in each, and more importantly because the use

of BIB-spiraling in the print-administered sessions greatly alleviated the effects of clustering the samples

of students within schools, for item-level data.

Table 3-7

Number of Students Per School for Each Session Type1

Sample Session Type

Number

of

Assessed

Students

Number

of

Schools

Mean

Number of

Students Per

Assessment

Per School

Mean

Number of

Students Per

Item Per

School

Age Class 9

Long-Term

Trend

Print Booklets 51-56

Tape Booklet 91

Tape Booklet 92

Tape Booklet 93

5,019

1,852

1,721

1,840

215

127

116

125

23.3

14.6

14.8

14.7

 3.9 - 7.82

14.6

14.8

14.7

Grade 4

Main

Print Mathematics

Print Science

Tape Mathematics Estimation

Print Mathematics Theme

10,830

11,578

2,115

4,004

445

421

120

230

24.3

27.5

17.6

17.4

5.6

4.5 - 7.42

17.6

8.7

Age Class 13

Long-Term

Trend

Print Booklets 51-56

Tape Booklet 91

Tape Booklet 92

Tape Booklet 93

5,493

1,928

1,866

1,864

221

128

125

124

24.9

15.1

14.9

15.0

4.1 - 8.32

15.1

14.9

15.0

Grade 8

Main

Print Mathematics

Print Science

Tape Mathematics Estimation

Print Mathematics Theme

Print Advanced Mathematics

11,521

11,971

2,244

4,227

2,365

411

346

104

175

253

28.0

34.6

21.6

24.2

9.3

6.5

5.6 - 9.42

21.6

12.1

9.3

Age Class 17

Long-Term

Trend

Print Booklets 51-56

Tape Booklet 84

Tape Booklet 85

4,669

1,848

1,691

186

133

122

25.1

13.9

13.9

4.2 - 8.42

13.9

13.9

Grade 12

Main

Print Mathematics

Print Science

Tape Mathematics Estimation

Print Mathematics Theme

Print Advanced Mathematics

Print Advanced Science

10,660

11,481

1,889

3,860

2,965

2,431

430

401

96

196

207

222

24.8

28.6

19.7

19.7

14.3

11.0

5.7

4.6 - 7.72

19.7

9.8

14.3

11.0

1 The numbers in this table reflect the full samples, consisting of schools in the S1, S2, and S3 samples. These sample types are described

in Section 3.4.

2 This number varied because some item blocks appeared more than others in the set of booklets used for this sample.
67

3.7 OVERSAMPLING OF SD/LEP STUDENTS FOR MAIN SAMPLE

MATHEMATICS AND SCIENCE ASSESSMENTS

As noted earlier, in the main assessments for mathematics and science, the procedures for

assessing SD and LEP students varied by sample type. The inclusion procedure used in sample type 1

differed somewhat from that used in sample types 2 and 3. SD/LEP students in sample type 3 were

offered accommodations not available to other students or to SD/LEP students in other sample types.

As a measure to ensure an adequate sample size of SD/LEP students from both sample types 2

and 3, oversampling procedures were applied for SD/LEP students at all three grades, in sample types 2

and 3 for mathematics, and in sample type 3 for science. In this way, comparisons of the effect of

offering accommodations to students would have enhanced power to detect effects.

The procedure for carrying out the oversampling was somewhat different for grade 4 than for

grades 8 and 12. This was because of the presence of the advanced mathematics and advanced science

samples at grades 8 and 12 only, which offered an opportunity to oversample in a way not possible at

grade 4. The general intent of the oversampling was, within each school assigned sessions of regular

mathematics in sample types 2 and 3, and regular science in sample type 3, to select SD/LEP students for

these assessments at twice the rate at which non SD/LEP students were sampled (or to include all

SD/LEP students if there were not sufficient numbers to permit sampling at twice the rate). There was no

oversampling of schools as part of the procedure.

At grade 4, the procedure was as follows. In each school where oversampling was to occur, the

initial desired sample of students was drawn for each session assigned, from the full list of eligible

students. Among those students not selected for any sessions in this way, the SD/LEP students were

identified. A sample from among these was drawn, using a sampling rate that would achieve the double

sampling rate required overall. In most cases, this involved selecting all such SD/LEP students in the

school. If the school was a sample type 3 school assigned to assess both mathematics and science, the

extra SD/LEP students so selected were split among mathematics and science in the same proportion as

the initial student sample for the school. Thus, if the school was assigned two sessions of science and one

of mathematics, two-thirds of these extra SD/LEP students were assigned to science, and one-third to

mathematics.

The sampling of additional SD/LEP students was carried out using designated line numbers,

indicated on the session assignment form used to generate the samples of students in each school. In this

way, the necessary information as to the selection probability of each student was retained for use in

weighting. No reliance was placed on information generated in the field. Field supervisors had only to

follow the prespecified sampling instructions.

At grades 8 and 12, a different approach was taken. As a result of the pattern of assigning

sessions to schools, it was the case that in every school in which there were students remaining who were

not selected for assessment in the initial sampling phase (so that there were in fact SD/LEP students

available for oversampling), a session of either advanced mathematics, or advanced science (at grade 12),

was assigned. This was the result of the scheme for assigning sessions to schools efficiently; it was not a

condition imposed in order to facilitate oversampling. The SD/LEP students assigned as an oversample

for the regular mathematics and (in the case of sample type 3) science assessments were those SD/LEP

students who were initially selected for the advanced mathematics and science samples, but who were not

eligible for those assessments because they had not taken the appropriate set of courses. Thus, for grades

8 and 12, the oversampling of SD/LEP students took place only among that subpopulation that was not

eligible for the advanced assessments.
68

It was assumed that there would be relatively few SD/LEP students who would qualify for the

advanced sessions, since nationally about 20 percent of all students were so eligible. To the extent to

which there were SD/LEP students who qualified for the advanced mathematics and science assessments,

however, the oversampling procedure for regular mathematics and science was not biased, because this

was taken into account in the weighting of the regular mathematics and science assessments. This was

possible because, for the students in the regular mathematics and science samples, those who were

SD/LEP were identified, and those who qualified for the advanced assessments were also identified.

All such additional SD/LEP students were included unless this would have led to the sampling

rate of SD/LEP students within the school being more than twice the rate of other students. In such cases,

a random subsample of the extra SD/LEP students was selected. As for grade 4, all information needed in

the field to carry out the oversampling was contained on the preprogrammed Session Assignment Form,

so that the complex weighting process could be carried without the possibility of error being introduced

in sampling information obtained from the field. Also, as in grade 4, in sample type 3 schools that were

assigned both regular mathematics and regular science sessions, the extra SD/LEP students sampled were

assigned in the appropriate proportions.

Since the aim was to oversample by a factor of two where possible, the overall rate of

oversampling of SD/LEP students was instead less than two. That is because in smaller schools there

were no students remaining who had not already been assigned to a session. Again, the weighting

procedures ensured that the results were not biased as a result of the relative under representation of

SD/LEP students from smaller schools.

Table 3-8 shows the results of the oversampling efforts for each grade and sample type for

mathematics and science. The weighted results show the proportion of the sample that would have been

SD/LEP students had no oversampling been attempted. The focus should be on sample types 2 and 3 for

mathematics and sample type 3 for science, since this is where the oversampling of SD/LEP students

occurred. The extent to which the unweighted percentage of SD/LEP students exceeds the weighted

percentage is a measure of the effectiveness of the oversampling. As can be seen, the procedure was

effective in increasing the sample of SD/LEP students considerably at grades 8 and 12 for both subjects,

but was not very effective at grade 4 for either subject. To have increased the sample of SD/LEP students

further at grade 4 would have required the assessment of additional schools.

Table 3-8

Percentage of Sampled Students Who Were Specified as SD/LEP

in the 1996 Main Samples - Mathematics and Science

Subject/ Grade 4 Grade 8 Grade 12

Sample Type1 Unweighted Weighted Unweighted Weighted Unweighted Weighted

Mathematics/S1 12.4% 13.2% 9.5% 10.2% 6.4% 6.8%

Mathematics/S2 17.9% 15.7% 19.9% 11.5% 15.3% 8.1%

Mathematics/S3 17.0% 15.4% 18.9% 12.1% 15.7% 8.5%

Total 15.8% 14.7% 16.1% 11.3% 12.7% 7.8%

Science/S2 16.5% 15.8% 13.1% 11.9% 9.5% 9.2%

Science/S3 17.7% 16.7% 18.9% 10.9% 13.6% 7.4%

Total 16.9% 16.2% 15.2% 11.4% 10.9% 8.3%

1 The numbers in this table reflect the full samples, consisting of schools in the S1, S2, and S3 samples. These sample types are

described in Section 3.4.
69

3.8 EXCLUDED STUDENTS

School staff completed an SD/LEP questionnaire for each sampled student identified as IEP

(with an individualized education program) or LEP. Some of these students were deemed unassessable by

school authorities and were excluded from the assessment. For the long-term trend samples, a distinct

sample of excluded students was identified at each age class. For the main samples, a distinct sample of

excluded students was identified for each subject and sample type combination.

The inclusion criteria for the main samples differed somewhat from those used for the long-term

trend samples. In addition, the inclusion criteria for the main samples differed by sample type. In sample

type 1 schools, the inclusion criteria for the main samples were identical to those used in 1990 and 1992,

and were intended to be somewhat more rigorously defined than those used in the long-term trend

samples. In sample type 2 schools, new 1996 inclusion criteria were used. In sample type 3 schools, the

new 1996 inclusion criteria were used and accommodations were offered to SD/LEP students.

For the long-term trend samples, the inclusion criteria were the same as in past long-term trend

assessments, dating back to the early 1980s.

For all samples, students were selected for specific sessions, and the school was then asked to

identify those to be excluded. Thus, only age-eligible students were considered for inclusion in the longterm trend tape-administered sessions, whereas both age- and grade-eligible students were considered for

inclusion in the print-administered long-term trend samples. The samples of excluded students for the

long-term trend samples were weighted in such a way as to account for this procedure appropriately (see

Chapter 10).

Table 3-9

Student Exclusion Rates by Age Class and School Type and Sample Type, Weighted

Age Class 9 Age Class 13 Age Class 17

Subject/

Sample Type1 Public

NonPublic Total PublicNonPublic Total PublicNonPublic Total

Main Samples

Mathematics/S1 5.6% 1.0% 5.2% 4.5% 0.2% 4.1% 3.4% 0.1% 3.0%

Mathematics/S2 9.1% 1.3% 8.1% 4.7% 0.1% 4.3% 3.5% 0.1% 3.2%

Mathematics/S3 4.4% 0.0% 3.9% 3.4% 0.0% 3.1% 3.1% 0.2% 2.8%

Science/S2 9.2% 0.3% 8.2% 4.7% 0.2% 4.3% 4.3% 0.4% 3.9%

Science/S3 6.5% 0.1% 5.9% 3.7% 0.3% 3.4% 2.8% 0.2% 2.6%

Estimation/All 6.7% 0.0% 5.8% 5.1% 0.0% 4.7% 4.0% 0.0% 3.5%

Theme/All 7.6% 0.6% 6.8% 4.4% 0.0% 4.0% 3.6% 1.4% 3.4%

Advanced Mathematics/All — —— 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%

Advanced Science/All — — — — —— 0.0% 0.0% 0.0%

Long-Term Trend Samples

Reading/Writing Print 6.6% 0.3% 5.9% 5.9% 0.4% 5.4% 5.3% 0.2% 5.0%

Mathematics/Science Tape 6.3% 0.6% 5.6% 4.8% 0.2% 4.3% 3.5% 0.4% 3.3%

1 The numbers in this table reflect the full samples, consisting of schools in the S1, S2, and S3 samples. These sample types are described in

Section 3.4.
70

Table 3-9 shows the rates of exclusion for each age class, subject, sample type (for math and

science), and school type for the long-term trend and main samples. The most marked effects are the

much higher rates of exclusion in public schools than in nonpublic, and the higher rates of exclusion at

lower grades. The former phenomenon is no doubt a function of the greater prevalence of special

education and language minority programs in public schools. The higher exclusion rates at lower ages,

which occurred also in other years, result from the greater proportion of students at these grades who are

excluded for reasons of limited English proficiency. In certain areas of the United States, fourth-grade

public-school students whose native language is Spanish are taught predominantly in Spanish, and in

these schools a very high proportion of sampled students are excluded. Factors that may limit the

comparability of these rates to those in previous years are the different inclusion criteria, oversampling of

SD/LEP in some subjects, the different subjects assessed, and the inclusion of only grade eligible

students in the main samples in 1996.

3.9 STUDENT PARTICIPATION RATES

Table 3-10 summarizes the rates of participation of invited students. The set of invited students

consists of the selected students, after removing the excluded students and, in the case of long-term trend

samples, removing those students selected for tape-administered sessions who were not age-eligible. For

a given session, a makeup session was called for when, for various reasons, more than a predetermined

tolerable number of invited students failed to attend the originally scheduled session to which they were

invited. The participation rates given in the table express the number finally assessed as a percentage of

those initially invited in the participating schools. Participation rates are shown for the main and longterm trend samples and for public and nonpublic schools separately in the case of the main samples.

Overall participation rates are also shown for comparable samples from the 1994 NAEP assessment. The

table shows that student participation rates in 1996 are similar to those experienced in 1994. The rates

increased slightly at age class 9 for both samples, and remained fairly steady for the other samples. At all

age classes, the participation rate of nonpublic-school students exceeds that of public-school students,

with the difference, both relative and absolute, increasing with age class.

Table 3-10

Student Participation Rates by Age Class and School Type, Unweighted1

1996 Public 1996 Nonpublic 1996 Combined 1994

Samples

Number

Invited

Participation

Rate

Number

Invited

Participation

Rate

Number

Invited

Participation

Rate2

Participation

Rate2

Age Class 9

 Long-Term Trend

 Main

9,715

24,082

95.4%

95.1%

1,204

5,834

96.8%

96.6%

10,919

29,916

95.5%

95.4%

94.2%

93.2%

Age Class 13

 Long-Term Trend

 Main

10,980

28,351

91.6%

92.3%

1,152

6,368

95.0%

96.7%

12,132

34,719

91.9%

93.1%

92.2%

91.0%

Age Class 17

 Long-Term Trend

 Main

9,051

34,199

83.4%

77.3%

717

7,473

91.9%

91.5%

9,768

41,672

84.0%

79.9%

84.1%

81.1%

1 The numbers in this table reflect the full samples, including all sample types (see Section 3.4).

2 Somewhat different inclusion criteria were used for the main samples than for the long-term trend samples in each year, and for the main

samples in 1994 versus 1996. The total rates for the main samples are based on a relatively greater contribution from nonpublic-school students.

Nonpublic-school students constitute about 18% of the 1996 main samples, 16% of the 1994 main samples, and 11% of the 1994 and 1996

long-term trend samples.
71

3.10 OVERALL STUDENT PARTICIPATION RATES

The combined impact of school nonparticipation and student absenteeism from sessions within

participating schools is summarized in Table 3-11. The table shows the percentages of students assessed,

from among those who would have been assessed if all initially selected schools had participated, and if

all invited students had attended either an initial or make-up session. The results show that, consistent

with earlier rounds of NAEP, the overall level of participation decreases substantially with the increase

in age and grade of the students.

So far in this chapter, only unweighted participation rates by age class and school type have been

presented. However, analysis is typically performed separately by age class and session type, and NCES

standards regarding acceptable potentials for bias are expressed in terms of weighted participation rates.

Therefore, Tables 3-12 and 3-13 show weighted participation rates by age class and session type for the

main and long-term trend samples, respectively. The main sample rates are for students in the reporting

populations. Note that for the main samples, the student participation rates are similar for different

session types at grades 4 and 8, but the student participation rates at grade 12 and the school participation

rates at all grades vary by session type. The differential school participation rates reflect the fact that,

more so than in previous years, different session types include different schools. This is due to the

assignment of schools to sample type, the fact that all session types were not assessed in all sample types,

and the specific sample types included in the reporting populations for each session type (see Chapter

10). For long-term trend, the participation rates are similar for different session types in the same age

class. They are also similar, in general, to the unweighted rates.

The procedures for substituting for nonparticipating schools or imputing for them through

weighting and the procedures for imputing for absent students through weighting were designed (so far

as feasible) to reduce the biases resulting from school and student nonparticipation. These procedures are

discussed in Chapter 10.

Table 3-11

Overall Participation Rates (School and Student Combined) by Age Class, Unweighted1

1996 Samples Age Class 9 Age Class 13 Age Class 17 Overall

Main Samples

 School participation

 Student participation

 Overall student participation

 Number of participating students

86.3%

95.4%

82.3%

28,527

83.3%

91.5%

76.2%

32,328

79.5%

79.9%

63.5%

33,286

82.9%

88.6%

73.4%

94,141

Long-Term Trend Samples

 School participation

 Student participation

 Overall student participation

 Number of participating students

85.2%

95.5%

81.4%

10,432

83.9%

91.9%

77.1%

11,151

81.4%

84.0%

68.4%

8,208

83.6%

90.8%

75.9%

29,791

Overall

 School participation

 Student participation

 Overall student participation

 Number of participating students

86.0%

95.4%

82.0%

38,959

83.5%

92.8%

77.5%

43,479

79.9%

80.7%

64.5%

41,494

83.1%

89.1%

74.0%

123,932

1 The numbers in this table reflect the full samples, including all sample types (see Section 3.4).
72

Table 3-12

Weighted Participation Rates by Age Class and Session Type, 1996 Main NAEP Reporting Samples

Participation

(Sample Type)

Mathematics

Print

Science

Print

Mathematics

Estimation

Print

Mathematics

Theme Print

Advanced

Mathematics

Print

Advanced

Science

Print

Grade 4

 School participation

 Student participation

 Overall participation

82.3%

95.3%

78.4%

77.8%

94.9%

73.8%

93.5%

96.7%

90.4%

77.9%

95.4%

74.4%

—

—

—

—

—

—

Grade 8

 School participation

 Student participation

 Overall participation

81.5%

92.9%

75.7%

79.7%

93.1%

74.3%

85.3%

93.8%

80.0%

86.8%

92.7%

80.4%

77.0%

95.6%

73.6%

—

—

—

Grade 12

 School participation

 Student participation

 Overall participation

76.2%

82.3%

62.7%

77.4%

77.5%

60.0%

63.9%

81.0%

51.7%

78.4%

78.2%

61.3%

77.6%

85.8%

66.6%

77.7%

86.5%

67.2%

Table 3-13

Weighted Participation Rates by Age Class and Session Type

1996 Long-Term Trend Samples

Participation

Reading/Writing

Print

Mathematics/Science

Tape

Age Class 9

 School participation

 Student participation

 Overall participation

83.5%

95.6%

79.9%

82.6%

95.4%

78.8%

Age Class 13

 School participation

 Student participation

 Overall participation

82.0%

92.2%

75.6%

80.8%

92.6%

74.8%

Age Class 17

 School participation

 Student participation

 Overall participation

81.7%

83.8%

68.5%

75.6%

84.1%

63.6%

3.11 SAMPLING TEACHERS

The teacher questionnaire was administered to teachers of fourth- and eighth-grade students

assessed in mathematics and science and twelfth-grade students assessed in advanced mathematics.

Teachers were given the questionnaire if they taught the student the subject in which the student was

assessed. The purpose of drawing these samples was not to estimate the attributes of the teacher
73

population, but to estimate the number (proportion) of students whose teachers had various attributes and

to correlate student characteristics and performance with the characteristics of their teachers.

The selected teachers were asked to complete a questionnaire concerning themselves and their

teaching practices, with specific references to each individual class period containing a student included

in the main assessment.

75

Chapter 4

ASSESSMENT INSTRUMENTS1

Stephen Lazer

Educational Testing Service

4.1 INTRODUCTION

In the 1996 assessment, four types of instruments were used to collect data about students,

teachers, and schools. Each assessed student received an assessment booklet containing both cognitive

and background questions. An SD/LEP Student Questionnaire was completed by school officials for

each sampled student identified as having a disability (SD) or classified as Limited English Proficient

(LEP), whether or not the students were able to participate in the assessment. The teachers of fourth-,

eighth-, and twelfth-grade students participating in the assessment were asked to complete a Teacher

Questionnaire. A School Characteristics and Policies Questionnaire was distributed to each

participating school.

This chapter begins with a discussion of the characteristics of the student booklets used for the

1996 main and long-term trend assessments and how the booklets were assembled. The contents of each

booklet and item block is presented in detail in a set of tables. Section 4.4 describes the student, teacher,

SD/LEP, and school questionnaires that were part of the 1996 assessment.

4.2 STUDENT BOOKLETS—MAIN ASSESSMENTS

4.2.1 Mathematics

Each student assessed in mathematics received a booklet containing a set of general background

questions, content questions, subject-specific background questions, and questions about his or her

motivation and familiarity with the assessment materials. The content questions were assembled into

sections or blocks. Students in the main assessment were given three 15-minute blocks. Those sampled

for the theme assessment completed one 15-minute block and one 30-minute block. Those sampled for

the advanced study at grade 8 completed three 20-minute blocks; at grade 12 advanced sample students

completed three 30-minute blocks. Students in the estimation sample completed one 15-minute block

from the main assessment and two paced-tape sections. The overall assessment time for each student was

approximately 63 minutes.

The assembly of blocks into booklets for the main assessment and their subsequent assignment to

sampled students was determined by a balanced incomplete block (BIB) design with spiraled

administration. The student booklets contained two five-minute background sections, a one-minute

background section, and three 15-minute blocks of items according to a BIB design.

The BIB design for the 1996 national mathematics assessment was focused by subject area, so

that students received booklets containing only blocks of mathematics questions (not science). The BIB

design also balances the order of presentation of the 15-minute blocks of items—every 15-minute block

 

1

 Stephen Lazer manages assessment development activities for the NAEP program at ETS.
76

appears as the first cognitive block in two booklets, as the second cognitive block in two other booklets,

and as the third cognitive block in another two booklets.

The design used in 1996 required that 13 blocks of mathematics items at each grade be assembled

into 26 booklets. Theme blocks were placed in two other booklets, and estimation blocks in one other

booklet. At grades 8 and 12, the advanced study was placed in one additional booklet.2 Once assembled,

the main assessment booklets were then spiraled and bundled. Spiraling involves interweaving the

booklets in a systematic sequence so that each booklet appears an appropriate number of times in the

sample. The bundles were designed so that each booklet would appear equally often in each position in a

bundle.

The final step in the BIB-spiraling procedure was the assigning of the booklets to the assessed

students. The students within an assessment session were assigned booklets in the order in which the

booklets were bundled. Thus, most students in an assessment session received different booklets. In the

assessment design, representative and randomly equivalent samples of students responded to each item at

a given grade level.

Tables 4-1, 4-2, and 4-3 provide the composition and number of booklets administered in the

1996 mathematics assessment. Table 4-4 gives details of the item blocks used in the main assessment,

including the number of cognitive items in each block and the booklets in which each block appeared;

Table 4-5 gives the same information for blocks in the special components of the assessment. Table 4-6

gives pertinent information about the background sections.

 

2

 See Chapter 2 for descriptions of these types of assessment blocks.
77

Table 4-1

Main Sample Booklet Configuration

Grade 4, Mathematics

Booklet Background Cognitive Motivation

Number Common Mathematics Blocks Background Block

 101 C13 M2 M3 M4 M7 MX

 102 C13 M2 M4 M5 M8 MX

 103 C13 M2 M5 M6 M9 MX

 104 C13 M2 M6 M7 M10 MX

 105 C13 M2 M7 M8 M11 MX

 106 C13 M2 M8 M9 M12 MX

 107 C13 M2 M9 M10 M13 MX

 108 C13 M2 M10 M11 M14 MX

 109 C13 M2 M11 M12 M15 MX

 110 C13 M2 M12 M13 M3 MX

 111 C13 M2 M13 M14 M4 MX

 112 C13 M2 M14 M15 M5 MX

 113 C13 M2 M15 M3 M6 MX

 114 C13 M2 M3 M5 M10 MX

 1151 C13 M2 M4 M6 M11 MX

 116 C13 M2 M5 M7 M12 MX

 117 C13 M2 M6 M8 M13 MX

 118 C13 M2 M7 M9 M14 MX

 119 C13 M2 M8 M10 M15 MX

 120 C13 M2 M9 M11 M3 MX

 1212 C13 M2 M10 M12 M4 MX

 122 C13 M2 M11 M13 M5 MX

 123 C13 M2 M12 M14 M6 MX

 124 C13 M2 M13 M15 M7 MX

 125 C13 M2 M14 M3 M8 MX

 126 C13 M2 M15 M4 M9 MX

 1273 C13 M2 M4 M16 M17 MX

 1284 C13 M2 M4 M21 — MX

 1294 C13 M2 M4 M22 — MX

 9215 C13 M2 M10 M12 M4 MX

1 This booklet was a large print version.

2 This booklet was also used for SD/LEP students who took a regular-print version.

3 This was an estimation booklet (involved paced audiotapes). 4 This was a theme booklet.

5 This was a bilingual booklet presented to some SD/LEP students. It contained the same blocks as Booklet Number 121.
78

Table 4-2

Main Sample Booklet Configuration

Grade 8, Mathematics

Booklet Background Cognitive Motivation

Number Common Mathematics Blocks Background Block

 101 C13 M2 M3 M4 M7 MX

 102 C13 M2 M4 M5 M8 MX

 103 C13 M2 M5 M6 M9 MX

 104 C13 M2 M6 M7 M10 MX

 105 C13 M2 M7 M8 M11 MX

 106 C13 M2 M8 M9 M12 MX

 107 C13 M2 M9 M10 M13 MX

 108 C13 M2 M10 M11 M14 MX

 109 C13 M2 M11 M12 M15 MX

 110 C13 M2 M12 M13 M3 MX

 111 C13 M2 M13 M14 M4 MX

 112 C13 M2 M14 M15 M5 MX

 113 C13 M2 M15 M3 M6 MX

 114 C13 M2 M3 M5 M10 MX

 1151 C13 M2 M4 M6 M11 MX

 116 C13 M2 M5 M7 M12 MX

 117 C13 M2 M6 M8 M13 MX

 118 C13 M2 M7 M9 M14 MX

 119 C13 M2 M8 M10 M15 MX

 120 C13 M2 M9 M11 M3 MX

 1212 C13 M2 M10 M12 M4 MX

 122 C13 M2 M11 M13 M5 MX

 123 C13 M2 M12 M14 M6 MX

 124 C13 M2 M13 M15 M7 MX

 125 C13 M2 M14 M3 M8 MX

 126 C13 M2 M15 M4 M9 MX

1273 C13 M2 M4 M16 M17 MX

1284 C13 M2 M4 M21 — MX

1294 C13 M2 M4 M22 — MX

1305 C13 M2 M20 M18 M19 MX

9215/6 C13 M2 M10 M12 M4 MX

1 This booklet was a large print version.

2 This booklet was also used for SD/LEP students who took a regular-print version.

3 This was an estimation booklet (involved paced audiotapes). 4 This was a theme booklet.

5 This was an advanced booklet.

6 This was a bilingual booklet presented to some SD/LEP students. It contained the same blocks as Booklet Number 121.
79

Table 4-3

Main Sample Booklet Configuration

Grade 12, Mathematics

Booklet Background Cognitive Motivation

Number Common Mathematics Blocks Background Block

 101 C13 M2 M3 M4 M7 MX

 102 C13 M2 M4 M5 M8 MX

 103 C13 M2 M5 M6 M9 MX

 104 C13 M2 M6 M7 M10 MX

 105 C13 M2 M7 M8 M11 MX

 106 C13 M2 M8 M9 M12 MX

 107 C13 M2 M9 M10 M13 MX

 108 C13 M2 M10 M11 M14 MX

 109 C13 M2 M11 M12 M15 MX

 110 C13 M2 M12 M13 M3 MX

 111 C13 M2 M13 M14 M4 MX

 112 C13 M2 M14 M15 M5 MX

 113 C13 M2 M15 M3 M6 MX

 114 C13 M2 M3 M5 M10 MX

 1151 C13 M2 M4 M6 M11 MX

 116 C13 M2 M5 M7 M12 MX

 117 C13 M2 M6 M8 M13 MX

 118 C13 M2 M7 M9 M14 MX

 119 C13 M2 M8 M10 M15 MX

 120 C13 M2 M9 M11 M3 MX

 1212 C13 M2 M10 M12 M4 MX

 122 C13 M2 M11 M13 M5 MX

 123 C13 M2 M12 M14 M6 MX

 124 C13 M2 M13 M15 M7 MX

 125 C13 M2 M14 M3 M8 MX

 126 C13 M2 M15 M4 M9 MX

1273 C13 M2 M4 M16 M17 MX

1284 C13 M2 M4 M21 — MX

1294 C13 M2 M4 M22 — MX

1305 C13 M2 M20 M18 M19 MX

1 This booklet was a large print version.

2 This booklet was used for SD/LEP students who took a regular-print version.

3 This was an estimation booklet (involved paced audiotapes). 4 This was a theme booklet.

5 This was an advanced booklet.
80

Table 4-4

1996 Mathematics Assessment, Main BIB

Block Designation Grade

MultipleChoice

Short

ConstructedResponse

Extended

ConstructedResponse Total Comments

3 S1M3 4 9 4 0 13 Trend (92)

S2M3 8 9 3 1 13 Trend (92)

S3M3 12 10 4 0 14 Trend (92)

4 S123M4a 4 14 0 0 14 Trend (90)

S123M4b 8 21 0 0 21 Trend (90)

S123M4c 12 22 0 0 22 Trend (90)

5 S12M5a 4 4 5 1 10 New

S12M5b 8 6 4 1 11 New

S3M5 12 4 5 1 10 New -- Calc

6 S123M6a 4 0 11 0 11 Trend (90)

S123M6b 8 0 16 0 16 Trend (90)

S123M6c 12 0 17 0 17 Trend (90)

7 S12M7a 4 3 4 1 8 New -- Manip

S12M7b 8 5 4 1 10 New -- Manip

S3M7 12 4 5 1 10 New -- Manip

8 S123M8a 4 14 1 0 15 Trend Calc (90)

S123M8b 8 16 2 0 18 Trend Calc (90)

S123M8c 12 17 4 0 21 Trend Calc (90)

9 S1M9 4 9 2 1 12 Trend (92)

S23M9b 8 5 3 1 9 Trend (92)

S23M9c 12 6 2 1 9 Trend (92)

10 S123M10a 4 0 6 0 6 Manipulatives (92)

S123M10b 8 0 7 0 7 Manipulatives (92)

S123M10c 12 3 6 1 10 Manipulatives (92)

11 S12M11a 4 11 5 0 16 Trend (92)

S12M11b 8 13 6 0 19 Trend (92)

S3M11 12 11 3 0 14 Trend (92)

12 S1M12 4 5 3 1 9 New Calculator

S23M12b 8 4 4 1 9 New Calculator

S23M12c 12 4 5 1 10 New Calculator

13 S1M13 4 6 5 1 12 Trend (92)

S2M13 8 6 4 1 11 Trend (92)

S3M13 12 3 5 1 9 Trend Prot (92)

14 S1M14 4 4 5 1 10 New Calculator

S23M14b 8 5 3 1 9 New Calculator

S23M14c 12 5 3 1 9 New Calculator

15 S1M15 4 3 6 1 10 New -- Ruler

S2M15 8 4 4 1 9 New -- Calculator/Protractor

S3M15 12 4 5 1 10 New -- Calculator
81

Table 4-5

1996 Mathematics Assessment, Estimation and Targeted Assessment

Block Designation Grade

MultipleChoice

Short

ConstructedResponse

Extended

ConstructedResponse Total Comments

16 S123M16a 4 20 0 0 20 Trend Estimation (90)

S123M16b 8 22 0 0 22 Trend Estimation (90)

S123M16c 12 22 0 0 22 Trend Estimation (90)

17 S1M17 4 10 3 0 13 New Estimation

S2M17 8 9 6 0 15 New Estimation

S3M17 12 16 0 0 16 New Estimation

18 S2M18 8 3 6 1 10 New Algebra

S3M18 12 3 6 2 11 New Advanced Mathematics

19 S2M19 8 6 5 1 12 New Algebra

S3M19 12 4 5 2 11 New Advanced Mathematics

 211 S1M21 4 1 4 3 8 New Theme

S23M21b 8 4 5 2 11 New Theme

S23M21c 12 4 5 2 11 New Theme

22 S1M22 4 0 3 3 6 New Theme

S2M22 8 4 4 2 10 New Theme

S2M23 12 4 2 1 7 New Theme

 1 Block 20 was a block composed of exercises from the main assessment used for linking.

Table 4-6

Background Sections of Student Mathematics Booklets

Number of Questions Placement in Student Booklet

Grade 4

General Background 24 Section 1

Mathematics Background 25 Section 2

Motivation 5 Section 61

Grade 8

General Background 26 Section 1

Mathematics Background 31 Section 2

Motivation 5 Section 61

Grade 12

General Background 35 Section 1

Mathematics Background 44 Section 2

Motivation 5 Section 61

1 Or Section 5 in theme booklets.
82

4.2.2 Science

Each student assessed in science received a booklet containing general background questions,

content questions, subject-specific background questions, and questions about his or her motivation and

familiarity with the assessment materials. The content questions were assembled into sections or blocks.

Students in the main assessment were given three 20-minute blocks at grade 4, and three 30-minute

blocks at grades 8 and 12. The last block in every book was a hands-on block. Those sampled for the

advanced study at grade 12 completed four 30-minute blocks. The overall assessment time for each

student was, on average, 120 minutes.

The assembly of blocks into booklets for the main assessment and their subsequent assignment to

sampled students was determined by a complex design with spiraled administration. The student booklets

contained two five-minute background sections, a one-minute background section, and three blocks of

items.

The design for the 1996 national assessment was focused by subject area, so that students

received booklets containing only blocks of science questions (not mathematics). The design also

balances the order of presentation of the blocks of items, except for the hands-on blocks, which always

appear in position three of a booklet. All other blocks appear an equal number of times in position one

and position two. Further, the design was set up to ensure that no student answered more than one themebased block (though some students did not receive any). This design allows for some balancing of the

impact of context and fatigue effects to be measured and reported, but makes allowance for the

difficulties and disruption of administering hands-on blocks. It also takes into account the limited breadth

of content coverage included in the theme blocks.

The design used in 1996 required that 15 blocks of science items at each grade be assembled into

37 booklets. At grade 12, the advanced study was composed of three additional booklets. Once

assembled, the main assessment booklets were then spiraled and bundled. Spiraling involves

interweaving the booklets in a systematic sequence so that each booklet appears an appropriate number

of times in the sample. The bundles were designed so that each booklet would appear equally often in

each position in a bundle.

The final step in the spiraling procedure was the assigning of the booklets to the assessed

students. The students within an assessment session were assigned booklets in the order in which the

booklets were bundled. Thus, most students in an assessment session received different booklets. In the

assessment design, representative and randomly equivalent samples of about 1,200 students responded to

each item at a given grade level.

Tables 4-7, 4-8, and 4-9 provide the composition and number of booklets administered in the

main 1996 science assessment. Table 4-10 provides the composition of booklets in the advanced science

study. Table 4-11 gives details of the item blocks used in the main assessment, including the number of

cognitive items in each block and the booklets in which each block appeared; Table 4-12 gives the same

information for blocks in the special components of the assessment. Table 4-13 gives pertinent

information about the background sections.
83

Table 4-7

Main Sample Booklet Configuration

Grade 4, Science

Booklet Cognitive Hands-On Background Motivation

Number Blocks Task1 Common Science Background Block

201 S7 S10 S3 C19 S2 SX

202 S7 S11 S4 C19 S2 SX

203 S7 S12 S5 C19 S2 SX

204 S7 S13 S6 C19 S2 SX

205 S10 S11 S3 C19 S2 SX

206 S12 S8 S4 C19 S2 SX

207 S10 S13 S5 C19 S2 SX

208 S10 S8 S6 C19 S2 SX

209 S11 S12 S3 C19 S2 SX

210 S13 S14 S4 C19 S2 SX

211 S11 S8 S5 C19 S2 SX

212 S11 S14 S6 C19 S2 SX

213 S13 S8 S3 C19 S2 SX

214 S8 S15 S4 C19 S2 SX

215 S12 S14 S5 C19 S2 SX

216 S12 S15 S6 C19 S2 SX

217 S8 S14 S3 C19 S2 SX

218 S14 S20 S4 C19 S2 SX

219 S8 S20 S5 C19 S2 SX

2202 S13 S20 S6 C19 S2 SX

221 S14 S15 S3 C19 S2 SX

222 S15 S21 S4 C19 S2 SX

223 S15 S9 S5 C19 S2 SX

224 S8 S21 S6 C19 S2 SX

225 S20 S21 S3 C19 S2 SX

226 S20 S9 S4 C19 S2 SX

227 S20 S7 S5 C19 S2 SX

228 S14 S9 S6 C19 S2 SX

229 S21 S9 S3 C19 S2 SX

230 S21 S7 S4 C19 S2 SX

231 S21 S10 S5 C19 S2 SX

232 S15 S7 S6 C19 S2 SX

233 S9 S13 S3 C19 S2 SX

234 S9 S10 S4 C19 S2 SX

235 S9 S11 S5 C19 S2 SX

236 S9 S12 S6 C19 S2 SX

237 S14 S7 S3 C19 S2 SX

1 Hands-on task blocks: Block S3 uses “A” kit - seeds; Block S4 uses “B” kit - unknown powders; Block S5 uses “C” kit -

floating pencil; and Block S6 uses “D” kit - markers. 2 This booklet was also used for SD/LEP students who took a regular-print version.
84

Table 4-8

Main Sample Booklet Configuration

Grade 8, Science

Booklet Cognitive Hands-On Background Motivation

Number Blocks Task1 Common Science Background Block

201 S7 S10 S3 C19 S2 SX

202 S7 S11 S4 C19 S2 SX

203 S7 S12 S5 C19 S2 SX

204 S7 S13 S6 C19 S2 SX

205 S10 S11 S3 C19 S2 SX

206 S12 S8 S4 C19 S2 SX

207 S10 S13 S5 C19 S2 SX

208 S10 S8 S6 C19 S2 SX

209 S11 S12 S3 C19 S2 SX

210 S13 S14 S4 C19 S2 SX

211 S11 S8 S5 C19 S2 SX

212 S11 S14 S6 C19 S2 SX

213 S13 S8 S3 C19 S2 SX

214 S8 S15 S4 C19 S2 SX

215 S12 S14 S5 C19 S2 SX

216 S12 S15 S6 C19 S2 SX

217 S8 S14 S3 C19 S2 SX

218 S14 S20 S4 C19 S2 SX

219 S8 S20 S5 C19 S2 SX

2202 S13 S20 S6 C19 S2 SX

221 S14 S15 S3 C19 S2 SX

222 S15 S21 S4 C19 S2 SX

223 S15 S9 S5 C19 S2 SX

224 S8 S21 S6 C19 S2 SX

225 S20 S21 S3 C19 S2 SX

226 S20 S9 S4 C19 S2 SX

227 S20 S7 S5 C19 S2 SX

228 S14 S9 S6 C19 S2 SX

229 S21 S9 S3 C19 S2 SX

230 S21 S7 S4 C19 S2 SX

231 S21 S10 S5 C19 S2 SX

232 S15 S7 S6 C19 S2 SX

233 S9 S13 S3 C19 S2 SX

234 S9 S10 S4 C19 S2 SX

235 S9 S11 S5 C19 S2 SX

236 S9 S12 S6 C19 S2 SX

237 S14 S7 S3 C19 S2 SX

1 Hands-On task blocks: Block S6 uses “D” kit - markers; Block S3 uses “E” kit - powders; Block S4 uses “F” kit - salt

solutions; and Block S5 uses “G” kit - soil tests. 2 This booklet was also used for SD/LEP students who took a regular-print version.
85

Table 4-9

Main Sample Booklet Configuration

Grade 12, Science

Booklet Cognitive Hands-On Background Motivation

Number Blocks Task1 Common Science Background Block

201 S7 S10 S3 C19 S2 SX

202 S7 S11 S4 C19 S2 SX

203 S7 S12 S5 C19 S2 SX

204 S7 S13 S6 C19 S2 SX

205 S10 S11 S3 C19 S2 SX

206 S12 S8 S4 C19 S2 SX

207 S10 S13 S5 C19 S2 SX

208 S10 S8 S6 C19 S2 SX

209 S11 S12 S3 C19 S2 SX

2102 S13 S14 S4 C19 S2 SX

211 S11 S8 S5 C19 S2 SX

212 S11 S14 S6 C19 S2 SX

213 S13 S8 S3 C19 S2 SX

214 S8 S15 S4 C19 S2 SX

215 S12 S14 S5 C19 S2 SX

216 S12 S15 S6 C19 S2 SX

217 S8 S14 S3 C19 S2 SX

218 S14 S20 S4 C19 S2 SX

219 S8 S20 S5 C19 S2 SX

220 S13 S20 S6 C19 S2 SX

221 S14 S15 S3 C19 S2 SX

222 S15 S21 S4 C19 S2 SX

223 S15 S9 S5 C19 S2 SX

224 S8 S21 S6 C19 S2 SX

225 S20 S21 S3 C19 S2 SX

226 S20 S9 S4 C19 S2 SX

227 S20 S7 S5 C19 S2 SX

228 S14 S9 S6 C19 S2 SX

229 S21 S9 S3 C19 S2 SX

230 S21 S7 S4 C19 S2 SX

231 S21 S10 S5 C19 S2 SX

232 S15 S7 S6 C19 S2 SX

233 S9 S13 S3 C19 S2 SX

234 S9 S10 S4 C19 S2 SX

235 S9 S11 S5 C19 S2 SX

236 S9 S12 S6 C19 S2 SX

237 S14 S7 S3 C19 S2 SX

1 Hands-On task blocks: Block S5 uses “G” kit - soil tests; Block S3 uses “H” kit - antacid; Block S4 uses “I” kit -

separation; and Block S6 uses “J” kit - pendulum. 2 This booklet was also used for SD/LEP students who took a regular-print version.
86

Table 4-10

Main Sample Booklet Configuration

Grade 12, Advanced Science

Booklet Cognitive Hands-On Background Motivation

Number Blocks Task Common Science Background Block

238 S19 S18 S17 S16 C19 S2 SX

239 S19 S17 S16 S18 C19 S2 SX

240 S19 S16 S18 S17 C19 S2 SX
87

Table 4-11

1996 Science Assessment, Main BIB

Block Designation Grade

MultipleChoice

Short

ConstructedResponse

Extended

ConstructedResponse Total Comments

3 S1S3 4 0 7 0 7 Hands-on

S2S3 8 0 4 2 6 Hands-on

S3S3 12 0 4 2 6 Hands-on

4 S1S4 4 1 0 7 7 Hands-on

S2S4 8 3 4 3 10 Hands-on

S3S4 12 0 1 3 4 Hands-on

5 S1S5 4 0 5 6 11 Hands-on

S23S5 8/12 0 6 0 6 Hands-on

6 S12S6A 4 0 0 4 4 Hands-on

S12S6B 8 0 5 2 7 Hands-on

S3S6 12 0 6 2 8 Hands-on

7 S1S7 4 0 10 0 10 Theme-based

S2S7 8 4 10 0 14 Theme-based

S3S7 12 5 7 3 15 Theme-based

8 S1S8 4 1 6 1 8 Theme-based

S23S8A 8 5 5 0 10 Theme-based

S23S8B 12 6 7 1 14 Theme-based

9 S12S9A 4 2 6 1 9 Theme-based

S12S9B 8 3 9 1 13 Theme-based

S3S9 12 4 8 2 14 Theme-based

10 S1S10 4 6 4 1 11 Concept/Problem-Solving

S2S10 8 8 7 1 16 Concept/Problem-Solving

S3S10 12 7 7 1 15 Concept/Problem-Solving

11 S1S11 4 6 5 0 11 Concept/Problem-Solving

S2S11 8 8 7 1 16 Concept/Problem-Solving

S3S11 12 7 5 3 15 Concept/Problem-Solving

12 S1S12 4 6 4 1 10 Concept/Problem-Solving

S23S12 8/12 8 6 2 16 Concept/Problem-Solving

13 S1S13 4 6 4 1 11 Concept/Problem-Solving

S23S13 8/12 8 7 1 16 Concept/Problem-Solving

14 S12S14A 4 5 5 0 10 Concept/Problem-Solving

S12S14B 8 7 11 0 18 Concept/Problem-Solving

S3S14 12 8 6 2 16 Concept/Problem-Solving

15 S12S15A 4 3 5 1 9 Concept/Problem-Solving

S12S15B 8 7 7 2 16 Concept/Problem-Solving

S3S15 12 0 5 2 7 In-depth

20 S1S20 4 6 2 3 11 Concept/Problem-Solving

S2S20 8 8 6 2 16 Concept/Problem-Solving

S3S20 12 7 6 3 16 Concept/Problem-Solving

21 S1S21 4 5 3 2 10 Concept/Problem-Solving

S2S21 8 7 7 2 16 Concept/Problem-Solving

S3S21 12 8 4 4 16 Concept/Problem-Solving
88

Table 4-12

1996 Science Assessment, Advanced Assessment

Block Designation Grade

MultipleChoice

Short

ConstructedResponse

Extended

ConstructedResponse Total Comments

16 S3S16 12 7 8 1 16 Advanced Block

17 S3S17 12 7 5 4 16 Advanced Block

18 S3S18 12 7 4 5 16 Advanced Block

19 S3S19 12 9 7 2 18 Advanced Block

Table 4-13

Background Sections of 1996 Student Science Booklets

Number of Questions Placement in Student Booklet

Grade 4

General Background 24 Section 4

Science Background 39 Section 5

Motivation 5 Section 6

Grade 8

General Background 26 Section 4

Science Background 42 Section 5

Motivation 5 Section 6

Grade 12

General Background 36 Section 4

Science Background 53 Section 5

Motivation 5 Section 6

4.3 STUDENT BOOKLETSLONG-TERM TREND ASSESSMENTS

There were several long-term trend samples in the 1994 assessment (see Chapter 1), each of

which required the use of special booklets. Tables 4-14, 4-15, and 4-16 summarize the contents of each

trend assessment booklet and show how many of each booklet were administered. Tables 4-20, 4-21, and

4-22 give details of the item blocks used in the long-term trend assessments, including the number of

cognitive and constructed-response items in each block and the booklets in which each block appeared.

Reading and Writing Long-Term Trend. Six booklets (numbered 51 to 56) containing reading and

writing items were administered to each age class. These booklets were identical to booklets used in

previous assessments of reading and writing and were spiraled for administration. Each booklet consisted

of a common background block (BZ) and three cognitive blocks (at least one reading block and at least

one writing block). In addition to cognitive items, the cognitive blocks also contained subject-related

background questions.

Mathematics and Science Long-Term Trend. Three booklets (91, 92, and 93) at ages 9 and 13 and

two booklets (84 and 85) at age 17, containing mathematics and science items, were identical to those

used in previous assessments to measure trends. Each booklet contained a common background block

(C1 or BZ) and three cognitive blocks. At ages 9 and 13, these booklets contained one reading block (R1,
89

R2, or R3), one mathematics block (M1, M2, or M3) and one science block (S1, S2, or S3). At age 17,

each booklet contained at least one mathematics block (M1 to M3) and at least one science block (S1 -

S3). Mathematics block M3 contained items that required the use of a calculator. All cognitive blocks

also contained subject-related background questions.

Table 4-14

Long-Term Trend Sample Booklet Contents and Number of Booklets Administered

Age Class 9

Subject

Area

Booklet

Number

Common

Background

Block

Subject

Area

Background

Block1 Cognitive Blocks

Number of

Booklets

Administered

Reading and

Writing

51

52

53

54

55

56

BZ

BZ

BZ

BZ

BZ

BZ

—

—

—

—

—

—

BC

BH

BC

BG

BM

BV

BL

BE

BK

BO

BG

BR

BQ

BR

BJ

BE

BN

1,186

1,165

1,178

1,180

1,169

1,184

Mathematics

and Science

91

92

93

C1

C1

C1

—

—

—

R1

S2

M2

M1

R2

S3

S1

M32

R3

2,388

2,512

2,435

1 Subject area background questions are included in cognitive blocks for this booklet.

2 Calculator needed for this block.
90

Table 4-15

Long-Term Trend Sample Booklet Contents and Number of Booklets Administered

Age Class 13

Subject

Area

Booklet

Number

Common

Background

Block

Subject

Area

Background

Block1 Cognitive Blocks

Number of

Booklets

Administered

Reading and

Writing

51

52

53

54

55

56

BZ

BZ

BZ

BZ

BZ

BZ

—

—

—

—

—

—

BM

BC

BH

BN

BG

BG

BK

BL

BE

BC

BO

BJ

BD

BQ

BR

BD

BE

BP

 919

 906

 923

 905

 928

 933

Mathematics

and Science

91

92

93

C1

C1

C1

—

—

—

R1

S2

M2

M1

R2

S3

S1

M32

R3

1,928

1,976

2,005

1

 Subject area background questions are included in cognitive blocks for this booklet.

2 Calculator needed for this block.

Table 4-16

Long-Term Trend Sample Booklet Contents and Number of Booklets Administered

Age Class 17

Subject

Area

Booklet

Number

Common

Background

Block

Subject

Area

Background

Block1 Cognitive Blocks

Number of

Booklets

Administered

Reading and

Writing

51

52

53

54

55

56

BZ

BZ

BZ

BZ

BZ

BZ

—

—

—

—

—

—

BM

BC

BH

BN

BG

BG

BK

BL

BE

BC

BO

BJ

BD

BQ

BR

BD

BE

BP

 927

 924

 917

 951

 939

 911

Mathematics

and Science

84

85

C1

C1

—

—

M1

S1

M2

S2

S3

M32

2,207

2,152

1 Subject area background questions are included in cognitive blocks for this booklet.

2 Calculator needed for this block.
91

Table 4-17

Long-Term Trend Sample Block Information, Age Class 9

Block Type

Total

Number

of Items

Number

of

Cognitive

Items

Number of

Open-Ended Items

Cognitive

Noncognitive

Booklets

Containing

Block

BZ Common Background 37 0 0 1 51 - 56

C1 Common Background 28 0 0 0 91 - 93

BC

BE

BG

BH

BJ

BK

BL

BM

BN

BO

BQ

BR

BV

Writing Background/Cognitive

Writing Background/Cognitive

Writing Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading and Writing

Background/Cognitive

23

11

 8

15

24

19

26

16

25

22

21

16

36

 1

 2

 2

11

13

11

 7

12

14

11

12

12

7 Rd.

1 Wr.

1

2

2

1

1

0

1

1

1

0

0

0

1 Rd.

1 Wr.

0

0

0

0

0

0

1

0

0

0

0

0

0

51, 53

52, 54

54, 55

52

53

53

51

55

55

54

51

52, 56

56

R1

R2

R3

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

20

20

17

 9

11

10

 0

 0

 1

0

0

0

91

92

93

M1

M2

M3

Mathematics

Background/Cognitive

Mathematics

Background/Cognitive

Mathematics

Background/Cognitive (Calc.)

26

26

19

26

26

16

 9

 9

10

0

0

0

91

93

92

S1

S2

S3

Science Background/Cognitive

Science Background/Cognitive

Science Background/Cognitive

23

25

31

18

25

20

 0

 0

 0

0

0

0

91

92

93
92

Table 4-18

Long-Term Trend Sample Block Information, Age Class 13

Block Type

Total

Number

of Items

Number of

Cognitive

Items

Number of

Open-Ended Items

Cognitive

Noncognitive

Booklets

Containing

Block

BZ Common Background 37 0 0 1 51 - 56

C1 Common Background 30 0 0 0 91 - 93

BC

BD

BE

BG

BH

BJ

BK

BL

BM

BN

BO

BP

BQ

BR

Writing Background/Cognitive

Writing Background/Cognitive

Writing Background/Cognitive

Writing Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

23

25

11

 8

18

24

17

27

16

23

21

15

23

19

 1

 1

 2

 2

13

14

 9

 6

12

12

10

 9

17

15

1

1

2

2

1

2

1

1

1

1

2

1

0

0

0

0

0

0

1

0

0

1

0

0

0

0

0

0

52, 54

51, 54

53, 55

55, 56

53

56

51

52

51

54

55

55

52

53

R1

R2

R3

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

31

19

28

12

10

13

 1

 0

 0

0

0

0

91

92

93

M1

M2

M3

Mathematics

Background/Cognitive

Mathematics

Background/Cognitive

Mathematics

Background/Cognitive (Calc.)

51

44

32

37

37

24

 9

 8

10

0

0

0

91

93

92

S1

S2

S3

Science Background/Cognitive

Science Background/Cognitive

Science Background/Cognitive

36

40

36

25

27

27

 0

 0

 0

0

0

0

91

92

93
93

Table 4-19

Long-Term Trend Sample Block Information, Age Class 17

Block Type

Total

Number

of Items

Number of

Cognitive

Items

Number of

Open-Ended Items

Cognitive

Noncognitive

Booklets

Containing

Block

BZ Common Background 48 0 0 1 51 - 56

C1 Common Background 48 0 0 0 84, 85

BC

BD

BE

BG

BH

BJ

BK

BL

BM

BN

BO

BP

BQ

BR

Writing Background/Cognitive

Writing Background/Cognitive

Writing Background/Cognitive

Writing Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

Reading Background/Cognitive

23

25

11

 8

19

17

17

32

16

32

24

25

17

20

 1

 1

 2

 2

13

 6

 9

 6

12

12

13

11

11

 9

1

1

2

2

1

2

1

1

1

1

1

1

1

0

0

0

0

0

2

1

0

2

0

1

0

0

0

0

52, 54

51, 54

53, 55

55, 56

53

56

51

52

51

54

55

56

52

53

M1

M2

M3

Mathematics Background/Cognitive

Mathematics Background/Cognitive

Mathematics Background/Cognitive

(Calculator)

49

49

35

35

35

24

10

 5

14

0

0

0

84

84

85

S1

S2

S3

Science Background/Cognitive

Science Background/Cognitive

Science Background/Cognitive

38

41

32

27

32

23

 0

 0

 0

0

0

0

85

85

84

4.4 STUDENT, TEACHER, AND SCHOOL QUESTIONNAIRES

4.4.1 Student Questionnaires

Each booklet in the main assessment included three student background questionnaires. The first,

consisting of general background questions, included questions about race/ethnicity, mothers and

fathers level of education, reading materials in the home, homework, attendance, academic expectations,

and which parents lived at home. The second, consisting of subject-area background questions, included

questions about instructional activities, courses taken, use of specialized resources such as calculators in

mathematics class, and views on the utility and value of the subject matter. Students were given five

minutes to complete each of these questionnaires, with the exception of the fourth graders, who were

given more time because the items in the general questionnaire were read aloud for them. The third
94

questionnaire followed the three cognitive blocks and contained five questions about students motivation

to do well on the assessment, their perceptions concerning the difficulty of the assessment, and their

familiarity with types of questions included.

The student questionnaires are described in detail in Chapter 2.

4.4.2 Teacher Questionnaires

To supplement the information on instruction reported by students, the mathematics teachers of

the students participating in the mathematics assessment were asked to complete a questionnaire about

their instructional practices, teaching backgrounds, and characteristics. The teacher questionnaire

contained two parts. The first part pertained to the teachers’ background and general training. The second

part pertained to specific training in teaching mathematics and the procedures the teacher uses for each

class containing an assessed student, as well as collecting information on teachers’ awareness and

knowledge of the NCTM Standards.

The Teacher Questionnaire, Part I: Background and General Training included questions

pertaining to gender, race/ethnicity, years of teaching experience, certification, degrees, major and minor

fields of study, course work in education, course work in specific subject areas, amount of in-service

training, extent of control over instructional issues, and availability of resources for their classroom.

The Teacher Questionnaire, Part II: Training in Mathematics and Classroom Instructional

Information included questions on the teacher’s exposure to various issues related to mathematics and

teaching mathematics through pre- and in-service training, ability level of students in the class, whether

students were assigned to the class by ability level, time on task, homework assignments, frequency of

instructional activities used in class, methods of assessing student progress in mathematics, instructional

emphasis given to the mathematics abilities covered in the assessment, and use of particular resources.

Because the sampling for the teacher questionnaires was based on participating students, the

responses to a particular teacher questionnaire do not necessarily represent all teachers of that subject

area at that grade level in the nation. Rather, they are teachers of the representative sample of students

assessed. It is important to note that in all NAEP reports, the student is always the unit of analysis, even

when information from the teacher or school questionnaire is being reported. Using the student as the

unit of analysis makes it possible to describe the instruction received by representative samples of

students. Although this approach may provide a different perspective from other studies simply reporting

information about teachers or schools, it is consistent with NAEP’s goal of providing information about

the educational context and performance of students.

The teacher questionnaires are described in detail in Chapter 2.

4.4.3 School Questionnaires

A School Characteristics and Policies Questionnaire was given to the principal or other

administrator of each school that participated in the assessment. This information provided an even

broader picture of the instructional context for students’ mathematics achievement. This questionnaire

included questions about background and characteristics of school principals, length of school day and

year, school enrollment, absenteeism, dropout rates, size and composition of teaching staff, policies

about grouping students, curriculum, testing practices and uses, special priorities and school-wide
95

programs, availability of resources, special services, community services, policies for parental

involvement, and school-wide problems.

School Characteristics and Policies questionnaires are described in detail in Chapter 2.

4.4.4 SD/LEP Student Questionnaires

The SD/LEP Student Questionnaire was completed by the teachers of those students who were

selected to participate in the assessment sample who were classified as Students with Disabilities (SD) or

were classified as Limited English Proficient (LEP). The questionnaire was completed for all SD or LEP

students, whether or not they actually participated in the assessment. This questionnaire asked about the

nature of the students disability and the special programs in which the student participated.

Schools were permitted to exclude certain students from the assessment. The same exclusion

criteria and rules used in the national assessment were also applied to the Trial State Assessment.

Although the intent was to assess all sampled students, students who were identified by school staff as

not capable of participating meaningfully were excluded. The NAEP guidelines for exclusion were

intended to assure uniformity of exclusion criteria from school to school as well as from state to state.

More information about the SD/LEP questionnaire and exclusion criteria are provided in

Chapters 2 and 5.

97

Chapter 5

FIELD OPERATIONS AND DATA COLLECTION

1

Lucy M. Gray, Mark M. Waksberg, and Nancy W. Caldwell

Westat, Inc.

5.1 INTRODUCTION

This chapter describes the field operations and data collection activities for the national

assessment component of the 1996 National Assessment of Educational Progress (NAEP). The national

assessment is comprised of main samples and long-term trend samples. Main NAEP samples typically

involve new assessment items, and may include new subject areas and innovative features; in long-term

trend, the procedures and items from previous years are carried forward so that trends in student

achievement can be measured over time. Both the main and long-term trend assessments are based on

probability samples of schools and students that allow for regional and national reporting only. The State

Assessment, the second major component of NAEP, comprises the state program that uses main

assessment materials and involves much larger sample sizes per state (or jurisdiction), so that results can

be reported for each participating state (for further technical information on the State Assessment, see the

Technical Report of the NAEP 1996 State Assessment Program in Mathematics, Allen, Jenkins, Kulick,

and Zelenak, 1997, and the Technical Report of the NAEP 1996 State Assessment Program in Science,

Allen, Swinton, Isham, and Zelenak, 1997).

The design of the national assessment component of NAEP is described in the remaining sections

of this chapter. For all components, NAEP guarantees the anonymity of participants, and student or

teacher names are never recorded on assessment booklets nor removed from the schools. NAEP results

are reported on the national level and by region of the country, not by school district, school, or

individual student. Only group statistics are reported, broken down by gender, race/ethnicity, and a host

of variables that illuminate teachers’ instructional practices.

5.1.1 Field Organization

The 1996 main assessment involved some new items and components including many innovative

features. For example, the science assessment differed from previous NAEP science assessments in that

every student performed an experiment. The mathematics assessment involved the use of mathematical

tools and a larger number of constructed-response items than in the past. Much of the mathematics

assessment has been used since 1990, thus providing “short-term trend” data. All students in a particular

assessment session received a booklet in the same subject (i.e., mathematics or science). Even though

many different booklets were used in a particular session, they were all for the same subject.

In most schools sampled for the main assessment, more than one session type was conducted. In

about one-third of the schools at each grade, only two session types, mathematics and mathematics

estimation, were possible. In the remaining two-thirds of schools, up to a maximum of four session types

 

1 Lucy M. Gray and Mark M. Waksberg assist in survey operations and field activities for the NAEP assessments, under the

direction of Nancy W. Caldwell.
98

for grade 4 or five session types at grades 8 or 12 could have been conducted. At grade 12, only one

advanced session—either mathematics or science—was conducted.

Historically, a small proportion (less than 10%) of the sampled students have been “excluded”

from NAEP assessment sessions because, according to school records, they are students with either

disabilities or limited English language proficiency who have been determined to be incapable of

participating meaningfully in the assessment. More recently, especially with the passage of the

Individuals with Disabilities Education Act, increased attention has been given to these students and to

including as many of them as possible in NAEP sessions (and in other testing situations as well). NAEP

has addressed these concerns, first in the 1995 field test and continuing with the 1996 operational

assessment, through a Special Study that uses both old and new “inclusion” criteria and (in some schools)

offers accommodations for testing students with disabilities and/or limited English proficiency

(SD/LEP). For the 1996 main assessment, a split-sample design was used, placing the sampled schools

into three subsamples, so that the impact of both the new SD/LEP criteria and the provision of

accommodations could be evaluated, while also collecting data with the old criteria to maintain

comparability with previous NAEP data bases. This Special Study was incorporated in the main

assessment but was not a part of the 1996 long-term trend assessment. The information in this chapter

and in Chapter 3 applies to all three sample types or subsamples.

For administrative purposes, the main and long-term trend assessments were conducted in

different schools. Responsibility for the assessments in long-term trend schools was given to one group

of assessment supervisors, while responsibility for the main assessment was assigned to another group of

supervisors. Since these supervisors worked in some of the same areas and sometimes in the same school

districts, careful coordination was required.

In order to reduce the burden on the participating schools, national assessment field staff

performed most of the work associated with the assessments. Introductory contacts and meetings were

held in the fall (1995) to enlist cooperation and explain the assessment procedures to district and school

representatives and to set a mutually agreed-upon assessment date for each school. The assessment

supervisor visited the school to select the sample of students a week or two before the assessment. The

assessment sessions were conducted by national assessment field staff, called exercise administrators,

under the direction of the assessment supervisor. At the conclusion of the assessment in a school, field

staff coded demographic information on the booklet covers and shipped the completed materials to

National Computer Systems (NCS), the processing subcontractor for NAEP (see Chapter 6 for more

detailed information on processing assessment materials).

5.2 PREPARING FOR THE ASSESSMENTS

5.2.1 Gaining the Cooperation of Sampled Schools

The process of gaining cooperation of the schools selected for the national assessment began in

late August 1995 with a series of letters and contacts with state and district-level officials. The National

Center for Education Statistics (NCES) first sent each jurisdiction a letter announcing NAEP plans for

1996. Westat then contacted the State Test Directors or NAEP State Coordinators in each sampled state

to notify them of the districts and schools selected in their states. In the 40 jurisdictions participating in

the State Assessment that also had schools sampled for the national assessment, the state received the list

of districts and schools sampled for both the national and state assessments.
99

From September through early December 1995, Westat sent lists of schools sampled for the national

assessment component and other NAEP materials to district superintendents, diocesan superintendents of

Catholic schools, and principals or heads of schools in other nonpublic schools, inviting their

participation. These initial mailings paved the way for telephone contacts by NAEP field supervisors who

were assigned the task of gaining cooperation and scheduling assessment dates.

The schedule for project activities for the 1996 main and long-term trend assessments was as

follows:

Date Activity

Mid-August 1995 Department of Education sent first letter to Chief State

School Officers about the 1996 assessment.

August 27 - 31, 1995 Training sessions were held for long-term trend assessment

supervisors.

September 7 - 9, 1995 Training sessions were held for main assessment schedulers.

Mid-September 1995 Westat sent state coordinators a list of their schools initially

selected for either or both major components.

Mid-to-Late September 1995 Westat sent samples and informational materials to districts

if not already sent by state coordinators.

Mid-September - December 1, 1995 Supervisors contacted districts and schools to secure

cooperation and to schedule assessments.

Supervisors conducted introductory meetings for the

national assessment, by telephone (or in person if requested

by districts/schools). Westat selected substitutes for

refusals.

Supervisors recruited, hired, and trained exercise

administrators.

October 9 - December 22, 1995 Fall long-term trend assessments were administered.

Early December 1995 Supervisors sent informational materials to principals and

school coordinators. Letter confirming assessment schedule

sent to each school from Westat.

December 9 - 13, 1995 Main assessment supervisor training session was held.

January 3 - March 29, 1996 Main assessments were administered.2

January 3 - March 8, 1996 Winter long-term trend assessments were administered.

March 11 - May 10, 1996 Spring long-term trend assessments were administered.

5.2.2 Supervisor Training

Training for assessment supervisors was multi-phased and involved separate sessions conducted

in August, September, and December 1995. All training was conducted by the Westat project director,

 

2

 Final makeup sessions were held April 1-5, 1996.
100

field director, and home office staff. Also in attendance were representatives from Educational Testing

Service (ETS), NCS, and NCES.

The first of these training sessions was held August 27-31, 1995 in Baltimore, Maryland for field

staff assigned to the long-term trend program for 1996. Attending the session were the long-term trend

field manager, the 11 field supervisors responsible for conducting the long-term trend assessments, and 4

troubleshooters.

After an introduction to the study, which included the background and history of NAEP, an

overview of the long-term trend assessments, and the 1995-1996 assessment schedule, the training

continued with a thorough (2 half-day sessions) presentation of NAEP contact/gaining cooperation

activities. This is a lengthy process of contacting states, districts, and schools regarding their

participation in and scheduling for NAEP; several demonstration phone calls, role plays, and exercises

were used to provide some practical experience during this part of the training. The long-term trend staff

was also trained on setting assessment schedules, recruiting/hiring/training exercise administrators, and

sample selection and preparation of Administration Schedules and other assessment materials. Several

practice exercises were used to demonstrate these topics. The training concluded with: discussions of

conducting the session and using the session script; preparing school worksheets and holding makeup

sessions; post-assessment activities; and Westat administrative procedures.

After an overview of NAEP and introductory remarks on the study schedule, the main assessment

group of about 25 supervisors received extensive training (similar to the August training for long-term

trend) in contacting the schools, gaining cooperation and scheduling the assessments; numerous

demonstrations, role plays and exercises were used. Other training topics included: supervisory

responsibilities; setting the assessment schedule; recruiting and training exercise administrators; and

administrative forms and procedures. The scheduling supervisors also received a full day of training on

using the reporting system installed on the laptop computers assigned to each of them for the gaining

cooperation/scheduling phase.

The 75 NAEP supervisors who were responsible for main assessment activities were trained

during a third session, held December 9-13, 1995. Training focused on a review of the preliminary

activities during the fall including results of initial contacts with districts and schools, scheduling of

assessments, the status of exercise administrators’ recruitment, and a thorough discussion of assessment

activities: sampling procedures; inclusion of SD/LEP students; teacher surveys; providing testing

accommodations; conducting science sessions; and administrative forms and procedures. Westat’s

classroom management videotape was also shown at this training session.

The main and state assessment field managers were present at the December session to support

training activities and answer questions concerning districts and schools that fell into the samples for

more than one component of the assessment. Each supervisor also met with the person who completed

the scheduling in their area, as a first step in preparing for the new supervisors’ contacts with each school

(and district, if needed).

5.2.3 Contacting Districts and Nonpublic Schools

Once the supervisors were trained in August and September, they began working on obtaining

cooperation. In states participating in the State Assessment, the assessment supervisor first spoke with

the State field manager to determine what contacts, if any, had already been made with districts about the

national assessment. The approach the supervisors took when calling superintendents depended on

whether the district had been notified about NAEP by the State Coordinator and whether the district also
101

had schools selected for the State Assessment. For districts that had been contacted by the State

Coordinator, the supervisor began by referring to that contact.

In previous NAEP assessments, the supervisors offered and usually held “introductory meetings”

with representatives from the superintendents’ offices and the selected schools, typically the

superintendent and the principals. These served as both an introduction to NAEP and a presentation on

what would be asked of the school. The meetings were also used to establish a schedule for the sampling

visits and the assessments in the schools.

However, over the years, these meetings have become somewhat redundant since many districts

have fallen into the NAEP sample more than one time. It has also become more and more difficult to

schedule these meetings, as district and school officials find it harder to allot time away from their

offices. Thus, beginning with the fall 1995 preparation for the 1996 study, the material was almost

always presented to the superintendents and principals during telephone calls rather than in formal

meetings. Generally, only if an in-person meeting was specifically requested by the district or school

officials, or if the supervisor felt that there was a better chance of convincing a district to participate in

person, was such a meeting held.

As the supervisors contacted superintendents, principals, and nonpublic school officials to

introduce NAEP and determine the schools’ cooperation status, they completed two forms and entered

the school status in the receipt control system installed on their laptop computers. The Results of Contact

Form was completed to document the discussion the supervisor had with each administrator concerning

the district’s willingness to participate and any special circumstances regarding the schools’ cooperation

or assessments.

The supervisor also completed portions of a School Control Form. This form was preprinted with

the number and types of assessment sessions assigned to the school, so that this information could then

be shared with the district/school official. Information gathered during the phone call, including the name

of the person designated to be the school coordinator, the number of students in the designated grade,

tentative dates for the sampling visit and assessment, and other information that could have some bearing

on the assessment, was recorded on the form. This information was used to update records in the home

office. In December, the forms were provided to the supervisors who would be conducting the

assessments.

A small number of in-person introductory meetings were held. The New York City and Los

Angeles City school districts have previously used these meetings to present information about the NAEP

assessments to the officials of all the selected schools and to encourage their participation, and wished to

continue that practice for the current assessment. A small number of other school districts also requested

such a meeting, involving representatives from their selected schools so that they would have a full

understanding of what the assessments entailed.

During the telephone presentation or the introductory meeting, the supervisor discussed

arrangements for the assessments with representatives from each school. Within the weeks scheduled for

the PSU, the supervisor had the flexibility to set each school’s assessment date in coordination with

school staff. The staff sometimes expressed preferences for a particular day or dates or had particular

times when the assessment could not be scheduled. Their preferences or restrictions depended on the

events that had already been scheduled on their school calendar. Using this information from the schools,

the supervisors set up the assessment schedule for each PSU.

The supervisor usually learned during the introductory contact whether a school required some

form of parental notification or permission. Three versions of standard NAEP letters were offered for the
102

school’s use, and each letter could be produced for selected students only or for all eligible students. The

first version informs parents about the assessment. The second assumes parental consent unless parents

send the form back stating that they do not want their child to participate in the assessment. The third

version requires that parents sign and return the form before students can be assessed. All versions of the

letter were available to the schools, although when the issue of parental permission came up in

discussion, supervisors offered the least restrictive version that met the requirements of the school or

district. In addition, Spanish language versions of the parent information letter were made available to the

schools. Schools could also send out their own letters and notices if they preferred not to use those

offered through NAEP. Information on whether the school required parent letters and the type of letter

used was recorded on the School Control Form.

5.2.4 Recruiting, Hiring, and Training Exercise Administrators

During the fall, while the supervisors were contacting their schools and scheduling assessments,

their other major responsibility was to recruit and hire exercise administrators, who would administer the

assessment sessions. Exercise administrators were recruited from many sources. Each supervisor was

given a PSU-by-PSU computerized list of exercise administrators and other field staff who had worked

previously on education studies for Westat. People who had served as exercise administrators before,

with good evaluations from their previous supervisors, were usually the first considered for hiring.

Subsequently, during contacts with the schools, the supervisors asked the school principals and other

staff to recommend potential exercise administrators. These referrals were frequently retired teachers or

substitutes. Finally, where necessary, ads were placed in local newspapers and the employment service

was notified.

Supervisors were told that, in general, four to five exercise administrators should be hired for

each PSU, although a variety of factors might influence the actual number. The number of schools in a

PSU, the size of the student sample in each school, distances to be traveled, the geography of the area,

and weather conditions during the assessment period were all factors taken into consideration by

supervisors in developing their plan for hiring exercise administrators.

A few supervisors, whose NAEP assignments contained contiguous PSUs, hired the same

exercise administrators to work in all their PSUs. Other supervisors, whose assignments comprised PSUs

that were not geographically connected, tended to hire teams of exercise administrators for each PSU.

Supervisors were encouraged to hire locally and to hire individuals with teaching experience and the

ability to handle classroom situations.

The scheduling supervisors, all of whom were experienced NAEP supervisors, had complete

responsibility for recruiting, hiring, and training all of the exercise administrators, including ones who

would report to different assessment supervisors. The training was standardized so that all supervisors

used a prepared script and exercises to train the exercise administrators.

Each exercise administrator received an exercise administrator manual, which covered the full

range of their job responsibilities. After studying the manual, they attended a half-day training session.

During the training, the supervisor reviewed all aspects of the exercise administrators’ job including

preparing materials, booklets, and Administration Schedules for assessments; the actual conduct of the

session; post-assessment collection of materials; coding booklet covers; recordkeeping; and

administrative matters. In January 1996, each exercise administrator attended a shorter, refresher training

session, conducted by the assessment supervisor, to gain further experience with the auxiliary materials,

such as mathematics manipulatives and science kits, to be used in specific assessment sessions.
103

5.3 SELECTING THE STUDENT SAMPLE FOR MAIN NAEP

5.3.1 Grade-Eligible Sample

After securing cooperation from the school, the first scheduled visit to the school was made to

select the sample of students to take part in the main assessments, and to conclude the arrangements for

the actual testing. This visit was made in January by the supervisor responsible for the assessments in the

school. Upon arriving at the school (rarely, sampling was done at the district office instead of in the

school), the supervisor first reviewed the list of grade-eligible students and confirmed with the school

coordinator that all eligible students were listed. If any eligible students were omitted, sampling could not

proceed until the list was completed.

Using the computer-generated Session Assignment Form (SAF) for the main assessment, which

was specific to the school, the supervisor selected the sample of students to be assessed. The SAF

documented the types of sessions to be administered, the anticipated number of students to be assessed,

the expected number of students eligible for the assessment, and a series of line numbers designating the

students to be sampled. Those eligible students on the school’s master list whose line numbers were

shown on the SAF were selected for the assessment. After making sure that all eligible students had been

listed, the supervisor numbered the students on the master list. If the total number of eligible students was

within the minimum and maximum limits indicated on the SAF, the supervisor could proceed to select

the sample. If the number was outside the limits, the supervisor called Westat for additional sampling

instructions. With either the original instructions or revised line numbers, the supervisor proceeded to

select the sample of students. The SAFs provided step-by-step instructions for sampling, indicating not

just the line number of each student to be selected, but the type of assessment session for which each

student was selected.

Once students were assigned to sessions, the supervisor and exercise administrators filled out an

Administration Schedule for each session. The Administration Schedule is the primary control document

for the assessment. It is used to list each sampled student and is the only link between booklets and

students. The sample was designed so that about 30 students were assigned to each session. The

supervisor discussed the final schedule of the sessions with the school coordinator and the date, time, and

location of each session were filled in on the Administration Schedules. Because student names were

recorded on the Administration Schedules, those forms remained in the schools after the sample was

drawn.

The supervisor then asked the school coordinator to identify any students in the sample with an

Individualized Education Program (IEP) (for reasons other than being gifted and talented) and/or who

were designated as LEP. Any student with either (or both) of these designations was to be indicated on

the Administration Schedules. The school was asked to complete an SD/LEP Student Questionnaire for

each student with this designation. This was to be completed by a teacher, counselor or other school

official who knew the designated student well.

The school coordinator was also asked to determine whether any of these students should be

excluded from NAEP based on the criteria for assessing SD/LEP students (discussed in detail in Section

5.3.2).

3

 Preliminary results indicate that less than half of the students with SD and/or LEP designations

were excluded from the assessment. If the school coordinator could not identify the excluded students

while the supervisor was at the school, the instructions were left with the Coordinator along with blank

copies of the SD/LEP Student Questionnaire. In those cases, the Coordinator consulted with other school

 

3 The criteria differs for the main and long-term trend assessments.
104

officials and informed the supervisor as to who was to be excluded when he/she returned for the

assessment.

At the end of the sampling visit, if requested by the school, the supervisor and/or exercise

administrators made lists of the sampled students for the teachers and/or completed appointment cards

notifying students about their assessment schedule. Teacher notification letters were also prepared in

some schools, which explained the assessment and listed the students who had been selected.

5.3.2 Sampling for Special Studies

Two special studies, requiring added steps in the sampling process, were included in the main

assessment for 1996. (The special studies were not a part of the long-term trend component.) One of

these special studies involved students eligible for advanced mathematics or science sessions. The other

involved applying two versions of the SD/LEP “inclusion” criteria for NAEP assessments and, in some

schools, offering accommodations for testing students designated as SD/LEP.

 Advanced Sessions

Samples of advanced mathematics or advanced science students were designated by separate

series of line numbers on the SAF as was done for the other session types scheduled for a school.

However, before these students could be listed on the Administration Schedule for the advanced session,

it was necessary to check each selected student’s eligibility using lists of students in advanced courses

prepared by the school. The definitions of advanced students were as follows:

• Grade 8 Mathematics: students enrolled in Algebra 1 or beyond at anytime during

the 1995-96 school year;

• Grade 12 Mathematics: students enrolled in Algebra 3, Pre-Calculus, Calculus and

Analytic Geometry, Calculus or AP Calculus; and

• Grade 12 Science: students enrolled in AP Biology, Chemistry 2 (AP), Physics 1,

Physics 2 without Calculus, and Physics 2 (AP).

The advanced sessions were available only at grade 8 in mathematics and grade 12 in

mathematics and science. Of the students designated for advanced sessions as per the line numbers on the

SAF, only those who were also eligible (according to the definitions above) were actually listed on the

Administration Schedules for the advanced sessions.

One further sampling step was applied in grade 8 and grade 12 schools regardless of whether the

school was actually scheduled for an advanced session. After the samples for all sessions were selected,

the supervisors compared the school’s lists of advanced students against the students selected for each

session to determine those students who were eligible for an advanced session. This did not mean that all

eligible students would take the advanced session (only those selected and eligible for an advanced

session were actually assessed at the advanced level), but the eligibility status was then recorded on the

Administration Schedules for all sessions to provide a source of information on the extent to which the

school offered advanced mathematics and science courses.
105

 SD/LEP Sampling and Inclusion Criteria

Because of increased interest throughout the education community in assessing as many students

as possible, NAEP has begun to evaluate the effects of using revised criteria for inclusion of SD/LEP

students and providing testing accommodations that are usually offered to these students by their schools.

For the 1996 study, the school sample was divided into three subsamples by the statisticians at Westat.

The purpose of the subsamples was to: collect data under the same conditions as previous studies in order

to maintain trend in mathematics within NAEP; evaluate the impact of a revised, more specific set of

inclusion criteria; and evaluate the combined effect of the new criteria and the use of accommodations

for testing students. The three subsamples of schools were defined as follows:

S1 (Sample 1) These schools used the criteria from 1990 and 1992, and

accommodations were not provided. Since the issue of “trend”

applies to mathematics and not to science, only mathematics

was assessed in these schools.

S2 (Sample 2) These schools used the new 1996 criteria, but accommodations

were not offered. This was designed to evaluate only the impact

of changing the criteria.

S3 (Sample 3) These schools applied the new 1996 criteria and the

accommodations most commonly used for achievement testing

were offered. For LEP students, the adaptations included a

bilingual (English-Spanish) version of the mathematics

assessment and a Spanish-language glossary for the science

assessment. For IEP students, the accommodations included:

small group or one-on-one assessments, untimed assessments,

and reading aloud. Large-print and Braille booklets were also

offered in some mathematics sessions.

During the sampling visit, after the samples of students were selected for each session, the

schools were asked to identify any sampled students whom the school considered to be SD/LEP. The

school was then asked to complete an SD/LEP Questionnaire regarding each of these students. These

basic steps were consistent with previous studies and did not vary among the three types of schools.

Further, the schools were asked to indicate which of the SD/LEP students should be included in

the assessment and which should not. Again, this step is consistent with previous studies, but in 1996, the

specific criteria and availability of accommodations varied among the three types of schools. Each school

based its decisions about assessing SD/LEP students on the specific criteria provided to that school.

To produce as large a sample as possible of students from which to evaluate the new criteria and

procedures, SD/LEP students were oversampled in certain session types in some schools. This was the

case for mathematics sessions at grades 4, 8, and 12 in S2 schools and in both mathematics and science

sessions at all three grades in S3 schools. The oversampled students were added to the appropriate

Administration Schedules (according to the instructions on the SAF) as part of the sampling process.

The SAF contained specific instructions on oversampling for the NAEP supervisor if it was

required in a particular school. Oversampling of SD/LEP students was performed only in S2 and S3

schools, and the “pool” from which the oversample was selected varied according to the grade to be

assessed and the types of sessions scheduled.
106

The pool was defined as:

• Grade 4 - any IEP and/or LEP students from the grade-eligible list who were not

selected for any session type; or

• Grade 8 or 12 with an advanced session - any IEP and/or LEP students preselected

for an advanced session but not eligible for that session; or

• Grade 8 or 12 with no advanced session - any IEP and/or LEP students from the

grade-eligible list who were not selected for any session type.

For the supervisors to select the oversample correctly, it was necessary to first complete the sampling for

every session in the school and to be sure that the school had provided IEP and/or LEP status for every

student on the grade-eligible list (i.e., not just for the selected students).

Once the oversample pool was established for a particular school, the supervisor numbered

(consecutively) the students in the pool. (This was essentially a renumbering and was done separately

from the original numbering of all students on the list). The oversample was then selected using the

oversampling line numbers specified on the SAF. Students were added to either regular mathematics or

regular science sessions according to the specifications on the SAF. The names of the oversampled

students were inserted at the end of the appropriate Administration Schedule. The school was asked to

complete an SD/LEP Questionnaire on each such student and to determine whether the student should be

included in the assessment and, for S3 schools, what specific testing accommodation(s), if any, are called

for in the students IEP and/or are normally provided for each student by the school.

The unweighted results of the 1996 assessments show that the sampling process generated, in

total, 15,871 students to be assessed in S1 schools, 48,769 to be assessed to S2 schools, and 41,513 in S3

schools. These counts include the SD/LEP students that the schools determined should participate in the

assessments. Accommodations were used in just over 200 S3 schools (about 30%) for approximately

1,050 students. The most frequently provided accommodations were small group, extended time

(untimed testing), and bilingual assessment booklets. These results are very preliminary, however,

because they are unweighted and cannot be used to compare results for S1, S2, and S3 schools (without

applying the weighting process). Detailed information and results of the SD/LEP special study will be

provided in a separate report.

5.4 CONDUCTING THE ASSESSMENT SESSIONS

The primary responsibility for conducting assessment sessions was given to the exercise

administrators. Supervisors were required to observe the first session each exercise administrator

conducted to ensure that they followed the procedures properly. Supervisors were also required to be

present in all schools with more than one small session to be conducted. The supervisor plays an

important role as the liaison between the national assessment and school staff ensuring that the

assessments go smoothly.

To ensure that sessions were administered in a uniform way, the exercise administrator was

provided with scripts for each session type. The scripts were to be read verbatim. The scripts began with

a brief introduction to the study. The exercise administrator was then directed to distribute the booklets,

being careful to match the student with the preassigned booklet.
107

After the booklets were distributed, some additional, scripted directions were read. Students were

asked to write in the NAEP school ID (except in grade 4, where NAEP staff entered the ID) and their

home ZIP code on the cover of the booklet, and given some general directions in completing the

assessment. For fourth grade students, all of the background questions were then read aloud by the

exercise administrator; at the upper grades, the first question, which asks the students’ race/ethnicity, was

read by the exercise administrator, and the students read the rest to themselves. After the background

questions were completed, the students were told that any further questions they might have could not be

answered by the exercise administrator, and that they were to begin the first cognitive section of the

assessment. This process (along with the script) was modified somewhat for science where the

background questions were at the end of the assessment booklet, and none of the items was read aloud at

grades 8 or 12.

During the sessions, the exercise administrators walked around the room monitoring the students

to make sure they were working in the correct section of their booklet and to discourage them from

looking at a neighbor’s booklet.

At the end of each assessment session, booklets were collected and students dismissed according

to the school’s policy. The exercise administrator was then responsible for completing the information at

the top of the Administration Schedule, totaling the number of participating students, and coding the

covers of all booklets, including those booklets assigned to absent students.

5.5 RESULTS OF THE MAIN NAEP ASSESSMENT

5.5.1 School and Student Participation

The unweighted school response rate for the main assessments in 1996 was 82 percent. The final

sample of cooperating schools included 604 schools at grade 4, 592 schools at grade 8, and 591 schools

at grade 12. Table 5-1 shows comparative response rates for the last four assessment periods.

Unlike the student response rates, there has been a small but steady decline in the main

assessment school response rates over the last several assessment periods. This has occurred despite

persistent efforts to convert schools and districts that indicate that they are not interested in participating

in the assessments. Both Westat field managers and ETS staff have been employed in these conversion

efforts.

Table 5-1

Comparison of Student and School Response Rates for Main NAEP, 1990-1996

1990 1992 1994 1996

Student response

Grade 4 92.9 93.4 93.2 95.5

Grade 8 89.0 88.8 91.0 93.2

Grade 12 80.8 80.8 81.1 80.1

School response

Grade 4 88.3 86.4 86.0 85.8

Grade 8 86.7 85.3 85.5 81.9

Grade 12 81.3 81.5 78.6 78.7
108

The most frequently stated reason for school and district refusals has been the increase in testing

throughout the jurisdictions and the resulting difficulty in finding time in the school schedule to conduct

the NAEP assessments. With so many states now mandating their own testing, school schedules are

becoming tighter, and administrators are finding it increasingly difficult to accommodate outside testing.

Despite the increased visibility and publicity surrounding NAEP, schools are reluctantly finding it

necessary to decline participation as a result of the increasing demands on their students’ time.

Of the 113,846 students sampled for the 1996 assessment, roughly five percent overall were

excluded by schools. Altogether, 94,157 students were assessed across all three grades: 28,528 students

were assessed at fourth grade, 32,339 were assessed at eighth grade, and 33,290 students were assessed at

twelfth grade. The overall student participation rate was 88.7 percent (after eliminating any withdrawn

and excluded students).

The response rate at which supervisors were required to conduct a makeup session was raised

from the standard that had been used in previous main assessments. The previous rates of

75 percent and 85 percent (in 1994 only) were changed to 90 percent for 1996, that is, any session (or

group of sessions within the same subject area) at which fewer than 90 percent of the eligible students

were assessed would require a makeup session (assuming that the school was willing to schedule one).

This change resulted in 129 schools conducting makeup sessions that would not have been required to do

so under the 85 percent rule. In these schools, an additional 595 students were assessed. These 595

students served to increase the overall response by less than one percentage point (0.6%). The greatest

increase was at the grade 12 level, where 413 students were assessed in these additional sessions, which

raised the response rate for this group of students by about one percentage point.

5.5.2 Assessment Questionnaires

Westat provided each school with a School Characteristics and Policies Questionnaire a few

weeks before the assessment was scheduled to be conducted (i.e., at the time of sampling). At the same

time, supervisors prepared an SD/LEP Student Questionnaire for each sampled student with either an IEP

and/or LEP designation, with the request that it be completed by someone at the school knowledgeable

about that student.

Selected teachers of fourth- and eighth-grade mathematics and science were asked to fill out

Teacher Questionnaires. The teachers asked to participate were the mathematics or science teachers of

those students selected for the assessment so that the teacher data could be linked to student performance

data. The Teacher Questionnaire for grade 4 was combined into one form since it is recognized that at

this grade level, the same teacher would probably teach all of the subjects. For grade 8, there were two

distinct questionnaires, one for mathematics teachers and the other for science teachers. At grade 12, a

teacher questionnaire was used only for advanced mathematics sessions.

The supervisor requested that the Teacher Questionnaires be distributed as quickly as possible

after the sampling so that they could be returned by the day of the assessment. Additional introductory

materials were included with the Teacher Questionnaires in response to questions that teachers have had

in the past about the importance of completing the questionnaires and about NAEP in general. Teachers

received a letter explaining the purpose of the Questionnaire, along with background materials about

NAEP.

If the Teacher Questionnaires were not complete at the time of the assessment, the supervisor left

a postage-paid envelope to NCS to be used to return the questionnaires. Table 5-2 shows the number of

questionnaires distributed and the number completed.
109

Table 5-2

Background Questionnaires Received for Schools, Teachers,

and SD/LEP Students in the 1996 Main Assessment1

School

Characteristics ———— Teacher Questionnaires —————

Grade

Assessed

and Policy

Questionnaire

Mathematics/Science

(grade 4 only) Mathematics Science

SD/LEP Student

Questionnaire

Grade 4

Number expected

Number received

Percent received

605

577

95%

1,601

1,601

100%

NA

NA

NA

NA

NA

NA

5,116

4,885

95%

Grade 8

Number expected

Number received

Percent received

592

554

94%

NA

NA

NA

1,400

1,365

98%

844

802

95%

5,048

4,770

94%

Grade 122

Number expected

Number received

Percent received

593

546

92%

NA

NA

NA

475

475

100%

NA

NA

NA

4,147

3,806

92%

1 The numbers in this table reflect the full samples, including S1, S2, and S3.

2 At grade 12, teacher questionnaires were used only for teachers of advanced mathematics. Thus, no data were collected from science teachers,

and the data shown here represent teachers of advanced mathematics only.

5.6 LONG-TERM TREND ASSESSMENTS

5.6.1 Overview

To provide continuity and comparability with past NAEP studies, the long-term trend component

(formerly referred to as the “bridge” assessments) replicates procedures and materials that have been

used since the inception of NAEP. Student eligibility in long-term trend is always based on criteria used

in years prior to 1988 (when the modal grade for students aged 17 changed from the grade 11 to grade

12). The 1996 schedule for long-term trend assessments was as follows: the fall assessment of age

13/grade 8 students was held in the 11-week period from October 9 through December 22, 1995; the

winter assessment of age 9/grade 4 students was held during the 10-week period from January 3 through

March 8, 1996; and the spring trend assessment of students who were age 17/grade 11 was conducted in

the 9-week period from March 11 through May 10, 1996. Students were assessed in reading, writing,

mathematics, and science.

Paced tape sessions were conducted with samples of age-eligible students only, as was done in

all previous years. Additional samples of age- and grade-eligible students were assessed with spiral

(print-administered) booklets, following procedures initiated in 1984. Six different types of sessions were

conducted: one print-administered and five separate tape-administered sessions. Depending on the size of

the school, up to four different session types, involving a total of about 80 students, might have been

conducted in a participating school.
110

5.6.2 Selecting the Student Sample

Procedures for sampling in long-term trend schools were very similar to those employed in the

schools selected for the main assessment. One to two weeks before the assessment, the supervisor visited

the school to select the sample. Lists of students were reviewed to ensure that all age- and grade-eligible

students were listed. The SAF for long-term trend schools specified a range for the expected number of

eligible students. If the total number of students was within this allowable range, the sampling could

proceed. Otherwise, the supervisor called Westat for additional sampling instructions. The SAF directed

the supervisor to assign students to long-term trend session types based on their line numbers from the

student list that the school had prepared. (The SAFs for the long-term trend sample were like those used

in long-term trend sampling for previous years, and were distinct from the SAFs for the main assessment

that were described earlier in this report.)

The only major variation within the sampling for long-term trend assessments was that, for the

tape sessions, only age-eligible students were selected. For these sessions, the supervisor selected from

the entire list of students (age- and grade-eligible), but then deleted those who were only grade-eligible

before recording the names of the students to be assessed on the Administration Schedules.

The criteria for excluding students were also different for the long-term trend schools (compared

to the main assessment), and again followed the criteria that were established previously for long-term

trend. For those students who were excluded, the school was asked to complete an Excluded Student

Questionnaire. If the school coordinator could not identify the excluded students while the supervisor

was at the school, a set of instructions for excluding students was left with the coordinator along with the

estimated number of questionnaires that would be needed.

5.6.3 Conduct of the Assessment

The conduct of the assessments in schools selected for the long-term trend program is essentially

the same as in the schools selected for the main assessment. Scripts are provided for the supervisors and

exercise administrators to use in administering the sessions. The major difference compared to main

assessment is that most of the sessions are tape administered. In these sessions, after the distribution of

the test booklets, the administrator is instructed to turn on the tape recorder. The remainder of the

instructions are contained on the tape, and the timing is determined by the length of time that the tape

runs.

5.6.4 Results of Long-Term Trend Assessments

The unweighted school response rate for the 1996 long-term trend assessments was 83 percent.

The final sample of cooperating schools included 240 schools at age 9/grade 4, and 238 schools at age

13/grade 8, and 191 schools at age 17/grade 11. Nearly 30,000 students were assessed in long-term trend,

or 91 percent of those eligible to be assessed.

Of the 36,371 students sampled for long-term trend assessments, eight percent were excluded by

the schools. Overall, 29,791 students were assessed across all three age/grade groups: 10,432 students

were assessed at age 9/grade 4, 11,151 students were assessed at age 13/grade 8, and 8,208 were assessed

at age 17/grade 11. Table 5-3 shows comparative response rates for the last four long-term trend

assessments.
111

Table 5-3

Comparison of Student and School Response Rates for Long-Term Trend NAEP, 1990-1996

1990 1992 1994 19961

Student response

Grade 4 92.4 94.0 94.2 95.5

Grade 8 90.4 90.8 92.2 91.9

Grade 12 81.2 82.8 84.1 84.0

School response

Grade 4 88.1 87.4 86.7 84.8

Grade 8 90.5 84.7 81.7 82.4

Grade 12 80.7 81.3 81.1 81.3

1 The numbers in this table reflect the full samples, including S1, S2, and S3.

5.6.5 Assessment Questionnaires

The School Characteristics and Policies Questionnaire and the Excluded Student Questionnaire

are forms that were distributed in the schools to be completed by school personnel. The School

Characteristics and Policies Questionnaire was provided to the school by the assessment supervisor at the

time of the sampling visit. This form was to be filled out by the principal or other staff member

knowledgeable about the school’s administrative policies and staff characteristics. The supervisors

collected the completed questionnaire when they returned to the school for the assessment.

An Excluded Student Questionnaire was to be filled out for every student who was sampled for

the assessment but excluded by the school. Following exclusion criteria used in previous long-term trend

assessments, schools could exclude students with limited English-speaking ability, those who were

educable mentally retarded, or functionally disabled students, if in the judgment of school staff or as

indicated in school records, they were unable to “participate meaningfully” in the assessment. After the

sample of students was drawn and Administration Schedules prepared, the supervisor requested that the

school coordinator identify any students who should be excluded. The supervisor then gave an Excluded

Student Questionnaire to the coordinator for every excluded student, with the request that it be completed

by someone in the school knowledgeable about the student. (Note that this varies somewhat from the

main assessment where questionnaires are assigned for all sampled students with an IEP and/or LEP, not

just for those who are excluded from the assessment.)

The supervisor attempted to collect all completed questionnaires (School Characteristics and

Policies Questionnaire and Excluded Student) on the assessment day. If the questionnaires were not

ready, and it was convenient for the supervisor or an exercise administrator to return to the school later to

pick them up, they would do so. Otherwise, the supervisor gave the coordinator a postage-paid envelope

to use to mail the forms to NCS. All (100%) of the School Characteristics and Policies Questionnaires

were completed and returned, and 95.9 percent of the Excluded Student Questionnaires were returned.

Once the assessments were completed in a school, the supervisor and exercise administrators

completed the coding of the front covers of the assessment booklets, filled out the necessary forms, and

shipped the booklets and forms to NCS. A copy of all forms was sent to Westat so that progress in the

field could be closely monitored.

The School Worksheet was used by the supervisor to summarize the results of the assessment

sessions in each school. The number of students to be assessed, the number actually assessed, and the
112

number absent were entered so that the supervisor could calculate whether a makeup session was

required. Attendance of less than 90 percent required a makeup. If a makeup was required for one or

more session types, the supervisor discussed the scheduling of the makeup with the school coordinator.

In long-term trend assessments prior to 1994, the percentage of students attending that would

necessitate a makeup session was 75 percent or below. For 1994, this rate was increased to 85 percent,

and it was raised again to 90 percent for 1996. By raising the rate to 90 percent, an additional 144

students were assessed in 73 schools, compared to the 85-percent-rule, increasing the overall student

response rate for 1996 by less than one-half of one percentage point (0.44%).

The top (original) copy of the School Worksheet, any Excluded Student Questionnaires

completed by the school, and the Administration Schedules (with the students’ names removed and left at

the school) were included with the booklets in the shipment to NCS. In addition, the supervisor included

a packing list with the materials, which inventoried the assessment materials assigned to and returned

from the school.

5.7 FIELD MANAGEMENT

Two field managers monitored the work of 25 scheduling supervisors who worked during fall

1995 to gain cooperation of districts and schools for the main assessment. During the assessment period,

these staff were expanded to about 75 supervisors and 6 field managers (4 of whom were located in

Westat’s home office). An additional field manager was assigned exclusively to long-term trend NAEP

throughout the gaining cooperation and assessment periods. All supervisors reported directly to their

field managers who, in turn, reported to Westat’s field director. All contacts were made at least weekly.

An automated management system was developed and maintained in Westat’s home office. The

scheduling supervisors working to contact schools during the fall used this system on their portable

computers. The system contained a record for each sampled school. A disposition code structure was

developed to indicate the status of each school’s participation (e.g., school cooperating, decision pending,

school refusal, district refusal, school closed, etc.). As a school’s status was determined, the scheduling

supervisors entered the status of the school onto their computers, and this information was downloaded

onto the home office system on a weekly basis. Disposition reports were then generated from the receipt

system once a week so that home office staff could review the progress of securing cooperation from the

sampled schools.

These reports were an invaluable tool for the sampling statisticians as well as for the field

director and field management staff. They provided the statisticians with the information needed to

determine whether the sample of schools was adequate to produce representative results. Based on the

information contained in these reports, the sampling statisticians selected substitute schools to replace

some of the non-cooperating schools.

After assessments were completed, the system was used to enter data from the School

Worksheets (for both Main NAEP and long-term trend) on the number of students to be assessed, the

number assessed, and the number absent for each school. Data on completed questionnaires received was

provided by NCS. The system was also used to alter school assessment dates, particularly when bad

weather required a change in schedule, and to monitor plans for and progress in conducting makeup

sessions. Reports were generated weekly during the assessment period that allowed the project staff to

monitor the progress of the assessments both in terms of checking that the schools were assessed on

schedule as well as assuring that a high response rate was achieved. The sampling statisticians used these

reports to monitor the sample yield by school, PSU, and age/grade level.
113

Progress of the assessments was constantly monitored through telephone reports held between

NAEP supervisors, field managers, and home office staff. During these phone conversations, the

supervisors’ schedules were reviewed and updated, and any problems that the supervisors were

experiencing were discussed. Much of the attention this year was focused on maintaining the schedule in

light of the many postponements due to severe winter weather.

The supervisors who traveled filled out a Work Schedule for a one- to two-week period, showing

their whereabouts, so that they could be contacted if necessary. It also allowed field managers and project

staff to review the supervisors’ schedules and the distribution of work.

Progress of the field work was also monitored during quality control visits made to the field by

Westat and ETS office staff.

115

Chapter 6

PROCESSING ASSESSMENT MATERIALS1

Patrick B. Bourgeacq, Bradley Thayer, and Timothy Robinson

National Computer Systems

6.1 INTRODUCTION

This portion of the report reviews the activities conducted by National Computer Systems (NCS)

for the NAEP 1996 main assessments in mathematics and science, and long-term trend assessments in

reading, mathematics, science, and writing. As a subcontractor to Educational Testing Service (ETS),

NCS was responsible for printing all of the NAEP student booklets and control documents; distributing

the assessment materials to the field; receiving, tracking, processing and editing the assessment materials

as they returned from the field; scoring all of the constructed-response items (in conjunction with ETS);

and delivering the assessment data files to ETS for analysis and reporting.

For this assessment, NCS was charged with processing and scoring the largest assessment in the

history of NAEP in the shortest amount of time. Further, image scanning processes, eliminating almost all

paper handling during scoring and improving monitoring and reliability scoring, increased to nearly twice

that of the 1994 assessment. Materials management and distribution of over one quarter of a million

science kits, receipt control processing for all receipts within 48 hours, image scanning throughput

increased nearly twice that of 1995, and professional scoring of over nine million constructed responses

highlight the challenges met by NCS for the 1996 NAEP national and state assessments.

NCS processed more than 134,000 booklets for the NAEP 1996 national assessment, as shown in

Tables 6-1 and 6-2. NCS also received and processed a total of over 2,300 school characteristics and

policies questionnaires, over 4,700 teacher questionnaires, and over 16,274 SD/LEP questionnaires for

the three grades, as shown in Table 6-3. Table 6-4 lists key events and dates in the NAEP schedule.

This chapter of the report reviews the activities conducted by NCS for the NAEP 1996 main and

long-term trend assessments.

 

1

 Bradley Thayer was the NCS project manager for 1996 NAEP, Patrick Bourgeacq was the NCS project director for 1996 NAEP

scoring, and Timothy Robinson was the NCS senior processing coordinator for 1996 NAEP.
116

Table 6-1

Processing and Scoring Totals for the 1996 NAEP Assessment

Booklets

Processed

Number of

ConstructedResponse

Items1

Number of

Responses

Scored2

Number of

Scorers and

Team

Leaders3

Length of

Training and

Scoring

Fall Trend

Reading/Writing 5,287 22 42,741 7 / 1 11/29/95 - 1/5/96

Fall Trend

Mathematics 5,727 28 88,782 11 / 0 12/18/95 - 1/5/96

Winter Trend

Reading/Writing 4,988 20 34,341 6 / 1 1/30/96 - 4/3/96

Winter Trend

Mathematics 5,442 29 86,322 9 / 1 3/25/96 - 3/29/96

Spring Trend

Reading/Writing 4,669 25 43,094 6 / 1 4/10/96 - 5/24/96

Spring Trend

Mathematics 3,570 29 86,478 5 / 1 5/13/96 - 5/22/96

Long-Term Trend

Writing Holistic N/A 6 63,793 56 / 7 6/10/96 - 6/16/96

Long-Term Trend

Writing Mechanics N/A 3 2,329 33 / 6 7/15/96 - 7/31/96

1990 Rescore

4th Grade Mathematics 749 12 8,988 198 / 17 3/13/96 - 5/6/96

1990 Rescore

8th Grade Mathematics 730 18 13,374 198 / 17 3/13/96 - 5/6/96

1990 Rescore

12th Grade Mathematics 725 21 15,225 198 / 17 3/13/96 - 5/6/96

1992 Rescore

4th Grade Mathematics 2,498 36 27,031 198 / 17 3/13/96 - 5/6/96

1992 Rescore

8th Grade Mathematics 2,498 44 33,077 198 / 17 3/13/96 - 5/6/96

1992 Rescore

12th Grade Mathematics 2,498 44 32,387 198 / 17 3/13/96 - 5/6/96

National 4th Grade

Mathematics Spiral 10,816 64 170,219 198 / 17 3/13/96 - 5/6/96

National 4th Grade

Mathematics Estimation 2,128 3 7,756 198 / 17 3/13/96 - 5/6/96

National 4th Grade

Mathematics Theme 4,038 13 29,750 198 / 17 3/13/96 - 5/6/96

National 8th Grade

Mathematics Spiral 11,554 69 195,764 198 / 17 3/13/96 - 5/6/96

1

 This is the number of discrete constructed-response items in assessment booklets.

2

 This is the number of student responses to the constructed-response items. These scored responses include those that were

rescored for reliability estimation.

3 Because readers scored items from all grades and all types of booklets, it is not possible to break the numbers down by how

many scored each classification of items.

(continued)
117

Table 6-1 (continued)

Processing and Scoring Totals for the 1996 NAEP Assessment

Booklets

Processed

Number of

ConstructedResponse

Items1

Number of

Responses

Scored2

Number of

Scorers and

Team

Leaders3

Length of

Training and

Scoring

National 8th Grade

Mathematics Estimation 2,267 6 17,027 198 / 17 3/13/96 - 5/6/96

National 8th Grade

Mathematics Theme 4,259 13 34,613 198 / 17 3/13/96 - 5/6/96

National 8th Grade

Mathematics Advanced 2,382 14 41,693 198 / 17 3/13/96 - 5/6/96

National 12th Grade

Mathematics Spiral 10,740 73 225,540 198 / 17 3/13/96 - 5/6/96

National 12th Grade

Mathematics Estimation 1,883 0 0 198 / 17 3/13/96 - 5/6/96

National 12th Grade

Mathematics Theme 3,892 12 29,210 198 / 17 3/13/96 - 5/6/96

National 12th Grade

Mathematics Advanced 2,987 15 56,101 198 / 17 3/13/96 - 5/6/96

Bilingual 4th Grade

Mathematics National 91 10 2,280 0 /4 5/2/96 - 5/2/96

Bilingual 8th Grade

Mathematics National 36 12 1,080 0 /4 5/3/96 - 5/3/96

National 4th Grade

Science Spiral 11,677 94 275,339 306 / 24 3/18/96 - 5/28/96

National 8th Grade

Science Spiral 12,079 125 322,261 306 / 24 3/18/96 - 6/7/96

National 12th Grade

Science Spiral 11,579 120 342,104 306 / 24 3/18/96 - 6/2/96

National 12th Grade

Science Advanced 2,449 36 110,207 306 / 24 3/18/96 - 6/2/96

1

 This is the number of discrete constructed-response items in assessment booklets.

2

 This is the number of student responses to the constructed-response items. These scored responses include those that were

rescored for reliability estimation.

3 Because readers scored items from all grades and all types of booklets, it is not possible to break the numbers down by how

many scored each classification of items.
118

Table 6-2

Student Participation and Session Information for the 1996 NAEP Assessment

Number

of

Sessions

Number of

Booklets for

Assessed

Students

Number of

Booklets

for Absent

Students

Number of

Booklets for

Excluded

Student

Number

of

Scanned

Sheets

Long-Term Trend

Fall 639 11,150 981 974 288,624

Winter 623 10,406 486 1,172 215,679

Spring 539 8,209 1,560 758 243,452

Main

Grade 4 total 1,693 28,531 1,354 2,293 711,291

Mathematics 982 16,953 796 1,198 418,509

Science 711 11,578 558 1,095 292,782

Grade 8 total 1,698 32,339 2,359 1,823 922,892

Mathematics 819 17,992 1,338 1,062 509,064

Advanced Mathematics 345 2,375 94 4 73,268

Science 534 11,972 927 757 340,560

Grade 12 total 2,196 33,306 8,266 1,704 1,143,332

Mathematics 848 16,424 4,103 969 536,256

Advanced Mathematics 380 2,965 456 11 99,112

Science 613 11,486 3,332 715 387,504

Advanced Science 355 2,431 375 9 120,460

Other

Rosters 23,535

Administration Schedules 24,575

Rescore Mathematics 1992

Grade 4 45,286

Rescore Mathematics 1992

Grade 8 61,311

Rescore Mathematics 1990 46,507
119

Table 6-3

Questionnaire Totals for the 1996 NAEP Assessment

Expected Received Percent

Main Assessment

Grade 4

SD/LEP Questionnaire 5,116 4,885 95%

School Characteristics Questionnaire 597 577 97%

Grade 8

SD/LEP Questionnaire 5,048 4,770 94%

School Characteristics Questionnaire 580 554 96%

Grade 12

SD/LEP Questionnaire 4,147 3,806 92%

School Characteristics Questionnaire 582 546 94%

Long-Term Trend

Fall

Excluded Students Questionnaire 979 947 97%

School Characteristics Questionnaire 239 224 94%

Winter

Excluded Students Questionnaire 1,187 1,145 96%

School Characteristics Questionnaire 247 235 95%

Spring

Excluded Students Questionnaire 768 721 94%

School Characteristics Questionnaire 186 173 93%

Main Teacher Questionnaires

Grade 4

Mathematics/Science 1,599 1,609 101%

Grade 8

Mathematics 1,401 1,359 97%

Science 1,310 1,270 97%

Grade 12

Advanced Mathematics 476 487 102%
120

Table 6-4

NCS Schedule for the 1996 NAEP Assessment

Task

Planned

Start

Planned

Finish

Actual

Start

Actual

Finish

FALL LONG-TERM TREND

Printing 6/15/95 9/19/95 6/15/95 9/19/95

Pre-packaging (barcoding, spiraling, and quality

control) 8/14/95 9/7/95 8/14/95 9/7/95

Session file to NCS from Westat 8/25/95 8/25/95 8/25/95 8/25/95

Westat supervisor training 8/27/95 8/30/95 8/27/95 8/30/95

Print Administration Schedule 9/1/95 9/5/95 9/1/95 9/1/95

Supervisor add file from Westat 9/5/95 9/5/95 9/5/95 9/5/95

Final packing specifications to packaging 9/5/95 9/5/95 9/5/95 9/5/95

Packing list and labels to packaging 9/5/95 9/6/95 9/6/95 9/6/95

Grade 8 school characteristics and policies

questionnaires arrive 9/7/95 9/7/95 9/7/95 9/19/95

Ship Administration Schedules 9/7/95 9/7/95 9/7/95 9/7/95

Information to Key Entry for screen set up 9/11/95 9/11/95 9/11/95 9/11/95

Bulk/session packaging 9/11/95 9/18/95 9/11/95 9/15/95

Packaging visit by ETS 9/13/95 9/14/95 9/14/95 9/14/95

Ship materials to supervisors 9/18/95 9/18/95 9/14/95 9/14/95

Materials due to supervisors 9/22/95 9/22/95 9/22/95 9/22/95

Processing specifications to Operations Department 10/2/95 10/2/95 10/2/95 10/2/95

Processing kick-off meeting 10/3/95 10/3/95 10/3/95 10/3/95

Requisitions for table leaders to HR 10/9/95 10/9/95 10/9/95 10/9/95

Requisitions for scorers to HR 10/9/95 10/9/95 10/9/95 10/9/95

Photocopy training materials 10/9/95 10/13/95 10/9/95 10/13/95

Test administration 10/9/95 12/19/95 10/10/95 12/19/95

Blue dot 10/10/95 10/20/95 10/20/95 10/20/95

Receiving 10/10/95 12/23/95 10/11/95 1/15/96

HR extends offer to table leaders 10/13/95 10/13/95 10/13/95 10/13/95

HR extends offers to scorers 10/16/95 10/16/95 10/16/95 10/16/95

General - Network Meeting 10/20/95 10/20/95 10/20/95 10/20/95

Processing 10/20/95 12/28/95 10/16/95 1/9/96

Scoring training preparation 10/9/95 11/17/95 10/9/95 10/17/95

Scoring training - writing 11/29/95 11/30/95 11/29/95 11/30/95

Scoring training - reading 12/4/95 12/5/95 12/4/95 12/5/95

Scoring reading/writing 11/29/95 1/5/96 11/29/95 1/5/96

Weights file shipped 1/12/96 1/15/96 1/15/96 1/15/96

Tape delivered 1/12/96 1/15/96 1/25/96 1/25/96

WINTER LONG-TERM TREND

Printing 6/15/95 9/19/95 6/15/95 9/19/95

Pre-packaging (barcoding, spiraling) 9/29/95 10/10/95 9/6/95 9/14/95

Bundle sheets delivered to packaging 10/25/95 10/25/95 9/1/95 9/1/95

All bundles through clean quality control 11/3/95 11/3/95 9/1/95 9/1/95

(continued)
121

Table 6-4 (continued)

NCS Schedule for the 1996 NAEP Assessment

Task

Planned

Start

Planned

Finish

Actual

Start

Actual

Finish

WINTER LONG-TERM TREND (continued)

Final packing specifications to packaging 11/17/95 11/17/95 11/16/95 11/16/95

HR extends offers to scorers 11/20/95 11/22/95 11/20/95 11/22/95

Session data file to NCS from Westat 11/22/95 11/22/95 11/22/95 11/22/95

Administration Schedule address file from Westat 11/28/95 11/28/95 11/22/95 11/22/95

Print Administration Schedules 12/1/95 12/1/95 11/30/95 11/30/95

Bulk/session address file from Westat 12/1/95 12/1/95 11/29/95 11/29/95

Ship Administration Schedules 12/4/95 12/4/95 12/1/95 12/1/95

Packing list and labels to packaging 12/4/95 12/5/95 12/1/95 12/1/95

Final packaging 12/6/95 12/8/95 12/6/95 12/7/95

Ship session materials 12/6/95 12/8/95 12/6/95 12/7/95

Processing specifications to Operations 12/18/95 12/18/95 12/18/95 12/18/95

Requisitions for scorers to HR 12/20/95 12/20/95 12/20/95 12/20/95

Materials due to supervisors 12/22/95 12/22/95 12/15/95 12/18/95

Test administration 1/3/96 3/15/96 1/2/96 3/15/96

Blue dot (s) 1/4/96 1/8/96 1/15/96 1/22/96

Receiving 1/4/96 3/20/96 1/9/96 3/20/96

Image definition ready 1/8/96 1/8/96 1/5/96 1/5/96

Processing 1/8/96 3/22/96 1/8/96 3/22/96

Photocopy training materials 1/9/96 1/12/96 1/9/95 10/13/95

Image test data ready 1/9/96 1/12/96 1/9/96 1/12/96

Image application ready 1/11/96 1/11/96 1/11/96 1/11/96

Scoring training preparation 1/29/96 1/31/96 1/29/96 1/29/96

Scoring training - writing 1/31/96 2/2/96 1/30/96 1/31/96

Scoring training - reading 1/31/96 2/2/96 2/6/96 2/6/96

Scoring reading/writing 2/5/96 4/5/96 2/7/96 4/2/96

Requisitions for mathematics scorers to HR 3/1/96 3/1/96 3/11/96 3/11/96

Scoring training - mathematics 3/25/96 3/25/96 3/25/96 3/25/96

Scoring - mathematics 3/25/96 4/5/96 3/25/96 3/29/96

Tape delivered 4/15/96 4/15/96 4/12/96 4/12/96

Weights file shipped 4/15/96 4/15/96 4/12/96 4/12/96

SPRING LONG-TERM TREND

Printing 6/15/95 9/19/95 6/15/95 9/19/95

Pre-packaging (barcoding, spiraling) 7/21/95 9/25/95 7/21/95 9/25/95

HR extends offers to table leaders 10/13/95 10/13/95 10/13/95 10/13/95

HR extends offers to scorers 11/20/95 11/22/95 11/20/95 11/22/95

Session file to NCS from Westat 2/5/96 2/5/96 2/5/96 2/5/96

Bulk/session address file from Westat 2/8/96 2/8/96 2/5/96 2/5/96

Administration Schedule address file from Westat 2/8/96 2/8/96 2/7/96 2/7/96

Packing list and labels to packaging 2/8/96 2/8/96 2/9/96 2/9/96

Final packing specifications to packaging 2/8/96 2/8/96 2/12/96 2/12/96

(continued)
122

Table 6-4 (continued)

NCS Schedule for the 1996 NAEP Assessment

Task

Planned

Start

Planned

Finish

Actual

Start

Actual

Finish

SPRING LONG-TERM TREND (continued)

Print Administration Schedules 2/12/96 2/12/96 2/9/96 2/9/96

Ship Administration Schedules 2/16/96 2/16/96 2/13/96 2/13/96

Package/ship session materials 2/23/96 2/23/96 2/23/96 2/23/96

Material due to supervisors 2/28/96 2/28/96 3/1/96 3/1/96

Processing specifications to Operations 3/8/96 3/8/96 3/8/96 3/8/96

Rescore booklets delivered to PSC 3/8/96 3/8/96 3/8/96 3/8/96

Photocopy training materials 3/9/96 3/12/96 10/9/95 10/13/95

Test administration 3/11/96 5/10/96 3/11/96 5/17/96

PSC approval of scoring sheets 3/15/96 3/15/96 3/13/96 3/13/96

Blue dot (s) 3/18/96 3/20/96 4/2/96 4/19/96

Processing 3/21/96 5/17/96 4/19/96 5/17/96

Requisitions for mathematics scorers to HR 4/1/96 4/1/96 4/1/96 4/1/96

Scoring training preparation 4/8/96 4/9/96 4/3/96 4/3/96

Scoring training - writing 4/10/96 4/11/96 4/10/96 4/11/96

Scoring reading/writing 4/10/96 5/24/96 4/11/96 4/11/96

Scoring training - reading 4/11/96 4/11/96 4/11/96 4/11/96

Scoring training - mathematics 5/13/96 5/13/96 5/13/96 5/13/96

Scoring mathematics 5/13/96 5/24/96 5/13/96 5/24/96

Project through clean post 5/17/96 5/17/96 5/17/96 5/17/96

Tape delivered 5/31/96 5/31/96 6/6/96 6/6/96

Weights file shipped 5/31/96 5/31/96 6/6/96 6/6/96

WRITING LONG-TERM TREND -

HOLISTIC SCORING

Requisition for scorers to HR 5/15/96 5/15/96 5/15/96 5/15/96

Requisition for table leaders to HR 5/15/96 5/15/96 5/15/96 5/15/96

PSC approves score sheet 5/15/96 5/15/96 5/15/96 5/15/96

HR makes offers to table leaders 5/20/96 5/31/96 5/28/96 6/4/96

HR makes offers to scorers 5/20/96 5/31/96 5/28/96 6/7/96

Samples drawn 5/24/96 5/24/96 5/20/96 5/24/96

Scoring preparation with J. Kennedy 6/4/96 6/7/96 6/5/96 6/7/96

Training and Scoring 6/10/96 6/14/96 6/10/96 6/16/96

Data tape delivered 7/1/96 7/1/96 6/27/96 6/27/96

WRITING LONG-TERM TREND -

MECHANICS SCORING

Requisition for scorers to HR 6/14/96 6/14/96 7/9/96 7/9/96

Requisition for table leaders to HR 6/14/96 6/14/96 7/9/96 7/9/96

HR makes offers to scorers 6/24/96 6/28/96 7/9/96 7/12/96

HR makes offers to table leaders 6/24/96 6/28/96 7/9/96 7/10/96

Samples drawn 6/24/96 6/24/96 6/24/96 6/24/96

(continued)
123

Table 6-4 (continued)

NCS Schedule for the 1996 NAEP Assessment

Task

Planned

Start

Planned

Finish

Actual

Start

Actual

Finish

WRITING LONG-TERM TREND -

MECHANICS SCORING (continued)

Responses copied for scoring 6/25/96 7/12/96 6/25/96 7/12/96

Training and scoring 7/15/96 7/26/96 7/15/96 7/31/96

Transcribe and proofread essays and scores 7/18/96 7/31/96 7/18/96 7/31/96

Data tape delivered 8/9/96 8/9/96 8/7/96 8/7/96

MAIN ASSESSMENT

Printing 9/2/95 12/11/95 9/2/95 12/11/95

Administration Schedule approved 9/15/95 9/15/95 9/21/95 9/21/95

Pre-packing specifications to packaging 10/2/95 10/2/95 10/2/95 10/2/95

Grade 8 teacher questionnaire roster delivered to NCS 10/12/95 10/12/95 10/16/95 10/16/95

Grade 4 mathematics/science teacher questionnaire

roster to NCS 10/12/95 10/12/95 10/17/95 10/17/95

PSC obtains copies of final blocks 10/16/95 10/16/95 10/16/95 10/16/95

Administration Schedule delivered to NCS 10/18/95 10/18/95 10/23/95 10/23/95

Grade 12 teacher questionnaire roster delivered to

NCS 10/20/95 10/20/95 10/16/95 10/16/95

Grade 8 school characteristics and policies

questionnaires at NCS 10/20/95 10/20/95 10/23/95 10/23/95

SD/LEP roster delivered to NCS 10/20/95 10/20/95 10/24/95 10/24/95

Grade 4 school characteristics and policies

questionnaires at NCS 10/20/95 10/20/95 10/25/95 10/25/95

Grade 12 school characteristics and policies

questionnaires at NCS 10/20/95 10/23/95 10/23/95 10/23/95

Grade 8 mathematics spiral material at NCS 10/23/95 11/2/95 10/18/95 11/3/95

Pre-packaging begins 10/23/95 12/20/95 10/16/95 12/1/95

Grade 4 mathematics spiral material at NCS 10/26/95 11/1/95 11/1/95 11/1/95

Grade 8 mathematics teacher questionnaire at NCS 10/30/95 10/30/95 10/25/95 10/25/95

Grade 8 science teacher questionnaire at NCS 10/30/95 10/30/95 11/1/95 11/1/95

Final valid score range for each item 11/1/95 11/1/95 10/24/95 11/17/95

PSC obtains rubrics from ETS 11/1/95 11/1/95 10/24/95 11/17/95

ETS/PSC define non-scorable codes 11/1/95 11/1/95 11/1/95 11/1/95

NCS/ETS meet to review items and schedule 11/2/95 11/3/95 11/2/95 11/3/95

Grade 4 mathematics/science teacher questionnaire

delivered to NCS 11/3/95 11/3/95 11/2/95 11/2/95

Grade 8 science spiral material at NCS 11/6/95 11/13/95 11/13/95 11/21/95

Grade 12 mathematics spiral material at NCS 11/14/95 11/21/95 11/21/95 12/1/95

General - sub-contractor’s meeting 11/16/95 11/17/95 11/16/95 11/17/95

SD/LEP questionnaire delivered to NCS 11/22/95 11/22/95 12/5/95 12/5/95

Grade 4 science spiral material at NCS 11/22/95 11/30/95 11/21/95 12/1/95

(continued)
124

Table 6-4 (continued)

NCS Schedule for the 1996 NAEP Assessment

Task

Planned

Start

Planned

Finish

Actual

Start

Actual

Finish

MAIN ASSESSMENT (continued)

All materials at NCS for packaging 11/29/95 12/1/95 12/1/95 12/15/95

Grade 12 science spiral material at NCS 12/1/95 12/11/95 12/1/95 12/4/95

NCS receive 95% session data from Westat 12/4/95 12/4/95 12/6/95 12/6/95

Westat training for main supervisors 12/9/95 12/13/95 12/9/95 12/13/95

Westat send Administration Schedule home address

file 12/11/95 12/11/95 12/11/95 12/11/95

Westat send NCS Wave 1 address file 12/11/95 12/11/95 12/11/95 12/11/95

Print Administration Schedule 12/11/95 12/13/95 12/13/95 12/13/95

WAVE 1 packing list and labels to packaging 12/13/95 12/13/95 12/14/95 12/14/95

Format/content Interrater Agreement Report 12/15/95 12/15/95 11/1/95 12/15/95

PSC obtains sample booklets 12/15/95 12/15/95 11/2/95 11/13/95

Purpose/use of T-Test and bridge reliability 12/15/95 12/15/95 11/2/95 11/3/95

PSC submit requisition for mathematics and science

scorers 12/15/95 12/15/95 1/6/96 1/16/96

PSC submit requisition for mathematics and science

team leaders 12/15/95 12/15/95 1/16/96 1/16/96

PSC submit requisition for mathematics and science

table leaders 12/15/95 12/15/95 1/30/96 1/30/96

Ship Administration Schedule, teacher questionnaire,

SD/LEP questionnaires, rosters 12/15/95 12/18/95 12/14/95 12/14/95

Ship bulk/Wave 1 material 12/20/95 12/20/95 12/15/95 12/18/95

Bulk/Wave 1 materials due to supervisors 12/26/95 12/26/95 12/26/95 12/29/95

Receiving 1/5/96 4/8/96 1/5/96 4/8/96

Test administration 1/3/96 4/5/96 1/3/96 4/5/96

Blue dot Grades 4, 8 and 12 mathematics 1/4/96 1/7/96 1/15/96 1/25/96

Blue dot Grades 4, 8, and 12 science 1/4/96 1/7/96 1/15/96 1/25/96

Blue dot advanced science 1/4/96 1/7/96 1/15/96 1/26/96

Blue dot Grade 12 estimation, theme, and advanced

mathematics 1/4/96 1/7/96 1/15/96 1/31/96

Processing 1/8/96 4/14/96 1/25/96 4/29/96

Blue dot SD/LEP questionnaires 1/9/96 1/11/96 2/20/96 2/27/96

Assignments of mathematics items to teams 1/15/96 1/15/96 1/2/96 2/23/96

Scoring calendar item-by-item for mathematics 1/15/96 1/15/96 1/2/96 2/15/96

Wave 2 addresses from Westat 1/17/96 1/17/96 1/17/96 1/17/96

Packaging Wave 2 materials (2 shifts) 1/18/96 1/26/96 1/19/96 1/23/96

Wave 2 packing and mailing labels to packaging 1/17/96 1/19/96 1/19/96 1/19/96

Blue dot school characteristics and policies

questionnaires and teacher questionnaires 1/22/96 1/25/96 2/9/96 2/26/96

Blue dot short-term rescore 1/25/96 1/25/96 3/5/96 3/11/96

Assignment of science items to teams 2/1/96 2/1/96 1/15/96 5/31/96

(continued)
125

Table 6-4 (continued)

NCS Schedule for the 1996 NAEP Assessment

Task

Planned

Start

Planned

Finish

Actual

Start

Actual

Finish

MAIN ASSESSMENT (continued)

Mathematics and science table leaders hired 2/1/96 2/1/96 1/15/96 2/28/96

Plan for staff range finding 2/1/96 2/1/96 1/15/96 2/1/96

PSC selects science team leaders 2/1/96 2/1/96 1/15/96 1/16/96

PSC selects mathematics team leaders 2/1/96 2/1/96 1/16/96 1/16/96

Blue dot rosters (all types) 2/1/96 2/1/96 1/26/96 2/12/96

Wave 2 materials due to supervisors 2/2/96 2/2/96 2/1/96 2/2/96

Scoring calendar item by item (science) 2/3/96 2/6/96 1/15/96 5/31/96

Wave 3 addresses from Westat 2/7/96 2/7/96 2/7/96 2/7/96

Segment 1 day 25% scorers hired 2/12/96 2/12/96 2/7/96 2/7/96

Wave 3 packing list/mailing labels to packaging 2/12/96 2/12/96 2/9/96 2/13/96

Packaging/ship Wave 3 materials 2/12/96 2/19/96 2/19/96 2/20/96

Pre-range finding paper selection – mathematics 2/12/96 3/8/96 2/5/96 3/8/96

Pre-range finding paper selection – science 2/12/96 3/15/96 2/5/96 3/8/96

Segment 1 day 50% scorers hired 2/19/96 2/19/96 2/9/96 2/9/96

Segment 1 day 75% scorers hired 2/26/96 2/26/96 2/15/96 2/15/96

Wave 3 materials due in supervisors 2/26/96 2/26/96 2/26/96 2/27/96

PSC selects mathematics table leaders 3/1/96 3/1/96 2/1/96 2/28/96

PSC selects science table leaders 3/1/96 3/1/96 2/1/96 2/28/96

Segment 2 day 25% scorers hired 3/1/96 3/1/96 2/8/96 2/8/96

Segment 3 day 25% scorers hired 3/1/96 3/1/96 2/8/96 2/8/96

Segment 3 evening 25% scorers hired 3/1/96 3/1/96 2/20/96 2/20/96

Segment 2 evening 25% scorers hired 3/1/96 3/1/96 2/22/96 2/22/96

Segment 1 day 100% scorers hired 3/4/96 3/4/96 3/6/96 3/6/96

Table leaders for mathematics hired 3/8/96 3/8/96 2/1/96 3/22/96

Segment 3 day 50% scorers hired 3/8/96 3/8/96 2/14/96 2/14/96

Segment 2 day 50% scorers hired 3/8/96 3/8/96 2/15/96 2/15/96

Segment 2 evening 50% scorers hired 3/8/96 3/8/96 3/6/96 3/6/96

Segment 3 evening 50% scorers hired 3/8/96 3/8/96 3/6/96 3/6/96

Mathematics and science scorers assigned to teams 3/11/96 3/11/96 3/11/96 3/11/96

Train/score mathematics - Segment 1(day shift) 3/13/96 4/3/96 3/13/96 4/5/96

Segment 2 day 75% scorers hired 3/15/96 3/15/96 3/5/96 3/14/96

Segment 3 day 75% scorers hired 3/15/96 3/15/96 3/6/96 3/6/96

Segment 2 evening 75% scorers hired 3/15/96 3/15/96 3/14/96 3/14/96

Segment 3 evening 75% scorers hired 3/15/96 3/15/96 3/14/96 3/14/96

Train/score science - Segment 1 (day shift) 3/18/96 4/5/96 3/18/96 4/5/96

Segment 2 evening 100% scorers hired 3/22/96 3/22/96 3/25/96 3/25/96

Segment 3 evening 100% scorers hired 3/22/96 3/22/96 3/25/96 3/25/96

Segment 2 day 100% scorers hired 3/22/96 3/22/96 4/1/96 4/1/96

Segment 3 day 100% scorers hired 3/22/96 3/22/96 4/1/96 4/1/96

Table leaders for science hired 4/5/96 4/5/96 2/12/96 2/28/96

(continued)
126

Table 6-4 (continued)

NCS Schedule for the 1996 NAEP Assessment

Task

Planned

Start

Planned

Finish

Actual

Start

Actual

Finish

MAIN ASSESSMENT (continued)

Train/score mathematics - Segment 2 (evening shift) 4/8/96 5/2/96 4/8/96 5/2/96

Train/score science - Segment 2 (evening shift) 4/8/96 5/2/96 4/8/96 5/2/96

Train/score mathematics - Segment 2 (day shift) 4/8/96 5/3/96 4/8/96 5/6/96

Train/score science - Segment 2 (day shift) 4/8/96 5/3/96 4/8/96 5/3/96

Mathematics through clean post 4/14/96 4/14/96 4/14/96 4/29/96

Science through clean post edit 4/28/96 4/28/96 4/26/96 4/26/96

Bilingual mathematics scoring 5/2/96 5/3/96 5/2/96 5/3/96

Grade 8 mathematics weights 5/4/96 5/6/96 5/3/96 5/3/96

Grade 4 mathematics weights 5/4/96 5/6/96 5/9/96 5/9/96

Grade 4 science data tape sent to ETS 5/31/96 5/31/96 5/30/96 5/30/96

School characteristics and policies questionnaires data

tape shipped to ETS 7/11/96 7/12/96 7/11/96 7/11/96

Teacher questionnaire data tape shipped to ETS 7/18/96 7/19/96 7/19/96 7/24/96

SD/LEP questionnaire data shipped to ETS 7/26/96 7/29/96 8/2/96 8/2/96

Grade 12 science data tape sent to ETS 6/7/96 6/7/96 6/4/96 6/4/96

Grade 12 advanced science data tape sent to ETS 6/7/96 6/7/96 6/10/96 6/10/96

Grade 8 science data tape sent to ETS 5/31/96 5/31/96 6/26/96 6/26/96

6.1.1 Innovations for 1996

Much of the information necessary for documentation of accurate sampling and for calculating

sampling weights is collected on the Administration Schedules that, until 1993, were painstakingly filled

out by hand by Westat administrative personnel. In 1994, for the first time, much of the work was

computerizedbooklets were preassigned and booklet ID numbers were preprinted on the Administration

Schedule. When Westat personnel received the documents, they filled in only the “exception”

information. This new method also permitted computerized updating of information when the

Administration Schedules were received at NCS, eliminating the need to sort and track thousands of

pieces of paper through the processing stream.

The introduction of image processing and image scoring further enhanced the work of NAEP.

Image processing and scoring were successfully piloted in a side-by-side study conducted during the 1993

NAEP field test, and so became the primary processing and scoring methods for the 1994 and 1996

assessments. Image processing allowed the automatic collection of handwritten demographic data from

the administrative schedules and the student test booklet covers through intelligent character recognition

(ICR). This service was a benefit to the jurisdictions participating in NAEP because they were able to

write rather than grid certain informationa reduction of burden on the schools. Image processing also

made image scoring possible, eliminating much of the time spent moving paper as part of the scoring

process. The images of student responses to be scored were transmitted electronically to the scoring

center, located at a separate facility from where the materials were processed. This process enhanced the

reliability and monitoring of scoring and allowed both NCS and ETS to focus attention on the intellectual

process of scoring student responses.
127

6.2 PRINTING

For the 1996 assessment, 255 unique documents were designed. NCS printed more than

1,900,000 booklets and forms, totaling over 58 million pages. Printing preparations began with the design

of the booklet covers in June 1995. This was a collaborative effort involving staff from ETS, Westat and

NCS. Since the goal was to design one format for use with all of the booklets, necessary data elements to

be collected for the different assessment types had to be agreed upon. After various iterations, the cover

design was finalized.

In a similar collaboration with ETS and Westat, NCS prepared administration schedules and

questionnaire rosters. The camera-ready copies for these documents were created and edited using NCS

Design ExpertTM software.

Printing of the NAEP documents began with the documents for the long-term trend assessments.

These included the 26 long-term trend assessment booklets, the Administration Schedule, the excluded

student questionnaire, three school characteristics and policies questionnaires, and the roster of

questionnaires. All materials for the long-term trend assessments were printed by September 19, 1995.

The printing of assessment booklets, questionnaires and tracking forms for the main and state assessments

followed. Printing of these documents was complete by December 11, 1995.

Details of the printing procedures are given in the Report of Processing and Professional Scoring

Activities (National Computer Systems, 1996).

6.3 PACKAGING AND SHIPPING

The distribution effort for the 1996 NAEP assessment involved packaging and mailing documents

and associated forms and materials to the Westat supervisors for the main and long-term trend

assessments. The NAEP Materials Distribution System (MDS), initially developed by NCS in 1990 to

control shipments to the schools and supervisors, was utilized again in 1996. Files in the MDS system

contained the names and addresses for shipment of materials, scheduled assessment dates, and a listing of

all materials available for use by a participant in a particular subject area. Changes to any of this

information were made directly in the MDS file either manually or via file updates provided by Westat.

Details of the accountability system and on-line bundle assignment and distribution system utilized for

NAEP are given in the Report of Processing and Professional Scoring Activities.

The bar code technology introduced by NCS in the 1990 assessments continued to be utilized in

document control. To identify each document, NCS utilized a unique ten-digit numbering system that

consisted of the three-digit booklet number or form type, a six-digit sequential number, and a check digit.

Each form was assigned a range of ID numbers. Bar codes reflecting this ID number were applied to the

front cover of each document by NCS bar code processes and high-speed ink jet printers.

Once all booklets from a subject area were bar coded, they were spiraled and bundled into groups

of 11 documents. For main samples in mathematics and science (done concurrently with the State

Assessment samples in mathematics and science), NCS spiraled the booklets according to the pattern

dictated by ETS in the bundle maps. Booklets were spiraled in such a manner that each booklet appeared

in the first position in a bundle approximately the same number of times and the booklets were evenly

distributed across the bundles. This assured that sample sizes of individual booklet types would not be

jeopardized if entire bundles were not used. Since the mathematics and science estimation and advanced
128

booklet bundles contained only one booklet type, these were bundled into groups of 11. The mathematics

bilingual booklets were bundled in groups of three.

Initially 5,161 individual sessions were shipped for the 1996 NAEP assessments. Approximately

600 additional shipments of booklets and miscellaneous materials were sent. All outbound shipments

were recorded in the NCS Outbound Mail Management system. This was accomplished by having a bar

code containing the school number on each address label. This bar code was read into the system, which

determined the routing of the shipment and the charges. Information was recorded in a file on the system

which, at the end of each day, was transferred by a PC upload to the mainframe. A computer program

could then access information to produce reports on all shipments sent, regardless of the carrier used.

These reports helped NCS phone staff trace shipments for Westat supervisors and assessment

administrators.

A toll-free telephone line was maintained for supervisors and school administrators to request

additional materials for the National assessments. To process a shipment, NCS phone staff asked the

caller for information such as primary sampling unit (PSU), school ID, assessment type, city, state, and

ZIP code. This information was then entered into the on-line short shipment system and the school’s

mailing address would be displayed on the screen to verify with the caller. The system allowed NCS staff

to change the shipping address for individual requests. The clerk proceeded to the next screen that

displayed the materials to be selected. After the requested items, due date and method of shipment were

entered, the system produced a packing list and mailing labels. Phone staff also took phone calls

concerning initial shipment delivery dates, tracing a shipment, and questions concerning NAEP.

Approximately 750 calls were received regarding the 1996 NAEP assessments.

Further information regarding packaging and shipping is provided in the Report of Processing

and Professional Scoring Activities (National Computer Systems, 1996).

6.4 PROCESSING

6.4.1 Overview

The following describes the various stages of work involved in receiving and processing the

documents used in the 1996 assessment. NCS staff created a set of predetermined rules and specifications

for the processing departments within NCS to follow. Project staff performed a variety of procedures on

materials received from the assessment administrators before releasing these materials into the NCS

NAEP processing system. Control systems were used to monitor all NAEP materials returned from the

field. The NAEP Process Control System (PCS) contained the status of sampled schools for all sessions

and their scheduled assessment dates. As materials were returned, the PCS was updated to indicate receipt

dates, to record counts of materials returned, and to document any problems discovered in the shipments.

As documents were processed, the system was updated to reflect processed counts. NCS report programs

were utilized to allow ETS, Westat, and NCS staff to monitor the progress in the receipt control

operations.

An “alert” process was used to record, monitor, and categorize all discrepant or problematic

situations. Throughout the processing cycle, alert situations were either flagged by computer programs or

identified during clerical check-in procedures. Certain alerts, such as missing demographic information

on the administration schedule, were resolved by opening staff retrieving the information from booklet

covers. These alerts, known as “Information Alerts,” were recorded directly into the PCS system by

opening personnel, eliminating the need for paper documentation. Since these problem situations were
129

categorized and tallied as they were key-entered into the PCS system, project staff were able to provide

timely reporting on clerical-type errors made during test administration. Alert situations that could not be

resolved by opening personnel were described on alert forms that were forwarded to project personnel for

resolution. Once resolved, the problems and resolutions were recorded online in the PCS system.

NCS’s Work Flow Management System was used to track batches of student booklets through

each processing step, allowing project staff to monitor the status of all work in progress. It was also used

by NCS to analyze the current work load, by project, across all work stations. By routinely monitoring

these data, NCS’s management staff was able to assign priorities to various components of the work and

to monitor all phases of the data receipt and processing.

6.4.2 Document Receipt

Shipments were to be returned to NCS packaged in their original boxes. As mentioned earlier,

NCS packaging staff applied a bar code label to each box indicating the NAEP school ID number. When a

shipment arrived at the NCS dock area, this bar code was scanned into a personal computer file, and the

shipment was forwarded to the receiving area. The file was then transferred to the mainframe and the

shipment receipt date was applied to the appropriate school within the PCS system, providing the status

of receipts regardless of any processing delays. Each receipt was reflected on the PCS status report

provided to the NCS receiving department and supplied to Westat via electronic file transfer and in hardcopy format. ETS also received a hard copy.

The PCS file could be manually updated to reflect changes. Receiving personnel also checked the

shipment to verify that the contents of the box matched the school and session indicated on the label.

Each shipment was checked for completeness and accuracy. Any shipment not received within two days

of the scheduled assessment date was flagged in the PCS system and annotated on the PCS report. The

administration status of these delayed shipments was checked and in some cases a trace was initiated on

the shipment.

A new requirement for NCS was to open all shipments within forty-eight hours of their receipt

and to key-enter preliminary processing information into the PCS system from the Administration

Schedule. The preliminary information was written on the Administration Schedule by Westat assessment

administrators and consisted of the following:

• School number

• Session number

• Original test date

• Total number assessed

This preliminary information, used to provide Westat with timely student response rates, was updated

with actual data when materials passed through processing error free. A completeness flag was also

applied to the PCS file by NCS opening staff if any part of the shipment was missing.

If multiple sessions were returned in one box, the contents of the package were separated by

session. The shipment was checked to verify that all booklets preprinted or handwritten on the

Administration Schedule were returned with the shipment and that all administration codes matched from

booklet cover to the Administration Schedule. If discrepancies were discovered at any step in this

process, the receiving staff issued an alert to facilitate tracking. If the administrator indicated that a makeup session was being held the documents were placed on holding carts until the make-up session
130

documents arrived. If no make-up session was indicated, Westat was contacted for the status of the

missing materials. If the missing materials were to be returned, the documents already received were held

until that time. If the materials were not being returned, processing continued and the appropriate

administration code was applied to the Administration Schedule.

6.4.3 Batching and Scanning Documents

Once all booklets listed on the Administration Schedule for a session were verified as present, the

entire session (both the Administration Schedule and booklets) was batched by grade level and session

type. Each batch was assigned a unique batch number. This number, created on the Image Capture

Environment system for all image-scannable documents and on the Work Flow Management system for

all key-entry and OMR-scannable documents2

, facilitated the internal tracking of the batches and allowed

departmental resource planning. All other scannable documents (School Characteristics and Policies

Questionnaires, Teacher Questionnaires, Students with Disabilities/Limited English Proficiency

[SD/LEP] Questionnaires, and rosters) were batched by document type in the same manner.

Because all assessment booklets were image-scannable, batch numbers for these documents were

created on the Image Capture Environment system. Sessions were sorted by grade level and automatically

uploaded to the Work Flow Management system after batch creation. The Administration Schedule for

these document types was used as a session header within a batch.

When batching mathematics documents, NCS needed to allow for having both image-scannable

and key-entry documents present in the same session, or having booklets listed on the Administration

Schedule that would not be present in processing. This was due to the testing accommodations of largeprint and Braille that were key-entry documents.

Large-print booklets had to be processed separately from the Administration Schedule and

scannable booklets in their session. A key-entry session header was created for these booklets that

contained the school ID number and session code from the Administration Schedule. Long-term trend

reading/writing booklets were processed through key-entry with the same type of key-entry session

header. The Administration Schedules from reading/writing sessions were processed in an

Administration-Schedule-only batch through the image scanning system. After the session that a largeprint booklet came with passed through image processing, session information was rejoined within the

processing computer programs. The same computerized match occurred with Trend reading/writing

materials once the Administration-Schedule-Only batch that contained a session’s administration

schedule passed through processing.

6.4.4 Questionnaires

The long-term trend assessments used one roster to account for all questionnaires. The roster of

questionnaires recorded the distribution and return of the School Characteristics and Policies

Questionnaires and the Excluded Student Questionnaires.

The main assessments utilized one roster to document and track the School Characteristics and

Policies Questionnaires and the Students with Disabilities/Limited English Proficiency (SD/LEP)

Questionnaire. In addition, the main and state assessments used the roster of Teacher Questionnaires to

record the distribution and return of Teacher Questionnaires.

 

2

 OMR is the acronym for Optical Mark Reading.
131

Some questionnaires may not have been available for return with the shipment. These were

returned to NCS at a later date in an envelope provided for that purpose. The questionnaires were

submitted for scanning as sufficient quantities became available for batching.

Receipt of the questionnaires was entered into the system using the same process as was used for

the Administration Schedules described in previous sections. The rosters were grouped with other rosters

of the same type from other sessions, and a batch was created on the Image Capture Environment system.

The batch was then forwarded to scanning where all information on the rosters was scanned into the

system.

6.4.5 Booklet Accountability

NCS used a sophisticated booklet accountability system to track all distributed booklets. Prior to

the distribution of NAEP materials, unique booklet numbers were read by bundle into a file. Specific

bundles were then assigned to particular supervisors or schools. This assignment was recorded in the

NAEP Materials Distribution System.

When shipments arrived at NCS from the field, all used booklets were submitted for processing

and a “processed documents” file was maintained. Unused booklets were submitted for security scanning

where booklet ID bar codes were read and recorded into a separate file. This file and the “processed

documents” file were later compared to the original bundle security file for individual booklet matching.

A list of unmatched booklet IDs was printed in a report used to confirm non-receipt of individual

booklets. At the end of the assessment period, supervisors returned all unused materials. These booklet

IDs were also read by the bar-code scanner and added to the bundle security file. All unused materials

received were then inventoried and sent to the NCS warehouse for storage while awaiting authorization

from ETS to salvage them.

6.4.6 Data Entry

The data entry process was the first point at which booklet-level data were directly available to

the computer system. Depending on the NAEP document, one of three methods was used to transcribe

NAEP data to a computerized form. The data on scannable documents were collected using NCS opticalscanning equipment that also captured images of the constructed-response items and ICR fields.

Nonscannable materials were keyed through an interactive online system. In both of these cases, the data

were edited and suspect cases were resolved before further processing.

All student booklets, questionnaires, and control documents were scannable. Throughout all

phases of processing, the student booklets were batched by grade and session type. The scannable

documents were then transported to a slitting area where the folded and stapled spine was removed from

the document. This process utilized an “intelligent slitter” to prevent slitting the wrong side of the

document. The documents were jogged by machine so that the registration edges of the NAEP documents

were smoothly aligned, and the stacks were then returned to the cart to be scanned.

During the scanning process, each scannable NAEP document was uniquely identified using a

print-after-scan number consisting of the scan batch number, the sequential number within the batch, and

the bar code ID of the booklet. These numbers were printed on each sheet of each document as it exited

the scanner. This permitted the data editors to quickly and accurately locate specific documents during
132

the editing phase. The print-after-scan number remained with the data record, providing a method for easy

identification and quick retrieval of any document.

The data values were captured from the booklet covers and Administration Schedules and were

coded as numeric data. Unmarked fields were coded as blanks and editing staff were alerted to missing or

uncoded critical data. Fields that had multiple marks were coded as asterisks (*). The data values for the

item responses and scores were returned as numeric codes. The multiple-choice single response format

items were assigned codes depending on the position of the response alternative; that is, the first choice

was assigned the code “1,” the second “2,” and so forth. The mark-all-that-apply items were given as

many data fields as response alternatives; the marked choices were coded as “1” while the unmarked

choices were recorded as blanks. The images of constructed-response items were saved as a digitized

computer file. The area of the page that needed to be clipped was defined prior to scanning through the

document definition process. The fields from unreadable pages were coded “X” as a flag for resolution

staff to correct. In addition to capturing the student responses, the bar code identification numbers used to

maintain process control were decoded and transcribed to the NAEP computerized data file.

As the scanning program completed scanning each stack, the stack was removed from the output

hopper and placed in the same order they were scanned on the output cart. The next stack was removed

from the input cart and placed into the input hopper, after which the scanning resumed. When the

operator had completed processing the last stack of the batch, the program was terminated. This closed

the dataset that automatically became available for the data validation (edit) process. The scanned

documents were then forwarded to a holding area in case they needed to be retrieved for resolution of edit

errors.

NCS again used the ICR engine to read various hand and machine printing on the front cover of

the assessment and supervisor documents for the 1996 NAEP assessments. Some information from

scannable student documents, such as the Administration Schedule, the roster of questionnaires, and some

questions in the School Characteristics and Policies Questionnaires, were read by the ICR engine and

verified by an online key-entry operator. In all, the ICR engine read approximately 15 million characters.

The ICR engine saved NAEP field staff and school personnel a significant amount of time because they

no longer had to enter these data by gridding rows and columns of data.

NCS also implemented new programs that allowed the scanners to read imprinted codes, known

as 2-out-of-5 codes, that were printed via a Xerox 4280 printer on the Administration Schedule. These

2-out-of-5 codes were imprinted at the same time the booklet ID numbers were printed on the

Administration Schedule and identified which booklet IDs were listed on that document. When the

scanning programs were unable to translate the 2-out-of-5 codes (thereby identifying the booklet ID

numbers on the document) image clips of the booklet ID numbers were displayed to online editing staff

for verification. This eliminated a significant amount of online editing time needed to process the NAEP

assessments.

To provide another quality check on the image scanning and scoring system, NCS staff stamped

blank booklets with a rubber stamp and assigned these booklets mock scores from the valid range. Each

unique item type scored via the image system had two quality control stamps per valid score. An example

of the stamp used is given below.

IMAGE SCORING

QUALITY ASSURANCE

SAMPLE
133

The quality control booklets were batched and processed together with student documents of the

same type. Because all of a specific item were batched together for transmission to the scoring facility,

the quality control-stamped responses were integrated with the student responses and transmitted

simultaneously to the scoring facility. During the scoring process, both student responses and the quality

control items were randomly displayed so scores could be applied.

When a person who was scoring responses (reader) later saw the quality control sample on the

monitor during scoring, he or she was to notify the team leader, who confirmed the score assigned by the

reader was the score listed on the sample. The quality control booklets were included in the pool of all

items to be drawn from for the 25 percent reliability rescore.

All image quality-assurance documents were created prior to the beginning of scoring and all predetermined score points were used. Because during the process of scoring, valid score points can be

changed or dropped completely, NCS provided ETS with documentation explaining what quality control

documents were produced and which score points on these items were no longer valid. When an image

quality control stamp was displayed to a reader that contained a score point that was no longer valid, the

reader gave the response a score point of zero.

A process of key entry and verification was used to make corrections to the non-scannable

long-term trend reading/writing documents and large print booklets. Teacher questionnaire and SD/LEP

questionnaire information was also corrected using key-entry methods. NCS used the Falcon system to

enter this data. The Falcon system is an on-line data-entry system designed to replace most methods of

data input such as keypunch, key-to-disk, and many of the microcomputer data-entry systems. The

terminal screens were designed to enhance operator speed and convenience. The fields to be entered were

titled to reflect the actual source document. Therefore, all key-entry fields were specific to the NAEP

student documents or questionnaire types being keyed.

6.4.7 Data Validation

Each dataset produced by the scanning system contained data for a particular batch. These data

had to be validated (or edited) for type and range of response. The data-entry and resolution system used

was able to simultaneously process a variety of materials from all age groups, subject areas, control

documents, and questionnaires as the materials were submitted to the system from scannable and nonscannable media.

The data records in the scan file were organized in the same order in which the paper materials

were processed by the scanner. A record for each batch header preceded all data records for that batch.

The document code field on each record distinguished the header record from the data records.

When a batch-header record was read, a pre-edit data file and an edit log were generated. As the

program processed each record within a batch from the scan file, it wrote the edited and reformatted data

records to the pre-edit file and recorded all errors on the edit log. The data fields on an edit log record

identified each data problem by the batch sequence number, booklet serial number, section or block code,

field name or item number, and data value. After each batch had been processed, the program generated a

listing or online edit file of the data problems and resolution guidelines. An edit log listing was printed at

the termination of the program for all non-image documents. Image “clips” requiring editing were routed

to online editing stations for those documents that were image scanned.

As the program processed each data record, it first read the booklet number and checked it against

the session code for appropriate session type. Any mismatch was recorded on the error log and processing
134

continued. The booklet number was then compared against the first three digits of the student

identification number. If they did not match, a message was written on the error log. The remaining

booklet cover fields were read and validated for the correct range of values. The school codes had to be

identical to those on the PCS record. All data values that were out of range were read “as is” but were

flagged as suspect. All data fields that were read as asterisks (*) were recorded on the edit log or online

edit file.

Document definition files described each document as a series of blocks that in turn were

described as a series of items. The blocks in a document were transcribed in the order that they appeared

in the document. Each block’s fields were validated during this process. If a document contained suspect

fields, the cover information was recorded on the edit log along with a description of the suspect data.

The edited booklet cover was transferred to an output buffer area within the program. As the program

processed each block of data from the dataset record, it appended the edited data fields to the data already

in this buffer.

The program then cycled through the data area corresponding to the item blocks. The task of

translating, validating, and reporting errors for each data field in each block was performed by a routine

that required only the block identification code and the string of input data. This routine had access to a

block definition file that had, for each block, the number of fields to be processed, and, for each field, the

field type (alphabetic or numeric), the field width in the data record, and the valid range of values. The

routine then processed each field in sequence order, performing the necessary translation, validation, and

reporting tasks.

The first of these tasks checked for the presence of blanks or asterisks (*) in a critical field. These

were recorded on the edit log or online edit file and processing continued with the next field. No action

was taken on blank fields for multiple-choice items because the asterisk code indicated a non-response.

The field was validated for range of response, and any values outside of the specified range were recorded

on the edit log or online edit file. The program used the item-type code to make a further distinction

among constructed-response item scores and other numeric data fields.

Moving the translated and edited data field into the output buffer was the last task performed in

this phase of processing. When the entire document was processed, the completed string of data was

written to the data file. When the program encountered the end of a file, it closed the dataset and

generated an edit listing for non-image and key-entered documents. Image-scanned items that required

correction were displayed at an online editing terminal.

6.4.8 Editing for Non-Image and Key-Entered Documents

Throughout the system, quality procedures and software ensured that the NAEP data were

correct. All student documents on the Administration Schedule were accounted for, as receipt control

personnel checked that the materials were undamaged and assembled correctly. The machine edits

performed during data capture verified that each sheet of each document was present and that each field

had an appropriate value. All batches entered into the system, whether key-entered or machine-scanned,

were edited for errors.

Data editing took place after these checks. This consisted of a computerized edit review of each

respondent’s document and the clerical edits necessary to make corrections based upon the computer edit.

This data-editing step was repeated until all data were correct.
135

The first phase of data editing was designed to validate the population and ensure that all

documents were present. A computerized edit list, produced after NAEP documents were scanned or key

entered, and all the supporting documentation sent from the field were used to perform the edit function.

The hard-copy edit list contained all the vital statistics about the batch: number of students, school code,

type of document, assessment code, suspect cases, and record serial numbers. Using these inputs, the data

editor verified that the batch had been assembled correctly and that each school number was correct.

During data entry, counts of processed documents were generated by type. These counts were

compared against the information captured from the Administration Schedules. The number of assessed

and absent students processed had to match the numbers indicated on the PCS.

In the second phase of data editing, experienced editing staff used a predetermined set of

specifications to review the field errors and record necessary corrections to the student data file. The

same computerized edit list used in phase one was used to perform this function. The editing staff

reviewed the computer-generated edit log and the area of the source document that was noted as being

suspect or as containing possible errors. The composition of the field was shown in the edit box. The

editing staff checked this piece of information against the NAEP source document. At that point, one of

the following took place:

Correctable error. If the error was correctable by the editing staff as per the editing

specifications, the correction was noted on the edit log for later correction via key-entry.

Alert. If an error was not correctable as per the specifications, an alert was issued to

NAEP project staff for resolution. Once the correct information was obtained, the

correction was noted on the edit log for key-entry correction.

Non-correctable error. If a suspected error was found to be correct as stated and no

alteration was possible according to the source document and specifications, the

programs were tailored to allow this information to be accepted into the data record. No

corrective action was taken.

The corrected edit log was then forwarded to the key-entry staff for processing. When all

corrections were entered and verified for a batch, an extract program pulled the corrected records into a

mainframe dataset. At this point, the mainframe edit program was initiated. The edit criteria were again

applied to all records. If there were further errors, a new edit listing was printed and the cycle was

repeated.

When the edit process produced an error-free file, the booklet ID number was posted to the

NAEP tracking file by age, assessment, and school. This permitted NCS staff to monitor the NAEP

processing effort by accurately measuring the number of documents processed by form. The posting of

booklet IDs also ensured that a booklet ID was not processed more than once.

6.4.9 Data Validation and Editing of Image-Processed Documents

The paper edit log for key-entered documents was replaced by online viewing of suspect data for

all image-processed documents. For rapid resolution, the edit criteria for each item in question appeared

on the screen along with the suspect item. Corrections were made immediately. The system employed an

edit/verify system that ultimately meant that two different people viewed the same suspect data and

operated on it separately. The “verifier” made sure the two responses (one from either the entry operator

or the ICR engine) were the same before the system accepted that item as being correct. The verifiers
136

could either overrule or agree with the original correction made if the two did not match. If the editor

could not determine the appropriate response, he or she escalated the suspect situation to a supervisor. For

errors or suspect information that could not be resolved by supervisory staff, a product-line queue was

created. This allowed supervisors to escalate edits to project staff for resolution. By having this productline queue, project staff were able to quickly locate edit clips within the image system, speeding up the

resolution process.

Once an entire batch was through the edit phase, it became eligible for the count-verification

phase. The Administration Schedule data were examined systematically for booklet IDs that should have

been processed (assessed administration codes). All documents under that Administration Schedule were

then inspected to ensure that all of the booklets were included.

With the satisfactory conclusion of the count-verification phase, the edited batch file was

uploaded to the mainframe, where it went through yet another edit process. A paper edit log was produced

and, if errors remained, was forwarded to another editor. When this paper edit was satisfied, the PCS and

Workflow Management system were updated. Because there was a possible time lag between a clean edit

in the image system and a clean edit in the mainframe systems, the batch was not archived until 48 hours

after the image edit phase was completed.

6.4.10 Data Transmission

Due to the rapid pace of scoring on an item-by-item basis, the NCS scoring specialists found it

necessary to continually monitor the status of work available to the readers and plan the scoring schedule

several weeks in advance. On Wednesday of each week, the NCS performance assessment specialist in

charge of each subject area planned the next two weeks’ schedule. That information was then provided to

the person in charge of downloading data to the scoring center. By planning the scoring schedule two

weeks in advance, the scoring specialists were able to ensure that readers would have sufficient work for

at least one week, after which the next download would occur to supplement the volume of any unscored

items and add an additional week’s work to the pool of items to score. Additionally, by scheduling two

weeks’ data transmission, flexibility was added to the scoring schedule, making it possible to implement

last-minute changes in the schedule once the items had been delivered to the scoring center. Depending

on the number of items to be transmitted, the actual downloading was conducted on Friday or was divided

into two smaller sessions for Thursday and Friday download. By the first week of May 1996, there was

sufficient space on the scoring servers to load all remaining unscored items to the scoring center.

Delivery of data to the scoring center was accomplished via several T1 transmission lines linking

the mainframe computers and the NAEP servers at the document-scanning site in the NCS main facility

with the scoring servers dedicated to distributing work to the professional readers at the scoring center.

The actual task of scheduling items for downloading was accomplished using a code written by the Image

Software Development team. This code enabled the person scheduling the download to choose a team of

readers and select the scheduled items from a list of all items that that team would be scoring throughout

the scoring project. This process was repeated for all teams of readers until all anticipated work was

scheduled. Once this task was completed, the scheduled job was tested to determine if there was

sufficient free disk space on the servers at the scoring center. If for any reason sufficient disk space was

not available, scheduled items could be deleted from the batch individually or as a group until the

scheduled batch job could accommodate all items on the available disk space at the scoring center. Once

it was determined that sufficient disk space was available, transmission of student responses commenced.

Data transmission was typically accomplished during off-shift hours to minimize the impact on systemload capacity.
137

6.5 DATA DELIVERY

The 1996 NAEP data collection resulted in several classes of data files  student, school,

teacher, weights, SD/LEP student, excluded student questionnaire data for long-term trends,

student/teacher match, and student-response information. Student-response information included response

data from all assessed students in 1996. Data resolution activities occurred prior to the submission of data

files to ETS and Westat to resolve any irregularities that existed. This section details additional steps

performed before creating the final data files to ensure capture of the most complete and accurate

information.

An important quality-control component of the image-scoring system was the inclusion, for

purposes of file identification, of an exact copy of the student edit record, including the student booklet

ID number, with every image of a student’s response to a constructed-response item. These edit files also

remained in the main data files residing on the NCS mainframe computer. By doing this, exact matching

of scores assigned to constructed-response items and all other data for each individual student was

guaranteed, since the booklet ID for each image was part of every image file. This ensured scores were

applied to the correct student’s record on the mainframe.

When all of the responses for an individual item had been scored, the system automatically

submitted all item scores assigned during scoring, along with their edit records, to a queue to be

transmitted to the mainframe. Project staff then initiated a system job to transmit all scoring data to be

matched with the original student records on the mainframe. A custom edit program matched the edit

records of the scoring files to those of the original edit records on the mainframe. As matches were

confirmed, the scores were applied to those individual files. After completion of this stage, all data

collected for an individual student was located in one single and complete record/file identified by the

edit record.

NCS processed the SD/LEP student questionnaires via OMR scanning. Edits performed on the

questionnaires assured that responses to questions fell within the valid range for that question. SD/LEP

questionnaires were then matched to a student record. SD/LEP Questionnaires that were not matched to a

student document were cross-referenced with the corresponding Administration Schedule, roster of

questionnaires, and student data files to correct, if necessary, the information needed to result in a match.

In 1996, NCS continued to use ICR technology to capture percentage figures written by school

personnel directly in boxes on the school characteristics and policies questionnaires rather than requiring

the school official to grid ovals in a matrix. The data were then verified by an edit operator.

To achieve the best possible student/teacher match rate, the same processes that were followed in

previous cycles were used in 1996. The first step was to identify teacher questionnaires not returned to

NCS for processing so as to exclude from the matching process the students of these teachers. Student

identification numbers that were not matched to a teacher questionnaire were cross-referenced with the

corresponding Administration Schedule and roster of teacher questionnaires to verify (and change, if

necessary) the teacher number, teacher period, and questionnaire number recorded on these control

documents. The NAEP school numbers listed on the roster of questionnaires and teacher questionnaire

were verified and corrected, if necessary. Once these changes were made, any duplicate teacher numbers

existing within a school were, if possible, cross-referenced for resolution with the roster of

questionnaires. Since this information was located together on a single, central control document, the

ability to match and resolve discrepant or missing fields was simplified.
138

After all data-processing activities were completed, data cartridges and/or diskettes were created

and shipped via overnight delivery to ETS and/or Westat. A duplicate archive file is maintained at NCS

for security and backup purposes.

6.6 MISCELLANEOUS

6.6.1 Storage of Documents

After batches of image-scanned documents had successfully passed the editing process, they were

sent to the NCS warehouse for storage. The long-term trend reading/writing booklets were sent to the

NCS scoring center to be scored on paper and to be used for holistic and mechanics scoring. After all

scoring had been completed for the long-term trend booklets, they were also sent to the warehouse for

storage. Due to the large number of rescore projects done with NAEP material, the documents were

unspiraled and sequenced by grade and booklet type after all of the processing/scoring was completed.

This will allow for efficient document retrieval to fill requests for specific booklet types for future

scoring projects. Unspiraled and sequenced booklets were then assigned a new inventory number by

grade and booklet type and sent back to the warehouse for storage. The storage locations of all documents

were recorded on the inventory control system. Unused materials were sent to temporary storage to await

completion of the entire assessment. Once the assessment was complete, NCS received authorization

from ETS to salvage unused materials after determining that a sufficient quantity of each form type was

retained in permanent archive.

6.6.2 Quality-Control Documents

ETS requires that a random sample of booklets and the corresponding scores/scoring sheets be

pulled for an additional quality-control check. For image-scanned documents, a scoring sheet is not used,

so ETS uses scores sent to them on a data tape to verify the accuracy of applied scores. During the

scoring of mathematics, a selected number of image-processed booklets were paper scored. If any of the

random sample of mathematics booklets used for paper scoring were selected as quality-control

documents, the scoring sheet was also sent to ETS. For non-scannable trend reading/writing booklets and

for the trend mathematics/science booklets that were scored via paper, both the booklet and its

corresponding score sheet were sent. All of these documents were selected prior to sending the booklets

to storage and were then sent to ETS to verify the accuracy and completeness of the data. A random

sample of all the questionnaires used in the 1996 NAEP assessment was also sent to ETS.

6.6.3 Alert Analysis

Table 6-5 identifies the different types of alerts to problems that were encountered in the

processing of NAEP data. For the 1996 main and long-term trend assessments, there was a total of 230

alerts; for the State Assessment, there was a total of 3,812 alerts.

Discrepancies were found in the receiving process that did not require an alert to be issued to

Westat. They did require a great deal of effort by the opening staff to resolve in order to provide the most

complete and accurate information. These are referred to as “info alerts.” These were categorized and

codes were assigned to them. They are listed in the left-hand column of Table 6-5.
139

Even though receipt-control staff were well trained in the resolution of many situations, there

were some problems that required resolution by NCS NAEP product line staff. These are referred to as

“problem alerts.” The various types of problem alerts were also categorized and coded. They are listed in

the right-hand column of Table 6-5. For any unusual situations, Westat was contacted to help with the

resolution of the alert.

Table 6-5

Alerts for the 1996 National and State Assessments

Information Alerts Problem Alerts

Code 52 not written on Administration

Schedules

Change of Administration Codes-A/S or

Booklets

The yes/no box not gridded on Rosters Incorrect Rosters/Questionnaires

Session Number not on Administration Schedules Administration Notes/Writing on Covers

Administration Codes not on A/S; but on booklets Duplicate Student / Booklet Number/

Administration Schedule

Administration Codes not on booklets; but on A/S All material not returned

Items returned for Westat Affected Testing - Problem

Writing on booklet covers Transcribed page(s) for student booklet(s)

Other Processed as is

Involves Inclusion Check List

Other

A/S = Administration Schedules

141

Chapter 7

PROFESSIONAL SCORING1

Patrick B. Bourgeacq, Bradley Thayer, and Timothy Robinson

National Computer Systems

7.1 INTRODUCTION

The 1996 national assessment required the scoring of constructed responses in mathematics and

science at grades 4, 8, and 12. Long-term trend assessments for 9-, 13-, and 17-year-olds in reading,

writing, mathematics and science continued at levels comparable to previous assessment years. In all,

over nine million constructed responses for the 1996 national and state assessments were scored.

More than 300 professional readers split between a full-time day shift and part-time evening shift

were hired for scoring. Veteran scorers were mixed with new hires screened for their ability to score

constructed responses, providing the scoring center with excellent groups of qualified readers. Regular

full-time staff, with the help of administrative assistants, bridged the shifts to ensure quality scoring

between the two groups. For the first time in a National Assessment of Educational Progress (NAEP)

assessment, National Computer Systems (NCS) provided a significant number of trainers who worked

with Educational Testing Service (ETS) staff to train teams of scorers on many items. Also, NCS used

lead scorers for the first time in a NAEP assessment to assist the table leaders with administrative duties

and monitoring quality of scoring. The help of lead scorers made it possible to score greater volumes of

responses with teams as large as 14 scorers without any apparent compromises in the quality of scoring.

As in previous years, the image system distributed responses among teams who scored them in

an efficient manner to maximize consistency and reliability. The system also provided enhanced tools to

display images, gather data, and monitor the quantity and quality of work.

The figures and tables on the following pages summarize the scope of professional scoring for

the 1996 NAEP.

 

1

 Bradley Thayer was the NCS project manager for 1996 NAEP, Patrick Bourgeacq was the NCS project director for 1996 NAEP

scoring, and Timothy Robinson was the NCS senior processing coordinator for 1996 NAEP.
142

 Figure 7-1

Image Scoring Flow Chart

Scoring Guides

Receipt of 

Assessed 

Documents

Entry of Items and 

Scoring Guides 

into Database 

Update Scoring 

Guides

Review Sample 

Responses

Orientation Training

Review Scoring 

Guides

Score and Review 

Practice Responses 

Reliability 

Satisfactory? 

No

Scoring

Yes

Scoring

First Scoring 

of Items

Reliability Scoring 

of Items

Back-Reading of 

Responses by 

Table Leader

Reliability 

Coefficient 

Reports

No

Continue Scoring

Yes

Select Anchor, 

Training and 

Calibration 

Papers

Reliability 

Satisfactory? 

Upload 

Files 
143

Figure 7-2

Paper Scoring Flow Chart

Scoring Guides

Receipt of

Assessed

Documents

Entry of Items and

Scoring Guides

into Database

Update Scoring

Guides

Review Sample

Responses

Orientation Training

Review Scoring

Guides

Score and Review

Practice Responses

Reliability

Satisfactory?

No

Scoring

Yes

Scoring

First Scoring

of Items

Reliability Scoring

of Items

Back-Reading of

Responses by

Table Leader

Reliability

Coefficient

Reports

No

Continue Scoring

Yes

Select Anchor,

Training and

Calibration

Papers

Reliability

Satisfactory?

Batch Scoring

Sheets

Scan Scoring

Sheets

Edits?

No

Yes

Resolve

Edits

Upload

Files

Warehouse

Scoring

Sheets

Input Scores for

Paper-Based Scoring
144

7.2 LONG-TERM TREND ASSESSMENTS

7.2.1 Mathematics

Items that contributed to long-term trend in mathematics items were scored as “right,” “wrong,”

or “omitted.” The scoring criteria identified the correct or acceptable answers for each item in each

block. The scores for these items included a “0” for no response, a “1” for a correct answer, or a “2” for

an incorrect or “I don’t know” response. The reading items that appeared in the mathematics/science

booklets were scored as “attempted,” or “omitted.” This scoring consisted of merely checking to see

whether the student had responded in any way to that item, in which case the item was determined to

have been reached or attempted. The scoring here was “0” for not attempting the item (blank) or “1” for

any writing in the space provided. This includes one reading item in an age 9 math booklet and one

reading item in an age 13 math booklet. The numbers of discrete constructed-response mathematics items

can be found in Table 7-1.

Since scoring of the long-term trend mathematics items was identical to previous years, no new

training papers were needed. Preparation for scoring included copying the scoring guides from previous

assessment years and drawing samples from previous years, retrieving the booklets listed in the samples,

and printing and matching scoring sheets for those booklets.

Because the mathematics items were scored as “right,” “wrong,” or “omitted,” lengthy training

for scoring these items was unnecessary. For each component (fall, winter, and spring), a different team

was trained to follow the procedures for scoring the mathematics items and became familiar with the

scoring standards, which listed general guidelines and also the correct answer for the items in each of the

blocks. Each season, the entire scoring was done in one or two weeks at the end of the administration

period. The number of booklets processed and the number of constructed responses scored for each age

level are reported in Table 6-1 in Chapter 6.

A different team scored each age level at the time of year the age level was assessed. The

booklets arrived in sessions, so each reader scored all items in all mathematics booklet types during the

course of the project. All scorers held the same qualifications as the readers for main and state

assessments. The number of readers, table leaders, and dates of scoring are reported in Table 7-2.

To establish the consistency of scoring across years, the readers rescored a subset of the

responses from previous assessments. Samples of 350 responses to each item from the 1990 assessment

and 250 from the 1994 assessment were drawn. The Performance Assessment Scoring Center (PSC)

score sheet scanning system gave real-time reports comparing the original scores to the scores assigned

by this year’s team. The team also second scored 33 percent of the current year sample to measure

consistency of scoring. The table leaders monitored daily interreader agreement reports and t-tests to

verify consistency of scores within year and across years. Summaries of the interreader agreement figures

can be found in Table 7-3.
145

Table 7-1

Number of Constructed-Response Items by Score-Point Levels

Subject 2- 3- 4- 5- 6-

Age/Grade Category Category Category Category Category Total

Mathematics Long-Term Trend

Age 9 29 0 0 0 0 29

Age 13 28 0 0 0 0 28

Age 17 29 0 0 0 0 29

Total 86 0 0 0 0 86

Reading Long-Term Trend

Age Class 9 9 0 1 2 2 14

Age Class 13 8 0 2 4 1 15

Age Class 17 8 0 3 4 1 16

Total 25 0 6 10 4 45

Writing Long-Term Trend

Age Class 9 0 2 4 0 0 6

Age Class 13 1 1 5 0 0 7

Age Class 17 3 0 6 0 0 9

Total 4 3 15 0 0 22

Writing Long-Term Trend (Holistic Scoring)

Age Class 9 0 0 0 0 2 2

Age Class 13 0 0 0 0 2 2

Age Class 17 0 0 0 0 2 2

Total 0 0 0 0 6 6

Bilingual Mathematics

Grade 4 1 7 1 1 0 10

Grade 8 1 9 2 0 0 12

Total 2 16 3 1 0 22

Mathematics New Items

Grade 4 0 18 14 7 0 39

Grade 4/8 0 5 0 0 0 5

Grade 8 5 30 5 6 0 46

Grade 8/12 0 4 1 2 0 7

Grade 12 2 29 4 8 0 43

Grade 4/8/12 0 0 0 0 0 0

Total 7 86 24 23 0 140

 Mathematics Short-Term Trend (Base Year 1990)

Grade 4 0 18 14 7 0 39

Grade 4/8 0 5 0 0 0 5

Grade 8 5 30 5 6 0 46

Grade 8/12 0 4 1 2 0 7

Grade 12 2 29 4 8 0 43

Grade 4/8/12 0 0 0 0 0 0

Total 7 86 24 23 0 140

(continued)
146

Table 7-1 (continued)

Number of Constructed-Response Items by Score-Point Levels

Subject 2- 3- 4- 5- 6-

Age/Grade Category Category Category Category Category Total

Mathematics Short-Term Trend (Base Year 1992)

Grade 4 5 4 2 2 0 13

Grade 4/8 1 1 2 5 0 9

Grade 8 5 4 1 3 0 13

Grade 8/12 1 1 0 0 0 2

Grade 12 2 3 4 4 0 13

Grade 4/8/12 0 3 1 0 0 4

Total 14 16 10 14 0 54

Science

Grade 4 7 52 10 2 0 71

Grade 4/8 2 47 8 1 0 58

Grade 8 1 41 17 0 0 59

Grade 8/12 2 50 8 0 0 60

Grade 12 4 83 32 6 1 126

Total 16 273 75 9 1 374

Table 7-2

Professional Scoring - Readers and Dates

Assessment

Number of

Table Leaders

Number of

Scorers Dates

Fall Long-Term Trend Reading/Writing 1 7 11/29/95-1/5/96

Fall Long-Term Trend Mathematics 0 11 12/18/95-1/5/96

Winter Long-Term Trend Reading/Writing 1 6 1/30/96-4/3/96

Winter Long-Term Trend Mathematics 1 9 3/25/96-3/29/96

Spring Long-Term Trend Reading/Writing 1 6 4/10/96-5/24/96

Spring Long-Term Trend Mathematics 1 5 5/13/96-5/24/96

Long-Term Trend Writing Holistic 7 56 6/10/96-61/6/96

Long-Term Trend Writing Mechanics 6 33 7/15/96-7/31/96

Mathematics Segment 1 Days 8 72 3/13/96-4/5/96

Mathematics Segment 2 Days 9 126 4/8/96-5/6/96

Mathematics Segment 2 Evenings 5 70 4/8/96-5/2/96

Bilingual Mathematics 0 4 5/2/96-5/3/96

Science Segment 1 Days 3 24 3/18/96-4/5/96

Science Segment 2 Days 3 27 4/8/96-5/3/96

Science Segment 2 Evenings 5 70 4/8/96-5/2/96

Science Segment 3 Days 12 156 5/6/96-6/7/96

Science Segment 3 Evenings 12 150 5/6/96-6/7/96
147

Table 7-3

Interreader Reliability Ranges

Number of

Unique Items

Number of Items in

Percentage Exact Agreement Range

Assessment1 Total 60-69% 70-79% 80-89% Above 90%

Fall Long-Term Trend Reading/Writing 13 0 0 3 10

Fall Long-Term Trend Mathematics 28 0 0 0 28

Winter Long-Term Trend Reading/Writing 11 0 0 0 11

Winter Long-Term Trend Mathematics 29 0 0 0 29

Spring Long-Term Trend Reading/Writing 14 0 0 6 8

Winter Long-Term Trend Mathematics 29 0 0 0 29

Long-Term Trend Writing Holistic2 6 001 5

Long-Term Trend Writing Mechanics 3 N/A N/A N/A N/A

4th Grade Mathematics 80 0 0 2 78

8th Grade Mathematics 102 0 0 4 98

12th Grade Mathematics 100 0 1 9 90

4th Grade Science 94 0 0 13 81

8th Grade Science 125 0 0 20 105

12th Grade Science 156 0 0 26 130

1

 Not all long-term trend items received second scoring. Figures are included here only for those that were second scored. 2

 Figures for long-term trend writing holistic include adjacent scores.

7.2.2 Reading and Writing (Primary Trait)

All of the writing items for the three long-term trend assessments (fall, winter, and spring) were

scored using the primary trait method. This method focused on the writer’s effectiveness in

accomplishing specific assigned tasks. The primary trait scoring criteria defined five levels of task

accomplishment:

1. not rated,

2. unsatisfactory,

3. minimal,

4. adequate, and

5. elaborated.

The scoring standard for each item described these levels in detail. Some of these items were also scored

for secondary traits, which involved indicating the presence or absence of elements that were of special

significance to a particular item (e.g., whether notes were made before writing or whether critical

information was filled out on a form).

The scoring guides for the constructed-response reading items focused on students’ abilities to

perform various reading tasks:

• identifying the author’s message or mood and substantiating their interpretation,

• making predictions based on given details, and

• comparing and contrasting.
148

The guides for the reading items varied somewhat, but typically included a range of scores denoting

inability to address the task, unsatisfactory responses, minimal ability in accomplishing the task,

satisfactory ability in addressing the task, or elaborated responses addressing the task fully. Some of the

reading items received scores for secondary traits based on what reactions or information the student

gave (i.e., whether the response was mostly content based, form based, a subjective reaction, or some

combination of the three).

The scoring guides for the constructed-response writing items focused on students’ abilities to

write in informative, persuasive, and narrative styles. The guides for the writing items were based on a

range of scores denoting unsatisfactory writing to address the task, minimal writing to address the task,

satisfactory writing to address the task, and elaborated writing to address the task.

The item known as “The Door” was scored for attemptedness only. The readers coded all blanks

as “0” and any attempt to answer as a “1.”

The numbers of discrete constructed-response reading and writing items can be found in Table 7-1.

As with mathematics, the scorers used the same training materials as in previous assessments for

reading and writing. Thus, there was no need to select new training material from current year responses.

Preparation for the three long-term trend scoring projects began with identifying samples from previous

years as indicated below. Scores assigned in assessment booklets from 1984 (reading responses) and

1988 (writing responses) had been masked in previous years to ensure that scoring for training, and

subsequent long-term trend reliability scoring, would be done without knowledge of the previous scores

given. The 1994 booklets required no masking because scores had never been written directly in the

booklets. Finally, clerical support staff members matched scoring sheets with the booklets selected for

rescore after they had been pulled from the warehouse.

The formal training for the long-term trend assessments was divided into two parts to

accommodate the reading and writing items. The reading/writing long-term trend scoring project started

with one team of seven readers with one table leader for fall trend. Six of the same scorers and the same

table leader continued for winter and spring trends. Dates for scoring the three seasons of trend booklets

are given in Table 7-2. During training each reader received a photocopied packet of materials used in the

1984 scoring of the reading items and the 1988 scoring of the writing items.

Prior to scoring any 1996 reading and writing trend material, a training reliability report was

generated using a 25 percent sampling of the 1984 assessment materials for reading and 1988 materials

for writing. Following the formal training sessions, the readers scored this material on scannable scoring

sheets produced for specific booklet types with the appropriate long-term trend items pre-printed on the

scoring sheets. These sheets were then routed to scanning under a special job number to ensure that this

material was labeled for training scoring only. The scoring coordinator was able to generate a computer

report that listed the individual and group percent agreement by item. The system automatically

compared the new score with the original score assigned in the 1984 or 1988 scoring and produced a

report on the training reliability. T-tests were also generated for each item to verify comparability of

scoring across years. The NCS scoring specialist then conferred with the appropriate ETS staff on this

training reliability agreement report before proceeding with scoring.

All readers for this project were experienced scorers with a minimum of a bachelor’s degree. One

team member had scored the same long-term trend items for the 1994 assessment. The team read

materials as they arrived at the scoring center, with occasional breaks in scoring when receipts were

slow. The table leader monitored consistency within the current year as well as across years on a daily
149

basis as indicated below. The number of booklets processed and constructed responses scored for each

age level can be found in Table 6-1 in Chapter 6.

Reliability studies were conducted for the scoring of the long-term trend reading and writing

items. For the 1996 booklets, 33 percent of the constructed-response items were scored by a second

reader to produce interreader reliability statistics. In addition, a long-term trend reliability study was

conducted to ensure that the scoring procedures were consistent with those used in 1984, 1988, and 1994.

For this study, 350 of the 1984 reading responses and 350 of the 1988 writing responses were sampled.

Also, 250 of both reading and writing responses from 1994 were sampled for rescore. The scoring of

these long-term trend samples was intermixed with the scoring of the 1996 reading and writing trend

material. The readers selected a bundle of approximately five of each booklet type each day and gridded

their scores on separate scannable scoring sheets for each item. These sheets were then scanned and

cross-referenced with the original data tape to extract information for long-term trend reliability

reporting. T-tests were generated daily to verify comparability of scoring across years. Note that only

primary trait scores were compared in the across-year rescore. Secondary traits and items scored for

attemptedness only were not second scored in the current year nor rescored in the long-term trend

sample. Composite ranges of interreader reliability figures can be found in Table 7-3.

7.2.3 Holistic Writing Scoring

Certain writing items included in the long-term trend assessment were scored holistically.

Holistic scoring is based on an overall judgment of writing fluency and structure. Because a six-point

holistic scale was used, no scores fell exactly in the middle of the scale. This was done to force readers to

categorize a student’s response into either the top half or the bottom half of the scale. Within the upper

and lower halves of the scale, the scores reflected the degree to which the student demonstrated fluency,

structure, or both, in responding to the prompt.

In the 1996 holistic scoring session, two items were scored from each of the three grade levels.

The items scored at grade 4 were

• flashlight, an imaginative task, and

• spaceship, a persuasive task.

At grades 8 and 11, the two items scored were

• food on the frontier, an informative task, and

• recreation opportunities, a persuasive task.

The responses scored were taken from six assessment years: 1984, 1988, 1990, 1992, 1994, and 1996.

Fifty-six readers participated in the 1996 holistic scoring session. They were organized into seven teams

of eight readers, with each team led by a table leader and assisted by one clerical aide.

The total number of responses scored, readers, table leaders, and dates are given in Table 6-1 in

Chapter 6. Detailed lists of responses broken down by prompt, grade, and year are given in the Report of

Processing and Professional Scoring Activities (NCS, 1996).
150

7.2.3.1 Materials Preparation

All grade-eligible student booklets with on-task scores were selected from each of the six

assessment years and assembled into bundles of 25 booklets each. Bundle header sheets were then

generated from the existing data file of student booklet identification numbers. The system assigned the

first 25 grade-eligible booklet ID numbers for each specific booklet type to a bundle. The next 25

booklets for that booklet type were assigned the next consecutive bundle number, and so on until all

booklets were assigned. The bundle header was placed on top of the bundle of booklets and these were

then rubber-banded together. In addition to the bundle number, the bundle header showed the name of the

item to be scored, the grade level, the assessment year, and the booklet ID numbers of the booklets in that

bundle.

For each bundle, two sets of scoring sheets were generatedone for the first scoring and one for

the second scoring. The scoring sheets indicated the bundle number, the name of the item to be scored,

the grade level, the assessment year, and the booklet ID number. Second scoring sheets were generated

for a random 25 percent of the booklets in each bundle. For both first and second scoring sheets, a

separate sheet was generated for each booklet in the bundle.

Since some of the booklets for grades 8 and 11 contained two items to be scored during the

holistic scoring, Recreation Opportunities and Food on the Frontier, two separate bundle headers and sets

of scoring sheets were created for these booklets to accommodate the two scorings of the same booklets.

The appropriate header and corresponding scoring sheets for both items were placed with the booklet

bundle prior to scoring. Upon completion of the holistic scoring for the Food on the Frontier item,

clerical staff removed the header and scoring sheets for this item and replaced them with the sheets for

the Recreation Opportunities item; then the responses for the Recreation Opportunities item were scored.

All bundles were separated by item on carts. A sheet identifying the item and year was attached

to each cart. Thus, all bundles could be readily identified and retrieved quickly and easily.

7.2.3.2 Training Preparation

Three days prior to the beginning of holistic scoring, the ETS writing coordinator met with the

table leaders and NCS administrative assistant to prepare training materials. They first reviewed the

training materials from the 1994 scoring session and augmented them with sample responses from the

1995-96 assessment. The training packets included a familiarization and range finder set each containing

six examples, an upper half and a lower half set each containing three examples, and three calibration

sets each containing four examples. The sets, together with their scoring guides and keys, can be found in

the Report of Processing and Professional Scoring Activities (NCS, 1996).

Following standard ETS procedures, the training of the readers was conducted immediately prior

to the beginning of scoring for each individual item.

7.2.3.3 Holistic Scoring Training

The training materials for the first item were distributed to each reader. Each reader read a copy

of the prompt and the holistic scoring guide for that item. The scoring guide was explained by the writing

long-term trend coordinator. A clerical aide was assigned to each of the seven teams to keep the materials
151

flowing in an organized and efficient manner. Training began on June 10, 1996. The general sequence of

training for each of the six prompts was as follows:

Scoring Guide and Prompt - The scoring guide was introduced and briefly discussed,

followed by a reading of the prompt and a general discussion of the scoring expectations.

Familiarization Set - Readers were asked to arrange the papers in the familiarization set

into ‘best to worst’ order, assigning a score to each paper, with no score being used more

than once. The writing long-term trend coordinator announced the scores given by the

table leaders for each of the individual papers in the set. The table leaders discussed with

their teams the rationale behind these scores, using the scoring guide.

Range Finders - The writing long-term trend coordinator read the six range finders (best

to worst) and discussed the rationale behind the scores given. If there was too much

discussion regarding ‘why,’ an alternate range finder for that score point was inserted

later. It was at this point the writing long-term trend coordinator asked the readers to

look only to their range finders and no longer to the scoring guide.

Upper Half - Readers first scored three papers representing the upper half. There was a

sample of a four-, a five-, and a six-score point paper included. A tally was kept of the

number of readers assigning each score point. After the tally, the writing long-term trend

coordinator announced the scores assigned in the preparation process and compared them

to the scores on the tally sheet.

Lower Half - The readers were then asked to score three lower-half papers. This sample

included a one-, a two-, and a three-score point paper. Again, a tally of the scores

assigned by the individual readers was kept. The table leaders discussed the rationale

behind the reader-assigned scores including both the upper and lower half score point

papers at this time and answered any team member questions.

Calibration Set 1 - Prior to the beginning of scoring, the writing long-term trend

coordinator assigned Calibration Set 1 consisting of four papers. After scoring

Calibration Set 1, a tally was kept and the writing long-term trend coordinator announced

the scores. The table leader and team discussed the readers’ scores and questions.

Calibration Sets 2 and 3 - A calibration set was used after a break of longer than 15

minutes (lunch, morning break, afternoon break, or overnight). The readers reviewed the

range finders and the table leaders provided a one-sentence review of each of the score

points. The readers scored the set of four papers and the scores given were tallied. The

tallies were kept for each of these calibration sets and can be found with the training

materials in the Report of Processing and Professional Scoring Activities (NCS, 1996).

7.2.3.4 Holistic Scoring

Holistic scoring occurred during the week of June 10, 1996. When scoring for an item began,

each reader was given a bundle to score. The reader entered his or her reader ID number in the

appropriate space on each scoring sheet and scored all the responses in the bundle. The readers were

directed to check the booklet number listed on the scoring sheet against the booklet ID number printed on
152

the front of the assessment booklet. If any discrepancies were found, they were brought to the table

leader for resolution.

When the first scoring of a bundle of booklets was completed, the reader handed the first scoring

sheets to the clerical support staff who placed the bundle of booklets, with the bundle header and second

scoring sheet still on top, back on the designated cart. NCS clerical staff distributed all bundles to the

readers to ensure an even flow of material and a distribution of second scoring among teams.

Reliability scoring was handled in the same manner as first scoring. When the second scoring

was completed, the second scoring sheets were placed on the table leader’s desk. All bundles of booklets

were placed on carts and removed from the scoring area. Clerical staff then sorted all scoring sheets and

routed them to the scanning area to be entered onto the database.

The table leaders read through each reader’s entire initial bundle of booklets and evaluated the

scores assigned by the reader. This process is known as backreading. They also periodically backread the

remaining bundles throughout the reading of each item. If discrepant scoring occurred, the table leader

brought it to the attention of the reader. If the problem recurred, the reader was retrained by the table

leader. If the problem continued, the writing long-term trend coordinator assisted with the re-training.

Twenty-five percent of the papers were read again by a reader from a different team. A

preliminary reliability measure was taken by the administrative assistant approximately two to three

hours after the beginning of scoring.

Scoring the booklets took between four and seven hours depending on the grade level and the

type of writing. The readers were trained and scored the items in the following order:

Informative: Food on the Frontier ..................Grade 11, followed by Grade 8

Persuasive: Recreation Opportunities...........Grade 11, followed by Grade 8

Spaceship ...................................Grade 4

Narrative: Flashlight ...................................Grade 4

The scoring was completed on Sunday, June 16, 1996. A more detailed log of daily activities can be

found in the Report of Processing and Professional Scoring Activities (NCS, 1996).

Table 7-4

Holistic Writing Scoring Reliability Figures

Grade Item Exact Agreement Adjacent Agreement

4 Flashlight 52.1% 89.6%

Spaceship 58.3% 93.5%

8 Food on the Frontier 53.1% 94.5%

Recreational Opportunities 58.2% 95.3%

11 Food on the Frontier 56.3% 94.5%

Recreational Opportunities 57.1% 94.9%

Total 56.1% 94.0%

After all the score sheets were scanned, the scanner operator produced a final report showing the

n-counts scored and the interreader agreement rates. The figures are given at the item level in Table 7-4.
153

They are given in more detail, broken down by prompt, grade, and year, in the Report of Processing and

Professional Scoring Activities (NCS, 1996).

7.2.4 Writing Mechanics

Mechanics scoring focused on the extent to which the writer can control the conventions of

written Englishgrammar, spelling, capitalization and punctuation. In addition, the procedures include

identifying sentence structures and word choice errors. A team of 33 readers and six table leaders scored

selected essays for each age/grade group from the writing long-term trend assessments conducted in years

1988, 1990, 1992, 1994, and 1996. Spaceship, an imaginative task, was scored at age 9/grade 4 and

recreation opportunities, a persuasive task, was scored at both age 13/grade 9 and age 17/grade 11.

7.2.4.1 Preparation

Essays to be scored from the 1996 assessment were selected according to ETS specifications as

follows. For each booklet in which each item appeared:

1. select all grade-eligible booklets with primary trait scores greater than 0 and less

than 7, in other words only on-task responses;

2. select every third booklet; and

3. select all Black students (based on the student’s response in the background

questions) not picked in Step 2.

As a result of this selection process, 1,593 essays were scored from the 1996 assessment. In addition, 10

percent of the essays previously scored for mechanics from the 1988, 1990, 1992, and 1994 assessments

were rescored for reliability. This sample was selected by locating specific booklets from a list generated

by ETS and resulted in a rescore of 736 essays for all four years.

In preparation for the scoring process, copies were made of each selected essay and its

corresponding booklet cover. The booklet cover, containing assessment year, age/grade, primary

sampling unit (PSU) and student ID information, was stapled to the essay. Papers were then grouped by

assessment year and grade into packets of 20. Packets were numbered consecutively and were identified

by headers. Three identical sets of packets (A, B, and C) were assembled since each essay had to be

scored independently by two readers and discrepancies had to be resolved by a table leader. Packets A

and B were used by the two readers and Packet C was used by the resolver. For prior year scoring, the

same sample was used as in 1994. Therefore, copies were made from the 1994 master sets rather than

returning to the original booklets. The master sets were labeled as D Packets and warehoused for

potential future use.

7.2.4.2 Writing Mechanics Training

Training of the six table leaders and 33 readers was conducted by the writing long-term trend

coordinator during the week of July 15, 1996. The training involved a detailed discussion of the scoring

guide. The writing coordinator presented the main sections of the guide:

• type of sentence construction,

• faulty sentence construction,

• punctuation, and

• word level categorization.
154

Copies of pertinent resource information were distributed, briefly reviewed, and reference materials were

identified. After discussion of each of the main sections of the guide, the group reviewed the scored

papers from the training packet, paying special attention to scores reflecting the category under

discussion.

To further train the readers, the trainer used a pool of responses from the1988 and 1990

assessments that has been scored for mechanics in 1990 but not used in the rescore sample in 1996.

Copies were made of these training essays to be used for practice. Each reader then individually scored a

selected group of essays. The scores were compared among the group, discussions were held when

discrepancies occurred, and again references were made to resource materials or to the scored sample

papers. When the group was comfortable with the decisions being made, the actual scoring began. Copies

of these sets and the scoring guidelines can be found in the Report of Processing and Professional

Scoring Activities (NCS, 1996).

7.2.4.3 Scoring

The actual scoring, resolution, data entry work and proofreading of writing mechanics began on

July 18 and was completed on July 31, 1996. In selecting packets for scoring, readers alternated among

the different grade levels and assessment years. The mechanics readers marked each paper with a series

of symbols, addressing the elements of sentence type, sentence construction, word choice, spelling,

punctuation, and capitalization. These symbols, written in red ink, designated each word or punctuation

mark in error and indicated sentence type or faulty sentence construction. Each essay was scored

independently by two different readers selecting either Packet A or B.

To track the movement of the packets, the NAEP internal tracking log was used. As readers and

resolvers worked on particular packets, the appropriate columns were initialed and dated. This enabled

NCS staff to see at a glance the status of each packet. The completed tracking logs have been warehoused

with the training materials. A sample of a blank tracking log can be found in the Report of Processing

and Professional Scoring Activities (NCS, 1996).

Resolution and quality control were conducted by table leaders who compared the scores marked

on copies A and B of each unique packet and resolved any discrepancies. After determining the

appropriate marks, the resolver used the unmarked copy C to record the final version. The copy with the

resolved marks was sent to the word processing area in NCS’s Creative Services department for

transcription as described below. To avoid confusion, unused copies were discarded or returned to

original readers with feedback information and with follow-up training, if deemed necessary.

To maintain a consistent scoring standard, the six table leaders, along with the NCS performance

assessment specialist, met twice daily to resolve any questions that arose during the scoring. Resolution

scoring allowed the table leaders to determine the accuracy of individual readers. If a reader was

confused by a facet of the coding, the table leader would approach the reader individually. If the table

leaders identified a trend in the coding, the issue would be broached at the re-calibration sessions. Twice

a week, for approximately 30 minutes on Wednesday mornings and Friday afternoons, the six tables

stopped scoring and re-calibrated; table leaders brought up pertinent scoring issues and readers asked for

clarification on coding decisions.

Resolved packets were sent to the NCS Creative Services department where the text of the

essays, along with the assigned marks and identification information, were entered into a data file. Essays

were typed exactly as they were written. Each essay was typed on a separate page and double-spaced

pages were printed for proofreading. The scoring team proofread the data entry work against the scored
155

papers. Student identification and PSU numbers were checked and discrepancies were resolved.

Corrections were indicated in red on the typed copy. If corrections were needed, all pages belonging to a

packet were returned to the Creative Services department for additional corrections. Complete and

correct packets were uploaded to the NCS mainframe. The data were reformatted according to ETS

specifications and a data file containing the scored information was sent to ETS on August 7, 1996.

7.3 MAIN NAEP ASSESSMENT

7.3.1 Selection of Training Papers

A pool of papers to be used for training for the NAEP main assessments was selected by NCS in

February and March of 1996. Persons identified as potential mathematics and science table leaders were

selected to copy student responses. Team leaders, with assistance of the potential table leaders, gave

tentative scores to the responses and selected 50 student responses from each dichotomous item, 75

responses from each 3-point item, 100 responses from each 4-point item and 125 from each 5-point item.

Because NCS staff screened the responses, the pool sent to the ETS test development specialists

contained a full range of point values.

NCS staff numbered the papers sequentially and copied the sets. NCS retained and filed the

originals and sent the copies to the appropriate subject area coordinators at ETS. ETS returned a list of

the anchor and practice sets with scores to NCS staff, who used the master file copies to create training

sets. NCS staff then masked the sequential reference numbers and wrote the actual scores on the anchor

papers and a new sequential reference number for the training sets.

The NCS copy center and PSC clerical staff shared responsibility for making multiple copies of

the sets for scorers. The master sets, team leader/trainer copy and table leader copies also had keys to the

training sets. When copying was complete, the master copy was placed in the appropriate file.

7.3.2 Calibration Policies

During scoring, the teams used calibration sets to calibrate on a daily basis and to calibrate

across longer periods of time. The table leader built pools of items for calibration, which were then

distributed to the scorers in sets of five or ten, depending upon the complexity of the item, whenever a

break of longer than 15 minutes occurred, such as after lunch or at the beginning of a new scoring day.

All readers on the team scored the same calibration sets, and the system compared the scores of each

reader to the scores assigned by the trainer and table leader. The table leader reviewed the interreader

agreement report with the trainer and the ETS subject area coordinator, discussed any discrepancies that

arose, and then proceeded with scoring.

Whenever a team returned to scoring an item after having worked on a different item in the

meantime, the team scored a calibration set of 75 responses and analyzed the results before proceeding.

This occurred in mathematics because many mathematics items were scored in two sessions since the

first sweep through the items was done while booklets were still arriving from the field. If the item had

fewer than 500 responses left out of a pool that contained both main and state samples, an extended

calibration was waived. In science, the teams did not begin scoring most items until all responses to the

item were available for scoring, so this type of calibration was not necessary.
156

Table leaders printed and archived hard copies of all calibration sets used for scoring. For more

information on the functionality of the calibration tool, see Section 7.3.3.3.

7.3.3 Table Leader Utilities

Two of the significant advantages of the image-scoring system were the ease of regulating the

flow of work to readers and the ease of monitoring scoring. The image system provided table leaders

with tools to determine reader qualification, to backread scores, to determine reader calibration, to

monitor interreader reliability, and to gauge the rate at which scoring was being completed. These

various tools are described below.

7.3.3.1 Reader-Qualification Tool

One of the utilities at a table leader’s disposal was a qualification algorithm used after training

on extended constructed-response items. The table leader would give identical qualification packets to

each reader. These packets contained 10 student responses to be independently scored by the readers.

After the readers finished, the table leader would enter each reader’s scores into the computer for

tabulation. The computer would calculate each reader’s percentage of exact, adjacent, and non-adjacent

agreement with the master key. If a reader attained a percentage of exact agreement above a predetermined threshold of 80%, the reader would be allowed to score. Readers not attaining the predetermined threshold were handled on a case-by-case basis—typically receiving individual training by

the trainer or the NCS table leader before being allowed to score. A table leader could cancel a reader’s

qualification to score an item if review of a reader’s work indicated inaccurate scoring and that

supplemental training was necessary. Note that reader qualification was required only on extended

constructed-response items involving 4 or more score point levels.

7.3.3.2 Backreading Tool

After scoring began, NCS table leaders reviewed each reader’s progress using a backreading

utility that allowed the table leader to review papers scored by each reader on the team. Typically, a table

leader reviewed responses scored by each reader in quantities similar to the amount second scored (i.e.,

6% for items with both state and national samples and more for items with only a national sample). Table

leaders noted the score the reader awarded each response as well as the score a second reader gave that

same response. This was done as an interreader reliability check. Alternatively, a table leader could

choose to review all student responses given a particular score to determine if the team as a whole was

scoring consistently. Both of these review methods used the same display screen and showed the ID

number of the reader and the scores awarded. If the table leader disagreed with the score given an item,

he or she discussed it with the reader for possible correction. Replacement of scores by the table leader

was done only with the knowledge and approval of the reader, thereby serving as a learning experience

for the reader. Additionally, neither score was changed in the case where the response was second

scored.

7.3.3.3 Calibration Tool

While backreading, a table leader could identify individual responses for inclusion in a pool of

calibration papers. These papers could be selected because they exemplified criteria set down in the
157

scoring rubrics or because they were unusual and pointed out less obvious aspects of the scoring

guidelines. After selecting a number of papers for inclusion in the calibration set, the table leader could

decide to route any number of these calibration papers to the scorers. A typical number of papers routed

to scorers during a mid-scoring calibration was 10, although the image system could accommodate as

many or as few as the table leader and trainer determine necessary to check the accuracy of scoring.

When all scorers had completed the calibration set, the table leader could then produce an interreader

reliability report on the scoring of the calibration set.

7.3.3.4 Tool for Monitoring Interreader Reliability

During the scoring of an item or the scoring of a calibration set, the table leader could monitor

progress using an interreader reliability tool. This display tool could be used in either of two modes:

1. to display information of all first readings versus all second readings, or

2. to display all readings of an individual that were also scored by another reader versus

the scores assigned by the other readers.

The information was displayed as a matrix with scores awarded during first readings displayed in rows

and scores awarded during second readings displayed in columns for Mode 1 and the individual’s scores

in rows and all other readers in columns for Mode 2. In this format, instances of exact agreement fell

along the diagonal of the matrix. For completeness, data in each cell of the matrix contained the number

and percentage of cases of agreement (or disagreement). The display also contained information on the

total number of second readings and the overall percentage of exact agreement on the item. Since the

interreader reliability reports were cumulative, a printed copy of the exact agreement of each item was

made every day and compared to previously generated reports. 

7.3.3.5 Tool for Monitoring the Rate of Scoring

The table leaders were able to monitor work flow using a status tool that displayed the number of

items scored, the number of items first-scored that still needed to be second-scored, the number of items

remaining to be second-scored, and the total number of items remaining to be scored. This allowed the

team leaders and performance assessment specialists to accurately monitor the rate of scoring and to

estimate the time needed for completion of the various phases of scoring.

7.3.3.6 Scoring Buttons

To assign a score, readers clicked the mouse over a button contained in the scoring window.

Since buttons were included only for valid scores, there is no need to edit for out-of-range scores.

Another recent development was the implementation of a tool that allowed the performance assessment

specialist to label scoring buttons with key phrases, correct responses or certain incorrect responses that

were to be tracked. This enhanced scoring as readers no longer had to mentally translate a student’s

response into a numerical value before choosing a scoring button on the image screen.
158

7.3.4 Main Mathematics Assessment

The mathematics portion of the 1996 main assessment included a total of 226 discrete

constructed-response items. Table 7-1 shows the types and number of constructed responses for the

mathematics assessment. A variety of constructed-response items were used to measure different

elements of students’ mathematical sophistication and understanding. These items were administered in

scannable assessment booklets. The bilingual booklets were key-entry documents. The items scored

included traditional computational items, short-answer constructed responses, extended constructed

responses, diagrams, geometric figures, and graphs. Each constructed-response item had a unique scoring

guide that identified the range of possible scores for the item and defined the criteria to be used in

evaluating student responses. Long-term trend items that focused on the students’ computational ability

were typically scored on a right/wrong basis. The scoring guides for the more complex items were

developed to be of diagnostic value, by including categories that reflected partial credit and/or different

kinds of incorrect answers that indicated particular misunderstandings. New items developed for the

1993 and 1995 NAEP field tests were scored on a partial-credit scale (3-point or 4-point) or on an

extended scale (5-point).

The operational assessment included, for the first time other than the field test, blocks that were

based on a theme. There were two different theme blocks administered at each grade. General

information on the number of constructed responses scored can be found in Table 6-1 in Chapter 6. Table

7-5 gives more detailed information by grade and booklet type (spiral, estimation, theme, and advanced).

7.3.4.1 Training

The training for each mathematics item was conducted by mathematics specialists from ETS and

NCS just prior to the scoring of that item. Training and scoring began on Wednesday, March 13, and

ended on Monday, May 6, 1996. The NCS mathematics performance assessment specialist and selected

NCS team leaders conducted all the training of the short-term trend items scored in March. The ETS

mathematics coordinator met with each trainer individually before the beginning of training to discuss

any questions and/or discrepancies. He returned to NCS at the beginning of April to review individually

the items assigned in April. The NCS training staff added another team leader to assist in training for the

evening shift in April.

Training involved explaining the item and its scoring guide and discussing responses that

represented the various score points in the guide. When this was completed, the readers scored and

discussed from 5 to 35 selected “practice papers” for each item. Next, readers practiced scoring by

gathering around a single image terminal and scoring several responses to the item. Once the trainer and

the table leader determined the individuals on the team understood the scoring guide, the table leader

qualified the scorers to enter the system using the reader-qualification tool, discussed in Section 7.3.3.1.
159

Table 7-5

Mathematics Constructed Responses Scored

Assessment

Proportions

Type Data 1990 1992 1996

Grand

Total

1996

National

1996

State

Grade 4

Regular Unique items

Responses first scored

Responses second scored

First and second scored

Average % exact agreement

12

0

8,988

8,988

95.9

36

0

27,031

27,031

93.8

64

2,037,866

122,272

2,160,138

96.8

112

2,037,866

158,291

2,196,157

95.7

7.9%

160,584

9,635

170,219

92.1%

1,877,282

112,637

1,989,919

Estimation Unique items

Responses first scored

Responses second scored

First and second scored

Average % exact agreement

0

0

0

0

N/A

0

0

0

0

N/A

3

7,317

439

7,756

93.3

3

7,317

439

7,756

93.3

Theme Unique items

Responses first scored

Responses second scored

First and second scored

Average % exact agreement

0

0

0

0

N/A

0

0

0

0

N/A

13

23,800

5,950

29,750

95.8

13

23,800

5,950

29,750

95.8

Total Unique items 12 36 80 128

Responses first scored 0 0 2,068,983 2,068,983

Responses second scored 8,988 27,031 128,661 164,680

First and second scored 8,988 27,031 2,197,644 2,233,663

Average % exact agreement 95.9 93.8 96.5 95.7

Average % exact agreement 95.9 93.8 96.5 95.7

Grade 8

Regular Unique items

Responses first scored

Responses second scored

First and second scored

Average % exact agreement

18

0

13,374

13,374

95.3

44

0

33,077

33,077

94.4

69

1,991,682

119,501

2,111,183

96.6

131

1,991,682

165,952

2,157,634

95.7

9.3%

184,683

11,081

195,764

90.7%

1,806,999

108.420

1,915,419

Estimation Unique items

Responses first scored

Responses second scored

First and second scored

Average % exact agreement

0

0

0

0

N/A

0

0

0

0

N/A

6

13,622

3,405

17,027

97.3

6

13,622

3,405

17,027

97.3

Theme Unique items

Responses first scored

Responses second scored

First and second scored

Average % exact agreement

0

0

0

0

N/A

0

0

0

0

N/A

13

27,690

6,923

34,613

93.9

13

27,690

6,923

34,613

93.9

(continued)
160

Table 7-5 (continued)

Mathematics Constructed Responses Scored

Assessment

Proportions

Type Data 1990 1992 1996 Grand Total 1996

National

1996

State

Grade 8

Advanced Unique items

Responses first scored

Responses second scored

First and second scored

Average % exact agreement

0

0

0

0

N/A

0

0

0

0

N/A

14

33,354

8,339

41,693

95.1

14

33,354

8,339

41,693

95.1

Total Unique items 18 44 102 164

Responses first scored 0 0 2,066,348 2,066,348

Responses second scored 13,374 33,077 138,168 184,619

First and second scored 13,374 33,077 2,204,516 2,250,967

Average % exact agreement 95.3 94.4 96.1 95.5

Grade 122

Regular Unique items

Responses first scored

Responses second scored

First and second scored

Average % exact agreement

21

0

15,225

15,225

94.8

44

0

32,387

32,387

92.1

73

212,774

12,766

225,540

95.6

138

212,774

60,378

273,152

94.4

N/A

N/A

N/A

N/A

N/A

N/A

N/A

N/A

Theme Unique items

Responses first scored

Responses second scored

First and second scored

Average % exact agreement

0

0

0

0

N/A

0

0

0

0

N/A

12

23,368

5,842

29,210

94.5

12

23,368

5,842

29,210

94.5

Advanced Unique items

Responses first scored

Responses second scored

First and second scored

Average % exact agreement

0

0

0

0

N/A

0

0

0

0

N/A

15

44,881

11,220

56,101

95.1

15

44,881

11,220

56,101

95.1

Total Unique items 21 44 100 165

Responses first scored 0 0 281,022 281,022

Responses second scored 15,225 32,387 29,829 77,441

First and second scored 15,225 32,387 310,851 358,463

Average % exact agreement 94.8 92.1 95.4 94.5

Grand Total - Unique items 51 124 282 457

1 4th grade mathematics had no advanced booklets.

2 12th grade estimation block had no constructed-response items.
161

7.3.4.2 Scoring

Mathematics scoring took place in two segments over two shifts as outlined in Table 7-2. Most

dichotomously scored short-term trend items were scored in Segment 1, with the remainder of the items

scored in Segment 2.

During scoring, the team leaders continued to compile notes on scoring decisions for the readers’

reference and guidance. Additionally, table leaders closely monitored interreader reliability using both

team and individual statistics as a reference. Consistently throughout the scoring of each item, the table

leaders also performed backreading duties in which they reviewed a sample of the responses scored by

each reader on the team. Lead scorers selected for their experience and accuracy in scoring assisted the

table leaders in backreading. The team leaders and performance assessment specialist continuously

monitored the progress of each team and noted all scoring-related decisions to ensure that training and

scoring progressed smoothly and in a timely manner.

The codes that were used for unscorable mathematics items were:

0 = Blank or random marks

8 = Completely crossed-out or erased

9 = “I don’t know,” refusal, off-task, illegible or language other than English

7.3.4.3 Reliability

A minimum of 25 percent of the mathematics responses for items involved only in the national

sample and 6 percent of the responses for items involved in the state and national samples were scored by

a second reader to obtain statistics on interreader reliability. Responses were automatically routed for

second scoring so that the reader could not discern any difference between a response being presented for

first or second scoring. The reliability figures were available to the table leader as soon as scoring began

and could be viewed on demand. These figures included a frequency distribution of all second scores for

the team and a frequency distribution for all second scores for individual members of the team who

scored the item. Ranges for interreader reliability figures for mathematics are reported in Table 7-3.

Average reliabilities, given in terms of percentage exact agreement, for each booklet type (spiral,

estimation, theme, and advanced) are reported in Table 7-5. This reliability information was also used by

the table leader in monitoring the capabilities of all readers and the uniformity of scoring across readers.

When scoring was completed for an item, a hard copy of the report was printed for analysis by ETS

project staff.

7.3.4.4 Short-Term Trend

The 1996 main assessment of mathematics included a number of items that had previously been

used in the 1990 and 1992 assessments. A list of these items is included in the tables in Appendix J. For

these items, the trainers used the same scoring guides and training sets as in 1992. Also, for those items

that originated in the 1992 assessment, 750 responses from 1992 were scanned into the scoring system

and rescored. For those items that originated in the 1990 assessment, the sample included 750 responses

each from 1990 and 1992. Table leaders used the management tools to distribute the rescore responses

out among the current-year material. Because the rescore responses could not be seeded into the current

materials, the table leaders divided the rescore material into thirds, inserting about 25 responses per

scorer at various intervals during scoring. Since the development group had loaded the original scores
162

from the previous years, the system was able to give real-time comparisons of scores. Table leaders,

trainers, and the NCS and ETS subject area specialists monitored the interreader agreement rates and ttests across years for all items used in the short-term trend.

7.3.4.5 Paper Scoring

Some mathematics items could not be scored on the image system because of printing problems

or difficulties with overlay templates. When scoring these items, professional readers coded any response

that could not be scored by viewing the image with a designated code such as “8” or “pull.” The

development group created a list of all responses coded for pulling, and the project coordinator led the

effort to pull these booklets from the warehouse. The booklets were then transported to the PSC, where

professional readers scored them using the same scoring guides, but marking their scores on scannable

scoring sheets. Clerical support staff then scanned the sheets and uploaded the file to the mainframe,

where the development team merged the data with the image scoring data.

As soon as the last item on any score sheet was completed, the score sheets were collected and

taken to a central clerical support area to be scanned on the NCS paper-based scoring system using

OpScan 5 scanners. As each sheet was processed, the scanning system compared the incoming data with

tables to ensure that all responses were scored with one and only one valid score, and that only raters

who were qualified to score an item were allowed to score it. Any discrepancies (e.g., no score assigned,

double gridding, out-of-range scores, or invalid scorer ID numbers) were flagged and resolved before the

data from that sheet were accepted into the scoring system. Interreader agreement reports were generated

twice a day.

All the scoring data were stored on personal computers at NCS after all the responses for a

subject area had been scanned. Upon completion of scoring, the scanner operator ran a query which

compared the sheets scanned with a table of records in the system to make sure that all score sheets were

accounted for. Once all edits were corrected, the personal computer file was renamed and put into an

export file, which automatically created the mainframe file. This file was then uploaded to the mainframe

to be merged with the mainframe student files.

7.3.4.6 Large Print

To accommodate students with visual impairment, field administrators had Braille and large print

versions of designated booklets available to sampled students who qualified for them. The scoring center

received no Braille material for scoring. Two large print booklets were received back for scoring. One of

the large print booklets was completely blank. The NCS performance assessment specialist in charge of

mathematics scored all the items in the other booklet and gave the scores to the development team to

enter into the database.

7.3.4.7 Bilingual Scoring

Some students who participated in the main fourth- and eighth-grade mathematics assessments

received bilingual booklets. All students who used bilingual booklets received the equivalent of Booklet

121 for their grade. The same blocks with the same items appeared in the bilingual booklets as in the

regular Booklet 121, with the exception that the bilingual booklets contained both Spanish and English

versions of each item. The items appeared in Spanish on the left-hand pages and in English on the right-
163

hand pages. The instructions told students to answer according to their preference. Altogether, the PSC

scored 91 bilingual booklets at grade 4 and 36 at grade 8.

Because of the small number of booklets involved, the PSC selected a team of four readers to

score all of the booklets on paper. The team consisted of two males and two females. Two of the scorers

were born in Chile, South America, and raised partly in Chile and partly in the United States. Both of

them have traveled and recently lived extended periods of time in South America, while making the

United States their permanent home. The third scorer comes from a Panamanian-American background.

Born and raised in a bilingual family, he holds a bachelor’s degree in Spanish, has lived extended periods

of time in Panama and Mexico, and has experience teaching university-level Spanish as well as working

several years in a business position that required daily telephone and correspondence contact with

Spanish-speaking clientele, mainly from Puerto Rico. The fourth scorer, while not a native speaker of

Spanish, holds a master’s degree in Spanish, has traveled and studied extensively in Mexico, the

Caribbean and South America, coordinated an adult education program for Spanish-speaking immigrants

from Mexican and Central American backgrounds for three years, taught university-level Spanish for

eight years, and worked in customer relations positions for four years dealing with Spanish-speaking

clientele, mainly from Puerto Rico.

The NCS mathematics specialist trained all four readers on the fourth-grade items on Thursday,

May 2, 1996, and the team scored all of the fourth-grade booklets the same day. On Friday, May 3, the

team learned the eighth-grade items, many of which overlapped with the fourth-grade items, and scored

all of the eighth-grade booklets on that day. Twenty-five percent of the booklets were scored by a second

reader to measure interreader agreement.

The team applied the same scoring guides that were used for regular scoring. However, two

scores were assigned for each item, one for the Spanish side and one for the English side. Since most

students answered relatively consistently on either one side or the other, most booklets received on-task

scores for one language and blanks for the other language. This procedure will allow analysts to separate

the data of those students who answered in English from those who answered in Spanish. Several

students wrote their answers in English on the side of the page where the item was written in Spanish. In

these cases, the scorers coded the score as a Spanish answer since that is where the student read the item

and wrote the answer. During the course of scoring, the team noted that over half of the fourth-grade

booklets came from the same school, and not a single student from that school answered any questions in

Section 3 of the booklet.

The same scanning procedures were performed as outlined in Section 7.3.4.5.

7.3.5 Main NAEP Science

The science portion of the 1996 NAEP included a total of 374 discrete constructed-response

items (see Table 7-1). It was scored over three segments and two shifts (see Table 7-2). Many kinds of

constructed-response items were utilized in the assessment to measure different elements of students’

conceptual understanding of scientific material as well as their practical reasoning ability. The items

scored included short-answer constructed responses and extended constructed responses. Each

constructed-response item had a unique scoring guide that identified the range of possible scores for the

item and defined the criteria to be used in evaluating student responses.
164

During the course of the project, each team scored short constructed-response items using a scale

that allowed for partial credit as follows:

1 = incorrect response

2 = partial understanding

3 = correct response

The readers scored extended constructed-response items on a scale of “1” to “4” as follows:

1 = incorrect response

2 = minimal understanding

3 = satisfactory level of comprehension

4 = correct response

7.3.5.1 Training

The training on each item was conducted by science specialists from ETS and NCS. The first teams

began training on March 18, 1996. Other teams were phased in throughout the project. Hands-on items

were scored a block at a time with a unique scoring guide for each item because of the related nature of the

items. The rest of the assessment was scored item-by-item so that each reader worked on only one set of

rubrics at a time. After scoring all available responses, a team would then proceed with training and scoring

the next item. Scoring was completed on June 7, 1996. Table 7-2 gives detailed information on the dates of

scoring and the number of readers and table leaders.

Training involved explaining the item and its scoring guide to the team and discussing responses

that represented the various score points in the guide. Typically, two or three anchor responses were chosen

for each score point. During this stage, readers and the table leader kept notes of scoring decisions. The

table leader was then responsible for compiling those notes and ensuring that all readers were in alignment.

When review of the anchor packet was completed, the readers scored and discussed 10 to 20 pre-scored

“practice papers” that represented the entire range of score points the item could receive. After the trainer

and table leader determined that the team had reached consensus, the table leader then released work on the

image-scoring system to the readers. The readers would initially take turns reading their first “live”

responses to the team or work in pairs as a final check before beginning work individually. Once the

practice session was completed, the formal scoring process began.

7.3.5.2 Scoring

All scoring for science was conducted via the image-based scoring system. During scoring, the

team leaders continued to compile notes on scoring decisions for the readers’ reference and guidance.

Additionally, table leaders closely monitored interreader reliability using both team and individual statistics

as a reference. Consistently throughout the scoring of each item, the table leaders also performed

backreading duties in which they reviewed a sample of the responses scored by each reader on the team.

Lead scorers selected for their experience and accuracy in scoring assisted the table leaders in backreading.

The table leaders and performance assessment specialist continuously monitored the progress of each team

and noted all scoring-related decisions to ensure that training and scoring progressed smoothly and in a

timely manner.
Table 7-6

Science Constructed Responses Scored

Type1 Assessment P

Grade Data Regular Hands-On Advanced Grand Total 1996 National 

4 Unique items

Responses first scoredResponses second scoredFirst and second scoredAverage % exact agreement

70

200,319

50,080

250,399

93.9

24

94,00423,501

117,505

93.8

0000

N/A

94

294,323

73,581

367,904

93.9

74.8%

220,271

55,068

275,339

8 Unique items

Responses first scoredResponses second scoredFirst and second scoredAverage % exact agreement

94

2,157,377

129,443

2,286,820

93.4

31

976,844

58,611

1,035,455

95.0

0

000

N/A

125

3,134,222

188,053

3,322,275

93.8

9.7%

304,020

18,241

322,261

12 Unique items

Responses first scoredResponses second scoredFirst and second scoredAverage % exact agreement

94

198,563

49,641

248,204

93.0

26

75,12018,78093,900

94.4

36

88,16622,041

110,207

94.5

156

361,849

90,462

452,311

93.6

Total Unique items 258 81 36 375

Responses first scored 2,556,260 1,145,968 88,166 3,790,394Responses second scored 229,163 100,892 22,041 352,096First and second scored 2,785,423 1,246,860 110,207 4,142,490Average % exact agreement 93.4 94.4 94.5 93.7

1 Regular and hands-on tasks include state and national constructed responses.
166

One advantage of utilizing an image-based scoring system is the ability to construct reader aids to

simplify scoring, thus increasing reader reliability. Prior to the start of the project, the ETS subject area

specialist and the NCS performance assessment specialist identified several items for the construction of

overlays. Overlays serve as templates to define boundaries in which correct responses must be located or

allow the placement of correct answers directly on the displayed image, and are displayed along with the

student response. A schematic representation of each overlay was included with the scoring guide and

sample papers for these items to familiarize readers with the use of the scoring aids during training.

General information on the number of constructed responses scored can be found in Table 6-1 in

Chapter 6. Table 7-6 gives more detailed information by grade and booklet type (spiral and advanced). The

codes that were used for unscorable science items were:

0 = Blank or random marks

8 = Completely cross-out or erased

9 = “I don’t know,” refusal, off-task, illegible or language other than English that could not

be translated

7.3.5.3 Reliability

A minimum of 25 percent of the science responses for items involved only in the national sample and 6

percent of the responses for items involved in the state samples were scored by a second reader to obtain

statistics on interreader reliability. Ranges for interreader reliability for science are reported in Table 7-3.

Average reliabilities, given in terms of percentage exact agreement, for each booklet type (spiral and

advanced) are reported in Table 7-6. This reliability information was also used by the team leaders to monitor

the capabilities of all readers and maintain uniformity of scoring across readers. Reliability reports could be

generated on demand by the table leader, scoring specialist, or performance assessment specialist when

needed, and they were displayed at a computer workstation. In addition to the immediate feedback provided

by the on-line reliability reports, each table leader could also review the actual responses scored by a reader

by using the backreading tool. In this way, the table leader monitored each reader carefully and corrected

difficulties in scoring almost immediately with a high degree of efficiency.
167

Chapter 8

CREATION OF THE DATABASE, QUALITY CONTROL OF

DATA ENTRY, AND CREATION OF THE DATABASE PRODUCTS1

 John J. Ferris, Katharine E. Pashley,

David S. Freund, and Alfred M. Rogers

Educational Testing Service

8.1 INTRODUCTION

The data processing, scoring and editing procedures described in Chapter 6 resulted in the

generation of disk and tape files containing various data for students (assessed and excluded), teachers,

schools, and SD/LEP (students with disabilities/students with limited English proficiency) information.

The weighting procedures described in Chapter 10 resulted in the generation of data files that included

the sampling weights required to make valid statistical inferences about the population from which the

1996 fourth-, eighth- and twelfth-grade NAEP samples were drawn. These files were merged into a

comprehensive, integrated database. The creation of the database is described in Section 8.2.1.

Section 8.2.2 describes a central repository or master catalog of this information. The master

catalog is accessible by all analysis and reporting programs and provides correct parameters for

processing the data fields and consistent labeling for identifying the results of the analyses.

To evaluate the effectiveness of the quality control of the data entry process, the corresponding

portion of the final integrated database was verified in detail against a sample of the original instruments

received from the field. The results of this procedure are given in Section 8.3.

The integrated database was the source for the creation of the NAEP item information database

and the NAEP secondary-use data files. These are described in Section 8.4.

8.2 CREATION OF THE DATABASE

8.2.1 Merging Files

The data processing conducted by National Computer Systems (NCS) resulted in the transmittal

to ETS of four data files for each of fourth, eighth and twelfth grade: one for the student background and

item response data and one file for each of the three questionnaires (Teacher Questionnaire, School

Characteristics and Policies Questionnaire, and SD/LEP Questionnaire). The sampling weights, derived

by Westat, Inc., comprised additional files for each grade—two sets for assessed students, two sets for

excluded students and for schools four sets at grade 4, five at grade 8, and six at grade 12. (See Chapter

10 for a discussion of the sampling weights.) These files at each grade were the foundation for the

 

1

 John J. Ferris was responsible for the evaluation of the quality of the database and the data entry process; Katharine E. Pashley

was responsible for database generation under the supervision of David S. Freund; Alfred M. Rogers created the secondary-use

data files.
168

analysis of the 1996 NAEP data. Before data analyses could be performed, these data files had to be

integrated into a coherent and comprehensive database.

 The database ultimately comprised four files per cohort: three student files (mathematics,

science, and long-term trend) and a single school file. The student files were separated by subject area to

improve maintenance and efficiency of the databases and data analyses. Each record on the student file

contained a student’s responses to the particular assessment booklet the student was administered (in the

case of excluded students, a booklet was assigned but the student response fields contain a special code

indicating no response), and the information from the questionnaire that the student’s teacher completed.

Additionally, for a student (assessed or excluded) who was identified as a student with a disability (SD)

or of limited English proficiency (LEP), the data from the SD/LEP Questionnaire are included. This

questionnaire is filled out for all students identified as SD and/or LEP, both assessed and excluded. (See

Chapter 2 for information regarding assessment instruments.) Also added to the student files were

variables with school-level information supplied by Quality Education Department, Inc. (QED) including

demographic information about schools such as race/ethnicity percentages. Since the teacher data are not

from a representative sample of teachers and since the focus of NAEP is to report student level results,

the teacher response data were added to the student records. The school data were on separate files that

could be analyzed on their own and could also be linked to the student files through the unique school ID

code.

The creation of the student data files for fourth, eighth, and twelfth grade began with the

reorganization of the data files received from NCS. This involved two major tasks:

1. the files were restructured, eliminating unused (blank) areas to reduce the size of the

files; and

2. in cases where students had chosen not to respond to an item, the missing responses

were recoded as either “omit” or “not reached,” as discussed in Part II of this report.

Next, the student response data were merged with the student weights files. The resulting file was then

merged with the SD/LEP and teacher data. In all merging steps, the 10-digit booklet ID (the three-digit

booklet number common to every booklet with the same block of items, a six-digit serial number unique

to the booklet a student was given and a single check digit, distinguishing bilingual booklets) was used as

the matching criterion.

The school file for each grade was created by merging the School Characteristics and Policies

Questionnaire file with the file of school weights and school variables, supplied by Westat. The primary

sampling unit (PSU) and school codes were used as the matching criteria. Since some schools did not

return a questionnaire, some of the records in the school file contained only school-identifying

information and sampling weight information.

When the student and school files for each grade had been created, the database was ready for

analysis. In addition, whenever new data values, such as composite background variables or plausible

values, were derived, they were added to the appropriate database files using the same matching

procedures described above.

For archival purposes and to provide data for outside users, restricted-use data files and

codebooks for each jurisdiction were generated from this database. The restricted-use data files contain

all responses and response-related data from the assessment, including responses from the student

booklets, Teacher Questionnaires, and School Characteristics and Policies Questionnaires, scale scores,

sampling weights, and variables used to compute standard errors.
169

8.2.2 Creating the Master Catalog

A critical part of any database is its processing control and descriptive information. Having a

central repository for this information, which may be accessed by all analysis and reporting programs,

will provide correct parameters for processing the data fields and consistent labeling for identifying the

results of the analyses. The NAEP master catalog file was designed and constructed to serve these

purposes for the NAEP database.

Each record of the master catalog contains the processing, labeling, classification, and location

information for each data field in the NAEP database. The control parameters are used by the access

routines in the analysis programs to define the manner in which the data values are to be transformed and

processed.

Each data field has a 50-character label in the master catalog describing the contents of the field

and, where applicable, the source of the field. The data fields with discrete or categorical response values

(e.g., multiple-choice items and professionally scored items, but not weight fields) have additional label

fields in the catalog containing 8- and 20-character labels for those response values. These short labels

can be used for reporting purposes as a concise description of the responses for the cognitive items.

The classification area of the master catalog record contains distinct fields corresponding to

predefined classification categories (e.g., mathematics content and process areas) for the data fields. For

a particular classification field, a nonblank value indicates the code of the subcategory within the

classification category for the data field. This classification area permits the grouping of identically

classified items or data fields by performing a selection process on one or more classification fields in the

master catalog.

According to NAEP design, it is possible for item data fields to appear in more than one

student sample and in more than one block within each sample. The location fields of the catalog record

contain age, block and, where applicable, the sequence within the block for each appearance of the data

field.

The master catalog file was constructed concurrently with the collection and transcription of the

State Assessment data so that it would be ready for use by analysis programs when the database was

created. As new data fields were derived and added to the database, their corresponding descriptive and

control information were entered into the master catalog. The machine-readable catalog files are

available as part of the secondary-use data files package for use in analyzing the data with programming

languages such as SAS and SPSS-X (see the NAEP 1996 Secondary-Use Data Files User Guide, Rogers,

Kline, & Schoeps, 1999).

8.3 QUALITY CONTROL OF NAEP DATA ENTRY FOR 1996

This section describes the evaluation of the data entry process for the 1996 national assessment.

As in past years, the NAEP database was found to be more than accurate enough to support the analyses

that were done. Overall, the observed error rates were comparable to those of past assessments, with the

possible exception of the Teacher Questionnaire data (see discussion below); they ranged from three

errors per 10,000 responses for the SD/LEP Student Questionnaire data to 33 errors per 10,000 responses

for the Teacher Questionnaire data.
170

The purpose of the analysis reported in this sction is to assess the quality of the data resulting

from the complete data entry system, beginning with the actual instruments collected in the field and

ending with the final machine-readable database used in the analyses. The process involved the selection

of instruments at random from among those returned from the field and the comparison of these

instruments, character by character, with their representations in the final database. In this way, we were

able to measure the error rates in the data as well as the success of the data entry system.

Of course the observed error rate cannot be taken at face value. For example, the sample of

teacher questionnaires that happened to be selected for close inspection contained 22 errors out of a total

of 6,741 characters. To conclude that the entire teacher questionnaire database has an error rate of

22/6741, or .0033, would be too optimistic; we may simply have been lucky (or unlucky) with this

particular random sample. What is needed is an indication of how bad the true error rate might be, given

what we observed. Such an indication is provided by confidence limits. Confidence limits indicate how

likely it is that a value falls inside a specified range in a specified context or distribution. In our analysis,

the specified range is an error rate between zero and some maximum value beyond which we are

confident at a specified level (traditionally 99.8%) that the true error rate does not lie; the specified

context or distribution turns out to be the cumulative binomial probability distribution. An example

should demonstrate this technique:

Let us say that 1,000 booklets were processed, each with 100 characters of data

transcribed for a total of 100,000 characters. Let us say further that five of these

characters were discovered to be in error in a random sample of 50 booklets that were

completely checked; in other words, five errors were found in a sample of 5,000

characters. The following expression may be used to establish the probability that the

true error rate is .0025 or less, rather than the single-value estimate of the observed rate

of one in a thousand (.001):

5000 0025 1 0025 0147

0

5

5000

j j

j j 



 



 ×− = =

× −

∑ .. .  	 


This is the sum of the probability of finding five errors plus the probability of finding

four errors plus. . . etc. . . plus the probability of finding zero errors in a sample of 5,000

with a true error rate of .0025; that is, the probability of finding five or fewer errors by

chance when the true error rate is .0025. Notice that we did not use the size of the

database in this expression. Actually, the assumption here is that our sample of 5,000

was drawn from a database that is infinite. The smaller the actual database is, the more

confidence we can have in the observed error rate; for example, had there been only

5,000 in the total database, our sample would have included all the data, and the

observed error rate would have been the true error rate. The result of the above

computation allows us to say, conservatively, that .0025 is an upper limit on the true

error rate with 98.53 percent (i.e., 1 - .0147) confidence; that is, we can be quite sure that

our true error rate is no larger than .0025.

Virtually all of the data collected for this assessment were machine-scanned. The only exception

was a set of six booklets used for the long-term trend reading and writing assessments; the format of

these booklets was kept the same for comparability with earlier assessments, so these booklets had to be

key-entered. As it happened, no errors at all were found in the sample of key-entered booklets selected

for quality control.
171

In the 1994 and 1996 assessments, the selection of booklets for this comparison took place at the

point of first entry into the recording process for data from the field. In earlier assessments, this selection

took place only after data had reached the final database, in order to assure that only relevant booklets

were involved in the quality control evaluation. The new selection process involves the risk that booklets

will be selected that ultimately will not appear in the final database, however, as in 1994, sufficient

numbers of booklets were in fact selected.

The individual instruments are briefly discussed in the following sections and a summary table

(Table 8-1) gives the upper 99.8 percent confidence limit for the error rate for each of the instruments as

well as the sampling information. The 99.8 percent confidence limit, and the selection rates noted, were

chosen to make these results comparable to those of previous administrations when the same parameters

were used.

Table 8-1

Summary of Quality Control Error Analysis for NAEP 1996 Data Entry

Main

Assessment

Student

Booklets

Long-Term

Trend

Assessment

Student Booklets

SD/LEP

Student

Questionnaires

Teacher

Questionnaires

School

Characteristics

and Policies

Questionnaires

Selection Rate 1/392 1/372 1/112 1/120 1/57

Different Booklets 203 26 2 4 3

Number of

Booklets Sampled

240 101 103 38 42

Number of

Characters

Sampled

30,134 12,082 13,098 6,741 6,633

Number of Errors 16 6 4 22 15

Observed

Error Rate

.0005 .0005 .0003 .0033 .0023

Upper 99.8%

Confidence Limit

.0011 .0015 .0011 .0058 .0045

8.3.1 Student Data

Data from about 94,000 students were processed across all samples in this assessment. Across all

the student data, roughly one booklet in 392 was selected for close examination, which is comparable to

the one in 400 target selection rate used in past assessments. The student data error rates were

consistently low in all subject areas and across all three grades. The overall quality of the data was very

high.

Data from some 30,000 additional students were also used in the study of long-term trends.

These data showed the same consistently low error rates, as indicated in the accompanying table.
172

8.3.2 SD/LEP Student Questionnaire Data

In this assessment, 13,098 SD/LEP questionnaires were scanned. The quality control sampling

rate was one in 112, a somewhat higher rate than that used in previous assessments. The data showed a

somewhat lower error rate than in previous assessments—comparable to that for the student data. The

few problems encountered involved the scanner’s mistaking an erasure for a genuine response or failing

to identify a multiple response as such.

8.3.3 Teacher Questionnaire Data

In this assessment, 4,585 teacher questionnaires were collected and scanned. About one

percent of these questionnaires was sampled for the quality control procedure. The error rates for these

questionnaires were higher than those of any other category of data, and also higher than those found for

teachers in past assessments. There was some evidence that the questionnaire layout was confusing to at

least some of the teachers in the part of the questionnaire that was intended to collect class period

information. While the majority of teachers did not have difficulty with this, consideration is being given

to a possible revision of the questionnaire, since a number of teachers also had similar problems in the

NAEP State Assessment.

8.3.4 School Characteristics and Policies Questionnaire Data

In this assessment, 2,404 School Characteristics and Policies Questionnaires were collected.

They were sampled at a rate of about 1 in 57. Fifteen scanning errors were found in these questionnaires,

which included both regular and long-term trend schools. Most of these errors came from a single booklet

that had been filled out in pen—something that frequently gives a scanner trouble.

8.4 NAEP DATABASE PRODUCTS

The NAEP database described to this point serves primarily to support analysis and reporting

activities that are directly related to the NAEP contract. This database has a singular structure and access

methodology that is integrated with the NAEP analysis and reporting programs. One of the directives of

the NAEP contract is to provide secondary researchers with a nonproprietary version of the database that

is portable to any computer system. In the event of transfer of NAEP to another client, the contract

further requires ETS to provide a full copy of the internal database in a format that may be installed on a

different computer system.

In fulfillment of these requirements, ETS provides two sets of database products: the item

information database and the secondary-use data files. The contents, format and usage of these products

are documented in the publications listed under the appropriate sections below.

8.4.1 The Item Information Database

The NAEP item information database contains all of the descriptive, processing, and usage

information for every assessment item developed and used for NAEP since 1970. The primary unit of this

database is the item. Each NAEP item is associated with different levels of information, including usage
173

across years and age cohorts, subject area classifications, response category descriptors, and locations of

response data on secondary-use data files.

The item information database is used for a variety of essential NAEP tasks: providing statistical

information to aid in test construction, determining the usage of items across assessment years and ages

for trend and “main” analyses, labeling summary analyses and reports, and organizing items by subject

area classifications for scaling analysis.

The creation, structure, and use of the NAEP item information database for all items used up to

and including the 1996 assessment are fully documented in the NAEP publications A Guide to the NAEP

Item Information Database (Rogers, Barone, & Kline, 1996) and A Primer for the NAEP Item

Information Database (Rogers, Kline, Barone, Mychajlowycz, & Forer, 1989).

The procedures used to create the 1996 version of the item information database are the same as

those documented in the guide. The version of the guide contains the subject area classification

categories for the cognitive items.

8.4.2 The Secondary-Use Data Files

The secondary-use data files are designed to enable any researcher with an interest in the NAEP

database to perform secondary analysis on the same data as those used at ETS. The data, documentation

and supporting files are distributed on CD-ROM media. For each sample in the assessment, the following

files are provided: the response data file; a printable codebook file; a file of control statements that will

generate an SPSS system file; a file of control statements that will generate a SAS system file; and a

machine-readable catalog file. Each codebook file is in portable document file (PDF) format, which may

be browsed, excerpted and printed using the Adobe Acrobat Reader program on a variety of platforms.

Each machine-readable catalog file contains sufficient control and descriptive information to permit the

user who does not have either SAS or SPSS to set up and perform data analysis.

The remainder of this section summarizes the procedures used in generating the data files and

related materials.

8.4.2.1 File Definition

The design of the 1996 assessment perpetuates two features of the 1990, 1992, and 1994

assessment design: the focused-BIB booklet design and the direct matching of teacher questionnaires to

student assessment instruments. In addition, the sample of students who were excluded from the

assessment is now incorporated into the appropriate assessed student subject area sample.

The focused-BIB design within the main assessment isolates the primary subject areas to separate

groups of booklets. This permits the division of the main sample into subject-specific subsamples. The

data files generated from these subsamples need only contain the data that are relevant to their

corresponding subject areas and are therefore smaller and more manageable than their counterparts in

previous assessments.

According to the design of the 1984, 1986, and 1988 assessments, only a sample of the teachers

of the assessed students were asked to fill out the teacher questionnaires. The large size of the

secondary-use main student files and the relatively low matching rate between students and teachers
174

made it impractical if not physically prohibitive to produce a complete file with student and teacher

information. Both the 1984 and 1986 secondary-use data packages had separate teacher data files which

could be linked to the student data files for analysis. The teacher file in the 1988 secondary-use data

package contained not only the teacher response data, but also the data from the students who could be

matched to teacher questionnaires. This type of file was more appropriate for the analysis of teacher data

because it defined the student as the unit of observation.

The intent of the 1996 assessment design was to collect data from mathematics or science

teachers of the main assessment students at specified grade levels who were administered mathematics or

science booklets. A portion of the teacher questionnaire contained questions that were directly related to

each matched student. This change in the design afforded a very high matching rate between student and

teacher data. Therefore, for those subject areas in each grade cohort for which teacher data were

collected, the teacher responses were appended to each student record in the secondary-use data files.

8.4.2.2 Definition of the Variables

The initial step in the variable definition process was the generation of a LABELS file of

descriptors of the variables for each data file to be created. Each record in a LABELS file contains, for a

single data field, the variable name, a short description of the variable, and processing control

information to be used by later steps in the data generation process. This file could be edited for deletion

of variables, modification of control parameters, or reordering of the variables within the file. The

LABELS file is an intermediate file only; it is not included on the released data files.

The variables on all data files are grouped and arranged in the following order: identification

information, weights, derived variables, proficiency scores (where applicable), and response data. On the

student data files, these fields are followed by the teacher response data and the SD/LEP student

questionnaire data, where applicable. The identification information is taken from the front covers of the

instruments. The weight data include sample descriptors, selection probabilities, and replicate weights for

the estimation of sampling error. The derived data include sample descriptions from other sources and

variables that are derived from the response data for use in analysis or reporting.

For each subject area of the main assessment, the item response data within each block were left

in their order of presentation. The blocks, however, were arranged according to the following scheme:

common background, subject-related background, the cognitive blocks in ascending numerical order, and

student motivation. The responses to cognitive blocks that were not present in a given booklet were left

blank, signifying a condition of “missing by design.”

In order to process and analyze the spiral sample data effectively, the user must also be able to

determine, from a given booklet record, which blocks of item response data were present and their

relative order in the instrument. This problem was remedied by the creation of a set of control variables,

one for each block, which indicated not only the presence or absence of the block but its order in the

instrument. These control variables are included with the derived variables.

8.4.2.3 Data Definition

To enable the data files to be processed on any computer system using any procedural or

programming language, it was desirable that the data be expressed in numeric format. This was possible,

but not without the adoption of certain conventions for reexpressing the data values.
175

During creation of the NAEP database, the responses to all multiple-choice items were

transcribed and stored in the database using the letter codes printed in the instruments. This scheme

afforded the advantage of saving storage space for items with 10 or more response options, but at the

expense of translating these codes into their numeric equivalents for analysis purposes. The response data

fields for most of these items would require a simple alphabetic-to-numeric conversion. However, the

data fields for items with 10 or more response choices would require “expansion” before the conversion,

since the numeric value would require two column positions. One of the processing control parameters

on the LABELS file indicates whether or not the data field is to be expanded before conversion and

output.

The ETS database contained special codes to indicate certain response conditions: “I don’t

know” responses, multiple responses, omitted responses, not-reached responses, and unresolvable

responses, which include out-of-range responses and responses that were missing due to errors in printing

or processing. The scoring guides for the mathematics and science constructed-response items included

additional special codes for ratings of “illegible,” “I don’t know,” “off task,” or non-rateable by the

scorers. All of these codes had to be reexpressed in a consistent numeric format.

The following convention was adopted and used in the designation of these codes: The

“illegible” response codes were converted to 5; the “off task” response codes were converted to 6; the “I

don’t know” and non-rateable response codes were converted to 7; the “omitted” response codes were

converted to 8; the “not reached” response codes were converted to 9; and the multiple response codes

were converted to 0; and. The out-of-range and missing responses were coded as blank fields,

corresponding to the “missing by design” designation.

This coding scheme created conflicts for those multiple-choice items that had seven or more

valid response options as well as the “I don’t know” response and for those constructed-response items

whose scoring guide had five or more categories. These data fields were also expanded to accommodate

the valid response values and the special codes. In these cases, the special codes were “extended” to fill

the output data field: the “I don’t know” and non-rateable codes were extended from 7 to 77, omitted

response codes from 8 to 88, etc.

Each numeric variable on the secondary-use files was classified as either continuous or discrete.

The continuous variables include the weights, proficiency scores, identification codes, and item

responses where counts or percentages were requested. The discrete variables include those items for

which each numeric value corresponds to a response category. The designation of “discrete” also

includes those derived variables to which numeric classification categories have been assigned. The

constructed-response items were treated as a special subset of the discrete variables and were assigned to

a separate category to facilitate their identification in the documentation.

8.4.2.4 Data File Catalogs

The CATALOG file is created by the GENCAT program from the LABELS file and the 1996

master catalog file. Each record on the LABELS file generates a CATALOG record by first retrieving the

master catalog record corresponding to the field name. The master catalog record contains usage,

classification, and response code information, along with positional information from the LABELS file:

field sequence number, output column position, and field width. Like the LABELS file, the CATALOG

file is an intermediate file and is not included on the released data files.
176

The information for the response codes, also referred to as “foils,” consists of the valid data

values for the discrete numeric fields, and a 20-character description of each. The GENCAT program

uses additional control information from the LABELS file to determine if extra foils should be generated

and saved with each CATALOG record. The first flag controls generation of the “I don’t know” or nonrateable foil; the second flag regulates omitted or not-reached foil generation; and the third flag denotes

the possibility of multiple responses for that field and sets up an appropriate foil. All of these control

parameters, including the expansion flag, may be altered in the LABELS file by use of a text editor, in

order to control the generation of data or descriptive information for any given field.

The LABELS file supplies control information for many of the subsequent secondary-use data

processing steps. The CATALOG file provides detailed information for those and other steps.

8.4.2.5 Data File Layouts

The data file layouts were the first user product to be generated in the secondary-use data files

process. The generation program, GENLYT, used a CATALOG file as input and produced a printable

file. The LAYOUT file is little more than a formatted listing of the CATALOG file.

Each line of the LAYOUT file contains the following information for a single data field:

sequence number, field name, output column position, field width, number of decimal places, data type,

value range, key or correct response value, and a short description of the field. The sequence number of

each field is implied from its order on the LABELS file. The field name is an 8-character label for the

field that is to be used consistently by all secondary-use data files materials to refer to that field on that

file. The output column position is the relative location of the beginning of that field on each record for

that file, using bytes or characters as the unit of measure. The field width indicates the number of

columns used in representing the data values for a field. If the field contains continuous numeric data, the

value under the number of decimal places entry indicates how many places to shift the decimal point

before processing data values.

The data type category uses five codes to designate the nature of the data in the field: Continuous

numeric data are coded “C;” discrete numeric data are coded “D;” constructed-response item data are

coded “OS;” if the item was dichotomized for scaling and “OE;” if it was scaled under a polytomous

response model. Additionally, the discrete numeric fields that include “I don’t know” response codes are

coded “DI.” If the field type is discrete numeric, the value range is listed as the minimum and maximum

permitted values separated by a hyphen to indicate range. If the field is a response to a scorable item, the

correct option value, or key, is printed; if the field is an assigned score that was scaled as a dichotomous

item using cut point scoring, the range of correct scores is printed. Each variable is further identified by a

50-character descriptor.

8.4.2.6 Data Codebooks

The data codebook is a printed document containing complete descriptive information for each

data field. Most of this information originates from the CATALOG file; the remaining data comes from

the COUNTS file and the IRT parameters file.

Each data field receives at least one line of descriptive information in the codebook. If the data

type is continuous numeric, no more information is given. If the variable is discrete numeric, the

codebook lists the foil codes, foil labels, and frequencies of each value in the data file. Additionally, if
177

the field represents an item used in IRT scaling, the codebook lists the parameters used by the scaling

program.

Certain blocks of cognitive items in the 1996 assessment that are to be used again in later

assessments for trend comparisons have been designated as nonreleased. In order to maintain their

confidentiality, generic labels have been substituted for the response category descriptions of these items

in the data codebooks and the secondary-use files.

The frequency counts are not available on the catalog file, but must be generated from the data.

The GENFREQ program creates the COUNTS file using the field name to locate the variable in the

database, and the foil values to validate the range of data values for each field. This program also serves

as a check on the completeness of the foils in the CATALOG file, as it flags any data values not

represented by a foil value and label.

The IRT parameter file is linked to the CATALOG file through the field name. Printing of the

IRT parameters is governed by a control flag in the classification section of the CATALOG record. If an

item has been scaled for use in deriving the proficiency estimates, the IRT parameters are listed to the

right of the foil values and labels, and the score value for each response code is printed to the immediate

right of the corresponding frequency.

The LAYOUT and CODEBOOK files are written by their respective generation programs to

print-image disk data files. Draft copies are printed and distributed for review before the production copy

is generated. The production copy combines the LAYOUT and CODEBOOK files for each sample in a

portable document file (PDF) format. This file may be browsed, excerpted and printed using the Adobe

Acrobat Reader program on a variety of platforms and operating systems.

8.4.2.7 Control Statement Files for Statistical Packages

An additional requirement of the NAEP cooperative agreement is to provide, for each

secondary-use data file, a file of control statements each for the SAS and SPSS statistical systems that

will convert the raw data file into the system data file for that package. Two separate programs, GENSAS

and GENSPX, generate these control files using the CATALOG file as input.

Each of the control files contains separate sections for variable definition, variable labeling,

missing value declaration, value labeling, and creation of scored variables from the cognitive items. The

variable definition section describes the locations of the fields, by name, in the file, and, if applicable, the

number of decimal places or type of data. The variable label identifies each field with a 50-character

description. The missing value section identifies values of those variables that are to be treated as

missing and excluded from analyses. The value labels correspond to the foils in the CATALOG file. The

code values and their descriptors are listed for each discrete numeric variable. The scoring section is

provided to permit the user to generate item score variables in addition to the item response variables.

Each of the code generation programs combines three steps into one complex procedure. As each

CATALOG file record is read, it is broken into several component records according to the information

to be used in each of the resultant sections. These record fragments are tagged with the field sequence

number and a section sequence code. They are then organized by section code and sequence number.

Finally, the reorganized information is output in a structured format dictated by the syntax of the

processing language.
178

The generation of the system files accomplishes the testing of these control statement files. The

system files are saved for use in special analyses by NAEP staff. These control statement files are

included on the distributed data files to permit users with access to SAS and/or SPSS to create their own

system files.

8.4.2.8 Machine-Readable Catalog Files

For those NAEP data users who have neither SAS nor SPSS capabilities, yet require processing

control information in a computer-readable format, the distribution files also contain machine-readable

catalog files. Each machine-readable catalog record contains processing control information, IRT

parameters, and foil codes and labels.

8.4.2.9 NAEP Data on Disk

The complete set of secondary-use data files described above are available on CD-ROM as part

of the NAEP Data on Disk product suite. This medium can be ideal for researchers and policy makers

operating in a personal computing environment.

The NAEP Data on Disk product suite includes two other components which facilitate the

analysis of NAEP secondary-use data. The PC-based NAEP data extraction software, NAEPEX, enables

users to create customized extracts of NAEP data and to generate SAS or SPSS control statements for

preparing analyses or generating customized system files. The NAEP analysis modules, which currently

run under SPSS7

 for Windows, use output files from the extraction software to perform analyses that

incorporate statistical procedures appropriate for the NAEP design.
179

Chapter 9

OVERVIEW OF PART II:

THE ANALYSIS OF 1996 NAEP DATA1

Nancy L. Allen and James E. Carlson

Educational Testing Service

9.1 INTRODUCTION

The purpose of this chapter is to summarize some information from previous chapters that is

integral to the analysis of NAEP data, to summarize the analysis steps used for all subjects, and to

indicate what information is in each of the remaining chapters. The overview of the analyses conducted

on the 1996 NAEP data focuses on the common elements of the analyses used across the subject areas of

the assessment. Some of this information is available only within this chapter. Details by subject area are

provided in Chapters 12 through 17.

The organization of this chapter is as follows:

• Section 9.2 provides a short overview of the NAEP design for 1996. To provide

additional background information, the section also provides a short description of

the samples selected for 1996. Chapters 1 through 8 provide this same information in

much more detail.

• Section 9.3 summarizes the steps in analysis common to all subject areas. Some of

this information is described in more detail in other chapters. The rest is included

only within this chapter. The topics covered are as follows:

• Section 9.3.1 briefly describes the preparation of the final sampling weights.

Detailed information about the weighting procedures is given in Chapter 10. Detailed

information about the sampling design is in Chapter 3.

• Section 9.3.2 provides information about the scoring reliability of constructedresponse items. It provides information about the reliability measures used with the

NAEP data during analysis. Chapter 7 contains information about the reliability

procedures used during the scoring process.

• Section 9.3.3 summarizes the information provided by the teacher questionnaires,

and indicates its use during the analysis process.

• Section 9.3.4 provides a description of the item properties examined for background

questions and for cognitive items. It includes a description of the classical item

statistics examined for both dichotomously and polytomously scored items. It also

 

1

 Nancy L. Allen was responsible for the psychometric and statistical analyses of national and state NAEP data. James E. Carlson

was responsible for psychometric and statistical analyses relating to special aspects and issues of NAEP. Eugene G. Johnson,

John Mazzeo, Spencer S. Swinton, and Rebecca Zwick also contributed to this chapter.
180

includes a description of the item-level results available from summary data tables.

Chapter 18 contains more information about the conventions used in creating these

summary tables. Finally, a thorough description of differential item functioning

analyses is provided.

• Section 9.3.5 summarizes the steps used to scale NAEP data. The steps include IRT

scaling of the items, generation of plausible values (conditioning), transforming the

results to the final reporting scale, creating composite scores if necessary, and

providing tables of reported statistics. Details of the theory behind these steps are

available in Chapter 11.

• Section 9.3.6 provides some information about previous results of dimensionality

analyses.

• Finally, Section 9.3.7 gives an introduction to hypothesis testing and drawing correct

conclusions about NAEP data. Specific information about which hypothesis test

procedures were used for different purposes is provided in Chapter 18.

• Section 9.4 contains a description of the information provided in Chapters 10

through 19 of this report.

9.2 SUMMARY OF THE NAEP DESIGN

As described in Chapter 1, the 1996 NAEP comprised three major components. One component

encompassed major assessments in mathematics and science, providing detailed information about

student proficiency at the fourth-, eighth-, and twelfth-grade levels of nonpublic and public schools.

Long-term trend assessments of science, mathematics, and reading at ages 9, 13, and 17, and the longterm trend writing assessment for grades 4, 8, and 11, constituted the second component. The third major

component was the State Assessment at the fourth- and eighth-grade levels in mathematics and at the

eighth-grade level in science. Technical details of the State Assessments are not included in this report

but are presented in the Technical Report of the NAEP 1996 State Assessment Program in Mathematics

(Allen, Jenkins, Kulick, & Zelenak, 1997) and the Technical Report of the NAEP 1996 State Assessment

Program in Science (Allen, Swinton, Isham, & Zelenak, 1997).

In addition to the three major components, special studies of advanced mathematics students at

the eighth- and twelfth-grade levels and advanced science students at the twelfth-grade level were

conducted. The results from and procedures used in these special studies are reported in separate

documents. Likewise, results from a study of holistic scores for the long-term trend writing task

responses are reported in a separate document. Results based on primary trait scores for the same writing

tasks are reported in the NAEP 1996 Trends in Academic Progress (Campbell, Voelkl, & Donahue,

1997), and the analyses contributing to those results are described in Chapters 14-17 of this document.

Finally, results for the items associated with specific mathematics themes are reported elsewhere.

Results from the analyses described in the following chapters were reported in the following

reports:

• The NAEP 1996 Mathematics Report Card for the Nation and the States, which

provides both public- and nonpublic-school data for major NAEP reporting
181

subgroups for all of the jurisdictions that participated in the State Assessment program, as

well as selected results from the 1996 national mathematics assessment.

• The NAEP 1996 Science Report Card for the Nation and the States, which provides both

public- and nonpublic-school data for major NAEP reporting subgroups for all of the

jurisdictions that participated in the State Assessment program, as well as selected results

from the 1996 national science assessment.

• The Cross-State Data Compendium for the NAEP 1996 Mathematics Assessment, which

includes jurisdiction-level results for all the demographic, instructional, and experiential

background variables included in the Mathematics Report Card and State Reports.

• The Cross-State Data Compendium for the NAEP 1996 Science Assessment, which

includes jurisdiction-level results for all the demographic, instructional, and experiential

background variables included in the Science Report Card and State Reports.

• The NAEP 1996 Trends in Academic Progress, which looks at trends in average

performance over time in the areas of mathematics, science, reading, and writing. [Please

note that a review of the 1996 long-term trend writing assessment data analyses has been

undertaken by NCES. Additional analyses may be required before revised results are

released.]

Because the samples of students included in the 1996 NAEP assessment are listed and described in

detail in Chapter 1, only a brief description of these samples is given here. The 1996 national samples were

of three general types: main NAEP samples, which were based on a common set of assessment procedures,

including grade-level samples; long-term trend samples, the purpose of which was to provide links to

earlier assessments; and special study samples, used to examine results for advanced mathematics and

science students.

To shorten the timetable for reporting results, the period for national main assessment data

collection was shortened in 1992, 1994, and 1996 from the five-month period (January through May) used

in 1990 and earlier assessments to a three-month period in the winter (January through March,

corresponding to the period used for the winter half-sample of the 1990 National Assessment). [Please note

that a review of the 1996 long-term trend writing assessment data analyses has been undertaken by NCES.

Additional analyses may be required before revised results are released.]

The 1996 analyses of long-term trend data extended the trend lines commencing in 1971 in

reading, 1973 in mathematics, 1969 in science, and 1984 in writing.

As described in Chapters 1 and 4, for each subject area in the main assessment, blocks of items

were used to create a large number of different assessment booklets according to a focused design. The

1996 mathematics assessment used a focused balanced incomplete block (focused-BIB) design while the

1996 science assessment used a more complex design due to the inclusion of blocks of items associated

with a specific theme or hands-on performance task. The focused-BIB design provided for booklets that

typically included three blocks of cognitive items in a single subject area, as well as background items. The

blocks of cognitive items for mathematics and science included both multiple-choice and constructedresponse items. In a focused-BIB design, each block of cognitive items appears in the same number of

booklets. To balance possible block position main effects, each block appears an equal number of times in

each position. In addition, the BIB design requires that each block of items be paired in a booklet with

every other block of items exactly once.
182

9.3 ANALYSIS STEPS

Because the analysis methods are not identical across subject areas or across major national and

long-term trend samples, a separate analysis chapter has been included for each major assessment and for

each long-term trend assessment. The procedures used depended on whether assessment items were

scored dichotomously (right versus wrong) or polytomously (more than two categories of response) and

whether links across grade levels were required. Basic procedures common to most or all of the subject

area analyses are summarized here. The order is essentially that in which the procedures were carried out.

9.3.1 Preparation of Final Sampling Weights

Because NAEP uses a complex sampling design (Chapter 3) in which students in certain

subpopulations have different probabilities of inclusion in the sample, the data collected from each

student must be assigned a weight to be used in analyses. The 1996 NAEP weights were provided by

Westat, Inc., the NAEP contractor in charge of sampling. Detailed information about the weighting

procedures is available in Chapter 10 and in The 1996 NAEP Sampling and Weighting Report (Wallace &

Rust, 1999).

9.3.2 Reliability of Scoring Constructed-Response Items

A minimum of 25 percent of the responses for science items involved only in the national

assessment and six percent of the responses for mathematics and science items involved in both the

national and state assessments were scored by a second reader to obtain statistics on interreader

(interrater) reliability. Ranges for percentage of exact agreement for state and national assessments,

together, of mathematics and science can be found in Table 9-1. Average percentage of exact agreement

for each booklet type (spiral and advanced) can be found in Tables 7-5 and 7-6 in Chapter 7. This

reliability information was also used by the team leaders to monitor the capabilities of all readers and

maintain uniformity of scoring across readers. More information about this use of the reliability

information is in Chapter 7.

Table 9-1

1996 Mathematics and Science State and National Assessments

Ranges of Percentage Exact Agreement Among Readers

Number of

Unique Items

Number of Items in Percentage

Exact Agreement Range

Assessment Total 70-79% 80-89% Above 90%

Mathematics

4

th grade 79 0 11 68

8

th grade 98 1 5 92

12th grade 96 1 9 86

Science

4

th grade 94 0 13 81

8

th grade 125 0 20 105

12th grade 156 0 26 130
183

In addition to reliability information calculated and used during the scoring process, several

additional reliability measures are calculated for constructed-response items after the item response data

has been placed on the NAEP database. They appear in Appendix I. These include a final percentage

exact agreement, the intraclass correlation, Cohen’s Kappa (Cohen, 1968), and the product-moment

correlation between the scores for the first and second readers. These measures are summarized in Zwick

(1988), Kaplan and Johnson (1992), and Abedi (1996). Each measure has advantages and disadvantages

for use in different situations. In this report, the percentage exact agreement is reported for all

constructed-response items, Cohen’s Kappa is reported for dichotomously scored constructed-response

items, and the intraclass correlation is reported for polytomously scored constructed-response items.

9.3.3 Teacher Questionnaires

Teachers of students who were in the fourth- and eighth-grade mathematics and science main

assessment samples and twelfth-grade advanced mathematics samples were asked to complete a two-part

questionnaire. The first part of the questionnaire pertained to the teacher’s background and training

(Parts I and IIA in Chapter 2). The second part pertained to the procedures used by the teacher for

specific classes containing assessed students (Part IIB in Chapter 2). See Chapter 2 for a description of

the teacher questionnaires.

To analyze the data from the teacher questionnaires with respect to the students’ data, each

teacher’s questionnaire had to be matched to all of the sampled students who were taught by that teacher.

In the subsequent chapters two separate match rates for each grade are given. The first is the percentage

of students that could be matched to both the first and second parts of the teacher questionnaire. For these

students, information is available not only about the background and training of their teachers, but also

about the methods used in the particular class they attended. The second match rate is the percentage of

students that could be matched to the first part of the teacher questionnaire. This match rate is larger

because more students could be matched with information about a teacher than with information about

the particular class they attended. Note that these match rates only reflect the student-level missing data.

They do not reflect the additional missing data due to item-level nonresponse on the part of teachers.

Variables derived from the teacher questionnaires were used as reporting variables at the student level

and as variables that contributed to conditioning for the appropriate samples.

Teachers of students who were in the grade 4 mathematics main assessment sample were asked

to complete a two-part questionnaire. As with the grade 8 teacher questionnaire, the first part pertained to

the teacher’s background and training. Unlike the grade 8 teacher questionnaire, the second part

pertained to only a single class that the teacher taught. In development of the questionnaires, it was

thought that fourth-grade teachers would teach one class in each subject. In practice, that was found to be

untrue for a number of teachers. A single student-teacher match rate matching students to the first part of

the questionnaire is reported for grade 4 in the following chapters.

9.3.4 Analysis of Item Properties: Background and Cognitive Items

The first step in the analysis of the 1996 data was item-level analysis of all instruments. Item

analyses were performed separately for each grade or age level on each item in each subject area. Each

block of items was analyzed separately, by age or grade, with the total score on the block (including the

analyzed item) used as the criterion score for statistics requiring such a score. In the cases where final

weights were not available, preliminary weights were used in these preliminary analyses. The item
184

analysis of cognitive items was repeated after scaling of the items was completed. The results for only

scaled items using final weights are reported in Chapters 12 through 17.

 Background Items

For each NAEP background item, the unweighted and weighted percent of students who gave

each response were examined, as well as the percent of students who omitted the item and the percent

who did not reach the item. The number of respondents was also tabulated. These preliminary analyses

were conducted within age/grade cohorts and within major reporting categories. If unexpected results

were found, the item data and the encoding of responses were rechecked.

 Cognitive Items

All NAEP cognitive items were subjected to analyses of item properties. These analyses included

conventional item analyses and incorporated examinee sampling weights. Item analysis was conducted at

the block level so that the “number-correct” scores for students responding to an item, selecting each

option of an item, omitting an item, or not reaching an item, is the average number of correct responses

for the block containing that item. Because of the inclusion of polytomously scored items in the cognitive

instruments, it was necessary to use special procedures for these items. The resulting statistics are

analogous to those for the dichotomously scored items, as listed below.

Dichotomously Scored Items. These items were analyzed using standard procedures that result

in a report for each item that includes:

 for each option of the item, for examinees omitting and not reaching the item, and for

the total sample of examinees:

⇒ the number of examinees,

⇒ the percentage of examinees,

⇒ the mean of number-correct scores for the block in which the item appears, and

⇒ the standard deviation of number-correct scores for the block in which the item

appears;

• the percentage of examinees providing a response that was “off-task;”

• p+, the proportion of examinees that received a correct score on the item (ratio of

number correct to number correct plus wrong plus omitted);

• , the inverse-normally transformed p+ scaled to mean 13 and standard deviation 4;

• the biserial correlation coefficient between the item and the number-correct scores

for the block in which the item appears; and

• the point-biserial correlation coefficient between the item and the number-correct

scores for the block in which the item appears.
185

Polytomously Scored Items. Enhanced procedures were employed for polytomously scored

items. Methods parallel to those used for dichotomously scored items resulted in values reported for each

distinct response category for the item. Response categories for each item were defined in two ways, one

based on the original codes for responses as specified in the scoring rubrics used by the scorers (and the

position of the item for which no response was given) and one based on a scoring guide developed by

subject area and measurement experts. For example, a constructed-response item with four response

categories would initially have seven categories (not-reached, omitted, “off-task,” and the four valid

response categories). Another set of statistics resulted from mapping the response categories (excluding

not-reached) into a new set of categories reflecting the scoring guide for the item. A constructed-response

item with ordered categories, for example, would be mapped into a set of integers in a corresponding

order. The scoring guide could result in the collapsing of (combining of) some response categories. The

response categories, based on the final scoring guide developed by subject area and measurement experts,

were used to calculate the polytomously scored item statistics.

The following statistics, analogous to those for dichotomously scored items, were computed:

• the percentage of examinees providing a response that was “off-task;”

• in place of p+, the ratio of the mean item score to the maximum-possible item score

was used;

• in place of , the ratio of the mean item score to the maximum-possible item score

underwent the same transformation as that used on p+ to get  for dichotomously

scored items;

• the polyserial correlation coefficient was used in place of the biserial; and

• the Pearson correlation coefficient was used in place of the point-biserial.

 Tables of Item-Level Results

Tables were created of the percentages of students choosing each of the possible responses to

each item within each of the samples administered in 1996. The results for each item were crosstabulated against the basic reporting variables such as region, gender, race/ethnicity, public/nonpublic

school, and parental education. All percentages were computed using the sampling weights. These tables

are referred to as the Test Question section of the electronically delivered summary data tables for each

sample (see Chapter 18 for a brief description of summary data tables). In the summary data tables, the

sampling variability of all population estimates was obtained by the jackknife procedure used by ETS in

previous assessments. Details of these procedures are presented in Chapter 10.

 Differential Item Functioning Analysis of Cognitive Items

Differential item functioning (DIF) analysis refers to procedures to assess whether items are

differentially difficult for different groups of examinees. DIF procedures typically control for overall

between-group differences on a criterion, usually by matching examinees from the two groups on overall

test scores. Between-group performance on each item is then compared within sets of examinees having

the same total test scores.
186

DIF analyses were conducted for items in the main assessments in mathematics and science that

had not previously been studied for differential item functioning. Each set of analyses involved three

reference group/focal group comparisons: male/female, White/Black, and White/Hispanic.

The DIF analyses of the dichotomous items were based on the Mantel-Haenszel chi-square

procedure, as adapted by Holland and Thayer (1988). The procedure tests the statistical hypothesis that

the odds of correctly answering an item are the same for two groups of examinees that have been

matched on some measure of proficiency (usually referred to as the matching criterion). The DIF

analyses of the polytomous items were based on the Mantel procedure (Mantel, 1963). These procedures

compare proportions of matched examinees from each group in each polytomous item response category.

The groups being compared are often referred to as the focal group (usually a minority or other group of

interest, such as Black examinees or female examinees) and the reference group (usually White

examinees or male examinees).

For both types of analyses, the measure of proficiency used is typically the total item score on

some collection of items. Since, by the nature of the BIB design, booklets comprise different

combinations of blocks, there is no single set of items common to all examinees. Therefore, for each

student, the measure of proficiency used was the total item score on the entire booklet. These scores were

then pooled across booklets for each analysis. This procedure is described by Allen and Donoghue (1994,

1996). In addition, because research results (Zwick & Grima, 1991) strongly suggest that sampling

weights should be used in conducting DIF analyses, the weights were used.

For each dichotomous item in the assessment, an estimate of the Mantel-Haenszel common oddsratio, expressed on the ETS delta scale for item difficulty, was produced. The estimates indicate the

difference between reference group and focal group item difficulties (measured in ETS delta scale units),

and typically run between about +3 and -3. Positive values indicate items that are differentially easier for

the focal group than the reference group after making an adjustment for the overall level of proficiency in

the two groups. Similarly, negative values indicate items that are differentially harder for the focal group

than the reference group. It is common practice at ETS to categorize each item into one of three

categories (Petersen, 1988): “A” (items exhibiting no DIF), “B” (items exhibiting a weak indication of

DIF), or “C” (items exhibiting a strong indication of DIF). Items in category “A” have Mantel-Haenszel

common odds ratios on the delta scale that do not differ significantly from 0 at the alpha = .05 level or

are less than 1.0 in absolute value. Category “C” items are those with Mantel-Haenszel values that are

significantly greater than 1 and larger than 1.5 in absolute magnitude. Other items are categorized as “B”

items. A plus sign (+) indicates that items are differentially easier for the focal group; a minus sign (-)

indicates that items are differentially more difficult for the focal group.

The ETS/NAEP DIF procedure for polytomous items uses the Mantel-Haenszel ordinal

procedure. The summary tables of identified polytomous items contain generalizations of the

dichotomous “A,” “B,” and “C” categories: “AA,” “BB,” or “CC.”

All analyses used rescaled sampling weights. A separate rescaled weight was defined for each

comparison as

where the total sample size is the total number of students for the two groups being analyzed (e.g., for the

White/Hispanic comparison, the total number of White and Hispanic examinees in the sample at that

grade), and the sum of the weights is the sum of the sampling weights of all the students in the sample for

Rescaled Weight = Original Weight x Total Sample Size

Sum of the Weights
187

the two groups being analyzed. Three rescaled weights were computed for White examineesone for the

gender comparison and two for the race/ethnicity comparisons. Two rescaled overall weights were

computed for the Black and Hispanic examineesone for the gender comparison and another for the

appropriate race/ethnicity comparison. The rescaled weights were used to ensure that the sum of the

weights for each analysis equaled the number of students in that comparison, thus providing an accurate

basis for significance testing.

In the calculation of total item scores for the matching criterion, both not-reached and omitted

items were considered to be wrong responses. Polytomous items were weighted more heavily in the

formation of the matching criterion, proportional to the number of score categories. For each item,

calculation of the Mantel-Haenszel statistic did not include data from examinees who did not reach the

item in question.

Each DIF analysis was a two-step process. In the initial phase, total item scores were formed, and

the calculation of DIF indices was completed. Before the second phase, the matching criterion was

refined by removing all “C” or “CC” items, if any, from the total item score. The revised score was used

in the final calculation of all DIF indices. Note that when analyzing an item classified as “C” or “CC” in

the initial phase, that item score is added back into the total score for the analysis of that item only.

Following standard practice at ETS for DIF analyses conducted on final forms, all “C” or “CC”

items were reviewed by a committee of trained test developers and subject-matter specialists. Such

committees are charged with making judgments about whether or not the differential difficulty of an item

is unfairly related to group membership. The committee assembled to review NAEP items included both

ETS staff and outside members with expertise in the field. The committee carefully examined each

identified item to determine if either the language or contents would tend to make the item more difficult

for an identified group of examinees. It was the committee’s judgment that none of the “C” or “CC”

items in the national assessment were functioning differentially due to factors irrelevant to test

objectives. Hence, none of the items were removed from scaling due to differential item functioning. As

pointed out by Zieky (1993):

It is important to realize that DIF is not a synonym for bias. The item response theory based

methods, as well as the Mantel-Haenszel and standardization methods of DIF detection, will identify

questions that are not measuring the same dimension(s) as the bulk of the items in the matching

criterion....Therefore, judgment is required to determine whether or not the difference in difficulty

shown by a DIF index is unfairly related to group membership. The judgment of fairness is based on

whether or not the difference in difficulty is believed to be related to the construct being

measured....The fairness of an item depends directly on the purpose for which a test is being used.

For example, a science item that is differentially difficult for women may be judged to be fair in a

test designed for certification of science teachers because the item measures a topic that every entrylevel science teacher should know. However, that same item, with the same DIF value, may be

judged to be unfair in a test of general knowledge designed for all entry-level teachers. (p. 340)

9.3.5 Scaling

Scales based on item response theory (IRT) were derived for each subject area. A single scale

was used for summarizing long-term trends at each age or grade level in each of the subject areas. Five

scales were created for mathematics main assessment data, one for each mathematics content strand, and

three scales were created for science data, one for each field of science. NAEP uses the methodology of

multiple imputations (plausible values) to estimate characteristics of the proficiency distributions.
188

Chapter 11 describes in detail the theoretical underpinnings of NAEP’s scaling methods and the required

estimation procedures. The basic analysis steps are outlined here.

1. Use the NAEP-BILOG/PARSCALE computer program (described in Chapter 11) to

estimate the parameters of the item response functions on an arbitrary provisional

scale. This program uses an IRT model incorporating the two- and three-parameter

logistic forms used in previous assessments for dichotomously scored items and the

generalized partial credit form for polytomously scored items. In order to select

starting values for the iterative parameter-estimation procedure for each dataset, the

program is first run to convergence, imposing the condition of a fixed normal prior

distribution of the proficiency variable. Once these starting values are computed, the

main estimation runs model ability as a multinomial distribution. That is, no prior

assumption about the shape of the proficiency distribution is made. In analyses

involving more than one population, estimates of parameters are made with the

overall mean and standard deviation of all subjects’ proficiencies specified to be 0

and 1, respectively.

2. Use a version of the MGROUP program (described in Chapter 11), which

implements the method of Mislevy (see Chapter 11 or Mislevy, 1991) to estimate

predictive proficiency distributions for each student on an arbitrary scale, based on

the item parameter estimates and the student’s responses to cognitive items and

background questions.

3. Use random draws from these predictive proficiency distributions (plausible values,

in NAEP terminology) for computing the statistics of interest, such as mean

proficiencies for demographic groups.

4. Determine the appropriate metric for reporting the results and transform the results

as needed. This includes the linking of current scales to scales from the past or the

selection of the mean and variance of new scales. After proficiency distributions for

the scaling are transformed, composite proficiency distributions are created for the

mathematics and science assessments.

5. Use the jackknife procedure to estimate the standard errors of the mean proficiencies

for the various demographic groups.

As explained in Chapter 11, the plausible values obtained through the IRT approach are not

optimal estimates of individual proficiency; instead, they serve as intermediate values to be used in

estimating subpopulation characteristics. Under the assumptions of the scaling models, these

subpopulation estimates are statistically consistent, which would not be true of subpopulation estimates

obtained by aggregating optimal estimates of individual proficiency.

 Scaling the Cognitive Items

The data from both the trend and the main assessment samples were scaled using IRT models.

For dichotomously scored items two- and three-parameter logistic forms of the model were used, while

for polytomously scored items the generalized partial credit model form was used. These two types of

items and models were combined in the NAEP scales. Item parameter estimates on a provisional scale

were obtained using the NAEP BILOG/PARSCALE program. The fit of the IRT model to the observed
189

data was examined within each scale by comparing the empirical item response functions with the

theoretical curves, as described in Chapter 11. Plots of the empirical item response functions and

theoretical curves were compared across assessments for long-term trend assessments. The DIF analyses

previously described provide information related to the model fit across subpopulations.

Long-Term Trend Scaling. Item parameters for science, mathematics, reading, and writing

trends were reestimated, separately for each age or age/grade group using the data from the 1994

assessment as well as data from the 1996 assessment. The resulting scales, based on these reestimated

item parameters, were then linked to the existing long-term trend scales. [Please note that a review of the

1996 long-term trend writing assessment data analyses has been undertaken by NCES. Additional

analyses may be required before revised results are released.]

Main Assessment Scaling. The main assessments of mathematics and science both have special

characteristics that determine the procedures that were followed for the scaling and of each subject. For

mathematics, a key consideration was the degree of similarity between the 1996 assessment and earlier

assessments in terms of the populations assessed and the characteristics of the assessment instrument

used. This was due to the fact that the mathematics scales were linked to existing mathematics scales. For

science, characteristics of items associated with particular themes and hands-on performance tasks were

of concern. The science scales were not linked to any previously defined scales.

The frameworks for the different subject areas dictate differences in the numbers of scales. For

mathematics and science, item parameter estimation was performed separately for each of several scales

defined in their frameworks, using data from each age/grade sample separately.

 Generation of Plausible Values for Each Scale

After the scales were developed, plausible values were drawn from the predictive distribution of

proficiency values for each student (this process is called conditioning). For the long-term trend scales,

the plausible values were computed separately for each age or age/grade group and year, and were based

on the student’s responses to the items going into the scale as well as on the values of a set of background

variables that were important for the reporting of proficiency scores. For the mathematics and science

main scales, vectors of multivariate plausible values were drawn from the joint distribution of

proficiency values for the assessed student. These multivariate plausible values were computed

separately for each grade and reflected the dependency between scale proficiencies by utilizing shared

variation among the scales. All plausible values were later rescaled to the final scale metric using

appropriate linear transformations.

The variables used to calculate plausible values for a given main assessment scale or group of

scales included a broad spectrum of background, attitude, and experiential variables and composites of

such variables. All standard reporting variables were included. To enhance numerical stability for the

main assessment scales, the original background variables were standardized and transformed into a set

of linearly independent variables by extracting principal components from the correlation matrix of the

original contrast variables. The principal components, rather than the original variables, were used as

independent variables to calculate plausible values for those scales. Trend scales used the same or similar

sets of conditioning variables that were used when the scales were originally constructed. Details of the

conditioning process and of the NAEP-BGROUP and NAEP-CGROUP computer programs that

implement the process are presented in Chapter 11.
190

 Transformation to the Reporting Metric

Mathematics short-term trend and science, mathematics, reading, and writing long-term trend

scales were linked to previous assessment scales via common population linking procedures described in

the subject-specific data analysis chapters. Essentially, the 1994 and 1996 data were calibrated together.

Data from the two assessments were scaled together in the same BILOG/PARSCALE run, specifying the

samples for each assessment as coming from different populations. For each scale, the mean and standard

deviation of the 1994 data from this joint calibration were matched to the mean and standard deviation of

the 1994 data as previously reported. This then linked the 1996 data to the previously established scale.

New scales were established for the science main assessment. Then the metrics for the newly established

science scales were set to have a mean of 150 and a standard deviation of 35.

The transformations were of the form

target = A  calibrated + B

where

target = scale level in terms of the system of units of the final scale used for

reporting;

calibrated = scale level in terms of the system of units of the provisional

NAEP-BILOG/PARSCALE scale;

A = SDtarget / SDcalibrated ;

B =Mtarget - A Mcalibrated ;

SDtarget = the estimated or selected standard deviation of the proficiency

distribution to be matched;

SDcalibrated = the estimated standard deviation of the sample proficiency distribution

on the provisional NAEP-BILOG/PARSCALE scale;

Mtarget = the estimated or selected mean of the proficiency distribution to be

matched; and

Mcalibrated = the estimated mean of the sample proficiency distribution on the

provisional NAEP-BILOG/PARSCALE scale.

After the plausible values were linearly transformed to the new scale, any plausible value less than 0 was

censored to 0. For all 1996 assessments other than the science main assessment, any value greater than

500 was censored to 500; for the science main assessment, any value greater than 300 was censored to

300. Fewer than 1 percent of the students in any sample were censored in this way. The final

transformation coefficients for transforming each provisional scale to the final reporting scale are given

in subsequent chapters.
191

 Definition of Composites for the Multivariate Scales

In addition to the plausible values for each scale, composites of the individual mathematics and

science main assessment scales were created as measures of overall proficiency within these subject

areas. These composites were weighted averages of the plausible values of the individual scales. The

weights reflected the relative importance of the scales and were provided in the frameworks developed

by the subject area committees. The weights are approximately proportional to the number of items in

each scale at a given grade level.

 Tables of Proficiency Means and Other Reported Statistics

Proficiencies and trends in proficiencies were reported by age or grade for a variety of reporting

categories. Additionally, for the main assessments, the percentages of the students within each of the

reporting groups who were at or above achievement levels were reported to provide information about

the distribution of achievement within each subject area. For the long-term trend assessments, the

percentages of the students within each of the reporting groups who were at or above anchor points were

reported for the same reason. All estimates based on proficiency values have reported variances or

standard errors based on proficiency values including the error component due to the latency of

proficiency values of individual students as well as the error component due to sampling variability.

These tables are part of the electronically delivered summary data tables.

9.3.6 Dimensionality Analysis

Over the years a number of studies have been conducted in order to seek answers to the question

of how many dimensions underlie the various NAEP assessment instruments, and whether there is a

sufficiently strong first dimension to support inferences about a composite scale in subjects such as

mathematics, science, and reading. In addition, for the 1992 mathematics and reading assessments, a

study was conducted (Carlson, 1993) to determine whether the increasing emphasis on extended

constructed-response items that are scored polytomously has any effect on the dimensionality. It was

determined that for the 1992 NAEP data, item type was not related to any of the dimensions identified.

 Previous Dimensionality Analyses of NAEP Data

In an early study, NAEP reading assessment data collected during the 1983-84 academic year

was examined for dimensionality by Zwick (1986, 1987). Zwick also studied simulated data designed to

mirror the NAEP reading item-response data but having known dimensionality. Analysis of the simulated

datasets allowed her to determine whether the BIB spiraling design artificially increases dimensionality.

Zwick found substantial agreement among various statistical procedures, and that the results using BIB

spiraling were similar to results for complete datasets. Overall she concluded that “it is not unreasonable

to treat the data as unidimensional (1987, p. 306).”

Rock (1991) studied the dimensionality of the NAEP mathematics and science tests from the

1990 assessment using confirmatory factor analysis. His conclusion was that there was little evidence for

discriminant validity except for the geometry scale at the eighth-grade level, and that “we are doing little

damage in using a composite score in mathematics and science (p. 2).”
192

A second-order factor model was used by Muthn (1991) in a further analysis of Rock’s

mathematics data, to examine subgroup differences in dimensionality. Evidence of content-specific

variation within subgroups was found, but the average (across seven booklets) percentages of such

variation was very small, ranging from essentially 0 to 22, and two-thirds of these percentages were

smaller than 10.

Carlson and Jirele (1992) examined 1990 NAEP mathematics data. Analyses of simulated onedimensional data were also conducted, and the fit to these data was slightly better than that to the real

NAEP data. Although there was some evidence suggesting more than one dimension in the NAEP data,

the strength of the first dimension led the authors to conclude that the data “are sufficiently unidimensional to support the use of a composite scale for describing the NAEP mathematics data, but that there is

evidence that two dimensions would better fit the data than one (p. 31).”

Carlson (1993) studied the dimensionality of the 1992 mathematics and reading assessments. The

relative sizes of fit statistics for simulated as compared to actual data suggested that lack of fit may be

more due to the BIB spiraling design of NAEP than the number of dimensions fitted. Kaplan (1995)

similarly found that the chi-squared goodness of fit statistic in the maximum likelihood factor analysis

model was inflated when data were generated using a BIB design. The sizes of the fit statistics for incomplete simulation conditions (a BIB design as in the actual NAEP assessment) were more like those of the

real data than were those of the case of simulation of a complete data matrix. Consistent with findings of

Zwick (1986, 1987), however, the incomplete design for data collection used in NAEP does not appear to

be artificially inflating the number of dimensions identified using these procedures.

9.3.7 Drawing Inferences from the Results

Drawing correct inferences from the results of the assessments depends on several components.

First, the hypothesis of no difference between groups must be tested statistically. For the 1996

assessment, the use of t-tests was introduced for most comparisons. These tests are more appropriate than

z-tests based on normal distribution approximations when the statistics that are being compared are from

distributions with thicker tails than those from the normal distribution. The statistical significance tests

used in NAEP are described in detail in Chapter 18.

A second component contributing to drawing correct inferences is the way in which error rates

are controlled when multiple comparisons are made. If we wish to make a number of comparisons in the

same analysis, say White students versus Black, Hispanic, Asian/Pacific Island, and American Indian

students, the probability of finding “significance” by chance for at least one comparison increases with

the family size or number of comparisons. By the Bonferroni inequality, for a family size of 4, for

example, the probability of a false positive (Type I error) using α = .05 is less than or equal to 4 .05

=.20, larger than most decision-makers would accept.

One general method for controlling error rates in multiple comparisons is based on the

Bonferroni inequality. In this method, the Bonferroni inequality is applied and α is divided by the family

size, n. Now α = .05/4 = .0125, and using α, the combined probability of one or more errors in the four

comparisons remains controlled at less than or equal to .05. Note that dividing the probability by n is not

the same as multiplying the critical value or the confidence band by n. Indeed, in moving from a family

size of 1 to 4, we increase the critical value only from 1.960 to 2.498, a 27.4 percent increase. Doubling

the family size again, to 8, increases the critical value to 2.735, an additional 9.5 percent increase. To

double the initial critical value to 3.92, the family size would have to be increased to 564.
193

The power of the tests thus depends on the number of comparisons planned. There may be cases

for which, before the data are seen, it is determined that only certain comparisons will be conducted. As

an example, with the five groups above, interest might lie only in comparing the first group with each of

the others (family size 4), rather than comparing all possible pairs of groups (family size 10). This means

that some possibly significant differences will not be found or discussed, but the planned comparisons

will have greater power to identify real differences when they occur.

In 1996, several other methods were used to increase the power of statistical tests. For cases

where comparisons of statistics in 2 × 2 tables were made, an adapted Bonferroni procedure varying the

family size for each consecutive test was used (Hochberg, 1988). For a very large number of

comparisons, as when comparing results for every state to the nation, a different criterion was used to

control the error rates due to the large number of comparisons. This criterion, the False Discovery Rate

(FDR), as described by Benjamini and Hochberg (1994), contrasts with the Familywise Error Rate

(FWE) criterion used in the Bonferroni method. For trends extending over several administrations, power

is gained by testing least-squares fitted linear and quadratic trends, rather than individual pairs of data

points. For example, if the linear regression coefficient is significantly greater than 0, and the quadratic

coefficient is not different from 0, the trend over time is positive, even though the Bonferroni test might

declare no individual pair of points significantly different. These ways to control error rates in multiple

comparisons are described in Chapter 18.

A third component contributing to drawing correct inferences is the limitation of comparisons to

those for which there are adequate data. In NAEP reports and data summaries, estimates of quantities

such as composite and content area proficiency means, percentages of students at or above the

achievement levels, and percentages of students indicating particular levels of background variables (as

measured in the student, teacher, and school questionnaires) are reported for the total population as well

as for key subgroups determined by the background variables. In some cases, sample sizes were not large

enough to permit accurate estimation of proficiency and/or background variable results for one or more

of the categories of these variables.

For results to be reported for any subgroup, a minimum sample size of 62 was required. This

number was arrived at by determining the sample size required to detect an effect size of 0.5 with a

probability of .8 or greater.2

 The effect size of 0.5 pertains to the “true” difference in mean proficiency

between the subgroup in question and the total population, divided by the standard deviation of

proficiency in the total population. In addition, subgroup members must represent at least five primary

sampling units (PSUs).

A fourth component contributing to drawing correct inferences is the limitation of comparisons

to those comparing statistics with standard errors that are estimated well. Standard errors of mean

proficiencies, proportions, and percentiles play an important role in interpreting subgroup results and

comparing the performances of two or more subgroups. The jackknife standard errors reported by NAEP

are statistics whose quality depends on certain features of the sample from which the estimate is

obtained. In certain cases, typically when the number of students upon which the standard error is based

is small or when this group of students all come from a small number of participating schools, the mean

squared error associated with the estimated standard errors may be quite large. In the summary reports,

estimated standard errors subject to large mean squared errors are followed by the symbol !.

 

2

 A design effect of 2 was assumed for this purpose, implying a sample design-based variance twice that of simple random

sampling. This is consistent with previous NAEP experience (Johnson & Rust, 1992).
194

The magnitude of the mean squared error associated with an estimated standard error for the

mean or proportion of a group depends on the coefficient of variation (CV) of the estimated size of the

population group, denoted as N. This coefficient of variation is estimated by:

where N is a point estimate of N and SE N  is the jackknife standard error of N .

Experience with previous NAEP assessments suggests that when this coefficient exceeds 0.2, the

mean squared error of the estimated standard errors of means and proportions based on samples for this

group may be quite large. Therefore, the standard errors of means and proportions for all subgroups for

which the coefficient of variation of the population size exceeds 0.2 are followed by ! in the tables of

all summary reports. These standard errors, and any confidence intervals or significance tests involving

these standard errors, should be interpreted with caution. (Further discussion of this issue can be found in

Johnson & Rust, 1992.)

A final component contributing to drawing correct inferences pertains to comparisons

involving extreme proportions. When proportions are close to zero or one, their distributions differ

greatly from t- or z-distributions. For this reason, hypothesis tests of the sort used by NAEP are not

appropriate in these cases. Under these conditions, no test is made. Chapter 18 includes the specific

definition of extreme proportion used in the analysis of 1996 data.

9.4 OVERVIEW OF CHAPTERS 10 THROUGH 19

The remaining chapters in Part II of this report are as follows:

Chapter 10: The 1996 National Assessment used a stratified multistage probability sampling

design that provided for sampling certain subpopulations at higher rates (see Chapter 3). Because

probabilities of selection are not the same for all assessed students, sampling weights must be used in the

analysis of NAEP data. Also, in NAEP’s complex sample, observations are not independent. As a result,

conventional formulas for estimating the sampling variance of statistics are inappropriate. Chapter 10

describes the weighting procedures and methods for estimating sampling variance that are necessitated

by NAEP’s sample design. Further detail on sampling and weighting procedures is provided in The 1996

NAEP Sampling and Weighting Report, published in 1999 by Westat, Inc., the NAEP contractor in

charge of sampling.

Chapter 11: A major NAEP innovation introduced by ETS is the reporting of subject-area results

in terms of IRT-based scales. Scaling methods can be used to summarize results even when students

answer different subsets of items. For purposes of summarizing item responses, NAEP developed a

scaling technique that has its roots in IRT and in the theories of imputation of missing data. Chapter 11

describes this scaling technique, the underlying theory, and the application of these methods to 1996

NAEP data. The final section of Chapter 11 gives an overview of the NAEP scales that were developed

for the 1996 assessment.

Chapter 12: The main short-term trend component of the 1996 mathematics analysis is described

in this chapter. A detailed analysis of the main assessment of mathematics was conducted for grades 4, 8,

and 12, including a study of the association between mathematics proficiency and student background

CV(N) = SE(N)

N

 


195

variables. The results from this component of the analysis can be compared with results from the 1992

mathematics analysis to examine short-term trends. At grades 8 and 12, background information and data

on instructional methods were collected from teachers and the relation of these variables to mathematics

proficiency was examined. The main assessment analyses are reported in NAEP 1996 Mathematics

Report Card for the Nation and the States (Reese, Miller, Mazzeo, & Dossey, 1997). A special study of

advanced eighth- and twelfth-grade mathematics students was conducted, and cognitive item responses

associated with specific mathematics themes for the main mathematics samples was studied.

Chapter 13: The main assessment analysis of the science data is detailed in Chapter 13. This

analysis included a study of the association of science knowledge with instructional techniques and

student background variables. At grade 8, background information and data on instructional methods

were collected from teachers and the relation of these variables to science proficiency was examined. The

science results appear in NAEP 1996 Science Report Card for the Nation and the States (O’Sullivan,

Reese, & Mazzeo, 1997). A special study of advanced twelfth-grade science students was also conducted.

Chapter 14: The reading trend results for the years 1971 through 1994 were extended to include

1996 at ages 9, 13, and 17. The results of the reading trend analysis, which include the percentages of

students at or above the reading scale anchor points established in 1984, are reported in NAEP 1996

Trends in Academic Progress: Achievement of U.S. Students in Science, 1969 to 1996; Mathematics,

1973 to 1996; Reading, 1971 to 1996; and Writing, 1984 to 1996 (Campbell, Voelkl, & Donahue, 1997).

Chapter 15: The long-term trend assessment analysis of the mathematics data is detailed in

Chapter 15. The results of the trend analysis, which provided links from 1973 through 1996 for ages 9,

13, and 17, are reported in NAEP 1996 Trends in Academic Progress: Achievement of U.S. Students in

Science, 1969 to 1996; Mathematics, 1973 to 1996; Reading, 1971 to 1996; and Writing, 1984 to 1996

(Campbell, Voelkl, & Donahue, 1997).

Chapter 16: The long-term trend assessment analysis of the science data is described in Chapter

16. The science trend results, which provide a link to 1970, 1973, 1977, 1982, 1986, 1990, 1992, and

1994 are reported in NAEP 1996 Trends in Academic Progress: Achievement of U.S. Students in Science,

1969 to 1996; Mathematics, 1973 to 1996; Reading, 1971 to 1996; and Writing, 1984 to 1996

(Campbell, Voelkl, & Donahue, 1997).

Chapter 17: [This chapter is intended to provide information about the 1996 long-term trend

assessment in writing; however, the data from this assessment are currently under review. After

additional examination and analyses, this chapter will be included in a revised web version of the

complete report.]

Chapter 18: The 1996 assessment analyses included changes in the methods, procedures, and

conventions used in making group comparisons. Chapter 18 highlights these changes and provides details

about which results were reported.

Chapter 19: This chapter presents basic data from the 1996 assessment, including the properties

of the measuring instruments and characteristics of the sample.

197

Chapter 10

WEIGHTING PROCEDURES AND ESTIMATION

OF SAMPLING VARIANCE1

Eugene G. Johnson and Jiahe Qian

Educational Testing Service

Leslie Wallace and Keith F. Rust

Westat, Inc.

10.1 INTRODUCTION

As was the case in previous assessments, the 1996 national assessment used a complex sample

design with the goal of securing a sample from which estimates of population and subpopulation

characteristics could be obtained with reasonably high precision (as measured by low sampling

variability). At the same time, it was necessary that the sample be economically and operationally feasible

to obtain. The resulting sample had certain properties that had to be taken into account to ensure valid

analyses of the data from the assessment.

The 1996 NAEP sample was obtained through a stratified multistage probability sampling design

that included provisions for sampling certain subpopulations at higher rates (see Chapter 3). To account for

the differential probabilities of selection, and to allow for adjustments for nonresponse, each student was

assigned a sampling weight. Section 10.2 discusses the procedures used to derive these sampling weights.

Another consequence of the NAEP sample design is its effect on the estimation of sampling

variability. Because of the effects of cluster selection (students within schools, schools within primary

sampling units) and because of the effects of certain adjustments to the sampling weights (nonresponse

adjustment and poststratification), observations made on different students cannot be assumed to be

independent of one another. In particular, as a result of clustering, ordinary formulas for the estimation of

the variance of sample statistics, based on assumptions of independence, will tend to underestimate the

true sampling variability. Section 10.3 discusses the jackknife technique used by NAEP to estimate

sampling variability. (The estimation of variability due to imperfect measurement of individual proficiency

is discussed in Chapter 11.)

The jackknife technique provides good quality estimates of sampling variability but requires

considerable computations. Section 10.4 suggests the use of design effects, combined with conventional

variance estimation formulas, as a simple approximation to sampling variability estimation.

Since the sample design determines the derivation of the sampling weights and the estimation of

sampling variability, it will be helpful to note the key features of the 1996 NAEP sample design. A

description of the design appears in Chapter 3, and the various assessment instruments are detailed in

Chapter 4.

 

1

 Keith F. Rust, and Leslie Wallace were responsible for the design and implementation of the weighting process for the 1996

NAEP assessments. Previous versions of this chapter were created with the significant contributions of Eugene Johnson,

Educational Testing Service. Jiahe Qian of Educational Testing Service made significant contributions to the sampling variability

sections of this chapter. The statistical programming for this chapter was overseen by Bruce Kaplan and provided by Phillip Leung.
198

The 1996 sample was a multistage probability sample consisting of four stages of selection for the

long-term trend samples and five stages of selection for the main samples. The first stage of selection, the

primary sampling units (PSUs), consisted of counties or groups of counties. The second stage of selection

consisted of elementary and secondary schools. For the long-term trend assessment, the assignment of

sessions to sampled schools comprised the third stage of sampling, and the fourth stage involved the

selection of students within schools and their assignment to sessions. For the main assessment, the

assignment of schools to sample type (see Chapter 3) comprised the third stage of sampling, the

assignment of sessions to sampled schools comprised the fourth stage of sampling, and the fifth stage

involved the selection of students within schools and their assignment to sessions.

The probabilities of selection of the first-stage sampling units were proportional to measures of

their size, while the probabilities for subsequent stages of selection were such that the overall probabilities

of selection of students were approximately uniform, with exceptions for certain subpopulations that were

oversampled by design. For the main assessment, schools with relatively high concentrations of Black

students and/or Hispanic students were deliberately sampled at twice the normal rate to obtain larger

samples of respondents from those subpopulations, in order to increase the precision in the estimation of

the characteristics of these subpopulations. Also for the main assessment, nonpublic school students were

sampled at three times the normal rate, again to increase the precision of estimates for this population

subgroup. For all assessment components, students from schools with smaller numbers of eligibles

received lower probabilities of selection, as a means of enhancing the cost efficiency of the sample.

The 1996 main assessment includes three student cohorts: students in grades 4, 8, and 12. The

main assessment of all grades was conducted in the spring of 1996 to provide a cross-sectional view of

students’ abilities in mathematics and science.

The 1996 assessment also included a number of additional samples that used the age definitions,

times of testing, and modes of administration used in previous assessments. These are referred to as longterm trend samples. The purpose of these samples was to provide the statistical linkage between the 1996

data and data from previous assessments. The long-term trend assessment represents two overlapping

samples of students, the first of specified grades (of any age) and the second of specified ages (in any

grade). Students were age-eligible if they were born in the appropriate year (1986, 1982, or October 1978

to September 1979). The corresponding grades for the long-term trend assessment were 4, 8, and 11. Each

student cohort is called an “age class.”

The full 1996 NAEP assessment thus includes a number of different samples from several

populations. Each of these samples has its own set of weights that are to be used to produce estimates of

the characteristics of the population addressed by the sample (the target population). Each main sample has

an additional set of weights to accommodate the reporting requirements. The various samples and their

target populations are as follows:

The Main Samples of Students. The target population for each of these samples (one for each

grade) consisted of all students who were in the specified grade and were deemed assessable by their

school. There were four distinct session types at grade 4 (mathematics, science, mathematics estimation,

and mathematics theme), five at grade 8 (mathematics, science, mathematics estimation, mathematics

theme, advanced mathematics), and six at grade 12 (mathematics, science, mathematics estimation,

mathematics theme, advanced mathematics, and advanced science). Each session type was conducted as

one or more distinct sessions within a school. Administration of each session type was always conducted

separately from other session types.

To facilitate analyses, two kinds of weights were produced. “Reporting weights” were produced

separately by grade and assessment type for analyses of the reporting samples that were defined for each
199

assessment. Several of the reporting samples included students from multiple sample types. “Modular

weights” were produced separately by grade, assessment type, and sample type, for analyses involving any

one sample type, or for comparing one sample type with another. Thus in total, across grades, session

types, and sample types, there were 15 sets of reporting weights and 32 sets of modular weights for

students in the main assessment.

Long-Term Trend Samples for Reading and Writing. These consist of samples comparable to the

1984 main assessment and address the subject areas of reading and writing. The samples were collected by

grade and age for age 9/grade 4, age 13/grade 8, and age 17/grade 11, using the age definitions and time of

testing from 1984. As in that assessment, print administration was used. Six assessment booklets were

administered at each age class. The respondents to the combined set of assigned booklets at a given age

class constitute a representative sample of the population of students who were in the specified grade or of

the specified age. The respondents to any one of the booklets also constitute a representative sample.

Long-Term Trend Samples for Mathematics and Science. These consist of samples comparable

to those used for the measurement of trends in 1986. The samples were collected by age only and using the

same age definitions and time of testing as in the long-term trend assessment in 1986. As in that

assessment, the administration of mathematics and science questions was paced with an audiotape. For

ages 9 and 13, three assessment booklets were administered to each age group while two booklets were

administered at age 17. The respondents to any one of the booklets assigned to a given age constitute a

representative sample of the population of all students of that age. Each booklet was administered in a

separate assessment session, but the booklets were combined for weighting and reporting.

For purposes of sampling and weighting, the assessment samples are categorized as “tapeadministered” or “print-administered” according to if paced audiotapes were used in the administration:

1. Tape-administered samples are samples that required audiotape pacing in the

assessment (the long-term trend assessments in mathematics and science). For these

samples, all students within a particular assessment session received the same booklet

and were paced through at least part of the booklet with an audiotape.

2. Print-administered samples are all main assessment samples and the long-term trend

assessments of reading and writing. For these samples, no audiotape pacing was

employed and the assessment booklets were spiraled through each assessment session

(that is, the different booklets that were part of a given session type were

systematically interspersed and assigned for testing in that order).

10.2 DERIVATION OF THE SAMPLE WEIGHTS

As indicated previously, NAEP uses differential sampling rates, deliberately oversampling certain

subpopulations to obtain larger samples of respondents from those subgroups, thereby enhancing the

precision of estimates of characteristics of these oversampled subgroups. As a result of the oversampling,

these subpopulations, corresponding to students from schools with high concentrations of Black and/or

Hispanic students, and from nonpublic schools, are overrepresented in the sample. Lower sampling rates

were introduced also for very small schools (those schools with only 1 to 19 eligible students). This

reduced level of sampling from small schools was undertaken in an approximately optimum manner as a

means of reducing variances per unit of cost (since it is relatively costly to administer assessments in these

small schools). Appropriate estimation of population characteristics must take disproportionate

representation into account. This is accomplished by assigning a weight to each respondent, where the
200

weights approximately account for the sample design and reflect the appropriate proportional

representation of the various types of individuals in the population.

Two sets of weights were computed for the 1996 main samples for each session type. “Modular

weights” were computed for analyses involving students in one sample type, or for comparing results

between sample types. Each assessment type by grade and sample type weights up separately to the target

population. “Reporting weights” were computed for analyses of the reporting samples defined in Table 10-

1. Many of the reporting samples include students from more than one sample type. For reporting samples

that include only one sample type (i.e., science), the reporting weights are identical to the modular weights.

The steps for computing these two sets of weights are identical, up to and including the step of “trimming”

the weights. The trimmed weights were poststratified separately by sample type to create the modular

weights. In a parallel procedure, the trimmed weights were scaled back using a “reporting factor” so that

the sample types included in each reporting sample, when combined, would weight up to the target

population. The resulting weights were poststratified (but not separately by sample type) to create the

reporting weights.

Table 10-1

Reporting Samples for the 1996 Main NAEP Samples

Assessment Type Grade Reporting Sample1

Mathematics All A1 + A2 + B1

Science All A2 + B2

Mathematics Estimation 4 or 8

12

A1 + B12

A1 + A2 + B12

Mathematics Theme All A2 + B23

Advanced Mathematics 8

12

A2 + A3 + B2

A3 + B3

Advanced Science 12 A2 + B2

1

 A indicates assessed non SD/LEP students, B indicates assessed SD/LEP students; and 1, 2,

or 3 indicates the sample type (see Chapter 3).

2

 In the actual data analysis, the 1996 Mathematics Estimation used the reporting samples of

A1 + A3 + B1 for 4th and 8th grades, and the reporting samples of A1 + A2 + A3 + B1 + B2

for 12th grade. The weights used in the data analysis were adjusted based on the information

of the target population of the non-SD/LEP and SD/LEP students separately.

3

 In the actual data analysis, the 1996 Mathematics Theme used reporting samples of A2 + A3

+ B2. The weights used in the data analysis were adjusted based on the information of the

target population of the non-SD/LEP and SD/LEP students separately.

The weighting procedures for 1996 included computing the student’s base weight, the reciprocal

of the probability that the student was selected for a particular session type. Such weights are those

appropriate for deriving estimates from probability samples via the standard Horvitz-Thompson estimator

(see Cochran, 1977). These base weights were adjusted for nonresponse and then subjected to a trimming

algorithm to reduce a few excessively large weights. The weights were further adjusted by a student-level

poststratification procedure to reduce the sampling error. This poststratification was performed by

adjusting the weights of the sampled students so that the resulting estimates of the total number of students

in a set of specified subgroups of the population corresponded to population totals based on information

from the Current Population Survey and U.S. Census Bureau estimates of the population. The

subpopulations were defined in terms of race, ethnicity, geographic region, grade, and age relative to

grade.
201

In addition, the weights at grade 8 were poststratified at the school level in an effort to correct an

imbalance in the school sample that was observed after the assessments were complete. This procedure has

not been used in prior NAEP assessments, and was introduced because at grade 8, the three sample types

contained relatively different proportions of schools with large numbers of Black and Hispanic students

(termed “high minority” schools). The different school distributions became apparent by reviewing student

distributions by sample type and race/ethnicity for the mathematics assessment. In mathematics at grade 8,

sample type 2 contained a larger proportion of Black students than sample type 1. Investigation revealed

that the differences in student distributions were due to different school distributions by percent minority

enrollment. Although the different proportions were within sampling error, a decision was made to adjust

the weights at grade 8 for each of the sample types separately so that in each case the resulting estimates of

number of schools by percent minority enrollment corresponded to totals based on information from the

sampling frame of schools.

The following sections provide an overview of the procedures used to derive the sampling weights.

Further details in the derivation of these weights can be found in The 1996 NAEP Sampling and Weighting

Report (Wallace & Rust, 1999).

10.2.1 Derivation of Reporting Weights for the Main Samples

Separate weights were computed for each assessment in the main samples (mathematics, science,

mathematics theme, mathematics estimation, advanced mathematics, and advanced science). In earlier

NAEP assessments, excluded students from all subjects were weighted together, separately from assessed

students. In 1996, excluded students were weighted with assessed students for each assessment. This

change in procedures was adopted because evidence indicated that exclusion rates may not be independent

of session type, and because this change simplified the weighting procedures, when accounting for sample

type. Reporting weights for the main samples were derived according to the steps outlined below.

10.2.1.1 Student Base Weight

The base weight assigned to a student is the reciprocal of the probability that the student was

selected for a particular assessment. That probability is the product of five factors:

1. the probability that the PSU was selected;

2. the conditional probability, given the PSU, that the school was selected;

3. the conditional probability, given the sample of schools in a PSU, that the school was

assigned the specified sample type;

4. the conditional probability, given the sample of schools in a PSU, that the school was

allocated the specified session type; and

5. the conditional probability, given the school, that the student was selected for the

specified session type.
202

Thus, the base weight for a student may be expressed as the product

WB = PSUWT • SCHWT • SMPTYPWT • SESSWT • STUSCHW

where PSUWT, SCHWT, SMPTYPWT, SESSWT, and STUSCHW are, respectively, the reciprocals of the

preceding probabilities.

Variations across the various 1996 assessments in probabilities of selection, and consequently of

weights, were introduced by design, either to increase the effectiveness of the sample in achieving its goals

of reporting for various subpopulations, or to achieve increased efficiency per unit of cost.

10.2.1.2 Session Nonresponse Adjustment (SESNRF)

Sessions were assigned to schools before cooperation status was final. The session nonresponse

adjustment was intended to compensate for session type nonresponse due to refusing schools or individual

session types not conducted. The first three digits of PSU stratum, called subuniverse (formed by crossing

the PSU major stratum and the first socioeconomic characteristic used to define the final PSU stratum; see

Section 3.2 for more detail) were used in calculating nonresponse adjustments. The adjustment factors

were computed separately within classes formed by subuniverse within sample type for mathematics and

science, and by subuniverse for the other assessment types. Occasionally, additional collapsing of classes

was necessary to improve the stability of the adjustment factors, especially for the smaller assessment

components. Most classes needing collapsing contained small numbers of cooperating schools.

Occasionally, classes with low response rates were collapsed.

In subuniverse s in session type h, the session nonresponse adjustment factor SESNRFhs was given

by

SESNRF

PSUWT SCHWT SMPTYPWT SESSWT G

PSUWT SCHWT SMPTYPWT SESSWT G hs

ii i hi i

i B

i C

i i i hi i

hs

hs

=

•• • •

•• • •

∈

∈

∑

∑

where

PSUWTi = the PSU weight for the PSU containing school i;

SCHWTi = the school weight for school i;

SMPTYPWTi = the sample type weight for school i;

SESSWThi = the session allocation weight for session type h in school i;

Gi = the estimated number of grade-eligible students in school i (the values of

Gi

 were based on QED data);

set Bhs = consists of all in-scope originally sampled schools allocated to session

type h in subuniverse s (excluding substitutes); and

set Chs = consists of all schools allocated to session type h in subuniverse s that

ultimately participated (including substitutes).
203

It should be noted that the nonresponse adjustments assume that nonresponse occurs at random

within the categories within which adjustments are made (see Little & Rubin, 1987). Some degree of bias

could result to the extent that this assumption is false.

10.2.1.3 School-Level Poststratification Adjustment at Grade 8 (SCHPSF)

As discussed earlier, the weights at grade 8 were poststratified so that the resulting estimates of

number of public schools by percent minority enrollment corresponded to totals based on information from

the sampling frame of schools.

Poststratification adjustments were calculated separately by sample type within assessment type.

For the descriptions of the information for stratification, see Section 3.3. Control totals were calculated as

the total estimated number of grade-eligible students (based on QED data) in public schools, by percent

minority enrollment category. The percent minority enrollment categories used to form adjustment cells

were 0-4, 5-14, 15-29, 30-36, 37-44, 45-54, 55-79, 80-94, and 95-100 for mathematics and science sample

type 2, and 0-4, 5-14, 15-29, 30-44, 45-79, and 80-100 for all other assessment type/sample type

combinations. The smaller sample sizes in the latter group did not support using more categories. The

poststratification factor for each class c and session type h is computed by

SCHPSF TOTAL

PSUWT SCHWT SMPTYPWT SESSWT SESNRF G

hc

c

i i i hi hs i

i Dhc

= •• • • • ∈

∑

where

TOTALc = the total number of grade-eligible students in class c, from the sampling

frame;

PSUWTi = the PSU weight for the PSU containing school i;

SCHWTi = the school weight for school i;

SMPTYPWTi = the sample type weight for school i;

SESSWThi = the session allocation weight for session type h in school i;

SESNRFhs = the session nonresponse adjustment factor for subuniverse s in session

type h;

Gi = the estimated number of grade-eligible students in school i (the values of

Gi

 were based on QED data); and

Set Dhc = consists of the public schools in class c that participated in session type h.

For some sample types in advanced mathematics and mathematics estimation, two or more

poststratification classes were collapsed into one to improve the stability of the adjustment factors. Private

schools and new schools (those schools added to the sample through the new school sampling procedure,

and so not included on the sampling frame) received school-level poststratification factors of 1.0.
204

10.2.1.4 Student Nonresponse Adjustment (STUNRF)

Student nonresponse adjustment factors were computed separately for each session type. For

students in the main samples, the adjustment classes were based on sample type (for mathematics and

science), subuniverse, modal age status, and race class (White or Asian/Pacific Islander, other). In some

cases, two or more nonresponse classes were collapsed into one to improve the stability of the adjustment

factors. For each class c in session type h, the student nonresponse adjustment factor STUNRFhc is

computed by

∑

∑

∈

∈

• • • • • •

• • • • • •

=

hc

hc

j B

j j j hj hj hj

j A

j j j hj hj hj

hc PSUWT SCHWT SCHPSFj SMPTYPWT SESSWT SESNRF STUSCHW

PSUWT SCHWT SCHPSFj SMPTYPWT SESSWT SESNRF STUSCHW

STUNRF

where

PSUWTj = the PSU weight for the PSU containing student j;

SCHWTj = the school weight for the school containing student j;

SCHPSFj = for grade 8, the school poststratification factor for the school containing

student j (set to 1.0 for grades 4 and 12);

SMPTYPWTj = the sample type weight for the school containing student j;

SESSWThj = the session allocation weight for the school containing student j in session

type h;

SESNRFhj = the session nonresponse adjustment factor for the school containing

student j in session type h;

STUSCHWhj = the within-school student weight for student j in session type h;

Set Ahc = consists of the students in class c who were sampled for session type h and

not excluded; and

Set Bhc = consists of the students in class c who were assessed in session type h.

Excluded students received nonresponse adjustments of 1.0.

10.2.1.5 Trimming of Weights

In a number of cases, students were assigned relatively large weights. One cause of large weights

was underestimation of the number of eligible students in some schools leading to inappropriately low

probabilities of selection for those schools. A second major cause is the presence of large schools (high

schools in particular) in PSUs with small selection probabilities. In such cases, the maximum permissible

within-school sampling rate (determined by the maximum sample size allowed per schoolsee

Chapter 3) could well be smaller than the desired overall within-PSU sampling rate for students. Large
205

weights arose also because very small schools were, by design, sampled with low probabilities. Other

large weights arose as the result of high levels of nonresponse coupled with low to moderate probabilities

of selection, and the compounding of nonresponse adjustments at various levels.

Students with notably large weights have an unusually large impact on estimates such as weighted

means. Since, under some simplifying assumptions, the variability in weights contributes to the variance of

an overall estimate by an approximate factor 1 + V2

, where V2 is the relative variance of the weights, an

occasional unusually large weight is likely to produce large sampling variances of the statistics of interest,

especially when the large weights are associated with students with atypical performance characteristics.

To reduce this problem, a procedure of trimming a few of the more extreme weights to values

somewhat closer to the mean weight was applied. This trimming can increase the accuracy of the resulting

survey estimates, substantially reducing V2

 and hence the sampling variance, while introducing a small

bias. The trimming algorithm was identical to that used since 1984, and had the effect of trimming the

weights of students from any school that contributed more than a specified proportion, ζ, to the estimated

variance of the estimated number of students eligible for assessment. The trimming was done separately

within sample type for mathematics and science, and overall for mathematics estimation and mathematics

theme. Trimming was not done for the advanced mathematics or advanced science assessments because

advanced students were expected to be concentrated in certain schools, so that the trimming algorithm was

not appropriate in these cases. In each case, the value of the proportion ζ was chosen to be 10/K, where K

was the number of schools in which a specified assessment was conducted. The number of schools where

weights were trimmed was no more than seven in any one assessment. The most extreme trimming factors

applied were of the order of 0.65; trimming affects the weights of only a very small proportion of the

assessed and excluded students.

Table 10-2 shows the distributions of eligible students based on the trimmed weights of assessed

students for the science samples in sample type 2 (the reporting population) for each grade. The

distributions are similar to those before trimming shown in Tables 10-6, 10-7, and 10-8. To the extent that

the characteristics in the table are related to student performance on the science assessment, there is a

small bias introduced in the assessment by trimming.
206

Table 10-2

Distribution of Populations of Eligible Students Based on Trimmed Weights of Assessed Students

in Participating Schools, 1996 Main NAEP Science Sample, Sample Type 2

Population Grade 4 Grade 8 Grade 12

Total population 3,389,669 3,365,499 2,491,555

Age category

At modal age or younger

Older than modal age

64.6%

35.4%

57.9%

42.1%

64.5%

35.5%

Race/ethnicity category

White

Black

Hispanic

Other

61.0%

14.3%

16.9%

 7.8%

64.2%

15.7%

13.6%

 6.6%

69.1%

12.3%

11.1%

 7.5%

Gender1

Male

Female

49.6%

 50.2%

50.2%

49.6%

48.3%

51.7%

SD

Yes

No

 5.2%

94.8%

 5.7%

94.3%

 3.1%

96.9%

LEP

Yes

No

 3.0%

97.0%

 1.9%

98.1%

 2.2%

97.8%

SD, LEP

SD yes, LEP yes

SD yes, LEP no

SD no, LEP yes

SD no, LEP no

 0.1%

 5.1%

 3.0%

91.8%

 0.1%

 5.6%

 1.8%

92.5%

 0.0%

 3.0%

 2.1%

94.8%

1

 For a very small percentage of students at grades 4 and 8, gender is unknown.

10.2.1.6 Reporting Factor

Each set of trimmed weights for a given sample type sums to the target population. Reporting

factors were assigned to students in order to scale back the trimmed weights so that final student

(reporting) weights within each reporting population (which may combine students from different sample

types) sum to the target population. The reporting factors assigned to students are specific to the reporting

populations defined in Table 10-1. Each assessed and excluded student in the reporting population

received a reporting factor as shown in Table 10-3 on the following page.
207

Table 10-3

Reporting Factors for Assessed and Excluded Students

Non-SD/LEP Students SD/LEP Students

———Sample Type——— ———Sample Type———

1 2 3 123

Grade 4

Mathematics 0.5 0.5 — 1 — —

Science — 1 — — 1 —

Mathematics theme — 1 — — 1 —

Mathematics estimation 1 — — 1 — —

Grade 8

Mathematics 0.5 0.5 — 1 — —

Science — 1 — — 1 —

Mathematics theme — 1 — — 1 —

Mathematics estimation 1 — — 1 — —

Advanced mathematics (B) — 0.35 0.65 — 1 —

Grade 12

Mathematics 0.5 0.5 — 1 — —

Science — 1 — — 1 —

Mathematics theme — 1 — — 1 —

Mathematics estimation 0.6667 0.3333 — 1 — —

Advanced mathematics (B) — — 1 — — 1

Advanced science (C) — 1 — — 1 —

10.2.1.7 Student-Level Poststratification

As in most sample surveys, the respondent weights are random variables that are subject to

sampling variability. Even if there were no nonresponse, the respondent weights would at best provide

unbiased estimates of the various subgroup proportions. However, since unbiasedness refers to average

performance over a conceptually infinite number of replications of the sampling, it is unlikely that any

given estimate, based on the achieved sample, will exactly equal the population value. Furthermore, the

respondent weights have been adjusted for nonresponse and a few extreme weights have been reduced in

size.

To reduce the mean squared error of estimates using the sampling weights, these weights were

further adjusted so that estimated population totals for a number of specified subgroups of the population,

based on the sum of weights of students of the specified type, were the same as presumably better

estimates based on composites of estimates from the 1993 and 1994 Current Population Survey and 1996

population projections made by the U.S. Census Bureau. This adjustment, called poststratification, is

intended especially to reduce the mean squared error of estimates relating to student populations that span

several subgroups of the population, and thus also to reduce the variance of measures of changes over time

for such student populations.

The poststratification in 1996 was done for the mathematics, science, mathematics estimation, and

mathematics theme assessments in each grade. The advanced mathematics and advanced science

assessments were not poststratified. Within each grade and assessment type group, poststratification

adjustment cells were defined in terms of race, ethnicity, and Census region as shown in Table 10-4. Note
208

that NAEP region was used in previous years instead of Census region. This change was made because the

data from the Current Population Survey and Census Projections are more reliable for Census regions than

for NAEP regions.

Table 10-4

Major Subgroups for Poststratification in 1996

Subgroup Race Ethnicity Census Region1

1

2

3

4

5

6

7

White

White

White

White

Any

Black

Other

Not Hispanic

Not Hispanic

Not Hispanic

Not Hispanic

Hispanic

Not Hispanic

Not Hispanic

Northeast

Midwest

South

West

Any

Any

Any

1 Census region is the same as the NAEP region in Table 3-2 used for stratification and

reporting, except that DE, DC, and MD moved from the Northeast to the South, OK and

TX moved from the West to the South, Central is called Midwest, and Southeast is called

South.

These subgroups were used as adjustment cells at grade 12. Each of these cells accounts for

between 5 percent (Subgroup 7) and 21 percent (Subgroup 3) of the population. For grades 4 and 8, each of

the seven subgroups was further divided into two eligibility classes: of modal age and not of modal age.

The procedure used at grade 12 was adopted because the independent estimates of the numbers of

students in the population did not provide consistent data on the numbers of twelfth grade students by age.

Specifically, the counts of twelfth grade students age 17 and older are not reliable because they include

adult education students. This procedure has been used since 1988. (See Rust, Bethel, Burke, & Hansen,

1990, and Rust, Burke, & Fahimi, 1992, for further details.)

Thus, there were 7 or 14 cells for poststratification. The poststratified weight for each student

within a particular cell was the student’s base weight, with adjustments for nonresponse and trimming, and

the reporting factor from Section 10.2.1.6, times a poststratification factor. The poststratification factor for

student j in session type h and poststratification adjustment class c is given by

∑∈

• • • • • =

Chc j

Bj j j j j j

c

hc W SESNRF SCHPSF STUNRF TRIMFCTR RPTFCTR

TOTAL PSFCTR

where

WBj = the base weight for student j (see Section 10.2.1.1);

TOTALc = the total number of grade-eligible students in class c, from the October

1993 and 1994 Current Population Surveys and 1996 population

projections;

SESNRFj = the session nonresponse adjustment factor for the school containing

student j in session type h;
209

SCHPSFj = for grade 8, the school poststratification factor for the school containing

student j (set to one for grades 4 and 12);

STUNRFj = the student nonresponse adjustment for student j;

TRIMFCTRj = the trimming factor for student j;

RPTFCTRj = the reporting factor for student j;

Set Chc = consists of the students in class c who were assessed in session type h,

except those at grade 12 who were age 17 or older.

Note that students at grade 12 who were age 17 or older received the poststratification factor according to

their adjustment class and session type even though they were not used in calculating the factor. Details of

the procedures used to obtain totals in the numerator of the adjustment factor are provided in Wallace and

Rust (1999).

10.2.1.8 The Final Student Reporting Weights

The final weight assigned to a student is the student full-sample reporting weight. This weight is

the student’s base weight after the application of the various adjustments described above. The student

full-sample reporting weight was used to derive all estimates of population and subpopulation

characteristics that have been presented in the various NAEP reports, including simple estimates such as

the proportion of students of a specified type who would respond in a certain way to an item and more

complex estimates such as mean proficiency levels. The distributions of the final student reporting weights

are given in Table 10-5.

As indicated earlier, under some simplifying assumptions the factor 1 + V2

 indicates the

approximate relative increase in variance of estimates resulting from the variability in the weights. The

factor V2

 for each sample is readily derivable from Table 10-5 by squaring the ratio of the standard

deviation to the mean weight. These factors, resulting from the combined effect of the variations in

weights introduced by design and from other causes, are discussed in Section 10.2.3.

10.2.2 Evaluation of Potential for Bias Resulting from School and Student Nonparticipation

Although school and student nonresponse adjustments are intended to reduce the potential for

nonparticipation to bias the assessment results, they cannot completely eliminate this potential bias with

certainty. The extent of bias remains unknown, of course, since there are no assessment data for the

nonparticipating schools and students.

Some insight can be gained about the potential for residual nonresponse bias, however, by

examining the weighted school- and student-level distributions of characteristics known for both

participants and nonparticipants, especially for those characteristics known or thought likely to be related

to achievement on the assessment. If the distributions for the full sample of schools (or students) without

the use of nonresponse adjustments are close to those for the participants with nonresponse adjustments

applied, there is reason to be confident that the bias from nonparticipation is small.
210

Table 10-5

Distribution of Final Student Reporting Weights, 1996 Main Samples

Sample

Numberof Cases Mean

Standard

Deviation Minimum

25th

Percentile Median

Grade 4

 Mathematics 6831 577.41 380.69 92.57 315.28 486.57  Science 8061 489.35 324.35 65.04 275.04 399.02  Mathematics Theme 2072 1903.80 1163.23 576.00 1133.85 1587.94  Mathematics Estimation 1130 3409.86 2052.49 685.71 1935.01 3010.68 

Grade 8

 Mathematics 7312 510.55 389.16 62.92 247.63 389.41  Science 8200 455.26 388.04 75.47 219.81 313.58  Mathematics Theme 2177 1714.81 1124.20 388.72 921.25 1258.58  Mathematics Estimation 1255 2974.61 1906.19 338.54 1398.58 2574.86  Advanced Mathematics 2339 346.90 246.57 100.98 184.65 234.99 

Grade 12

 Mathematics 7020 415.75 257.34 64.39 226.94 337.48 

Science 7963 380.19 222.13 64.80 226.82 302.07 

 Mathematics Theme 2097 1442.96 862.82 313.34 879.14 1117.08  Mathematics Estimation 1472 1949.24 1050.36 488.10 1170.72 1743.95  Advanced Mathematics 2972 235.55 127.79 77.16 142.67 182.49  Advanced Science 2436 241.49 137.63 58.23 134.85 190.13 
211

There are several school-level characteristics available for both participating and nonparticipating

schools. The tables below show the combined impact of nonresponse and of the nonresponse adjustments

on the distributions of schools (weighted by the estimated number of eligible students enrolled) and

students, by the type of school (public, Catholic, other nonpublic) the size of the school as measured by the

estimated number of eligible students enrolled, and the urban/rural nature of the place where the school is

located. Three size classes have been defined for each grade. The data are for the science assessment,

sample type 2 (the reporting population). Science from sample type 2 was chosen because it is the largest

assessment at each grade. It is assumed that other large assessments would behave similarly.

Several student-level characteristics are available for both absent and assessed students. The tables

that follow show the impact of school nonresponse and nonresponse adjustments, and student nonresponse

and nonresponse adjustments on the distributions of eligible students for each grade. This discussion also

focuses on the science assessment for sample type 2, since it is the largest. The distributions are presented

by age category (at or below modal age, and above modal age), race category (White, Black, Hispanic, and

other), gender, SD, and LEP.

Table 10-6 shows the weighted marginal distributions of students for each of the three

classification variables for each grade, using weighted eligible schools. The distributions before school

nonresponse adjustments are based on the full sample of in-scope schools for science--those participating,

plus those refusals for which no substitute participated. The distributions after school nonresponse

adjustments are based only on participating schools for science, with school nonresponse adjustments

applied to them.

It can be seen from the tables that, even though the level of school nonparticipation is as high as

22.6 percent for grade 12 (see Table 3-11) and somewhat lower for the other grades, for the most part, the

distributions for the three characteristics considered remain similar. Exceptions may be medium and large

schools, and midsize cities and urban fringe of large cities at grade 12; and urban fringe of large cities and

rural nonMSAs at grade 4.
212

Table 10-6

Distribution of Populations of Eligible Students Based on Full Weighted Sample of Eligible Schools,

Before and After School Nonresponse Adjustments, 1996 Main NAEP Science Samples, Sample Type 2

Grade 4 Grade 8 Grade 12

Population Before After Before After Before After

Total population 3,777,554 3,777,554 3,198,390 3,198,390 2,762,448 2,762,448

School type

Catholic

Other Nonpublic

Public

 5.7%

 5.6%

88.7%

 7.2%

 4.3%

88.5%

 6.0%

 4.7%

89.3%

 6.8%

 4.6%

88.7%

 5.5%

 4.2%

90.3%

 5.9%

 3.4%

90.7%

School size1

1

2

3

18.8%

46.4%

34.8%

19.8%

47.2%

33.0%

10.8%

55.7%

33.5%

11.6%

55.8%

32.6%

 5.0%

70.5%

24.5%

 4.7%

67.0%

28.3%

School location

Large city

Midsize city

Urban fringe/large city

Urban fringe/midsize city

Large town

Small town

Rural MSA

Rural nonMSA

18.3%

19.4%

23.9%

13.7%

 0.3%

10.5%

 2.3%

11.5%

19.9%

19.7%

24.0%

11.9%

 0.5%

 8.8%

 1.8%

13.5%

17.7%

18.1%

23.1%

15.8%

 0.7%

 9.9%

 5.1%

 9.6%

17.9%

18.1%

21.8%

16.7%

 0.7%

 8.8%

 6.1%

 9.9%

15.4%

16.3%

23.3%

15.3%

 1.5%

14.4%

 3.7%

10.2%

16.8%

18.3%

21.2%

14.4%

 0.9%

15.5%

 3.7%

 9.5%

1 Distributions by school size are not comparable to previous assessments, since students were eligible by grade only (instead of by grade

or age) in 1996. School size = number of eligible students enrolled:

12 3

Grade 4 1-49 50-99 100 +

Grade 8 1-49 50-299 300 +

Grade 12 1-49 50-399 400 +

Table 10-7 shows the distributions of the same three classification variables, plus additional

distributions of student-level characteristics, using weighted eligible students. The distributions before

student nonresponse adjustments are based on assessed and absent science students (with base weights

adjusted for school nonparticipation). The distributions after student nonresponse adjustments are based on

assessed science students only, with the student nonresponse adjustments also applied to them.
213

Table 10-7

Distribution of Populations of Eligible Students Before and After Student Nonresponse Adjustments

1996 Main NAEP Science Samples, Sample Type 2

Grade 4 Grade 8 Grade 12

Population Before After Before After Before After

Total population 3,419,493 3,419,493 3,428,867 3,428,867 2,496,241 2,496,241

School type

Catholic

Other Nonpublic

Public

 7.7%

 4.5%

87.8%

 7.8%

 4.6%

87.6%

 6.1%

 4.3%

89.6%

 6.3%

4.4%

89.3%

 6.4%

 3.6%

90.0%

 8.1%

 4.2%

87.6%

School location

Large city

Midsize city

Urban fringe/large city

Urban fringe/midsize city

Large town

Small town

Rural MSA

Rural nonMSA

18.9%

19.2%

25.2%

12.1%

0.5%

8.9%

1.8%

13.3%

18.9%

19.3%

25.2%

12.0%

0.5%

9.0%

1.8%

13.3%

13.2%

15.9%

20.8%

19.3%

 0.5%

11.6%

 7.1%

11.6%

13.2%

15.9%

20.8%

19.4%

 0.5%

11.5%

 7.1%

11.7%

17.0%

18.2%

21.4%

13.6%

 1.0%

15.5%

 3.9%

 9.4%

16.9%

16.9%

21.7%

14.8%

 1.0%

15.2%

 4.1%

 9.6%

Age category

At modal age or younger

Older than modal age

64.6%

35.4%

64.5%

35.5%

57.5%

42.5%

57.6%

42.4%

64.3%

35.7%

64.5%

35.5%

Race/ethnicity category

White

Black

Hispanic

Other

61.4%

14.4%

16.6%

 7.6%

61.2%

14.2%

16.8%

 7.8%

64.9%

15.7%

13.0%

 6.4%

64.7%

15.4%

13.3%

 6.5%

69.4%

12.6%

11.0%

 7.1%

69.1%

12.2%

11.1%

 7.5%

Gender1

Male

Female

49.6%

50.2%

49.6%

50.2%

50.3%

49.4%

49.6%

50.1%

48.7%

51.2%

48.4%

51.6%

SD

Yes

No

 5.3%

94.7%

 5.3%

94.7%

 6.1%

93.9%

 5.8%

94.2%

 3.4%

96.6%

 3.0%

97.0%

LEP

Yes

No

 3.0%

97.0%

 3.0%

97.0%

 1.9%

98.1%

 1.9%

98.1%

 2.1%

97.9%

 2.2%

97.8%

SD, LEP

SD yes, LEP yes

SD yes, LEP no

SD no, LEP yes

SD no, LEP no

 0.1%

 5.2%

 2.9%

91.8%

 0.1%

 5.2%

 2.9%

91.8%

 0.1%

 6.0%

 1.8%

92.1%

 0.1%

 5.7%

 1.8%

92.5%

 0.1%

 3.3%

 2.1%

94.6%

 0.0%

 3.0%

 2.1%

94.8%

1

 Gender is unknown for a small percentage of students.

The rates of student nonparticipation were 5.1 percent for grade 4, 6.9 percent for grade 8, and

22.5 percent for grade 12 (see Table 3-11). The table shows that with one exception at grade 12, for the

distributions of type of school attended and place where the school is located, the combined effect of

student nonparticipation and the subsequent nonresponse adjustments have resulted in very little change in

distribution. The changes in the distribution of school type at grade 12 reflect the relatively high
214

nonresponse rate of grade 12 public school students (22.7 percent versus 8.5 percent for nonpublic school

students; see Table 3-9).

Table 10-8 shows the weighted distributions of eligible students in participating schools, using the

base weights of assessed and absent students unadjusted for school-level nonresponse. Tables 10-7 and 10-

8 show that both school and student-level nonresponse and nonresponse adjustments have little effect on

the distributions of eligible students by age, race/ethnicity, gender, SD and LEP. All of the distributions in

the tables are similar.

Table 10-8

Distribution of Populations of Eligible Students Before School and Student Nonresponse Adjustments

 1996 Main NAEP Science Samples, Sample Type 2

Population Grade 4 Grade 8 Grade 12

Total population 2,635,218 2,743,713 1,935,174

Age category

At modal age or younger

Older than modal age

64.3%

35.7%

56.3%

43.7%

63.9%

36.1%

Race/ethnicity category

White

Black

Hispanic

Other

60.8%

14.6%

16.9%

 7.7%

66.6%

14.0%

13.0%

6.5%

70.3%

12.4%

10.3%

 6.9%

Gender1

Male

Female

49.7%

50.1%

50.3%

49.5%

48.8%

51.2%

SD

Yes

No

 5.3%

94.7%

 6.2%

93.8%

 3.4%

96.6%

LEP

Yes

No

 3.2%

96.8%

 2.0%

98.8%

 2.0%

98.0%

SD, LEP

SD yes, LEP yes

SD yes, LEP no

SD no, LEP yes

SD no, LEP no

 0.1%

 5.2%

 3.1%

91.7%

 0.1%

 6.1%

 1.9%

91.9%

 0.1%

 3.3%

 1.9%

94.7%

1

 Gender is unknown for a small percentage of students.

When comparing the distributions in Table 10-7 before and after student nonresponse adjustments,

we expect the distributions by age category and race/ethnicity to be similar because these variables were

used to determine student nonresponse adjustment classes. However, the distributions by gender, SD, and

LEP are also similar. To the extent that nonrespondents would perform like respondents with the same

characteristics (defined by the classification variables in the tables), the bias in the assessment data is

small.

Further information about potential nonresponse bias can be gained by studying the absent

students. NAEP proficiency estimates are biased to the extent that assessed and absent students within the
215

same weighting class differ in their distribution of proficiency. It seems likely that the assumption that

absent students are similar in proficiency to assessed students is reasonable for some absent

studentsnamely, those whose absence can be characterized as random. Conversely, it seems likely that

students with longer and more consistent patterns of absenteeismsuch as truants, dropouts, near

dropouts, and the chronically illare unlikely to be as proficient as their assessed counterparts.

In the 1996 assessments, schools were asked to classify each absent student into one of nine

categories. The results of this classification for the science assessment are shown in Table 10-9. The

discussion focuses on the science assessment because it is the largest. It is assumed that the other large

assessments would behave similarly.

Table 10-9 shows that, as anticipated, the majority of absence from the assessment was the result

of an absence from school of a temporary and unscheduled nature. The table shows that absence among

twelfth graders occurs at about four times the rate of absence among fourth or eighth graders. The

proportion of absence classified as temporary differs somewhat by grade, but is of the same magnitude for

grades 8 and 12. These two facts taken together suggest strongly that a substantial proportion of the

temporary absences among twelfth grade students is not a result of illness, because such absences are

occurring at almost three times the rate that they do among fourth or eighth grade students. Whereas it

might be reasonable to regard temporary absence due to illness as independent of proficiency, for other

temporary absences, this appears less tenable. The data in the table give support to the contention that, at

grade 4, student absences are unlikely to introduce any significant bias into NAEP estimates. The absentee

rate is low; most absences are temporary, and three quarters of the remaining absences are a result of

parental refusal.

Table 10-9

Weighted Distribution of Absent Students by Nature of Absenteeism for All Grades

1996 Science Assessment, Sample Type 2

Nature of Absenteeism Grade 4 Grade 8 Grade 12

Temporary absence1 78.2% 68.0% 63.6%

Long-term absence2 1.6% 1.4% 0.8%

Chronic truant 0.2% 2.7% 1.4%

Suspended or expelled 0.7% 6.4% 0.4%

Parent refusal 16.4% 13.1% 9.2%

Student refusal 0.0% 3.2% 12.0%

In school, did not attend session 0.0% 3.0% 7.0%

In school, not invited3 0.0% 0.8% 0.0%

Other 0.3% 0.9% 3.9%

Missing 2.6% 0.4% 1.8%

Total absentee sample 384 569 2,269

Total sample size 7,689 8,343 9,807

Overall absentee rate 5.0% 6.8% 23.1%

1 Absent less than two weeks due to illness, disability, or excused absence.

2

 Absent more than two weeks due to illness or disability.

3

 In school, but not invited to assessment session due to disruptive behavior.

At grades 8 and 12, however, a significant component of absenteeism is not temporary or due to

parental refusal. Chronic truants, those suspended, and those in school but not invited, constitute the

obvious candidates for potential bias. These groups comprise 12.1 percent of absent students at grade 8 (or
216

0.8 percent of the total sample) and 8.8 percent of absent students at grade 12 (or 2.0 percent of the total

sample). Thus their potential for introducing significant bias under the current procedures is minor.

10.2.3 Derivation of Modular Weights for the Main Samples

As discussed earlier, modular weights were computed to facilitate analyses involving students

from a single sample type. The same procedures were used to derive modular and reporting weights up

through the weight trimming step described in Section 10.2.1.5. After trimming, weighting continued in

two parallel processes. Final student reporting weights were the result of one of these processes, and

modular weights were the result of the other.

Modular weights differ from reporting weights in two ways. First, they do not contain the

reporting factor described in Section 10.2.1.6. The second difference lies in the manner in which the

weights were poststratified. The modular weights were poststratified as described in Section 10.2.1.7,

except that each sample type within each grade and session was poststratified separately. The same initial

adjustment cells were used: 7 cells based on race/region for each session/sample type combination at grade

12, and 14 cells based on race/region and eligibility class (of modal age, not of modal age) for each

session/ sample type combination at grades 4 and 8. Some adjustment factors are quite variable for the

same adjustment cell across different sample types for the same grade and session. This indicates that the

individual samples by sample type may not be particularly stable, especially for the smaller sessions of

mathematics theme and mathematics estimation.

The modular weight is the student’s base weight after the application of the various adjustments

described in Section 10.2.1, except for the reporting factor, and with the new poststratification factor

described above. The distributions of the modular weights are given in Table 10-10. Note that modular

weights are identical to reporting weights for a particular grade/session/sample type combination when that

sample type is the only one included in the reporting population for that grade and session.

10.2.4 Derivation of Student Weights for the Long-Term Trend Samples

Final student weights were derived for the long-term trend samples in a manner similar to that used

in 1994. The procedure was identical to that used to derive reporting weights, described in Section 10.2.1,

except as noted below.

As in 1994, excluded students for all subjects in the long-term trend samples were weighted

together, separately from assessed students.

Base weight. A student’s base weight is the reciprocal of the product of four factorsall of the

factors used for reporting weights except sample type weight (SMPTYPWT).

Session nonresponse adjustments. Session nonresponse adjustments were calculated separately at

each age class for the spiral assessment, the tape assessment, and excluded students, within classes formed

by subuniverse. The formula for the adjustment does not contain sample type weight (SMPTYPWT), and

for excluded students, it does not contain session allocation weight (SESSWT). Gi is the estimated number

of age- plus grade-eligible students in the school for the spiral assessment and excluded students, and the

estimated number of age-eligible students for the tape assessment. For excluded students, Sets B and C are

not specific to any particular session.
Table 10-10

Distribution of Modular Weights, Main Samples

Grade/Session/

Sample Type

Number of

Cases Mean

Standard

Deviation Minimum

25th

Percentile Median

75

Perce

Grade 4

Mathematics/1 3808 1035.39 615.46 163.33 579.23 903.43 1366Mathematics/2 3691 1068.73 626.06 207.26 637.04 930.71 1305Mathematics/3 4077 967.54 670.28 211.91 513.62 759.53 1208Science/2 8061 489.35 324.35 65.04 275.04 399.02 598Science/3 4600 857.54 591.01 108.77 463.61 646.17 1032Mathematics Theme/2 2072 1903.80 1163.23 576.00 1133.85 1587.94 2276Mathematics Theme/3 2239 1761.80 1061.66 178.33 1063.45 1373.85 2247Mathematics Estimation/1 1130 3490.86 2052.49 685.71 1935.45 3010.68 4316Mathematics Estimation/3 1109 3556.97 2235.80 1006.89 2080.57 2916.57 416

Grade 8

Mathematics/1 4107 908.97 620.95 106.84 405.18 770.50 124Mathematics/2 4004 932.35 802.10 146.84 465.54 637.59 1098Mathematics/3 4128 904.35 571.76 165.42 505.73 704.53 1175Science/2 8200 455.26 388.04 75.47 219.81 313.58 562Science/3 4520 825.92 634.43 113.78 386.38 611.60 1026Mathematics Theme/2 2177 1714.81 1124.20 388.72 921.25 1258.58 232Mathematics Theme/3 2264 1648.91 1050.29 459.90 889.19 1384.34 2006Mathematics Estimation/1 1255 2974.61 1906.19 338.54 1398.58 2574.86 3952Mathematics Estimation/3 1097 3403.04 1856.24 663.36 1873.01 3079.88 4303Advanced Mathematics/2 805 890.46 661.32 288.53 511.84 645.93 89Advanced Mathematics/3 1562 557.79 386.61 185.10 286.90 385.01 684

Grade 12

Mathematics/1 3732 789.88 495.08 119.49 415.49 600.65 1100Mathematics/2 3913 750.91 451.42 103.41 434.29 614.34 1036Mathematics/3 3672 828.88 521.21 65.20 467.94 683.04 107Science/2 7963 380.19 222.13 64.80 226.82 302.07 500Science/3 4179 718.80 465.56 66.96 406.63 587.94 884Mathematics Theme/2 2097 1442.96 862.82 313.34 879.14 1117.08 1789Mathematics Theme/3 1944 1524.34 789.35 434.13 922.15 1248.22 199Mathematics Estimation/1 1090 2616.75 1353.39 667.28 1641.53 2331.77 3387Mathematics Estimation/2 431 6810.82 3947.11 2181.87 4076.82 5516.38 8886Mathematics Estimation/3 458 6384.23 4477.96 2416.15 3611.22 4792.51 6366Advanced Mathematics/3 2972 235.55 127.79 77.16 142.67 182.49 324Advanced Science/2 2436 241.49 137.63 58.23 134.85 190.13 323
218

School-level poststratification. There was no school-level poststratification for the long-term trend

assessments.

Student nonresponse adjustments. Student nonresponse adjustments were calculated separately at

each age class for the spiral assessment and the tape assessment within classes formed by subuniverse and

modal grade status (at or above modal grade, below modal grade). For excluded students at each age class,

the adjustments were calculated within classes formed by subuniverse. The formula for the adjustment does

not contain sample type weight (SMPTYPWT) or the school-level poststratification factor (SCHPSF), and for

excluded students, it does not contain session allocation weight (SESSWT). For excluded students, Set A

consists of all excluded students in class C, and Set B consists of the excluded students in class C for whom

an excluded student questionnaire was completed.

Trimming. Trimming was done separately for the spiral assessment, the tape assessment, and

excluded students at each age class.

Reporting factor: There was no reporting factor for the long-term trend assessments.

Student-level poststratification. Poststratification adjustments were calculated separately at each

age class for the spiral assessment, the tape assessment, and excluded students. Adjustment cells were

formed by race/region (as described in Section 10.2.1.7) and eligibility class (eligible by grade and of modal

age, eligible by age only, and eligible by grade but not of modal age). Thus 21 cells were used for the spiral

assessment and excluded students at each age class. Seven cells (by race/region only) were used for the tape

assessment at each age class. For each cell the poststratification factor is a ratio whose denominator is the

sum of weights (after adjustments for nonresponse and trimming) of assessed and excluded students, and

whose numerator is an adjusted estimate of the total number of students in the population who are members

of the cell.

Final student weights. The final weight assigned to each student is the student’s base weight after

application of the various adjustments described above. The distributions of the final student weights for the

long-term trend samples are given in Table 10-11.

10.2.5 Other Weights

Special weighting adjustments were developed for certain subsets of the fourth-grade and eighthgrade students assessed in the main samples. The weights for these subsets, with these adjustments applied,

were used in equating the results of the national and state assessments for subjects they had in common.

Also, weights appropriate for analyzing school-level data files were developed.

10.2.5.1 Weights for Equating National and State-by-State Samples

Weights for Equating National and State-by-State Assessments. The fourth-grade mathematics and

eighth-grade mathematics and science assessments conducted in February 1996 in the NAEP 1996 State

Assessment consisted of identical assessment material to that administered in the corresponding national

main sample sessions. Technical details of the NAEP 1996 State Assessments are given in Allen, Jenkins,

Kulick, and Zelenak (1997) and Allen, Swinton, Isham, and Zelenak (1998). The national and state-by-state

assessments were equated so that state and national results could be reported on a common scale. The

equating was achieved by using from each assessment that part of the sample representing a common

population. For the national samples, this consisted of those fourth-grade or eighth-grade public-school
Table 10-11

Distribution of Final Student Weights, Long-Term Trend Samples

Sample

Number

of Cases Mean

Standard

Deviation Minimum

25th

Percentile Median

75th

Percentil

Age Class 9

Reading/Writing 5019 923.09 442.60 227.42 599.94 827.79 1140.65 Mathematics/Science 5414 613.85 288.34 189.55 420.00 562.09 696.40 Excluded Students 1117 431.66 359.05 121.12 231.89 327.31 448.31 

Age Class 13

Reading/Writing 5493 855.31 369.72 54.00 840.26 840.26 1073.19 Mathematics/Science 5658 594.48 284.74 461.32 516.88 516.88 670.97 Excluded Students 933 465.54 347.50 38.77 342.48 342.48 482.85 

Age Class 17

Reading/Writing 4669 944.55 546.50 203.58 796.32 796.32 1207.55 Mathematics/Science 3539 900.51 453.46 287.24 782.92 782.92 1022.00 Excluded Students 713 549.66 338.57 160.95 471.56 471.56 621.88 
220

students from a participating state (including the District of Columbia) who were assessed in the main

sample mathematics or (for grade 8) science assessment reporting samples.

Although each sample of students received appropriate weights from the weighting procedure used

for the national assessment, in an effort to increase the precision of the equating process, an additional

weighting adjustment was developed and applied to each subsample by grade and subject, solely for use in

equating. For each subsample, the distributions of the main sample reporting weights for three categorical

variables were adjusted to agree closely with those obtained from the weighted aggregate sample from the

state assessments from the participating states. The first two variables were NAEP region (Northeast,

Southeast, Central, and West) and race/ethnicity (White nonHispanic, Black nonHispanic, Hispanic, and

other). For fourth-grade mathematics, the third variable was mathematics skill (good, not sure, other). For

eighth-grade mathematics, the third variable was the student’s mathematics course (eighth-grade

mathematics, pre-algebra, algebra, other). For eighth-grade science, the third variable was the student’s

science course (earth science only, physical science only or only earth science and physical science, other).

The categorical variables and control totals for each of the assessed grades and subjects are presented in

Tables 10-12 and 10-13.

Table 10-12

First and Second Categorical Variables Used for Raking1

Raking

Dimensions

Fourth Grade Mathematics

Control Total

Eighth Grade Mathematics

Control Total

Eighth Grade Science

Control Total

First Dimension

NAEP Region

1. Northeast 631,451 480,785 475,079

2. Southeast 733,191 723,032 726,322

3. Central 427,577 433,541 432,580

4. West 932,339 904,369 915,815

Total 2,724,558 2,541,727 2,549,797

Second Dimension

Race/Ethnicity

1. White nonHispanic 1,690,310 1,596,274 1,590,164

2. Black nonHispanic 408,725 377,660 372,715

3. Hispanic 454,883 408,197 419,467

4. Other 170,641 159,595 167,451

Total 2,724,558 2,541,727 2,549,797

1

 Numbers may not add up exactly due to rounding.

The equating of each weight distribution was achieved using a procedure known as iterative

proportional fitting (described by Little & Rubin, 1987). At the end of the fitting, adjustment factors were

derived and multiplied to the main sample weights for each subgroup to force their distribution to agree

with that from the aggregated state samples, for each of these three variables in turn. This process was then

repeated, and the final set of adjusted weights was compared with the state sample weights on all three

distributions, and found to be in very close agreement. Table 10-14 shows the distribution of the

adjustment factors for each of the grades and subjects assessed.
221

Table 10-13

Third Categorical Variable Used for Raking1

Control Totals

Grade 4

Mathematics Skill

1. Good 1,734,093

2. Not sure 634,732

3. Other 355,733

Total 2,724,558

Grade 8

Mathematics Type Course

1. Eight Grade Mathematics 1,018,743

2. Pre-Algebra 714,925

3. Algebra 639,393

4. Other 168,665

Total 2,541,727

Science Type Course

1. Earth Sciences only 737,898

2. Physical Sciences only or Earth Science and Physical Science 525,048

3. Other 1,286,851

Total 2,549,797

1

 Numbers may not add up exactly due to rounding.

Table 10-14

Percentiles of Raking Adjustments

Distribution Grade 4 Mathematics Grade 8 Mathematics Grade 8 Science

Minimum 0.587 0.708 0.721

10th Percentile 0.699 0.741 0.810

25th Percentile 0.777 0.837 0.847

Median 0.891 0.958 0.983

75th Percentile 1.025 1.074 1.161

90th Percentile 1.119 1.230 1.409

Maximum 1.524 1.728 2.355

10.2.5.2 School Weights

The sampling procedures used to obtain national probability samples of assessed students also

gave rise indirectly to several national probability samples of schools (from which the students were

subsequently sampled). So that the school samples can be utilized for making national estimates about

schools, appropriate nonresponse adjusted survey weights have been developed.

For the first time in 1996, the school weights for the main assessments were computed separately

by subject within grade. The school weights were a direct byproduct of the student weighting process. The

weight for school i in session h is given by

Whi = PSUWTi • SCHWTi • SESSWThi • SESNRFhi • SCHPSFhi
222

where PSUWTi, SCHWTi, SESSWThi, SESNRFhi, and SCHPSFhi are defined in Section 10.2.1.

The school weights for the long-term trend assessments in 1996 were computed using the same

procedures used in earlier years. The weights were computed separately by age class. The school base

weight was the product of PSUWT and SCHWT. School nonresponse adjustments were then applied to

these base weights. The values of the adjustment factors are not subject specific. In fact, they are identical

to the school nonresponse adjustment factors used for excluded students. Schools that did not participate in

any of the sessions that they were assigned were treated as nonrespondents, but schools that conducted at

least one of their assigned sessions were treated as respondents.

Fifteen samples of schools were weighted to be nationally representative in the main assessments,

and three samples were weighted to be nationally representative in the long-term trend assessments. In the

main samples, the population of schools represented is that of schools with grade 4 (for the grade 4

assessments), grade 8 (for the grade 8 assessments), or grade 12 (for the grade 12 assessments). In the

long-term trend samples, the school population at age class 9 is that of schools having eligible students and

at least one of the grades 2 through 5, the school population at age class 13 is that of schools having

eligible students and at least one of the grades 6 through 9, while the school population at age class 17 is

that of schools having eligible students and at least one of the grades 9 through 12.

10.2.5.3 Jackknife Replicate Weights

In addition to the weights that were used to derive all estimates of population and subpopulation

characteristics, other sets of weights, called jackknife replicate weights, were derived to facilitate the

estimation of sampling variability by the jackknife variance estimation technique. These weights and the

jackknife estimator are discussed in the next section.

10.3 PROCEDURES USED BY NAEP TO ESTIMATE SAMPLING VARIABILITY

A major source of uncertainty in the estimation of the value in the population of a variable of

interest exists because information about the variable is obtained on only a sample from the population. To

reflect this fact, it is important to attach to any statistic (e.g., a mean) an estimate of the sampling

variability to be expected for that statistic. Estimates of sampling variability provide information about

how much the value of a given statistic would be likely to change if the statistic had been based on another,

equivalent, sample of individuals drawn in exactly the same manner as the achieved sample.

Another important source of variability is that due to imprecision in the measurement of individual

proficiencies. For the 1996 assessment, proficiencies in all subject areas were summarized through item

response theory (IRT) models, but not in the way that these models are used in standard applications where

each person responds to enough items to allow for precise estimation of that person’s proficiency. In

NAEP, each individual responds to relatively few items so that individual proficiency values are not well

determined. Consequently, the variance of any statistic based on proficiency values has a component due

to the imprecision in the measurement of the proficiencies of the sampled individuals in addition to a

component measuring sampling variability. The estimation of the component of variability due to

measurement imprecision and its effect on the total variability of statistics based on proficiency values are

discussed in Chapter 11.
223

The estimation of the sampling variability of any statistic must take into account the sample

design. In particular, because of the effects of cluster selection (students within schools, schools within

PSUs) and because of effects of nonresponse and poststratification adjustments, observations made on

different students cannot be assumed to be independent of each other (and are, in fact, generally positively

correlated). Furthermore, to account for the differential probabilities of selection (and the various

adjustments), each student has an associated sampling weight, which should be used in the computation of

any statistic and which is itself subject to sampling variability. Ignoring the special characteristics of the

sample design and treating the data as if the observations were independent and identically distributed, will

generally produce underestimates of the true sampling variability, due to the clustering and unequal

sampling weights.

The proper estimation of the sampling variability of a statistic based on the NAEP data is

complicated and requires techniques beyond those commonly available in standard statistical packages.

Fortunately, the jackknife procedure (see, e.g., Wolter, 1985; Kish & Frankel, 1974; Rust, 1985) provides

good quality estimates of the sampling variability of most statistics, at the expense of increased

computation, and can be used in concert with standard statistical packages to obtain a proper estimate of

sampling variability.

The jackknife procedure used by NAEP has a number of properties that make it particularly suited

for the analysis of NAEP data. When properly applied, a jackknife estimate of the variability of a linear

estimator (such as a total) will be the same as the standard textbook variance estimate specified for the

sample design (if the first-stage units were sampled with replacement and approximately so otherwise).

Additionally, if the finite sampling corrections for the first stage units can be ignored, the jackknife

produces asymptotically consistent variance estimates for statistics such as ratios, regression estimates or

weighted means and for any other nonlinear statistic that can be expressed as a smooth function of

estimated totals of one or more variables (Krewski & Rao, 1981).

Through the creation of student replicate weights (defined below), the jackknife procedure allows

the measurement of variability attributable to the use of poststratification and other weight adjustment

factors that are dependent upon the observed sample data. Once these replicate weights are derived, it is a

straightforward matter to obtain the jackknife variance estimate of any statistic.

The jackknife procedure in this application is based upon the development of a set of jackknife

replicate weights for each assessed student (or excluded student, or school depending upon the file

involved). The replicate weights are developed in such a way that, when utilized as described below,

approximately unbiased estimates of the sampling variance of an estimate result, with an adequate number

of degrees of freedom to be useful for purposes of making inferences about the parameter of interest.

The estimated sampling variance of a parameter estimator t is the sum of M squared differences

(where M is the number of replicate weights developed):

Var t t t  i

i

M

 = − =

∑

1

2

,

where ti denotes the estimator of the parameter of interest, obtained using the ith set of replicate weights,

SRWTi, in place of the original sample of full sample estimates WT.

For each of the three sample types (see Section 3.4 and Chapter 5 for a description of the three

sample types) in the main assessment samples, 62 replicate weights were developed using the procedures

outlined below. Similar procedures were followed for the long term trend samples. However, since those
224

samples were based on fewer PSUs (52 rather than 94), the long-term trend samples have fewer replicate

weights (36 instead of 62). Full details of the generation of replicate weights for all samples are given in

Wallace and Rust (1999).

Of the 62 replicate weights formed for each record from a main assessment sample, 36 act to

reflect the amount of sampling variance contributed by the noncertainty strata of PSUs, with the remaining

26 replicate weights reflecting the variance contribution of the certainty PSU samples.

The derivation of the 36 replicate weights reflecting the variance of the noncertainty PSUs

involves first defining pairs of PSUs in a manner that models the design as one in which two PSUs are

drawn with replacement per stratum. This definition of pairs is undertaken in a manner closely reflective of

the actual design, in that PSUs are pairs that are drawn from strata within the same subuniverse, and with

similar stratum characteristics. The same definition of pairs was used for each of the age/grade classes in

the main assessment, since all were drawn from the same sample of noncertainty PSUs. The 72

noncertainty PSUs, drawn one from each of 72 strata, were formed into 36 pairs of PSUs, where the pairs

were composed of PSUs from adjacent strata within each subuniverse (thus the strata were relatively

similar on socioeconomic characteristics such as proportion minority population, population change since

1980, per capita income, educational attainment, and unemployment rate). Whereas the actual sample

design was to select one PSU with probability proportional to size from each of 72 strata, for variance

estimation purposes the design is regarded as calling for the selection of two PSUs with probability

proportional to size with replacement from each of 36 strata. This procedure likely gives a small positive

bias to estimates of sampling error.

The student replicate weight for the i

th pair of noncertainty PSUs, for the 36 pairs corresponding to

values of i from 1 to 36, is computed as follows:

1. Let WB be the base weight of a student, as described in Section 10.2.1, which accounts

for the various components of the selection probability for the student.

2. At random, one PSU in each pair is denoted as PSU number 1, while the other is

denoted as PSU number 2. The i

th replicate base weight Wbi is given by:

W

i

W i

WB i

bi B =











0

2

if the student belongs to PSU number 1 of pair 

if the student belongs to PSU number 2 of pair 

if the student is from neither PSU in pair 

*

3. The i

th student replicate weight SRWTi is obtained by applying the various school and

student nonresponse adjustments, the weight trimming, and the poststratification to the

i

th set of replicate base weights, using procedures identical to those used to obtain the

final student weights WT from the set of base weights WB.

In brief, the procedure for deriving the sets of Wbi values from the WB values reflects the sampling

of PSUs, schools, sessions, and students. By repeating the various weight adjustment procedures in each

set of replicate base weights, the impact of these procedures on the sampling variance of the estimator, t, is

appropriately reflected in the variance estimator Var t defined above.

The procedure for obtaining the 26 sets of replicate weights to estimate the sampling variance from

the certainty PSUs is analogous, but somewhat more complex. The first stage of sampling in this case is at

the school level, and the derivation of replicate weights must reflect appropriately the sampling of schools

within certainty PSUs. Since each of the three grade classes in the main assessment involved different
225

samples of schools, the procedure for forming replicate base weights was individualized to each of these

sample components. In common across these three samples were the 22 certainty PSUs used, and the fact

that 26 replicate weights were formed in each case.

For a given sample, the 22 certainty PSUs constituted strata, with a sample of schools drawn

systematically within each. Using the schools listed in order of sample selection within each stratum,

successive schools were paired or formed into triples. These pairs and triples numbered more than 26, so

that each replicate weight was in general formed by perturbing the weights of students from more than a

single pair or triple. These aggregates of pairs and triples were in general assigned in proportion to the size

of the PSU. Thus generally speaking, the largest PSUs were assigned three replicates each, the next largest

were assigned two replicates each, and the remaining self-representing PSUs were assigned one replicate

each. When splitting the larger PSUs, the schools were split into groups of (as close as possible) equal size,

based on the ordering at the time of sample selection. One group was assigned to each replicate. Within

each PSU (or partial PSU in the case of the large split PSUs) schools were alternately numbered 1 or 2

starting randomly. If, however, there were exactly three schools sampled in the PSU the schools were

randomly numbered 1, 2, or 3. The method of forming replicate base weights in strata where there were not

exactly three schools was the same as for the noncertainty strata (except that members of a pair, i, could

come from more than a single “stratum”).

When a stratum contained three schools, students in these schools had their weights perturbed for

two sets of replicates, say i1 and i2, as follows:

W

i

W i

W i

bi B

B

=











0

1 5

if the student is in school number 1 of a PSU belonging to set 

if the student is in school number 2 or 3 of a PSU belonging to set 

if the student does not belong to a PSU in set 

. *

W

W i

i

W i

bi

B

B

=











1 5

0

. * if the student is in school number 1 or 2 of a PSU belonging to set 

if the student is in school number 3 of a PSU belonging to set 

if the student does not belong to a PSU in set 

The actual pattern of replicate base weight assignment used for each of the samples is given in

Wallace and Rust (1999).

The nonresponse, trimming, and poststratification adjustments were applied to each set of replicate

base weights to derive the final replicate weights in each case, exactly as in the noncertainty PSUs. In fact

these procedures were applied to the full set of weights from all parts of the given sample together, just as

for the full sample weights. That is, for example, poststratification factors were derived from the full set of

data for each replicate, not separately for certainty and noncertainty PSUs.

This estimation technique was used by NAEP to estimate all sampling errors presented in the

various reports. A further discussion of the variance estimation procedure used by NAEP, including a

discussion of alternative jackknife estimators that were also considered, appears in Johnson (1989).

We noted above (as discussed in Chapter 11) that a separate estimate of the contribution to

variance due to the imprecision in the measure of individual proficiencies is made and added to the

jackknife estimate of variance. That variance component could have been approximately reflected in the

jackknife variance estimates simply by separately applying the IRT computations to each jackknife

replicate. Because of the heavier IRT computational load, this was not done. Less work was involved by

the simple procedure of making separate estimates of this component to be added to the jackknife variance
226

estimates. Also, a separate measure of this component of variance is then available, which would not be so

if it were reflected in the jackknife variance estimate.

10.4 APPROXIMATING THE SAMPLING VARIANCE USING DESIGN EFFECTS

In practical terms, the major expenditure of resources in the computation of a jackknife variance

estimate occurs in the preparation of estimates for each of the pseudoreplicates. In the 1996 assessment,

this implies that the statistic of interest has to be recomputed up to 63 times, once for the overall estimate t,

and once for each of the up to 62 pseudoreplicates ti. Because this is a considerable increase in the amount

of computation required, relative to a conventional variance estimate, it is of interest to see how much the

jackknife variance estimates differ from their less computationally intensive, simple random sampling

based, analogues.

The comparison of the conventional and the jackknife methods of variance estimation will be in

terms of a statistic called the design effect, which was developed by Kish (1965) and extended by Kish and

Frankel (1974). The design effect for a statistic is the ratio of the actual variance of the statistic (taking the

sample design into account) over the conventional variance estimate based on a simple random sample

with the same number of elements. The design effect is the inflation factor to be applied to the

conventional variance estimate in order to adjust error estimates based on simple random sampling

assumptions to account approximately for the effect of the sample design. The value of the design effect

depends on the type of statistic computed and the variables considered in a particular analysis as well as

the combined clustering, stratification, and weighting effects occurring among sampled elements. While

stratification drives down the sampling variance, the effects of clustering and weighting that drive

variances up are generally sufficient to produce variance estimates that are larger than variances based on

simple random sampling assumptions. Consequently, the design effects will be greater than one. In NAEP,

the underestimates are the result of ignoring the effects of clustering and unequal probabilities of selection

in the variance calculations.

Since most of the analyses conducted by NAEP are based on the results of scaling models that

summarize performance of students across a learning area, we consider the design effects to be expected

for analyses based on these scale scores. For reasons given in Chapter 11, NAEP provides each individual

with a set of “plausible values,” each of which is a random draw from the distribution of the potential scale

scores for that individual. Since our current interest is on the effect of the sampling design on estimation

and inference, we will restrict our attention to a single measure of an individual’s proficiency, the first

plausible value of the individual’s scale score.

A key statistic of interest is the estimated mean proficiency of a subgroup of the population. An

estimate of the subgroup mean proficiency is the weighted mean of the first plausible values of proficiency

of the sampled individuals who belong to the subpopulation of interest. Let Y be the weighted mean of the

plausible values of the sampled members of the subpopulation. The conventional estimate of the variance

of Y is

Var Y

wy Y

N W con

i i

i

N

 

  =

− =

+

∑ 2

1 ,

where N is the total number of sampled individuals in the subpopulation for which plausible values are

available, wi is the weight of the i

th individual, yi is a plausible value from the distribution of potential

proficiencies for that individual, and W+ is the sum of the weights across the N individuals.
227

The design effect for the subgroup mean proficiency estimate is

deff Y Var Y Var Y    = JK con /

where VARJK(Y ) is the jackknife variance of Y . (As has been pointed out previously, VARJK(Y ) as

computed does not measure the variability of Y due to imprecision in the measurement of the proficiencies

of the sampled individuals. The estimation of this very important source of variability is discussed in

Chapter 11.) Of the factors that determine deff(Y ), the effects of stratification are usually less than one,

which means the efficiency of a stratified sampling is better than a simple random sampling; whereas the

clustering effects are always larger than one. The clustering effects can be approximated by

1 1 + −   m ρ,

where m is the average cluster size and ρ is the intracluster correlation (Cochran, 1977). Therefore, the

large cluster size or large intracluster correlation will inflate the clustering effects.

Values of the design effects for subgroup mean proficiencies are displayed, by grade, in Tables 10-

15 and 10-19 for the 1996 main assessments of mathematics and science respectively. Design effects are

shown for the population as a whole (Total) as well as for a variety of demographic subgroups: gender;

race/ethnicity (White, Black, Hispanic, Asian/Pacific Islander, other); type of location (Central City, Urban

Fringe/Large Town, Rural/Small Town); parental education (did not graduate high school, graduated high

school, post-high school, graduated college, unknown); and type of school (public, nonpublic). These

particular demographic variables were selected because (1) they are major variables in NAEP reports and

(2) they reflect different types of divisions of the population that might have different levels of sampling

variability. Note that the tables of the design effects provided in the NAEP Technical Reports previous to

1994 are computed for the mean item scores, proportion-correct statistics, which can not be compared with

the design effects for proficiency scale scores directly.

The 1996 main mathematics assessment contains the three sample types S1, S2 and S3 (see

Section 5.3). To conserve trend in the main mathematics assessment, the reporting samples were made up

of A1 and B1 portions in S1 samples and A2 in S2 samples (see Table 10-1). The advantage of including

A2 in the reporting samples is to obtain more accurate scale scores, yet a trade-off is that the clustering

effects for the reporting samples becomes lager than the clustering effects for only sample S1.

The larger intracluster correlation in A1 and A2 and larger cluster sizes in sample type S1 are two

factors that contribute to the increase in clustering effects. First, the reporting sample for the main

mathematics assessment from sample type S2 has only non-SD/LEP students, A2. Compared with the

SD/LEP students, the non-SD/LEP students are relative homogenous in scale scores. The high

homogeneity in clusters implies a large intracluster correlation, ρ1. Given other conditions, the large

clustering effects that are due to a large intracluster correlation expand the design effects in the reporting

samples. To check this conclusion, the design effects for subsamples of A1, A2, and B1 were calculated

separately and are displayed in Tables 10-16, 10-17 and 10-18. The estimated design effects for

subsamples of A1 and A2 are much larger than those for subsample B1 that contains SD/LEP students in

sample type S1. Secondly, the cluster sizes tended to be larger in sample type S1 because the schools in

sample type S1 only provided samples for the main mathematics assessment, whereas the schools in

sample type S2 provided samples for both the main mathematics and science assessments. Therefore, the

design effects in A1 are found to be larger than the design effects in A2 and both subsamples of A1 and A2

contributed students to the reporting samples.
228

Tables 10-20 and 10-21 provide equivalent information for the long-term trend samples. Table 10-20

provides, for each age class and demographic subgroup, the average of the design effects for mean reading

and writing proficiencies for the students selected for the long-term trend assessments of reading and

writing. Table 10-21 provides the average of the design effects for the mean mathematics and science

proficiencies for the students selected for the long-term trend assessments of mathematics and science.

Finally, for comparison with the national mathematics and science results, Table 10-22 shows the

average design effects for state-level mean mathematics proficiency, averaged across all jurisdictions

participating in the grade 4 and grade 8 1996 State Assessment in mathematics. The results in Table 10-23

are the average design effects for state-level mean science proficiency, averaged across all jurisdictions

participating in the grade 8 1996 State Assessment in science.

The tables show that the design effects are predominantly larger than 1, indicating that standard variance

estimation formulas will be generally too small, usually markedly so. Although the design effects appear

somewhat different for certain subgroups of the population, they are, perhaps, similar enough (at least within

a subject and grade) to select an overall composite value that is adequate for most purposes. In choosing a

composite design effect, some consideration must be made about the relative consequences of

overestimating the variance as opposed to underestimating the variance. For example, if an overestimate of

the variance is viewed as severe an error as an underestimate, the composite design effect should be near to

the center of the distributions of the design effects. Possible composites of this type are the mean and

median design effects across the combined distribution of all design effects. Larger design effects should be

used if it is felt that it is a graver error to underestimate the variability of a statistic than to overestimate it.

For example, Johnson and King (1987) examine estimation of variances using design effects (among other

techniques) under the assumption that the consequences of an underestimate are three times as severe as

those of an overestimate of the same magnitude. Adopting a loss function that is a weighted sum of absolute

values of the deviations of predicted from actual with underestimates receiving three times the weight of

overestimates, produces the upper quartile of the design effects as the composite value. This assumes that

the distribution of design effects is roughly independent of the jackknife estimates of variance, so that the

size of a design effect does not depend on the size of the variance.

Table 10-24 gives the values of these potential composites, by grade, for the mathematics and

science assessments, and across those assessments. Tables 10-25 and 10-26 gives composite values for the

1996 State Assessment of mathematics (grades 4 and 8), and the 1996 State Assessment of science (grade 8),

respectively. Table 10-27 shows composite values for the 1996 State Assessment of mathematics and the

1996 State Assessment of science, combined. The state assessments tend to have smaller design effects than

the matching national assessment, due to the lesser degree of clustering in the state assessment samples (i.e.,

the average cluster size of m is smaller). Table 10-28 gives the values of the composites for the two longterm trend samples.

We note that the Var Y con  as defined above is an estimate of S2

/N where S2 represents the unit

variance for a simple random sample for the population of students from which the sample is drawn. This is

an appropriate estimate of the increase in variance over simple random sampling from that population due to

the effects of weighting. However, the computer packages used for estimating the variance may not reflect

the weights in estimating the unit variance, as given above, but instead may provide an estimate of a unit

variance of the form

y Y

N

i i

N

− ∑=  2

1

2 .

In this case, the unweighted estimate of unit variance would be appropriate for the denominator of a design

effect measure of the increase in variance over the unit variance as estimated by the computer package. If
229

there is no correlation between the wi and yi, there would be little difference between the two.

Table 10-15

Design Effects by Demographic Subgroup and Grade

for Mean Mathematics Proficiencies1

Subgroup Grade 4 Grade 8 Grade 12

Total 5.00 5.82 6.50

Male 3.28 4.48 3.50

Female 3.19 3.13 4.56

White 4.74 5.92 5.14

Black 6.32 3.89 5.95

Hispanic 4.04 2.72 2.42

Asian/Pacific Islander 3.86 4.45 6.21

Other race/ethnicity 1.46 1.09 15.38

Urban 9.29 7.27 11.53

Suburban 7.10 10.50 6.36

Rural 6.09 5.92 8.00

PARED < HS 1.54 1.33 1.59

PARED = HS 2.12 1.89 2.28

PARED > HS 1.18 2.34 1.31

PARED = College 4.76 5.05 5.17

PARED = Unknown 4.36 1.13 1.06

Public school 5.24 5.87 4.42

Nonpublic school 7.00 8.53 18.24

1

Design effects are based on the conventional and jackknife variances of subgroup means of the first plausible values of proficiency.

Table 10-16

Design Effects of Subsample A1 by Demographic Subgroup and Grade

for Mean Mathematics Proficiencies1

Subgroup Grade 4 Grade 8 Grade 12

Total 9.38 7.33 7.48

Male 5.61 5.04 4.62

Female 4.95 4.13 4.14

White 5.13 6.89 5.21

Black 15.14 3.93 7.63

Hispanic 2.08 2.86 1.91

Asian/Pacific Islander 2.46 7.96 4.54

Other race/ethnicity 1.12 0.96 30.13

Urban 16.94 6.13 12.20

Suburban 6.84 15.34 5.72

Rural 8.25 8.01 13.42

PARED < HS 1.28 1.55 1.03

PARED = HS 2.70 1.76 2.74

PARED > HS 1.32 2.28 1.57

PARED = College 7.92 7.96 5.88

2

PARED = Unknown  

Public school 9.49 7.07 5.43

Nonpublic school 12.24 9.18 20.54

1 Design effects are based on the conventional and jackknife variances of subgroup means of the first plausible values of proficiency. 2

 Insufficient data to compute design effects
230

Table 10-17

Design Effects of Subsample A2 by Demographic Subgroup and Grade

for Mean Mathematics Proficiencies1

Subgroup Grade 4 Grade 8 Grade 12

Total 7.95 3.98 5.98

Male 4.75 3.50 3.19

Female 4.78 2.02 5.32

White 4.39 3.95 3.50

Black 3.43 1.80 1.92

Hispanic 3.26 2.31 3.00

Asian/Pacific Islander 3.92 2.07 1.61

Other race/ethnicity 0.60 1.41 2.05

Urban 7.25 6.77 5.53

Suburban 13.77 4.80 5.36

Rural 4.24 4.25 3.94

PARED < HS 1.44 1.52 1.76

PARED = HS 1.81 2.18 1.41

PARED > HS 1.19 0.92 1.89

PARED = College 5.55 2.10 3.27

2

PARED = Unknown  

Public school 7.79 4.02 5.08

Nonpublic school 5.99 7.58 6.08

1 Design effects are based on the conventional and jackknife variances of subgroup means of the first plausible values of proficiency. 2

 Insufficient data to compute design effects

Table 10-18

Design Effects of Subsample B1 by Demographic Subgroup and Grade

for Mean Mathematics Proficiencies1

Subgroup Grade 4 Grade 8 Grade 12

Total 2.88 1.23 1.77

Male 2.42 1.38 1.38

Female 1.31 1.15 1.98

White 1.95 1.54 1.10

Black 1.60 0.98 1.77

Hispanic 3.73 0.81 0.78

Asian/Pacific Islander 1.90 0.80 8.32

Other race/ethnicity 0.87 1.13 13.01

Urban 3.96 1.26 1.91

Suburban 2.76 1.55 1.19

Rural 1.67 1.27 1.86

PARED < HS 1.38 0.69 1.46

PARED = HS 1.46 0.98 0.89

PARED > HS 1.21 0.98 1.21

PARED = College 1.45 2.00 0.72

2

PARED = Unknown  

Public school 2.85 1.10 1.49

Nonpublic school 0.31 0.35 5.99

1 Design effects are based on the conventional and jackknife variances of subgroup means of the first plausible values of proficiency. 2

 Insufficient data to compute design effects
231

Table 10-19

Design Effects by Demographic Subgroup and Grade

for Mean Science Proficiencies1

Subgroup Grade 4 Grade 8 Grade 12

Total 3.83 4.98 4.80

Male 2.43 3.35 3.77

Female 2.43 4.23 3.25

White 3.71 5.09 4.73

Black 4.17 2.01 2.79

Hispanic 3.08 3.57 4.60

Asian/Pacific Islander 4.00 3.07 2.81

Other race/ethnicity 2.45 2.38 2.18

Urban 11.40 7.17 11.86

Suburban 13.97 8.51 8.60

Rural 6.17 6.66 5.07

PARED < HS 1.23 1.75 1.91

PARED = HS 1.79 2.35 3.11

PARED > HS 1.16 1.66 2.43

PARED = College 2.64 4.50 2.95

PARED = Unknown 2.77 3.72 1.74

Public school 3.97 4.43 4.94

Nonpublic school 5.23 9.31 6.96

1 Design effects are based on the conventional and jackknife variances of subgroup means of the first plausible values of proficiency.

Table 10-20

Design Effects by Demographic Subgroup and Age Averaged Over Mean

Reading and Writing Proficiencies for the Reading and Writing Long-Term Trend Samples1

Subgroup Age 9 Age 13 Age 17

Total 1.86 2.13 2.13

Male 1.08 1.81 1.79

Female 2.00 1.85 2.08

White 1.51 1.24 1.84

Black 1.31 3.93 2.48

Hispanic 2.17 1.98 1.21

Asian/Pacific Islander 1.53 1.78 2.22

Other race/ethnicity 1.23 1.75 3.75

Urban 2.16 6.65 2.58

Suburban 3.37 4.35 2.08

Rural 3.74 3.33 8.14

PARED < HS 0.89 1.66 1.06

PARED = HS 1.68 1.09 1.03

PARED > HS 1.12 1.22 1.86

PARED = College 1.21 2.09 1.42

PARED = Unknown 2.08 2.04 0.68

Public school 2.05 1.89 2.41

Nonpublic school 1.96 2.28 2.16

1 Design effects are based on the conventional and jackknife variances of subgroup means of the first plausible values of proficiency.
232

Table 10-21

Design Effects by Demographic Subgroup and Age Averaged Over Mean

Mathematics and Science Proficiencies for the Mathematics and Science Long-Term Trend Samples1

Subgroup Age 9 Age 13 Age 17

Total 3.02 3.54 2.04

Male 2.88 1.76 1.86

Female 1.37 2.83 1.45

White 2.49 3.95 2.04

Black 2.83 1.79 1.17

Hispanic 2.06 2.06 1.75

Asian/Pacific Islander 2.29 2.72 1.71

Other race/ethnicity 1.79 2.18 1.8

Urban 4.42 6.18 5.16

Suburban 4.36 7.73 2.42

Rural 6.32 8.52 6.35

PARED < HS 0.84 2.09 1.8

PARED = HS 1.51 1.79 2.39

PARED > HS 1.88 1.08 1.7

PARED = College 2.67 2.14 1.77

PARED = Unknown 0.71 1.86 1.47

Public school 2.72 3.57 1.86

Nonpublic school 4.36 10.89 4.74

1 Design effects are based on the conventional and jackknife variances of subgroup means of the first plausible values of proficiency.

Table 10-22

Average Design Effects by Demographic Subgroup for Mean

State Mathematics Proficiencies Averaged Across State Samples1

Subgroup Grade 4 Grade 8

Total 4.15 3.45

Male 2.65 2.36

Female 2.66 2.36

White 3.06 2.56

Black 1.98 1.93

Hispanic 1.84 1.59

Asian/Pacific Islander 1.57 1.40

Other race/ethnicity 1.63 1.76

Urban 5.99 4.88

Suburban 4.23 3.59

Rural 4.19 3.08

PARED < HS 1.37 1.26

PARED = HS 1.65 1.79

PARED > HS 1.37 1.44

PARED = College 2.89 2.37

PARED = Unknown 2.13 1.47

Public school 4.02 3.42

Nonpublic school 7.06 5.60

1 Design effects are based on the conventional and jackknife variances of subgroup means

of the first plausible values of proficiency.
233

Table 10-23

Average Design Effects by Demographic Subgroup for Mean

State Science Proficiencies Averaged Across State Samples1

Subgroup Grade 8

Total 3.53

Male 2.43

Female 2.33

White 2.55

Black 1.92

Hispanic 1.63

Asian/Pacific Islander 1.34

Other race/ethnicity 1.70

Urban 5.13

Suburban 3.52

Rural 2.99

PARED < HS 1.34

PARED = HS 1.75

PARED > HS 1.45

PARED = College 2.26

PARED = Unknown 1.51

Public school 3.41

Nonpublic school 6.50

1 Design effects are based on the conventional and jackknife variances of

subgroup means of the first plausible values of proficiency.

Table 10-24

Within-Grade Mean, Median, and Upper Quartile of the Distribution of Design Effects

for National Main Assessments by Subject Area and Across Subject Areas

Statistic Grade 4 Grade 8 Grade 12

Mean Mathematics Proficiencies

(Distribution Across Demographics Subgroups)

Upper Quartile 6.09 5.92 6.50

Mean 4.48 4.52 6.09

Median 4.88 5.44 6.08

Mean Science Proficiencies

(Distribution Across Demographics Subgroups)

Upper Quartile 4.17 5.09 4.94

Mean 4.25 4.37 4.36

Median 3.90 4.47 4.67

Across Subject Areas

(Distribution Across Subject Areas and Demographic Subgroups)

Upper Quartile 5.24 5.92 6.36

Mean 4.36 4.45 5.23

Median 3.92 4.33 4.58
234

Table 10-25

Mean, Median, and Upper Quartile of the Across-State Average Design Effects

for Mean State Mathematics Proficiency (Distribution Across Demographic Subgroups)

Statistics Grade 4 Grade 8

Upper Quartile 4.15 3.42

Mean 3.02 2.57

Median 2.66 2.36

Table 10-26

Mean, Median, and Upper Quartile of the Across-State Average Design Effects

for Mean State Science Proficiency (Distribution Across Demographic Subgroups)

Statistics Grade 8

Upper Quartile 3.41

Mean 2.63

Median 2.30

Table 10-27

Mean, Median, and Upper Quartile of the Across-State Average Design Effects

for Mean State Proficiency (Distribution Across Demographic Subgroups and Across Subjects)

Statistics Grade 4 Grade 8

Upper Quartile 4.15 3.42

Mean 3.02 2.60

Median 2.66 2.35

Table 10-28

Mean, Median, and Upper Quartile of the Distribution of Design Effects

for the Long-Term Trend Samples1

Statistic Age 9 Age 13 Age 17

Reading and Writing Long-Term Trend

(Distribution Across Demographic Subgroups of Average

of Design Effects for Reading and Writing Mean Proficiencies)

Upper Quartile 2.08 3.09 2.48

Mean 1.81 2.44 2.40

Median 1.67 1.94 2.08

Mathematics and Science Long-Term Trend

(Distribution Across Demographic Subgroups of Average

of Design Effects for Mathematics and Science Mean Proficiencies)

Upper Quartile 3.02 3.95 2.39

Mean 2.70 3.70 2.42

Median 2.58 2.45 1.83

1 Design effects are based on the conventional and jackknife variances of subgroup means of the first plausible values of

proficiency.
235

Chapter 11

SCALING PROCEDURES1

Nancy L. Allen, Eugene G. Johnson, Robert J. Mislevy, and Neal Thomas

Educational Testing Service

11.1 INTRODUCTION

The primary method by which results from the 1996 National Assessment of Educational

Progress (NAEP) were disseminated was scale-score reporting. With scaling methods, the performance

of a sample of students in a subject area or subarea can be summarized on a single scale or series of

scales even when different students have been administered different items. This chapter presents an

overview of the scaling methodologies employed in the analyses of the data from NAEP surveys in

general and from the 1996 assessment in particular. Details of the scaling procedures specific to the

subject areas of science, mathematics, reading, and writing are presented in Chapters 12 through 17.

11.2 BACKGROUND

The basic information from an assessment consists of the responses of students to the items

presented in the assessment. For NAEP, these items are constructed to measure performance on sets of

objectives developed by nationally representative panels of learning area specialists, educators, and

concerned citizens. Satisfying the objectives of the assessment and ensuring that the tasks selected to

measure each goal cover a range of difficulty levels typically require many items. For example, the

mathematics assessment required 164 items at grade 8. Depending on the subject areas, a mixture of

multiple-choice, short constructed-response, and extended constructed-response items were used.

Multiple-choice and short constructed-response items were used in all assessments but writing. Extended

constructed-response items, scored on a multipoint scale, were presented in the main mathematics and

science assessments and in the long-term trend writing assessment. To reduce student burden, each

assessed student was presented only a fraction of the full pool of items through multiple matrix sampling

procedures.

The most direct manner of presenting the assessment results is to report separate statistics for

each item. However, because of the vast amount of information, having separate results for each of the

items in the assessment pool hinders the comparison of the general performance of subgroups of the

population. Item-by-item reporting masks similarities in trends and subgroup comparisons that are

common across items.

An obvious summary of performance across a collection of items is the average of the separate

item scores. The advantage of averaging is that it tends to cancel out the effects of peculiarities in items

 

1

 Nancy L. Allen is responsible for the psychometric and statistical analysis of national and state NAEP data. Eugene G. Johnson

is a senior psychometrician, contributing to the design of NAEP and to discussions of sampling issues. Previously he was

responsible for the psychometric and statistical analysis of NAEP data. Robert J. Mislevy is a technical consultant contributing in

the area of item response theory. Neal Thomas was a technical consultant to the NAEP analysis staff, contributing in the area of

imputed values.
236

that can affect item difficulty in unpredictable ways. Furthermore, averaging makes it possible to

compare more easily the general performances of subpopulations.

Despite their advantages, there are a number of significant problems with average item scores.

First, the interpretation of these results depends on the selection of the items; the selection of easy or

difficult items could make student performance appear to be overly high or low. Second, the average

score is related to the particular items comprising the average, so that direct comparisons in performance

between subpopulations require that those subpopulations have been administered the same set of items.

Third, because this approach limits comparisons to average scores on specific sets of items, it provides

no simple way to report trends over time when the item pool changes. Finally, direct estimates of

parameters or quantities such as the proportion of students who would achieve a certain score across the

items in the pool are not possible when every student is administered only a fraction of the item pool.

While the average score across all items in the pool can be readily obtained (as the average of the

individual item scores), statistics that provide distributional information, such as quantiles of the

distribution of scores across the full set of items, cannot be readily obtained without additional

assumptions.

These limitations can be overcome by the use of response scaling methods. If several items

require similar skills, the regularities observed in response patterns can often be exploited to characterize

both respondents and items in terms of a relatively small number of variables. These variables include a

respondent-specific variable, called proficiency, which quantifies a respondent’s tendency to answer

items correctly (or, for multipoint items, to achieve a certain score) and item-specific variables that

indicate characteristics of the item such as its difficulty, effectiveness in distinguishing between

individuals with different levels of proficiency, and the chances of a very low proficiency respondent

correctly answering a multiple-choice item. (These variables are discussed in more detail in the next

section.) When combined through appropriate mathematical formulas, these variables capture the

dominant features of the data. Furthermore, all students can be placed on a common scale, even though

none of the respondents takes all of the items within the pool. Using the common scale, it becomes

possible to discuss distributions of proficiency in a population or subpopulation and to estimate the

relationships between proficiency and background variables.

It is important to point out that any procedure of aggregation, from a simple average to a complex

multidimensional scaling model, highlights certain patterns at the expense of other potentially interesting

patterns that may reside within the data. Every item in a NAEP survey is of interest and can provide

useful information about what young Americans know and can do. The choice of an aggregation

procedure must be driven by a conception of just which patterns are salient for a particular purpose.

The scaling for the main assessments in mathematics and science was carried out separately

within the content strands or fields specified in the frameworks for those subjects, respectively. This

scaling within subareas was done because it was anticipated that different patterns of performance or

different trends over time might exist for these essential subdivisions of the subject areas. By creating a

separate scale for each of these content areas, potential differences in subpopulation performance

between the content areas are preserved.

The creation of a series of separate scales to describe performance within a subject area does not

preclude the reporting of a single index of overall performance in the subject areathat is, an overall

subject area composite. A composite is computed as the weighted average of the content area scales,

where the weights correspond to the relative importance given to each content area as defined by the

framework. The composite provides a global measure of performance within the subject area, while the
237

constituent content area scales allow the measurement of important interactions within educationally

relevant subdivisions of the subject area.

11.3 SCALING METHODOLOGY

This section reviews the scaling models employed in the analyses of data of the 1996 assessment,

and the multiple imputation or “plausible values” methodology that allows such models to be used with

NAEP’s sparse item-sampling design. The reader is referred to Mislevy (1991) for an introduction to

plausible values methods and a comparison with standard psychometric analyses, to Donoghue (1994),

Mislevy, Johnson and Muraki (1992), and Beaton and Johnson (1992) for additional information on how

the models are used in NAEP, and to Rubin (1987) for the theoretical underpinnings of the approach. It

should be noted that the imputation procedure used by NAEP is a mechanism for providing plausible

values for the unobserved proficiencies and not for filling in blank responses to background or cognitive

variables.

While the NAEP procedures were developed explicitly to handle the characteristics of NAEP

data, they build on other research, and are paralleled by other researchers. See, for example Dempster,

Laird, and Rubin (1977); Little and Rubin (1983, 1987); Andersen (1980); Engelen (1987); Hoijtink

(1991); Laird (1978); Lindsey, Clogg, and Grego (1991); Zwinderman (1991); Tanner and Wong (1987);

and Rubin (1987, 1991).

11.3.1 The Scaling Models

Three distinct scaling models, depending on item type and scoring procedure, were used in the

analysis of the data from the 1996 assessment. Each of the models is based on item response theory (IRT;

e.g., Lord, 1980). Each is a “latent variable” model, defined separately for each of the scales, which

expresses respondent’s tendencies to achieve certain scores (such as correct/incorrect) on the items

contributing to a scale as a function of a parameter that is not directly observed, called proficiency on the

scale.

A three-parameter logistic (3PL) model was used for the multiple-choice items (which were

scored correct/incorrect). The fundamental equation of the 3PL model is the probability that a person

whose proficiency on scale k is characterized by the unobservable variable k will respond correctly to

item j:

Px a b c c c

a b j k jjj j P j

j k j

 j k



 

1  1

1 17

 1

 ,,, exp .  ,  

   (11.1)

where

xj is the response to item j, 1 if correct and 0 if not;

aj where aj>0, is the slope parameter of item j, characterizing its sensitivity to

proficiency;

bj is the threshold parameter of item j, characterizing its difficulty; and

cj where 0cj<1, is the lower asymptote parameter of item j, reflecting the chances of

students of very low proficiency selecting the correct option.
238

Further define the probability of an incorrect response to the item as

P Px a b c P j j k jjj j k 0 1     0 1   ,,, .   (11.2)

A two-parameter logistic (2PL) model was used for the short constructed-response items that

were scored correct or incorrect. The form of the 2PL model is the same as Equations (11.1) and (11.2)

with the cj parameter fixed at zero.

In addition to the multiple-choice and short constructed-response items, a number of extended

constructed-response items were presented in the assessments of mathematics and science; and only

extended constructed-response items were presented in the long-term trend writing assessment. Each of

these items was scored on a multipoint scale with potential scores ranging from 0 to 3 or from 0 to 4.

Some short constructed-response items were scored on a three-point scale (0-2). Items that are scored on

a multipoint scale are referred to as polytomous items, in contrast with the multiple-choice and short

constructed-response items, which are scored correct or incorrect and referred to as dichotomous items.

The polytomous items were scaled using a generalized partial credit model (Muraki, 1992). The

fundamental equation of this model is the probability that a person with proficiency k on scale k will

have, for the jth item, a response xj that is scored in the ith of mj ordered score categories:

Px i a b d d

a bd

a bd

j k j j j jm P

j k j jv

v

i

j k j jv

v

g

g

m ji k j j

 

 



 

	




 



 

	


 -  =

= =

-



 







, , , , ,..., , 

exp . ,

exp . ,

1 1

0

0 0

1

1 7

1 7

 

 

 

  (11.3)

where

mj is the number of categories in the response to item j

xj is the response to item j, with possibilities 0,1,...,mj-1

aj is the slope parameter;

bj is the item location parameter characterizing overall difficulty; and

dj,i is the category i threshold parameter (see below).

Indeterminacies in the parameters of the above model are resolved by setting dj,0 = 0 and setting

dj i

i

mj

, .  =

-

 0

1

1

 Muraki (1992) points out that bj - dj,i is the point on the k scale at which the plots of Pj,i-1(k)

and Pji(k) intersect and so characterizes the point on the k scale at which the response to item j has

equal probability of falling in response category i-1 and falling in response category i.

When mj = 2, so that there are two score categories (0,1), it can be shown that Pji(k) of Equation

(11.3) for i=0,1 corresponds respectively to Pj0(k) and Pj1(k) of the 2PL model (Equations (11.1) and

(11.2) with cj=0).
239

Close examination of the 3PL and generalized partial credit models indicate that both models

have a linear indeterminacy of the theta scale. In other words, if the item parameters were estimated in a

different metric, the value of k could be transformed to make (11.1) and (11.3) true. For the purposes of

reporting item parameter estimates and other intermediary estimates, the linear indeterminacies apparent

in (11.1) and (11.3) may be resolved by an arbitrary choice of the origin and unit size in a given scale. In

most cases, a provisional scale standardizing the theta distribution to have mean 0 and standard deviation

1 is employed. Final results for each content area were linearly transformed from the scale to a 0-to500 (for mathematics) or a 0-to-300 scale (for science), as described in the subject area chapters in this

report.

A basic assumption of item response theory is the conditional independence of the responses by

an individual to a set of items, given the individual’s proficiency. That is, conditional on the

individual’sk, the joint probability of a particular response pattern x = (x1,...,xn) across a set of n items

is simply the product of terms based on (11.1), (11.2), and (11.3):

P x item parameters P k ji k

u

i

m

j

n ji

j

   ,    =

-

=



0

1

1

(11.4)

where Pji(k) is of the form appropriate to the type of item (dichotomous or polytomous), mj is equal to 2

for the dichotomously scored items, and uji is an indicator variable defined by

u

if response x isincategoryi

otherwise ji

j

 






1

0

It is also typically assumed that response probabilities are conditionally independent of

background variables (y), given k, or

P x item parameters y p x item parameters   k k , ,, .     (11.5)

After x has been observed, equation 11.4 can be viewed as a likelihood function, and provides a

basis for inference about k or about item parameters. Estimates of item parameters were obtained by the

NAEP BILOG/PARSCALE program, which combines Mislevy and Bock’s (1982) BILOG and Muraki

and Bock’s (1991) PARSCALE computer programs2

, and which concurrently estimates parameters for all

items (dichotomous and polytomous). Donoghue (1994) reports on the effect of having both dichotomous

and polytomous items within a scale. The item parameters are then treated as known in subsequent

calculations. In subject areas with multiple scales (main mathematics and science), the parameters of the

items constituting each of the separate scales were estimated independently of the parameters of the other

scales. Once items have been calibrated in this manner, a likelihood function for the proficiency k is

induced by a vector of responses to any subset of calibrated items, thus allowing k-based inferences

from matrix samples. The likelihood function for the proficiency k is called the posterior distribution of

the thetas for each student.

In almost all NAEP IRT analyses, missing responses at the end of each block of items a student

was administered were considered “not reached,” and treated as if they had not been presented to the

respondent. Missing responses to dichotomous items before the last observed response in a block were

considered intentional omissions, and treated as fractionally correct at the value of the reciprocal of the

 

2

 See Muraki and Bock (1997) for the current version of PARSCALE.
240

number of response alternatives, if the item was a multiple-choice item. These conventions are discussed

by Mislevy and Wu (1988). With regard to the handling of not-reached items, Mislevy and Wu found that

ignoring not-reached items introduces slight biases into item parameter estimation when not-reached

items are present and speed is correlated with ability. With regard to omissions, they found that the

method described above provides consistent limited-information maximum likelihood estimates of item

and ability parameters under the assumption that respondents omit only if they can do no better than

responding randomly.

Occasionally, extended constructed-response items were the last item in a block of items.

Because considerably more effort was required of the student to answer these items, nonresponse to an

extended constructed-response item at the end of a block was considered an intentional omission (and

scored as the lowest category, 0) unless the student also did not respond to the item immediately

preceding that item. In that case, the extended constructed-response item was considered not reached and

treated as if it had not been presented to the student.

Although the IRT models are employed in NAEP only to summarize performance, a number of

checks are made to detect serious violations of the assumptions underlying the models. Checks are made

to detect multidimensionality of the construct being measured and certain condition dependencies. DIF

analyses are used to examine issues of dimensionality, and what are called 2

 statistics in the IRT

literature are used to flag responses with serious departures from the IRT model. The latter statistics

might better be called item fit statistics since they do not really have 2

 distributions. These checks

include comparisons of empirical and theoretical item response functions to identify items for which the

IRT model may provide a poor fit to the data. When warranted, remedial efforts, such as collapsing

categories of polytomous items or combining items into a single item, are made to mitigate the effects of

such violations on inferences.

Scaling areas in NAEP are determined a priori by grouping items into content areas for which

overall performance is deemed to be of interest, as defined by the frameworks developed by the National

Assessment Governing Board (NAGB). A proficiency scale k is defined a priori by the collection of

items representing that scale. What is important, therefore, is that the models capture salient information

in the response data to effectively summarize the overall performance on the content area of the

populations and subpopulations being assessed in the content areas. NAEP has routinely conducted

differential item functioning (DIF) analyses to guard against potential biases in making subpopulation

comparisons based on the proficiency distributions.

The local independence assumption embodied in Equation (11.4) implies that item response

probabilities depend only on and the specified item parameters, and not on the position of the item in

the booklet, the content of items around an item of interest, or the test-administration and timing

conditions. However, these effects are certainly present in any application. The practical question is

whether inferences concerning aggregate performance in the scaling area that are based on the IRT

probabilities obtained via (11.4) are robust with respect to the ideal assumptions underlying the IRT

model. Our experience with the 1986 NAEP reading anomaly (Beaton & Zwick, 1990) has shown that for

measuring small changes over time, changes in item context and speededness conditions can lead to

unacceptably large random error components. These can be avoided by presenting items used to measure

change in identical test forms, with identical timings and administration conditions. Thus, we do not

maintain that the item parameter estimates obtained in any particular booklet configuration are

appropriate for other conceivable configurations. Rather, we assume that the parameter estimates are

context-bound. This is the reason that the long-term trend booklets and administration procedures have

not changed since the early 1980s and only a limited number of blocks of items are released after each

main assessment cycle. It was also the reason we prefer common population equating to common item
241

equating whenever equivalent random samples are available for linking. In common item equating, items

are assumed to be measuring exactly the same thing for two or more populations, despite any differences

in context or administration. In common population equating, results for two or more samples from the

same population are matched to one another when linking the scales. Therefore, the data from the State

Assessment were calibrated separately from the national NAEP data. In this case, the administration

procedures differed somewhat between the State Assessment and the national NAEP.

In practice, PARSCALE item fit statistics are used as a way to identify items that need further

examination. Most of the statistics of this type that are available for use in this setting have distributions

that are unknown. Therefore, they cannot be used for final decisions about the fit of the items to the IRT

model. Because of the lack of statistical tests for IRT model fit, the fit of the IRT models to the observed

data was examined within each scale by comparing the empirical item response functions (IRFs) with the

theoretical curves. The primary means of accomplishing this is to generate plots of empirical versus

theoretical item response curves. The theoretical curves are plots of the response functions based on the

estimates of the item parameters. The empirical proportions are calculated from the posterior

distributions of the thetas for each student who received the item. For dichotomous items, the sum of the

values of the posterior distributions at a point on the theta scale for each student who answered an item

correctly plus the sum of a fractional portion of the values of the posterior distribution at that point on the

theta scale for each student who omitted the item is parallel in meaning to the number of students who

actually answered the item correctly plus a fraction of the number of students who omitted the item. The

sum of the values of the posterior distributions for all students receiving the item at each point on the

theta scale is parallel in meaning to the empirical number of students at that point on the theta scale who

received the item. The plotted values are sums of these individual posteriors at each point on the theta

scale for those who got the item correct plus a fraction of the omitters divided by the sum of the

posteriors of those administered the item, in the case of dichotomous items, and for those who scored in

the category of interest over the sum for those who received the item, in the case of polytomous items.

Figure 11-1 contains a plot of the empirical and theoretical IRFs for a dichotomous item. In the

plot, the horizontal axis represents the theta (proficiency) scale, the vertical axis represents the

probability of a correct response. The solid curve is the theoretical IRF based on the item parameter

estimates and Equation (11.1). The centers of the diamonds represent the empirical proportions correct as

described above. The size of the diamonds are proportional to the sum of the posteriors at each point on

the theta scale for all of those who received the item; this is related to the number of students

contributing to the estimation of that empirical proportion correct.

Figure 11-2 contains a plot of the empirical and theoretical IRFs for a polytomous item from the

1997 Arts (Theatre) National Assessment. As for the dichotomous item plot in Figure 11-1, the

horizontal axis represents the proficiency scale, but the vertical axis represents the probability of having

a response fall in each category. The solid curves are the theoretical IRFs based on the item parameter

estimates and Equation (11.3). The centers of the diamonds represent the empirical proportions of

students with responses in each category and are proportional to the sum of the posteriors at each point

on the theta scale for the students who received the item.

For good fitting items, the empirical and theoretical curves are close together. Therefore, items

for which this is not true are examined carefully. Examples of plots for specific items are provided in the

subject-area chapters. When the same items are presented in two assessment years, the empirical curves

for the two years can be compared. Normally, these curves differ somewhat due to the sampling of

students for each of the two years. Figure 12-1 contains a plot for an item with curves of this type. When

the empirical curves differ dramatically, one cause might be a change in the meaning of the item due to

instructional or societal changes across the years. This type of item is ordinarily treated as two different
242

items—one for each of the assessment years. Figure 12-4 contains the plot for an item that has been

treated in this way.

To summarize, using current methodologies in psychometrics, assumption of conditional

independence and the assumption that the data fit the models in Equations 11.1 and 11.3 are examined

and controlled in NAEP in several ways. They are examined by considering tests of DIF, item fit

statistics, and plots of empirical and theoretical IRFs. They are controlled by treating missing and “not

reached” responses in reasonable ways, maintaining the context and administration of items across

assessments, collapsing categories of polytomous items when appropriate, combining items into a single

item, or making decisions about the inclusion or exclusion of an item in a scale based on data. The

identification and amelioration of violations of IRT assumptions is an area of ongoing research in

educational measurement.

Figure 11-1

Example Cross-Sectional Dichotomous Item (R016102, Age 13/Grade 8)

Exhibiting Good Model Fit*

*Note: The plot compares empirical and model-based estimates of the item response function

(IRF). The smooth curve represents the model-based estimate at each provisional proficiency

level. The diamonds represent the empirical proportion of 1994 age 13/grade 8 students

answering correctly at each point on the theta scale.
243

Figure 11-2

Example Polytomous Item (HC00004, Grade 8) Exhibiting Good Model Fit*

*Note: The plot compares empirical and model-based estimates of the item category

response functions (ICRFs). The smooth curve represents the model-based estimate at

each provisional proficiency level. The diamonds represent the empirical proportion of

1997 grade 8 students with responses in each category at each point on the theta scale.

11.3.2 An Overview of Plausible Values Methodology

Item response theory was developed in the context of measuring individual examinees’ abilities.

In that setting, each individual is administered enough items (often 60 or more) to permit precise

estimation of his or her , as a maximum likelihood estimate, 

 , for example. Because the uncertainty

associated with each is negligible, the distribution of , or the joint distribution of with other

variables, can then be approximated using an individual’s 

 values as if they were values.

This approach breaks down in the assessment setting when, in order to provide broader content

coverage in limited testing time, each respondent is administered relatively few items in a subject area

scale. A first problem is that the uncertainty associated with individual s is too large to ignore, and the

features of the 

 distribution can be seriously biased as estimates of the distribution. (The failure of

this approach was verified in early analyses of the 1984 NAEP reading survey; see Wingersky, Kaplan,

& Beaton, 1987.) A second problem, occurring even with test lengths of 60, arises when test forms vary

across and within assessments as to the numbers, formats, and content of the test items. The measurement

error distributions thus differ even if underlying distributions do not, causing 

 distributions to exhibit
244

spurious changes and comparisons in apparent population distributionseasily greater than actual

differences over time or across groups. Although this latter problem is avoided in traditional standardized

testing by presenting students with parallel test forms, controlled tightly across time and groups, the same

constraints cannot be imposed in the design and data-collection phases of the present NAEP. Plausible

values were developed as a way to estimate key population features consistently, and approximate others

no worse than standard IRT procedures would, even when item booklet composition, format, and content

balances change over time. A detailed development of plausible values methodology is given in Mislevy

(1991). Along with theoretical justifications, that paper presents comparisons with standard procedures,

discussions of biases that arise in some secondary analyses, and numerical examples.

The following provides a brief overview of the plausible values approach, focusing on its

implementation in the 1996 NAEP analyses.

Let y represent the responses of all sampled examinees to background and attitude questions,

along with variables based on the sampling design such as the school where the student is enrolled, and

let represent the vector of proficiency values. If were known for all sampled examinees, it would be

possible to compute a statistic t(,y), such as a scale or composite subpopulation sample mean, a sample

percentile point, or a sample regression coefficient, to estimate a corresponding population quantity T. A

function U(,y)e.g., a jackknife estimate—would be used to gauge sampling uncertainty, as the

variance of t around T in repeated samples from the population.

Because the scaling models are latent variable models, however, values are not observed even

for sampled students. To overcome this problem, we follow Rubin (1987) by considering as “missing

data,” and approximate t(,y) by its expectation given (x,y), the data that actually were observed, as

follows:

t xy t yxy

t yp xyd

*, , ,

, ,.

 

   



 

 

 

(11.6)

It is possible to approximate t

*

 using random draws from the predictive conditional distribution

of the scale proficiencies given the item responses xi, background variables yi, and model parameters for

sampled student i. These values are referred to as imputations in the sampling literature, and plausible

values in NAEP. The value of for any respondent that would enter into the computation of t is thus

replaced by a randomly selected value from the respondent’s conditional distribution. Rubin (1987)

proposes that this process be carried out several timesmultiple imputationsso that the uncertainty

associated with imputation can be quantified. The average of the results of, for example, M estimates of t,

each computed from a different set of plausible values, is a Monte Carlo approximation of (11.6); the

variance among them, B, reflects uncertainty due to not observing , and must be added to the estimated

expectation of U(,y), which reflects uncertainty due to testing only a sample of students from the

population. Section 11.5 explains how plausible values are used in subsequent analyses.

It cannot be emphasized too strongly that plausible values are not test scores for individuals in

the usual sense. Plausible values are offered only as intermediary computations for calculating integrals

of the form of Equation (11.6), in order to estimate population characteristics. When the underlying

model is correctly specified, plausible values will provide consistent estimates of population

characteristics, even though they are not generally unbiased estimates of the proficiencies of the

individuals with whom they are associated. The key idea lies in the contrast between plausible values and
245

the more familiar estimates of proficiency (e.g., maximum likelihood estimate or Bayes estimate) that are

in some sense optimal for each examinee: Point estimates that are optimal for individual examinees have

distributions that can produce decidedly nonoptimal (specifically, inconsistent) estimates of population

characteristics (Little & Rubin, 1983). Plausible values, on the other hand, are constructed explicitly to

provide consistent estimates of population effects. For further discussion see Mislevy, Beaton, Kaplan,

and Sheehan (1992).

11.3.3 Computing Plausible Values in IRT-Based Scales

Plausible values for each respondent r are drawn from the predictive conditional distribution

p(r|xr,yr,,), where  and are regression model parameters defined in this subsection. This

subsection describes how, in IRT-based scales, these conditional distributions are characterized, and how

the draws are taken. An application of Bayes’ theorem with the IRT assumption of conditional

independence produces

p xy r r r , ,  , 	 Px y  rr r , ,,p y r r, ,  Px p y   r  r r r,, (11.7)

where, for vector-valued r, P(xr|r) is the product over scales of the independent likelihoods induced by

responses to items within each scale, and p(r|yr, ,) is the multivariateand generally

nonindependentjoint density of proficiencies for the scales, conditional on the observed value yr of

background responses, and the parameters  and . The provisional scales are determined by the item

parameter estimates that constrain the population mean to zero and standard deviation to one. The item

parameter estimates are fixed and regarded as population values in the computation described in this

subsection.

In the analyses of the data from the main assessments, a normal (Gaussian) form was assumed for

p(r|yr,,), with a common variance-covariance matrix, , and with a mean given by a linear model

with slope parameters, , based on the first approximately 200 principal components of several hundred

selected main-effects and two-way interactions of the complete vector of background variables. The

included principal components will be referred to as the conditioning variables, and will be denoted y

c

.

(The complete set of original background variables used in the analyses of each subject area are listed in

Appendix C.) The following model was fit to the data within each subject area:

 =  ’ yc 
 (11.8)

where is multivariately normally distributed with mean zero and variance-covariance matrix . The

number of principal components of the background variables used for each sample was sufficient to

account for 90 percent of the total variance of the full set of background variables (after standardizing

each variable). As in regression analysis,  is a matrix each of whose columns is the effects for one scale

and  is the matrix variance-covariance of residuals between scales.

A model similar to (11.8) was used for the long-term trend assessments, with the difference that

y

c

 consisted of main effects and interactions from the smaller set of background variables (rather than

principal components of those variables) available in the long-term trend assessments.
246

Maximum likelihood estimates of  and , denoted by 

and  , are obtained with extensions

of Sheehan’s (1985) MGROUP computer program using the EM algorithm described in Mislevy (1985).

The EM algorithm requires the computation of the mean, r, and variance-covariance matrix, p

r , of the

predictive conditional distribution in (11.7) for respondent r when there are p scales within a subject

area. For subject areas with multiple scales, the CGROUP version of the MGROUP program was used to

compute the moments using higher order asymptotic corrections to a normal approximation (Thomas,

1993). For the long-term trend assessments, each of which have a single scale, the more precise but

computationally intensive BGROUP version of MGROUP was used. BGROUP uses numeric quadrature

to evaluate the predictive conditional distribution moments required by the E-step of the EM algorithm

for one- and two-dimensional applications (Thomas, 1993). For estimation of group means on a single

scale, CGROUP and BGROUP results will be nearly identical to those from the original MGROUP

program. CGROUP and BGROUP yield better estimates of correlations between scales, and hence better

estimates of composite scale means. BGROUP will, theoretically, yield better estimates than CGROUP,

but because of the heavy computational demands of the methodology used, its function is limited to

bivariate scales. Hence CGROUP is used for assessments involving more than two scales.

After completion of the EM algorithm, the plausible values for all sampled respondents is drawn

in the following three-step process. First, a value of  is drawn from a normal distribution with mean

being 

and variance being the variance of . Second, conditional on the generated value of  and the

fixed value of = 

, the predictive conditional distribution mean r and the predictive conditional

distribution variance r of respondent r are computed from Equation 11.7 using the EM algorithm (see

Thomas, 1993). Finally, the r are drawn independently from a multivariate normal distribution with

mean r and variance r approximating the distribution in (11.7). These three steps are repeated five

times producing five sets of imputation values for all sampled respondents.

11.4 INFERENCES ABOUT PROFICIENCIES

When survey variables are observed without error from every respondent, usual variance

estimators quantify the uncertainty associated with sample statistics from the only source of uncertainty,

namely the sampling of respondents. Item-level statistics for NAEP cognitive items meet this

requirement, but proficiency values do not. The IRT models used in their construction posit an

unobservable proficiency variable to summarize performance on the items in the subarea. The fact that

values are not observed even for the respondents in the sample requires additional statistical analyses

to draw inferences about distributions and to quantify the uncertainty associated with those inferences.

As described above, Rubin’s (1987) multiple imputations procedures were adapted to the context of

latent variable models to produce the plausible values upon which many analyses of the data from the

1996 assessment were based. This section describes how plausible values were employed in subsequent

analyses to yield inferences about population and subpopulation distributions of proficiencies.

11.4.1 Computational Procedures

Even though one does not observe the value of respondent r, one does observe variables that

are related to it: xr, the respondent’s answers to the cognitive items he or she was administered in the area

of interest, and yr, the respondent’s answers to demographic and background variables. Suppose one

wishes to draw inferences about a number T(,Y) that could be calculated explicitly if the and y values
247

of each member of the population were known. Suppose further that if values were observable, we

would be able to estimate T from a sample of N pairs of and y values by the statistic t(,y) [where

(,y)  (1,y1,...,N,yN)], and that we could estimate the variance in t around T due to sampling

respondents by the function U(,y). Given that observations consist of (xr,yr) rather than (r,yr), we can

approximate t by its expected value conditional on (x,y), or

t

*

 (x,y) = E[t(,y)|x,y] =  t(,y) p(|x,y) d.

It is possible to approximate t

*

 with random draws from the conditional distributions p(i|xi,yi),

which are obtained for all respondents by the method described in Section 11.3.3. Let 

m be the mth such

vector of plausible values, consisting of a multidimensional value for the latent variable of each

respondent. This vector is a plausible representation of what the true vector might have been, had we

been able to observe it.

The following steps describe how an estimate of a scalar statistic t(,y) and its sampling variance

can be obtained from M (>1) such sets of plausible values. (Five sets of plausible values are used in

NAEP analyses.)

1. Using each set of plausible values 

m in turn, evaluate t as if the plausible values

were true values of . Denote the results tm , for m=1,...,M.

2. Using the jackknife variance estimator defined in Chapter 10, compute the estimated

sampling variance of tm , denoting the result Um.

3. The final estimate of t is

t

t

M

m

m

M

*   . =



1

4. Compute the average sampling variance over the M sets of plausible values, to

approximate uncertainty due to sampling respondents

U U

M

m

m

M

* .  =



1

5. Compute the variance among the M estimates tm , to approximate the betweenimputation variance

B

t t

M

m

m

M

 

=    * .  

 

2

1 1

6. The final estimate of the variance of t

*

 is the sum of two components

VU M B  - * . 1 1  

In this equation, (1+M-1)B is the estimate of variance due to the latency of . Due to the

excessive computation that would be required, NAEP analyses did not compute and
248

average jackknife variances over all five sets of plausible values, but only on the first set.

Thus, in NAEP reports, U*

 is approximated by U1.

11.4.2 Statistical Tests

The variance described in Section 11.4.1 is used to make statistical tests comparing NAEP

results. This section describes the relationships between these tests and the variance components

described above. Chapter 18 contains details of the hypothesis tests used in this assessment.

Suppose that if values were observed for sampled students, the statistic (t - T)/U1/2 would

follow a t-distribution with d degrees of freedom. Then the incomplete-data statistic (t* - T)/V1/2 is

approximately t-distributed, with degrees of freedom (Satterthwaite, 1941; Johnson & Rust, 1993)

given by

v

f

M

f

d



 

1

1

1 2 2  

where f is the proportion of total variance due to not observing values:

f  M B V - 1 1   / .

When B is small relative to U*, the reference distribution for incomplete-data statistics differs

little from the reference distribution for the corresponding complete-data statistics. This is the case with

main NAEP reporting variables. If, in addition, d is large, the normal approximation can be used to flag

“significant” results.

For k-dimensional t, such as the k coefficients in a multiple regression analysis, each Um and U*

is a covariance matrix, and B is an average of squares and cross-products rather than simply an average of

squares. In this case, the quantity (T-t*

) V -1 (T-t*), is approximately F distributed, with degrees of

freedom equal to k and , with  defined as above but with a matrix generalization of f:

f = (1+M-1) Trace (BV-1)/k .

By the same reasoning as used for the normal approximation for scalar t, a chi-square distribution on k

degrees of freedom often suffices for multivariate t.

11.4.3 Biases in Secondary Analyses

Statistics t

*

 that involve proficiencies in a scaled content area and variables included in the

conditioning variables y

c

 are consistent estimates of the corresponding population values T. This includes

interrelationships among scales within a content area that have been treated in the multivariate manner

described above in Section 11.3.3. Statistics involving background variables y that were not conditioned

on, or relationships among proficiencies from different content strands or fields, are subject to asymptotic

biases whose magnitudes depend on the type of statistic and the strength of the relationships of the

nonconditioned background variables to the variables that were conditioned on and to the proficiency of

interest. That is, the large sample expectations of certain sample statistics need not equal the true

population parameters.
249

The direction of the bias is typically to underestimate the effect of nonconditioned variables. For

details and derivations see Beaton and Johnson (1990), Mislevy (1991), and Mislevy and Sheehan (1987,

Section 10.3.5). For a given statistic t

*

 involving one content area and one or more nonconditioned

background variables, the magnitude of the bias is related to the extent to which observed responses x

account for the latent variable , and the degree to which the nonconditioned background variables are

explained by conditioning background variables. The first factorconceptually related to test

reliabilityacts consistently in that greater measurement precision reduces biases in all secondary

analyses. The second factor acts to reduce biases in certain analyses but increase it in others. In

particular,

• High shared variance between conditioned and nonconditioned background variables

mitigates biases in analyses that involve only proficiency and nonconditioned

variables, such as marginal means or regressions.

• High shared variance exacerbates biases in regression coefficients of conditional

effects for nonconditioned variables, when nonconditioned and conditioned

background variables are analyzed jointly as in multiple regression.

The large number of background variables that have been included in the conditioning vectors for the

1996 assessments allows a large number of secondary analyses to be carried out with little or no bias, and

mitigates biases in analyses of the marginal distributions of in nonconditioned variables. Kaplan and

Nelson’s analysis of the 1988 NAEP reading data (some results of which are summarized in Mislevy,

1991), which had a similar design and fewer conditioning variables, indicates that the potential bias for

nonconditioned variables in multiple regression analyses is below 10 percent, and biases in simple

regression of such variables is below five percent. Additional research (summarized in Mislevy, 1990)

indicates that most of the bias reduction obtainable from conditioning on a large number of variables can

be captured by instead conditioning on the first several principal components of the matrix of all original

conditioning variables. This procedure was adopted for the 1992, 1994, and 1996 main assessments by

replacing the conditioning effects by the first K principal components, where K was selected so that 90

percent of the total variance of the full set of conditioning variables (after standardization) was captured.

Mislevy (1990) shows that this puts an upper bound of 10 percent on the average bias for all analyses

involving the original conditioning variables.

11.4.4 A Numerical Example

To illustrate how plausible values are used in subsequent analyses, this subsection gives some of the

steps in the calculation of the 1992 grade 4 reading composite mean and its estimation-error variance.

This illustration is an example of the calculation of NAEP means and variances and can be used to

understand their calculation for any NAEP assessment.

The weighted mean of the first plausible values of the reading composite for the grade 4 students

in the sample is 217.79, and the jackknife variance of these values is 0.833. Were these values true 

values, then 217.79 would be the estimate of the mean and 0.833 would be the estimation-error variance.

The weighted mean of the second plausible values of the same students, however, is 217.62; the third,

fourth, and fifth plausible values give weighted means of 217.74, 218.24, and 218.05. Since all of these

figures are based on precisely the same sample of students, the variation among them is due to

uncertainty about the students’ s, having observed their item responses and background variables.

Consequently, our best estimate of the mean for grade 4 students is the average of the five plausible

values: 217.89. Taking the jackknife variance estimate from the first plausible value, 0.833, as our
250

estimate U* of sampling variance, and the variance among the five weighted means, .063, as our estimate

B of uncertainty due to not observing , we obtain as the final estimate V of total error variance 0.833 +

(1+5-1) .063 = 0.908.

It is also possible to partition the estimation error variance of a statistic using these same

variance components. The proportion of error variance due to sampling students from the population is

U*

/V, and the proportion due to the latent nature of is (1+M-1)B/V. The results are shown in Table 11-1.

The value of U*

/V roughly corresponds to reliability in classical test theory and indicates the amount of

information about an average individual’s present in the observed responses of the individual. It should

be recalled again that the objective of NAEP is not to estimate and compare values of individual

examinees, the accuracy of which is gauged by reliability coefficients. The objective of NAEP, rather, is

to estimate population and subpopulation characteristics, and the marginal estimation methods described

above have been designed to do so consistently regardless of the values of reliability coefficients.

Table 11-1

Estimation Error Variance and Related Coefficients for the 1992 Grade 4 Reading Composite

(Based on Five Plausible Values)

Proportion of Variance Due to...

U* (1+5-1)B V

Student

Sampling:

U*/V

Latency of :

(1+5-1)B/V

0.833 0.076 0.908 0.92 0.08

Chapters 12 through 17 and Appendix E provide values of the proportion of variance due to

sampling and due to the latent nature of for all 1996 scales and composites for the populations as a

whole and, in the appendix, for selected subpopulations. It will be seen that the proportion of variance

due to the latency of varies somewhat among subject areas, tending to be largest for the long-term trend

writing assessment, where there is low correlation between tasks and each student responded to only one

or at most two tasks. The proportion of variance due to latency of is smallest for the composites of the

main assessment subjects, where the number of items per student is largest. Essentially, the variance due

to the latent nature of is largest when there is less information about a student’s proficiency. (Note the

distinction between estimation error variance of a parameter estimate and the estimate of the variance of

the distribution. The former depends on the accuracy of measurement; the large-sample model-based

expected value of the latter does not.) Given fixed assessment time, this decrease in information will

occur whenever the amount of information per unit time decreases as can happen when many short

constructed-response or multiple-choice items are replaced by a few extended constructed-response

items.

11.5 DESCRIBING STUDENT PERFORMANCE

Since its beginning, a goal of NAEP has been to inform the public about what students in

American schools know and can do. While the NAEP scales provide information about the distributions

of proficiency for the various subpopulations, they do not directly provide information about the meaning

of various points on the scale. Traditionally, meaning has been attached to educational scales by normreferencingthat is, by comparing students at a particular scale level to other students. In contrast,

NAEP achievement levels and scale anchors describe selected points on the scale in terms of the types of
251

skills that are likely to be exhibited by students scoring at that level. The achievement level process was

applied to the science composite. Scale anchoring of certain percentiles of the student proficiency

distribution was applied to the long-term trend assessment composite using the 1996 data. The

achievement level process for mathematics was completed for the 1990 assessment when the NAEP

mathematics framework was revised, so the results were directly applied to the 1996 results. In addition,

each item was mapped to a point on the scale in which it belonged, so that the content of each item

provides information about what students at each score level can do in a probabistic sense.

11.5.1 Achievement Levels

NAGB has determined that achievement levels shall be the first and primary way of reporting

NAEP results. Setting achievement levels is a method for setting standards on the NAEP assessment that

identify what students should know and be able to do at various points on the composite. For each grade

of each subject, three levels were definedbasic, proficient, and advanced. Based on initial policy

definitions of these levels, panelists were asked to determine operational descriptions of the levels

appropriate with the content and skills assessed in the assessment. With these descriptions in mind, the

panelists were then asked to rate the assessment items in terms of the expected performance of

marginally acceptable examinees at each of these three levels. These ratings were then mapped onto the

NAEP scale to obtain the achievement level cutpoints for reporting. Further details of the achievement

level-setting process for science appear in Appendix G.

11.5.2 Performance Descriptions Based on Composite Scales

A procedure known as scale anchoring was used to develop descriptions of student performance

at selected points on the composite scales. The scale points that were selected for anchoring reflect three

levels of knowledge and abilities corresponding to lower-, middle-, and higher-performing students for

each subject.

Around each percentile point, a band was built to define a range of scale scores. Students

described as being at a particular level were within a five percentile point range on either side of the

specified scale point. For example, the 50th percentile was defined as the region between the 45th and

55th percentile points on the scale. A question was identified as anchoring at a percentile point on the

scale if it was answered successfully by at least 65 percent of the students within that percentile band.

(The criterion was set at 74 percent for multiple-choice questions to correct for the possibility of

answering correctly by guessing.)

After defining the bands of the scale to be anchored, the next step in the process was to identify:

(1) questions answered correctly for dichotomously scored questions, or (2) questions answered at a

particular score level for partial credit constructed-response questions. Because the extended

constructed-response questions were scored according to four levels of performance, each extended

constructed-response question was treated as three distinct questions corresponding to scores of Partial or

better, Essential or better, and Extensive. These distinct score levels were then analyzed in the same

manner as questions scored dichotomously, as either correct or incorrect. Thus, for example, an extended

constructed-response question might anchor at the 50th percentile for Partial or better responses and at

the 90th percentile for Essential or better responses.

A committee of subject area experts, including teachers for the grades involved, college

professors, state curriculum supervisors, and researchers, was assembled to review the sets of questions
252

identified for each percentile band. The committee was divided into three groups, one for each grade.

Each group examined and analyzed questions that anchored at the 25th, 50th, and 90th percentiles to

determine the specific knowledge and abilities associated with each question.

Committee members were also provided with the sets of questions at each grade that “did not

anchor” to inform their decisions about what students could do by seeing examples of what they could

not do. Drawing on their knowledge of the subject area, committee members were asked to summarize

student performance by describing the knowledge, skills, and abilities demonstrated by students in each

of the score bands.

The performance descriptions are cumulative; that is, the abilities described for the lower

performing students are considered to be among the abilities of students performing at higher points on

the scale. Therefore, the full description of student’s knowledge and abilities in the middle scale band

would include those abilities described at the lower band. Similarly, the abilities of students performing

at the higher scale band include the abilities described for students at the middle and lower bands.

11.5.3 Item Mapping Procedures

In order to map items (questions) to particular points on each subject area scale, a response

probability convention had to be adopted that would divide those who had a higher probability of success

from those who had a lower probability. Establishing a response probability convention has an impact on

the mapping of assessment items onto the scales. A lower boundary convention maps the items at lower

points along the scales, and a higher boundary convention maps the same items at higher points along the

scales. The underlying distribution of skills in the population does not change, but the choice of a

response probability convention does have an impact on the proportion of the student population that is

reported as “able to do” the items on the scales.

There is no obvious choice of a point along the probability scale that is clearly superior to any

other point. If the convention were set with a boundary at 50 percent, those above the boundary would be

more likely to get an item right than get it wrong, while those below that boundary would be more likely

to get the item wrong than right. While this convention has some intuitive appeal, it was rejected on the

grounds that having a 50/50 chance of getting the item right shows an insufficient degree of mastery. If

the convention were set with a boundary at 80 percent, students above the criterion would have a high

probability of success with an item. However, many of the students below this criterion show some level

of achievement that would be ignored by such a stringent criterion. In particular, those in the range

between 50 and 80 percent correct would be more likely to get the item right than wrong, yet would not

be in the group described as “able to do” the item.

In a compromise between the 50 percent and the 80 percent conventions, NAEP has adopted two

related response probability conventions: 74 percent for multiple-choice items (to correct for the

possibility of answering correctly by guessing), and 65 percent for constructed-response items (where

guessing is not a factor). These probability conventions were established, in part, based on an intuitive

judgment that they would provide the best picture of students’ knowledge and skills.

Some additional support for the dual conventions adopted by NAEP was provided by Huynh

(1998, 1994). He examined the IRT information provided by items, according to the IRT model used in

scaling NAEP items. Following Bock (1972), Huynh decomposed the item information into that provided

by a correct response [Pji () *Ij ()] and that provided by an incorrect response [(1-P ()) *I ()]. Huynh

showed that the item information provided by a correct response to a constructed-response item is
253

maximized at the point along the scale at which two-thirds of the students get the item correct (for

multiple-choice items with four options, information is maximized at the point at which 75 percent get

the item correct). Maximizing the item information, I (), rather than the information provided by a

correct response [P () * I ()], would imply an item mapping criterion closer to 50 percent. Maximizing

the item information, I (), takes into account both responses that are correct and those that are incorrect,

however.

For dichotomously scored items the information function as defined by Birnbaum (1968, p. 463)

is defined for the jth item as

I

aP P c

P c j

j j k j k j

j k j



 

      

    



1 7

1

2

0 1

2

1

2

. ,

where the notation is the same as that used in Equations (11.1) and (11.2). The item information function

was defined by Samejima (1969) in general for polytomously scored items, and has been derived for

items scaled by the generalized partial credit model (Muraki, 1993; Donoghue, 1994) as (in a slightly

different, but equivalent form)

I a i P iP j j ji k

i

m

ji k

i

j j m

           
















 



 =

-

=

-

1 7   2 2

0

1

0

1 2

. .

11.6 OVERVIEW OF THE 1996 NAEP SCALES

The following IRT scale-score analyses were carried out for the 1996 NAEP assessment:

• Mathematics: Five IRT scales linked back to the 1990 and 1992 main assessment of

mathematics and one unidimensional IRT mathematics scale linking 1996 results to results

from mathematics assessments in 1973, 1976, 1982, 1986, 1990, 1992, and 1994. The first

five scales, along with a composite scale, are associated with the 1996 main assessment,

while the unidimensional scale is associated with the 1996 long-term trend assessment.

• Science: Three newly developed IRT scales for the main assessment of science and one

unidimensional scale linking 1996 results to results from science assessments in 1969, 1973,

1977, 1982, 1986, 1990, 1992, and 1994. The first three scales, along with a composite scale,

are associated with the 1996 main assessment, while the unidimensional scale is associated

with the 1996 long-term trend assessment.

• Long-Term Trend Reading: One IRT scale linking 1996 results to results from reading

assessments in 1971, 1975, 1979, 1984, 1988, 1990, 1992, and 1994. This scale is associated

with the 1996 long-term trend assessment.

• Long-Term Trend Writing: One polytomous item scale linking 1996 writing results to the

1984, 1988, 1990, 1992, and 1994 assessments. This scale is associated with the 1996 longterm trend assessment. [Please note that a review of the 1996 long-term trend writing

assessment data analyses has been undertaken by NCES. Additional analyses may be

required before revised results are released.]

Details follow in Chapters 12 through 17.

255

Chapter 12

DATA ANALYSIS FOR THE MATHEMATICS ASSESSMENT1

Frank Jenkins, Hua-Hua Chang, and Edward Kulick

Educational Testing Service

12.1 INTRODUCTION

This chapter describes the analyses performed on the responses to the cognitive and background

items in the 1996 mathematics assessment. This chapter focuses on the methods and procedures used to

estimate IRT-based scale score distributions for subgroups of students. This includes a wide array of

topics, such as the scoring of constructed-response items, classical item characteristics, item

responsetheory (IRT) analysis of mathematics scales, and estimation of subgroup means by the

imputation of plausible values. The statistical bases of the IRT and plausible values methodology

described in this chapter are given in Chapter 11. These analyses led to the results presented in NAEP

1996 Mathematics Report Card for the Nation and the States (Reese, Miller, Mazzeo, & Dossey, 1997).

For a description of the state analyses, see the Technical Report of the NAEP 1996 State Assessment

Program in Mathematics (Allen, Jenkins, Kulick, & Zelenak, 1997).

The student samples that were administered mathematics items in the 1996 national assessment

are shown in Table 12-1. (See Chapters 1 and 3 for descriptions of the target populations and the sample

design used for the assessment.). These samples were defined only by grade (4, 8, or 12) and not by age

of the student. Data from the Math Main samples comprised the spiraled balanced incomplete block

(BIB) design. The present chapter contains information about the scaling of data from these samples. The

long-term trend analyses are presented in a separate chapter (see Chapter 15). The other samples (MathEstimation, Math-Theme, and Math-Advanced) will be analyzed and presented in separate focus reports.

A brief description of the analyses of these samples is presented in Section 12.3. Technical

documentation detailing the analyses of the 1996 State Assessment of mathematics is provided in the

Technical Report of the NAEP 1996 State Assessment Program in Mathematics (Allen, Jenkins, Kulick,

& Zelenak, 1997).

 

1

 Frank Jenkins was the primary person responsible for the planning, specifications, and coordination of the national mathematics

analyses. He was assisted by Hua-Hua Chang. Data analysis and scaling were coordinated by Ed Kulick with help from Steve Wang

and Xiaohui Wang and additional assistance from David Freund.
256

Table 12-1

NAEP 1996 Mathematics Student Samples

Sample

Booklet

Number Mode

Cohort

Assessed Time of Testing1

Number

Assessed

 4 [Math Main]

 8 [Math Main]

12 [Math Main]

 1-26

 1-26

 1-26

Print

Print

Print

Grade 4

Grade 8

Grade 12

1/3/96 - 3/29/96 (Winter)

1/3/96 - 3/29/96 (Winter)

1/3/96 - 3/29/96 (Winter)

6,627

7,146

6,904

4 [Math-Estimation]

8 [Math-Estimation]

12 [Math-Estimation]

127

127

127

Tape

Tape

Tape

Grade 4

Grade 8

Grade 12

1/3/96 - 3/29/96 (Winter)

1/3/96 - 3/29/96 (Winter)

1/3/96 - 3/29/96 (Winter)

2,023

2,183

1,849

4 [Math-Theme]

8 [Math-Theme]

12 [Math-Theme]

128,129

128,129

128,129

Print

Print

Print

Grade 4

Grade 8

Grade 12

1/3/96 - 3/29/96 (Winter)

1/3/96 - 3/29/96 (Winter)

1/3/96 - 3/29/96 (Winter)

3,790

4,027

3,735

8 [Math-Advanced]

12 [Math-Advanced]

130

130

Print

Print

Grade 8

Grade 12

1/3/96 - 3/29/96 (Winter)

1/3/96 - 3/29/96 (Winter)

2,337

2,965

1 Final makeup sessions were held April 1-5, 1996.

LEGEND:

Math Mathematics Print Printed administration

Main Main spiral BIB assessment Tape Audiotape administration

Estimation Main estimation assessment Theme Assessment with theme booklets

Advanced Assessment with advanced booklets

12.2 DESCRIPTION OF STUDENT SAMPLE, ITEMS, AND ASSESSMENT

BOOKLETS

The data from the main BIB mathematics assessment (from samples 4[Math Main], 8 [Math

Main], and 12[Math Main]) were used for main analyses comparing the levels of mathematics

achievement for various subgroups of the 1996 target populations. In previous assessments the

mathematics samples were defined as age/grade cohorts (e.g., students who were either in the fourth

grade or 9 years old). Starting with the 1996 assessment, cohorts were defined solely by grade. The

sampled students in each of these three grade cohorts were assessed in the winter. The samples in the

main assessment are presented in Table 12-1.

The pool of items used in the 1996 mathematics assessment contained a range of constructedresponse and multiple-choice questions measuring performance on sets of objectives (National

Assessment Governing Board, 1994). The framework for the objectives is described in Chapter 2. A total

of 360 distinct mathematics items addressing these objectives were scaled (see Table 12-2). The number

of items per grade was 144, 164, and 165 respectively for grades 4, 8, and 12 (before scaling). Tables 12-

3, 12-5, and 12-7 give, for each grade, the numbers of items by item type and block (before scaling). For

item counts after scaling, taking into account items that were dropped or collapsed, see Tables 12-4, 12-

6, and 12-8. These tables indicate that there was a nearly fourfold increase in the number of

polytomously-scored constructed-response items over the numbers presented in the 1992 assessment. In

1996, the number of items was 30 in grade 4, 30 in grade 8, and 33 in grade 12. The items were classified
257

into five content strands: numbers and operations; measurement; geometry; data analysis, statistics, and

probability; and algebra and functions. These five content strands (renamed number sense, properties,

and operations; measurement; geometry and spatial sense; data analysis, statistics, and probability; and

algebra and functions in the 1996 analysis) constituted the scales used in 1996 reporting.2

 The items

were partitioned into 13 blocks and these blocks were then assigned to 26 booklets according to a BIB

design. Each booklet contained relatively few items from each of the five categories.

Table 12-2

National Main BIB: Numbers of Scaled Mathematics Items

Common Across Grade Levels, by Content Strand Scale

Grade(s)

Number Sense,

Properties,

and

Operations Measurement

Geometry

and

Spatial

Sense

Data Analysis,

Statistics,

and

Probability

Algebra

and

Functions Total

4 only 40 20 12 11 8 91

8 only 20 13 12 10 18 73

12 only 23 11 17 22 32 105

4 and 8 10 2 9 3 7 31

8 and 12 8 9 6 9 6 38

4 and 12 0 0 0 0 0 0

4, 8, and 12 9 3 4 3 3 22

Total Grade 4 59 25 25 17 18 144

Total Grade 8 47 27 31 25 34 164

Total Grade 12 40 23 27 34 41 165

Total 110 58 60 58 74 360

In the main samples, each student was administered a booklet containing three blocks of

mathematics cognitive items, a block of background questions common to all booklets for a particular

grade level, and a block of mathematics-related background questions common to all mathematics

booklets for a particular grade level. At the end of each booklet there was a short block consisting of five

questions concerning the student’s motivation and his or her perception of the difficulty of the cognitive

items. The BIB design by which the 13 blocks of mathematics cognitive items were assigned to the 26

booklets for each grade level is discussed in Chapter 4. The 13 blocks were not intended to be parallel

measuring instruments. For example, several blocks contained only the items designed for calculator

usage, and some blocks contained items for ruler and protractor usage. In addition, the proportion of

items sampled from the five categories were not exactly the same among the 13 blocks.

The 360 unique items were constructed according to several formats, some traditional and some

more innovative: multiple-choice items, constructed-response items scored dichotomously, constructedresponse items scored polytomously, and cluster items.3

 The multiple-choice items conformed to the

familiar format of a stem followed by several possible answers, with only one answer being correct.

 

2

 The content strand number sense, properties, and operations was called numbers and operations in the 1990 and 1992

assessments. The content strand geometry and spatial sense was called geometry in the 1990 and 1992 assessments. 3

 A cluster item is an aggregation of a group of items (in the case of NAEP mathematics, typically three to five items) that are

related to a single content strand, topic, or stimulus, and are developed and scored as a single unit (see Wainer & Kiely, 1987, for

further details and examples of different types of cluster items).
258

Constructed-response items that were scored dichotomously were questions that required an open-ended

response (e.g., explaining why the previous question was answered as it was). These questions were read

by raters who determined whether or not the response was correct. Constructed-response items that were

scored polytomously were open-ended questions that required several stages of reasoning or problem

solving. They were also read by raters but were given a score reflecting degree of correctness rather than

simply judged right or wrong. Cluster items were derived from a set of three to five multiple-choice items

that related to the same basic stem. The cluster score was the number of constituent cluster items that the

respondent answered correctly.

As Table 12-3 indicates, of the 144 items at grade 4, 79 were multiple-choice items, 16 were

constructed response items that were scored dichotomously, 48 were constructed-response items that

were scored polytomously, and 1 was a cluster item. As shown in Table 12-5, of 163 items at grade 8, 91

were multiple-choice items, 21 were constructed-response items that were scored dichotomously, 48

were constructed-response items that were scored polytomously, and 3 were cluster items. For grade 12,

Table 12-7 indicates that of the 166 items administered, 92 were multiple-choice items, 21 were

constructed-response items that were scored dichotomously, 50 were constructed-response items that

were scored polytomously, and 3 were cluster items.

Tables 12-4, 12-6, and 12-8 show comparable information for each grade after the scaling was

carried out.

Table 12-3

1996 NAEP Mathematics Block Composition by Content Strand

and Item Type, Grade 4, As Defined Before Scaling

MultipleChoiceConstructed-Response Items Scored

Polytomously Cluster Total

Block Items 2-category 3-category 4-category 5-category Items Items

M3 9 4 0 0 0 0 13

M4 14 0 0 0 0 0 14

M5 4 0 5 0 1 0 10

M6 0 7 2 2 0 0 11

M7 3 0 4 0 1 0 8

M8 14 0 1 0 0 0 15

M9 6 2 0 0 1 1 10

M10 0 1 4 1 0 0 6

M11 11 0 0 2 3 0 16

M12 5 0 3 0 1 0 9

M13 6 1 2 2 1 0 12

M14 4 0 5 0 1 0 10

M15 3 1 5 0 1 0 10

Total 79 16 31 7 10 1 144
259

Table 12-4

1996 NAEP Mathematics Block Composition by Content Strand

and Item Type, Grade 4, As Defined After Scaling*

MultipleChoiceConstructed-Response Items Scored

Polytomously Cluster Total

Block Items 2-category 3-category 4-category 5-category Items Items

M3 9 4 0 0 0 0 13

M4 14 0 0 0 0 0 14

M5 4 0 5 0 1 0 10

M6 0 7 2 2 0 0 11

M7 3 2 2 0 1 0 8

M8 14 0 1 0 0 0 15

M9 6 2 0 1 0 1 10

M10 0 1 4 1 0 0 6

M11 11 0 0 2 3 0 16

M12 5 0 3 0 1 0 9

M13 6 1 2 2 1 0 12

M14 4 0 5 1 0 0 10

M15 3 2 4 0 1 0 10

Total 79 19 28 9 8 1 144

* Counts reflect items that were dropped and collapsed.

Table 12-5

1996 NAEP Mathematics Block Composition by Content Strand

and Item Type, Grade, As Defined Before Scaling

MultipleChoiceConstructed-Response Items Scored

Polytomously Cluster Total

Block Items 2-category 3-category 4-category 5-category Items Items

M3 8 2 0 1 1 1 13

M4 21 0 0 0 0 0 21

M5 6 0 4 0 1 0 11

M6 0 11 4 1 0 0 16

M7 4 0 5 0 1 0 10

M8 16 1 0 1 0 0 18

M9 5 3 0 0 1 0 9

M10 0 1 5 1 0 0 7

M11 13 0 1 2 3 0 19

M12 4 0 4 1 0 0 9

M13 6 3 1 0 1 0 11

M14 5 0 3 0 1 1 10

M15 3 0 4 0 1 1 9

Total 91 21 31 7 10 3 163
260

Table 12-6

1996 NAEP Mathematics Block Composition by Content Strand

and Item Type, Grade 8, As Defined After Scaling*

MultipleChoiceConstructed-Response Items Scored

Polytomously Cluster Total

Block Items 2-category 3-category 4-category 5-category Items Items

M3 8 2 0 2 0 1 13

M4 21 0 0 0 0 0 21

M5 6 1 4 0 0 0 11

M6 0 11 4 1 0 0 16

M7 4 1 5 0 0 0 10

M8 16 1 0 1 0 0 18

M9 5 3 0 1 0 0 9

M10 0 1 5 1 0 0 7

M11 13 0 1 2 3 0 19

M12 4 2 3 0 0 0 9

M13 6 3 1 1 0 0 11

M14 5 1 1 1 0 1 9

M15 3 0 4 1 0 1 9

Total 91 26 28 11 3 3 162

* Counts reflect items that were dropped and collapsed.

Table 12-7

1996 NAEP Mathematics Block Composition by Content Strand

and Item Type, Grade 12, As Defined Before Scaling

MultipleChoiceConstructed-Response Items Scored

Polytomously Cluster Total

Block Items 2-category 3-category 4-category 5-category Items Items

M3 10 3 1 0 0 0 14

M4 22 0 0 0 0 0 22

M5 4 0 5 0 1 0 10

M6 0 13 2 2 0 0 17

M7 3 0 4 0 1 1 9

M8 17 2 1 1 0 0 21

M9 6 2 0 1 0 0 9

M10 3 0 5 1 1 0 10

M11 11 1 2 0 0 0 14

M12 4 0 4 0 1 1 10

M13 3 0 0 3 3 0 9

M14 5 0 4 0 1 1 11

M15 4 0 5 0 1 0 10

Total 92 21 33 8 9 3 166
261

Table 12-8

1996 NAEP Mathematics Block Composition by Content Strand

and Item Type, Grade 12, As Defined After Scaling*

MultipleChoiceConstructed-Response Items Scored

Polytomously Cluster Total

Block Items 2-category 3-category 4-category 5-category Items Items

M3 10 3 1 0 0 0 14

M4 22 0 0 0 0 0 22

M5 4 0 6 0 0 0 10

M6 0 13 2 2 0 0 17

M7 3 0 4 1 0 1 9

M8 17 2 1 1 0 0 21

M9 6 2 0 1 0 0 9

M10 3 0 5 2 0 0 10

M11 11 1 2 0 0 0 14

M12 4 0 4 1 0 1 10

M13 3 0 0 4 2 0 9

M14 4 1 3 1 0 1 10

M15 4 1 5 0 0 0 10

Total 91 23 33 13 2 3 165

* Counts reflect items that were dropped and collapsed.

12.3 SPECIAL MATHEMATICS ASSESSMENTS

There were three special studies in the 1996 NAEP assessment—estimation, theme, and advanced

studies. The block structure of the special study booklets is provided only in Table 12-9. The estimation and

the two theme blocks have a linking BIB block, M4, in the first position. This structure held for all three

grades, but the actual items in the blocks varied over grades. In addition to the cognitive blocks, each of the

special study booklets included three blocks in common with the main assessment:

a general student background block (BM1),

a mathematics background block (MB1), and

a motivation block (M2).

Table 12-9

Block Structure of the Special Study Booklets

Study Booklet Cognitive Blocks

Estimation 127 M4, M16, M17

Theme 128 M4, M21

Theme 129 M4, M22

Advanced Mathematics1 130 M20, M18, M19

1 Grades 8 and 12 only
262

The cognitive portion of the estimation booklet was administered in two sections: first there was a regular

mathematics block in common with the main assessment that was self-paced; and second, there were two

estimation blocks (M16 and M17) in which items were administered by a paced audio tape. The theme and

advanced booklets were self-paced. Note that the theme booklets contained a single non-theme block (M4)

in common with the main assessment.

The special studies were not part of the main assessment; analysis for these booklets will appear

in separate focus reports. The major findings for these studies will be derived from an analysis of the

‘reporting’ samples, which are subsamples of the assessment’s total sample. The nonreporting segments

of the total samples were added to the assessments in order to study the effects of changing inclusion

rules and accommodations for students of limited English proficiency (LEP) and students with

disabilities (SD) (See Section 12.4).

The estimation and theme samples were drawn from the same population as the main assessment.

The grades 8 and 12 advanced booklets were drawn from a population of students considered to be high

mathematics achievers. Students for the grade 8 study of advanced mathematics were sampled from a

population of students who were enrolled in an advanced class (defined as algebra 1 or beyond) during

the 1995-96 school year. For grade 12, the advanced study students were also sampled from a population

of students who took an advanced course during the school year. Grade 12 advanced courses were;

Algebra 3, Pre-calculus, Calculus and Analytical Geometry, Calculus, and AP Calculus. Table 12-10 lists

the number of items in the special assessment blocks in the three grades. More detail is available in the

procedural appendices of reports on the estimation, theme and advanced analyses.

Table 12-10

Number of Cognitive Items in the

1996 Special Mathematics Assessment Blocks

Estimation

Blocks

Theme

Blocks

Advanced

Blocks

M16 M17 M21 M22 M20 M18 M19

 Grade 4 20 14 8 6 —1 —1 —1

 Grade 8 22 16 12 15 13 10 12

 Grade 12 22 16 11 7 15 11 11

1

 There were no advanced blocks administered to grade 4 students.

12.4 ASSESSING THE EFFECT OF CHANGING INCLUSION CRITERION AND

ACCOMMODATIONS FOR SD/LEP STUDENTS

NAEP samples include SD/LEP students in at least the same percentages in which they are found

in the general school population. Although a substantial percentage of these students are included in the

assessment, schools are allowed to exclude some of these students from NAEP when the students are

judged to be incapable of meaningfully participating in a large scale assessment. To facilitate the

consistent implementation of NAEP’s intention to include as many students as possible, NAEP provides

specific criteria that staff in sampled schools can use to determine who should be included in the

assessment.
263

In the 1996 assessment, procedures for dealing with SD and LEP students were modified, based

on recommendations from the U.S. Department of Education. First, inclusion criteria were revised with

the intention of making them clearer, more inclusive, and more likely to be applied consistently across

schools. Second, a variety of assessment accommodations and adaptations were offered to

students with disabilities whose individualized education programs (IEPs)

specified such accommodations, and

LEP students who were unable to take the assessment in English.

In order to assess the impact of new inclusion criteria and accommodations, a three-sample

design was instituted at all three grades. The first sample comprised students who were subject to the

1994 inclusion rules and, as was the case in past NAEP assessments, were offered no accommodations.

Students in the second sample were subject to the 1996 inclusion rules but the SD/LEP students were

offered no accommodations. The third sample had students who were comprised students who were

assessed under conditions that will probably be used for future NAEP assessments—new inclusion rules

and some accommodations being offered to the included SD/LEP students. See Chapter 3, Sections 3.4 to

3.8, for more details.

Results of this study can be found in the focus report dealing with the comparison of the three

accommodation samples.

12.5 ITEM ANALYSIS

This section contains a detailed description of the item analysis performed using the national

main BIB sample data. The analysis examines items within blocks. In preparation for this analysis,

constructed-response items with more than two categories and cluster items were polytomously scored,

two-category constructed-response items were dichotomously scored, and derived background variables

were calculated. Item statistics such as mean percent correct, average score, item to total score

correlations and percent responding in each item category were calculated.

Tables 12-11, 12-12, and 12-13 show the number of items, mean proportion correct, mean item to

total score correlation, and alpha reliability for each block administered at each grade level for the main

assessment. These values were calculated within block only for those items used in the scaling process.

The table also gives the number of students who were administered the block and the percent not

reaching the last item in the block. Student weights were used, except for reporting the sample sizes. The

results for the blocks administered at each grade level indicated that the blocks differ in number of items,

average difficulty, reliability, and percent not reaching the last item.

Tables 12-11 through 12-13 show the item analyses by block position within a booklet. Each

booklet had three cognitive blocks, and each block appeared in three different booklets—once in the first,

once in the second, and once in the third position. In some past assessments (e.g., 1992 science), students

responded differently to the items depending on the block position. The IA tables evidence few

systematic differences in item summary statistics as a function of block position. For grade 4 (Table 12-

11) the weighted average item score had a slight tendency to be higher when a block was in the first

rather than the third position. For grades 8 and 12 no systematic difference can be seen.
264

Table 12-11

Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the Mathematics Main Sample, Grade 4, As Defined After Scaling

Block Block

Statistic Position M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 

Number of scale items

Total 13 14 10 11 8 15 13 6 16 9 Multiple choice 9 14 4 0 3 14 9 0 11 5 Constructed response (dichotomous) 4 0 0 11 2 1 2 6 5 0 Constructed response (polytomous) 0 0 6 0 3 0 2 0 0 4 

Unweighted sample size 1 502 498 516 511 509 501 525 497 519 506 

2 515 515 501 494 519 509 510 516 518 503 3 522 497 516 509 493 499 511 497 495 517 

ALL 1539 1510 1533 1514 1521 1509 1546 1510 1532 152

Weighted average item score 1 .51 .46 .45 .45 .38 .58 .48 .43 .50 .47 

2 .49 .45 .47 .43 .38 .59 .46 .41 .51 .49 3 .49 .44 .44 .42 .38 .61 .46 .40 .49 .46 

ALL .50 .45 .46 .44 .38 .59 .47 .41 .50 .47 

Weighted alpha reliability 1 .67 .63 .75 .69 .70 .71 .67 .61 .75 .7

2 .70 .66 .74 .71 .72 .74 .68 .63 .74 .73 .72 .61 .75 .72 .63 .75 .64 .66 .77 .72 

ALL .69 .63 .75 .71 .69 .74 .66 .63 .75 .7

Weighted average r-polyserial 1 .59 .52 .68 .65 .70 .55 .64 .81 .59 .67 

2 .61 .55 .68 .67 .73 .58 .65 .81 .59 .66 3 .63 .53 .67 .68 .64 .58 .61 .83 .60 .6

ALL .61 .53 .68 .66 .69 .57 .63 .82 .59 .67 

Weighted proportion of students 1 .81 .91 .88 .81 .81 .79 .98 .85 .84 .8

Reaching the last item 2 .79 .91 .89 .87 .87 .80 .96 .91 .90 .85 

3 .81 .91 .89 .79 .82 .82 .99 .90 .90 .87 

ALL .81 .91 .89 .82 .83 .81 .98 .89 .88 .85 
Table 12-12

Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the Mathematics Main Sample, Grade 8, As Defined After Scaling

Block Block

Statistic Position M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 

Number of scale items

Total 17 21 12 16 10 18 9 7 19 9 Multiple choice 12 21 7 0 4 16 5 0 13 4 Constructed response (dichotomous) 3 0 1 16 1 2 3 7 6 2 Constructed response (polytomous) 2 0 4 0 5 0 1 0 0 3 

Unweighted sample size 1 558 556 543 571 555 541 546 543 538 547 

2 535 558 548 556 561 566 542 534 551 530 3 544 538 554 542 554 540 548 568 569 542 

ALL 1637 1652 1645 1669 1670 1647 1636 1645 1658 1619 

Weighted average item score 1 .51 .58 .55 .59 .56 .50 .36 .56 .63 .34 

2 .50 .60 .54 .61 .56 .53 .35 .55 .65 .33 3 .51 .58 .53 .62 .54 .50 .37 .55 .64 .34 

ALL .51 .59 .54 .61 .55 .51 .36 .55 .64 .34 

Weighted alpha reliability 1 .75 .76 .72 .82 .68 .79 .56 .65 .76 .60 

2 .75 .74 .70 .85 .66 .80 .63 .54 .79 .66 3 .74 .78 .69 .82 .71 .76 .64 .58 .80 .68 

ALL .75 .76 .70 .83 .68 .79 .61 .59 .79 .64 

Weighted average r-polyserial 1 .66 .55 .63 .67 .66 .63 .58 .82 .62 .65 

2 .66 .53 .61 .73 .63 .63 .64 .76 .64 .68 3 .66 .56 .61 .67 .70 .60 .64 .78 .65 .68 

ALL .66 .55 .62 .69 .66 .62 .62 .79 .64 .67 

Weighted proportion of students 1 .98 .87 .97 .90 .78 .72 .96 .90 .90 .81 

Reaching the last item 2 .93 .86 .96 .93 .76 .71 .94 .89 .87 .82 

3 .95 .88 .94 .92 .84 .75 .96 .92 .93 .85 

ALL .95 .87 .96 .91 .79 .72 .95 .90 .90 .83 
266

Table 12-13

Descriptive Statistics for Item Blocks by Position Within Test Booklet and Overall

Occurrences for the Mathematics Main Sample, Grade 12, As Defined After Scaling

Block Block

Statistics Position M3 M4 M5 M6 M7 M8 M9 M10 M11 M12 

Number of scaled items

Total 14 22 10 17 9 21 9 10 14 10 Multiple choice 10 22 4 0 3 17 6 3 11 4 Constructed response (dichotomous) 4 0 0 17 0 4 2 6 3 0 Constructed response (polytomous) 0 0 6 0 6 0 1 1 0 6 

Unweighted sample size 1 525 530 519 536 539 548 530 526 523 540 

2 515 519 539 523 525 529 545 547 524 516 3 530 534 522 520 511 535 527 540 523 525 

ALL 1570 1583 1580 1579 1576 1612 1602 1613 1570 1590 

Weighted average item score 1 .43 .70 .36 .55 .45 .55 .52 .47 .48 .41 

2 .43 .70 .37 .56 .43 .55 .49 .45 .51 .42 3 .45 .70 .39 .56 .44 .56 .48 .43 .49 .41 

ALL .44 .70 .37 .56 .41 .55 .49 .45 .49 .40 

Weighted alpha reliability 1 .74 .76 .73 .83 .75 .80 .63 .52 .72 .70 

2 .69 .77 .71 .81 .74 .82 .59 .58 .69 .70 3 .76 .75 .72 .81 .77 .80 .58 .62 .72 .68 

ALL .73 .76 .72 .82 .77 .81 .60 .58 .71 .76 

Weighted average r-polyserial 1 .69 .61 .72 .71 .70 .63 .65 .57 .63 .63 

2 .64 .61 .70 .68 .70 .63 .63 .61 .61 .60 3 .70 .59 .70 .66 .73 .63 .60 .63 .63 .60 

ALL .68 .60 .71 .69 .74 .63 .63 .61 .63 .63 

Weighted proportion of students 1 .65 .84 .70 .78 .86 .48 .96 .88 .68 .78 

Reaching the last item 2 .67 .81 .70 .74 .86 .45 .94 .88 .70 .77 

3 .69 .78 .76 .65 .84 .50 .92 .87 .67 .83 

ALL .67 .81 .72 .72 .85 .48 .94 .88 .68 .79 
267

As described in Chapter 9, in NAEP analyses (both conventional and IRT-based), a distinction is

made between missing responses at the end of each block (not-reached) and missing responses prior to

the last completed response (omitted). Not-reached items are those occurring after the last item the

student completed in a block. Items that were not reached are treated as if they had not been presented to

the examinee, while omitted items are regarded as incorrect. The proportion of students attempting the

last item of a block (or, equivalently, 1 minus the proportion not reaching the last item) is often used as

an index of the degree of speededness of the block of items.

Standard practice at ETS is to treat all students who did not respond to the last item as if they had

not reached that item. For multiple-choice and short constructed-response items, this convention

produced a reasonable pattern of results, in that the proportion of students reaching the last item does not

differ markedly from the proportion attempting the next-to-last item. However, for the blocks that ended

with extended constructed-response items, this convention resulted in an implausibly large drop in the

number of students attempting the final item (see Koretz et al., 1993). Therefore, for blocks that ended

with an extended constructed-response item, students who attempted the next-to-last item but did not

respond to the last item were classified as having intentionally omitted that item.

Tables 12-11 to 12-13 also contain information about the effect of the position of blocks within

booklets on the average percent correct for items within each block presented to the BIB samples for

each grade. The averages for the grade-only portion of the focused-BIB samples show that the order of

blocks within booklets did not have a large or consistent effect on scale scores in the mathematics

focused-BIB assessment.

12.5.1 Constructed-Response Items

As indicated in Tables 12-3 to 12-8, over 40 percent of the mathematics items were constructedresponse. Constructed-response items that were scored dichotomously were given a right/wrong scoring.

The categories of responses for the items and the number of responses that were rescored for each item

are indicated in Appendix I. The percent agreement for the raters and Cohen’s Kappa, a reliability

estimate appropriate for items that are dichotomized, are also given in the tables. The sample sizes listed

in the tables correspond to the samples used in calculating the rater reliability.

In general, the rater reliability of the scoring for dichotomized responses was quite high. Cohen’s

Kappa reliabilities ranged over items from 0.76 to 1.00 for grade 4, from 0.90 to 1.00 for grade 8, and

from 0.67 to 1.00 for grade 12.

Chapter 7 discusses the definition of the item ratings and describes the process by which teams

of raters scored the constructed-response items. This discussion includes the rating definitions for short

and extended constructed-response items as well as the range of interrater reliabilities that occurred.

Extended constructed-response items were scored on a scale from 1 to 5 to reflect degrees of knowledge.

In scaling, this scale is shifted to 0 to 4. Rating information on extended constructed-response items can

be found in Appendix I, which lists the sample sizes, percent agreement, and Cohen’s Kappa reliability

index.
268

12.5.2 Differential Item Function Analysis

A differential item functioning (DIF) analysis of the main-assessment mathematics items was

done in order to guide committees in identifying biased items that should be examined more closely for

possible bias. Sample sizes were large enough to compare male and female students, White and Black

students, and White and Hispanic students.

The DIF analyses of the dichotomous items were based on the Mantel-Haenszel chi-square

procedure, as adapted by Holland and Thayer (1988), which is described in Chapter 9. The procedure

tests the statistical hypothesis that the odds of correctly answering an item are the same for two groups of

examinees that have been matched on some measure of mathematics ability (usually referred to as the

matching criterion). The DIF analyses of the polytomous items were based on the Mantel procedure

(1963) and the Somes (1986) chi-square test (see also Chapter 9). These procedures compare proportions

of matched examinees from each group in each polytomous item response category. The groups being

compared are often referred to as the focal group (usually a minority or other group of interest, such as

Black examinees or female examinees) and the reference group (usually White examinees or male

examinees).

For each dichotomous item in the assessment, an estimate was produced of the Mantel-Haenszel

common-odds ratio, expressed on the ETS delta scale for item difficulty. The estimates indicate the

difference between reference group and focal group item difficulties (measured in ETS delta scale units),

and typically run between about +3 and -3. Positive values indicate items that are differentially easier for

the focal group than the reference group after making an adjustment for the overall level of mathematics

ability in the two groups. Similarly, negative values indicate items that are differentially harder for the

focal group than the reference group. It is common practice at ETS to categorize each item into one of

three categories (Peterson, 1988): “A” (items exhibiting no DIF), “B” (items exhibiting a weak indication

of DIF), or “C” (items exhibiting a strong indication of DIF). Items in category “A” have MantelHaenszel common-odds ratios on the delta scale that do not differ significantly from 0 at the alpha = 0.05

level or are less than 1.0 in absolute value. Category “C” items are those with Mantel-Haenszel values

that are significantly greater than 1 and larger than 1.5 in absolute magnitude. Other items are categorized

as B items. A plus sign (+) indicates that items are differentially easier for the focal group; a minus sign

(- ) indicates that items are differentially more difficult for the focal group.

The ETS/NAEP DIF procedure for polytomous items incorporates both the MH ordinal

procedure and the generalized MH statistic. The summary tables of identified polytomous items contain

generalizations of the dichotomous “A,” “B,” and “C” categories namely “AA,” “BB,” and “CC”

respectively. Analogous to the dichotomous case, only the “CC” items are considered to have a strong

indication of DIF and are flagged for scrutiny by the subject matter committee

For each block of items at each grade four DIF comparisons were made: male/female,

White/Asian American, White/Black, and White/Hispanic. The first subgroup in each comparison is the

reference group; the second subgroup is the focal group.

Following standard practice at ETS for DIF analyses conducted on final test forms, all

“C” and “CC” items were reviewed by a committee of trained test developers and subject-matter

specialists. As indicated by Tables 12-14 and 12-15, two dichotomous items and five polytomous

items met the criteria to be considered by the DIF committee. Such committees are charged with

making judgments about whether or not the differential difficulty of an item is unfairly related to

group membership. See Appendix J for a list of the “C” and “CC” items.
269

The committee assembled to review NAEP items included both ETS staff and outside members

with expertise in the field. It was the committee’s judgment that none of the “C” or “CC” items for the

national data were functioning differentially due to factors irrelevant to test objectives. In other words, all

of the items that were classified as “C” or “CC” items measured concepts in the assessment framework

and specifications that could not be measured in another way. Hence, none of the items were removed

from scaling due to differential item functioning.

Table 12-14

DIF Category by Grade for Dichotomous Items

DIF Analysis

Grade Category1 Male/Female White/Black White/Hispanic

 4 CBAA+

B+

C+

0

1

11

7

1

0

0

2

9

5

4

0

 0

0

10

9

1

0

 8 CBAA+

B+

C+

 0

2

13

9

0

0

0

0

7

15

2

0

 0

0

13

8

3

0

12 CBAA+

B+

C+

0

3

14

5

0

0

0

1

8

12

0

1

 0

0

11

10

0

1 1 Positive values of the index indicate items that are differentially easier for the focal group (female,

Black, or Hispanic students) than for the reference groups (male or White students). “A+” or “A-”

means no indication of DIF, “B+” means a weak indication of DIF in favor of the focal group, “B-“

means a weak indication of DIF in favor of the reference group and “C+” or “C-” means a strong

indication of DIF.
270

Table 12-15

DIF Category by Grade for Polytomous Items

DIF Analysis

Grade Category1 Male/Female White/Black White/Hispanic

 4 CCBBAAAA+

BB+

CC+

0

1

13

10

3

0

0

1

14

11

1

0

0

0

12

15

0

0

 8 CCBBAAAA+

BB+

CC+

0

0

10

16

1

0

3

2

14

7

1

0

1

2

13

10

1

0

12 CCBBAAAA+

BB+

CC+

0

2

9

18

1

0

0

3

14

10

2

1

0

1

15

12

2

0 1 Positive values of the index indicate items that are differentially easier for the focal group (female,

Black, or Hispanic students) than for the reference groups (male or White students). “A+” or “A-”

means no indication of DIF, “B+” means a weak indication of DIF in favor of the focal group, “B-“

means a weak indication of DIF in favor of the reference group and “C+” or “C-” means a strong

indication of DIF.

12.5.3 Estimation of Item Parameters

The BILOG/PARSCALE computer program was used to estimate the item parameters for the main

assessment and for the special estimation study. For dichotomous multiple-choice and dichotomized

constructed-response items, a three-parameter IRT model was used. Extended constructed-response and

cluster items were polytomously scored and were analyzed with a generalized partial credit model (Muraki,

1992).

Recall from section 12.5 that for calibration, items that were missing prior to the last completed

item in a block were considered omitted and scored as wrong. Also, items that were not reached were

treated as if they were not presented to the examinees (and not counted as wrong.) However, there is an

exception for blocks that end with an extended constructed-response item. In these blocks, students who

respond to the next-to-last item but do not respond to the last item are classified as having omitted the last

item (i.e., the last item is counted as wrong). Responses to extended constructed-response items that were

off-task were also treated as omitted. The multi-category constructed-response items had two, three, four or

five categories of partial credit. Scoring levels were labeled as listed in Table 12-16.
271

Table 12-16

Labels for Score Levels of Polytomous Items

Score 3-Category Item 4-Category Item 5-Category Item

4 Correct

3 Correct 4th Category

2 Correct 3rd Category 3rd Category

1 2nd Category 2nd Category 2nd Category

0 Incorrect, off-task, or

omitted

Incorrect, off-task, or

omitted

Incorrect, off-task, or

omitted

Note that the categories falling between “incorrect” and “correct” represent increasing levels of a partially

correct response.

The item parameter estimation was done separately within grade, but the final mathematics scale

estimates were transformed to conform with the cross-grade scales of the 1990 and 1992 assessments.

Within each grade, items were scaled using the grade-only sample of students available from the 1992 and

the 1996 assessments. The 1992 data were included at the scaling step of the analysis to assure that

parameter estimates for items administered at both time points would not drift drastically between

assessments. Item parameters were estimated separately for each of the five content strands and the

mathematics scale means and variances for samples from the two assessment years were allowed to be

different. In the final BILOG/PARSCALE run, the prior distributions of the population abilities were free to

be estimated and the overall distribution was centered at zero. In general, if an item was common to both

assessment years, identical item parameters were used for both assessments. The appropriateness of the use

of the identical parameters across assessments was examined by comparing the fit of the empirical item

response functions against the estimated IRT item response functions. If IRT parameters did not fit the data,

parameters specific to the assessment year were used. (See Chapter 11 for further descriptions of the scaling

process.) The calibration was based on student weights that were rescaled so that the their sum equaled the

unweighted sample size of the 1996 sample. Also, weights for the 1992 data were restandardized to give

equal weight to the two assessment years included in the scaling (see Appendix K).

Items that received special treatment in the scaling procedure are listed in Table 12-17, along with

the reason for special treatment. Items were dropped, combined into clusters, split between assessment years

and collapsed. If items had empirical item response functions that were nonmonotonic, they were dropped.

If several items had highly correlated responses (conditional on theta) they were combined into a single

polytomous item called a cluster item. If items were administered in both 1992 and 1996 but showed

evidence of having a distinct item response functions for each assessment year, the item is treated as two

separate items and parameters estimated separately for each assessment year. If polytomous items had

sparse or nonmonotonic responses in one or more categories, the items were collapsed so that some

response categories were combined into a single category. Only about ten percent of the total scaled items

were given special treatment.
272

Table 12-17

Items from the 1996 Assessment in Mathematics Receiving Special Treatment

Grade

NAEP

ID Block

Content

Strand Treatment

4 M010531 M8 1 1992 and 1996 responses split into items

M010531Y and M010531Z

4 M040801 M9 2 1992 and 1996 responses split into items

M040801Y and M040801Z

4 M041001 M9 2 1992 and 1996 responses split into items

M041001Y and M041001Z

4 M041201 M9 3 1992 and 1996 responses split into items

M041201Y and M041201Z

Categories in both items collapsed: 0,1,2,3,4 becomes

0,1,2,3,3,

4 M068001 M7 1 Collapsed categories: 0,1,2 becomes 0,0,1

4 M068003 M7 3 Collapsed categories: 0,1,2 becomes 0,0,1

4 M072701 M14 5 Collapsed categories: 0,1,2,3,4 becomes 0,1,2,3,3

4 M074701 M15 1 Collapsed categories: 0,1,2 becomes 0,0,1

8 M013531 M8 1 1992 and 1996 responses split into items

M013531Y and M013531Z

8 M018201 M4 1 1992 and 1996 responses split into items

M018201Y and M018201Z

8 M018901 M4 4 1992 and 1996 responses split into items

M018901Y and M018901Z

8 M045901 1992 3 Collapsed categories: 0,1,2,3,4 becomes 0,1,2,3,3

1992 only

8 M050261 M3 4 Collapsed categories: 0,1,2,3,4 becomes 0,0,0,1,2

8 M051001 M3 3 1992 and 1996 responses split into items

M051001Y and M051001Z

8 M051101 M3 1 1992 and 1996 responses split into items

M051101Y and M051101Z

Categories in both items collapsed: 0,1,2,3,4 becomes

0,1,2,3,3

8 M051201 M13 1 1992 and 1996 responses split into items

M051201Y and M051201Z

8 M052201 M13 2 Collapsed categories: 0,1,2,3,4 becomes 0,1,2,3,3

8 M053101 M9 4 Collapsed categories: 0,1,2,3,4 becomes 0,1,2,2,3

8 M054301 1992 5 Collapsed categories: 0,1,2,3,4 becomes 0,1,2,3,3

1992 only
273

Table 12-17 (continued)

Items from the 1996 Assessment in Mathematics Receiving Special Treatment

Grade

NAEP

ID Block

Content

Strand Treatment

8 M055501 1992 1 Collapsed categories: 0,1,2,3,4 becomes 0,1,2,3,3

1992 only

8 M066301 M5 5 Collapsed categories: 0,1,2 becomes 0,1,1

8 M067501 M5 4 Collapsed categories: 0,1,2,3,4 becomes 0,1,2,2,2

8 M068201 M7 3 Collapsed categories: 0,1,2,3,4 becomes 0,0,1,1,1

8 M069301 M12 5 Collapsed categories: 0,1,2 becomes 0,1,1

8 M069601 M12 1 Collapsed categories: 0,1,2 becomes 0,1,1

8 M070001 M12 4 Collapsed categories: 0,1,2,3 becomes 0,1,2,2

8 M0732CL M14 4 Collapsed categories: 0,1,2,3,4 becomes 0,0,1,2,3

8 M073401 M14 1 Dropped due to bad item fit

8 M073501 M14 4 Collapsed categories: 0,1,2 becomes 0,1,1

8 M073601 M14 1 Collapsed categories: 0,1,2,3,4 becomes 0,1,2,3,3

8 M0757CL M15 5 Collapsed categories: 0,1,2,3 becomes 0,0,1,2

8 M076001 M15 2 Collapsed categories: 0,1,2,3,4 becomes 0,1,2,3,3

12 M073402 M14 1 Dropped

12 M025301 M5 1 Dropped 1992 only

12 M024901 M5 2 Dropped 1992 only

12 M070501,

M070502

M12 4 Combined into cluster item M0705CL

12 M071701,

M071702

M7 5 Combined into cluster item M0717CL

12 M071401 M5 1 Collapsed: 0,1,2,3,4 becomes 0,1,1,1,2

12 M056601 M3 1 1992 and 1996 responses split into items

M056601Y and M056601Z

12 M062401 M10 3 1992 and 1996 responses split into items

M062401Y and M062401Z

Both items collapsed: 0,1,2,3,4 becomes 0,1,2,3,3

12 M073901 M14 3 Collapsed categories: 0,1,2 becomes 0,1,1

12 M012731 M8 3 1992 and 1996 responses split into items

M012731Y and M012731Z

12 M058701 M11 3 1992 and 1996 responses split into items

M058701Y and M058791Z

12 M071801 M7 4 Collapsed categories: 0,1,2,3,4 becomes 0,1,1,2,3
274

Table 12-17 (continued)

Items from the 1996 Assessment in Mathematics Receiving Special Treatment

Grade

NAEP

ID Block

Content

Strand Treatment

12 M070601 M12 4 Collapsed categories: 0,1,2,3,4 becomes 0,1,2,3,3

12 M0732CL M14 4 Collapsed categories: 0,1,2,3,4 becomes 0,0,1,2,3

12 M018901 M4 4 1992 and 1996 responses split into items

M018901Y and M018901Z

12 M021501 M6 4 1992 and 1996 responses split into items

M021501Y and M021501Z

12 M013131 M8 4 1992 and 1996 responses split into items

M013131Y and M013131Z

12 M053401 M9 4 1992 and 1996 responses split into items

M053401Y and M053401Z

12 M061905 M10 4 1992 and 1996 responses split into items

M061905Y and M061905Z

12 M060701 M13 5 1992 and 1996 responses split into items

M060701Y and M060701Z

Both items collapsed: 0,1,2,3,4 becomes 0,1,2,3,3

12 M074101 M14 5 Collapsed categories: 0,1,2,3,4 becomes 0,1,2,3,3

12 M077001 M15 5 Collapsed categories: 0,1,2,3,4 becomes 0,1,1,1,1

12 M021601 M6 5 1992 and 1996 responses split into items

M021601Y and M021601Z

12 M021602 M6 5 1992 and 1996 responses split into items

M021602Y and M021602Z

12.5.4 Evaluating the Fit of the IRT Model

During the course of estimating an IRT model, individual items were evaluated to determine how

well the item response model fit the data. This was done by visual inspection of plots comparing

empirically based and theoretical item response functions. Specifically, for dichotomous items these plots

consisted of nonmodel-based estimates of the expected proportion correct for each level of mathematics

ability compared to the proportion correct for each level of mathematics ability as predicted by the

theoretical item response function. For polytomous extended constructed-response items, similar plots

were produced for each item category response function (see Chapter 9 for a fuller explanation of these

plots).

In making decisions about excluding items from the final scales, a balance was sought between

being too stringent, hence deleting too many items and possibly damaging the content representativeness

of the pool of scaled items, and being too lenient, hence including items with model fit poor enough to

endanger the types of model-based inferences made from NAEP results. Items that clearly did not fit the
275

model were not included in the final scales; however, a certain degree of misfit was tolerated for a

number of items included in the final scales.

For most items, the model fit well. In a few cases, poor fit with the data led to special treatment

or deletion of the item. Figures 12-1, 12-4, 12-5 and 12-6 give item response plots of dichotomous items.

In the plots, the x-axis depicts scale score, and the y-axis the probability of a correct response. The solid

line is the logistic model prediction, and the symbols (usually a circle or diamond) are the nonmodelbased predicted proportions. The size of the symbols are proportional to the estimated number of students

at a particular scale score level. The symbols are ordinarily larger in the middle of the theta scale, where

most students’ scale scores fall. The item parameter values are also included in the plot.

Item response plots for polytomously scored items are given in Figures 12-2, 12-3, 12-7, and

12-8. These are similar to the plots for dichotomous items except that there are several solid lines, one for

each item category, with each line indicating the probability of responding in the respective item

category. As before, the circles or diamonds indicate the empirical response function, with the size of the

circles and diamonds proportional to the estimated number of students at a scale score level.

In the plots good fit of the model to the data is indicated when the model-based functions (solid

lines) coincide with the empirical functions (circles, diamonds, or other symbols).When the empirical

plot is far away from the model based line, there is poor fit of the model to the data.

Four examples of fit are illustrated. First there is good model fit which is shown by Figure 12-1

for a dichotomous item and Figure 12-2 for a polytomous item. In both cases empirical and theoretical

lines coincide.

Second is an example of an item that displayed non-logistic empirical functions and was

dropped. Figure 12-3 shows a polytomous item that was dropped from the assessment.

Third is an example of an item that changed function from one assess year to another. Figure 12-

4 shows that the empirical item functions for two assessment years (diamonds for 1992 and circles for

1996) are distinctly different. Figures 12-5 and 12-6 show the result of estimating item parameters

separately for the two years. This ‘splitting’ of the item across years results in quite good fit for each

year.

The fourth example is of a poor fitting polytomous item that was modified by collapsing

categories. Figure 12-7 shows a 5-category item which evidences poor fit mostly in the upper category,

due in large part to a low number of respondents. As a result, the upper two categories were collapsed

resulting in a 4-category item which, as Figure 12-8 illustrates, fits satisfactorily.
276

Figure 12-1

Dichotomous Item Exhibiting Good Model Fit*

*Circles (1996 data) and diamonds (1992 data) indicate estimated conditional probabilities obtained without assuming a logistic

form; the solid curve indicates estimated item response function assuming a logistic form.
277

Figure 12-2

Polytomous Item Exhibiting Good Model Fit*

a= 0.5634

b= 0.6933

d1= 1.9463

d2= -0.3616

d3= -1.0074

d4= -0.5773

fit= 3.55

a = 0.5634

b = 0.6933

d1 = 1.9463

d2 = -0.3616

d3 = -1.0074

d4 = -0.5773

*Circles indicate estimated conditional probabilities obtained without assuming a model-based form; the solid curves indicate

estimated item response function assuming a model-based form.
278

Figure 12-3

Polytomous Item Exhibiting Bad Model Fit That

Was Deleted from the Assessment*

a= 0.6338

b= 1.7533

d1= 0.1185

d2= -01185

fit= 78.73

a = 0.6338

b = 1.7533

d1 = 0.1185

d2 = -0.1185

*Circles indicate estimated conditional probabilities obtained without assuming a model-based form; the solid curves indicate

estimated item response function assuming a model-based form.
279

Figure 12-4

Dichotomous Item Exhibiting Different Empirical Item Functions for Different Assessment Years*

*Diamonds represent 1992 data; circles represent 1996 data. Circles and diamonds indicate estimated conditional probabilities

obtained without assuming a logistic form; the solid curve indicates estimated item response function assuming a logistic form.
280

Figure 12-5

Dichotomous Item Fit Separately to the 1992 Data and Exhibiting Good Model Fit*

*Diamonds indicate estimated conditional probabilities obtained without assuming a logistic form; the solid curve indicates

estimated item response function assuming a logistic form.
281

Figure 12-6

Dichotomous Item Fit Separately to the 1996 Data and Exhibiting Good Model Fit*

*Circles indicate estimated conditional probabilities obtained without assuming a logistic form; the solid curve indicates

estimated item response function assuming a logistic form.
282

Figure 12-7

Polytomous Item Exhibiting Poor Model Fit in the Upper Category*

a = 0.3692

b = 2.2569

d1 = 2.6374

d2 = 1.9363

d3 = 0.0322

d4 = -4.6059

*Circles indicate estimated conditional probabilities obtained without assuming a model-based form; the solid curves indicate

estimated item response function assuming a model-based form.
283

Figure 12-8

Polytomous Item With the Upper Two Categories Collapsed, Now Exhibiting Good Model Fit*

a= 0.3824

b= 0.6994

d1= 1.0777

d2= 0.3514

d3= -1.4291

fit= 3.15

a = 0.3824

b = 0.6994

d1 = 1.0777

d2 = 0.3514

d3 = -1.4291

*Circles indicate estimated conditional probabilities obtained without assuming a model-based form; the solid curves indicate

estimated item response function assuming a model-based form.

12.5.5 Derived Background Variables

Derived variables are variables which use information from more than one background question.

They were used for two purposes: as conditioning variables and as reporting variables used to define

subgroups. Some of these variables are common to all the subject areas; others are specific to the 1996

mathematics assessment. Derived variables used for conditioning and reporting are described in Appendix C.
284

12.5.6 Generation of Plausible Values

For the entire sample, multivariate plausible values for content strand scales were generated for

each grade group separately using the multivariate conditioning program CGROUP as revised by Thomas

(1993). As with the scaling, student weights were used at this stage of the analysis. Instead of using selected

background variables for conditioning variables (as had been done prior to the 1990 assessment), principal

components of the background variables were used. The principal components used accounted for 90

percent of the variance of the original conditioning variables. Principal components were employed to

remedy problems of extreme collinearity among some of the original conditioning variables.

Research based on data from the 1990 Trial State Assessment in mathematics suggests that results

obtained using the 90 percent subset of components will differ only slightly from those obtained using the

full set (Mazzeo, Johnson, Bowker, & Fong, 1992). Table 12-18 lists the number of principal components

included in conditioning, as well as the proportion of variance accounted for by the conditioning model for

each grade.

Table 12-18

Proportion of Scale Score Variance Accounted for by the Conditioning Model

for the Mathematics Main Assessment

Proportion of Scale Score Variance

Grade

Number of

Conditioning

Contrasts

Number of

Principal

Components

Number

Sense,

Properties,

and

Operations Measurement

Geometry

and

Spatial

Sense

Data

Analysis,

Statistics,

and

Probability

Algebra

and

Functions

 4 895 321 .70 .71 .61 .75 .69

 8 1,027 362 .71 .75 .68 .77 .73

12 812 314 .76 .79 .81 .70 .83

The codings of the original mathematics-specific conditioning variables, before principal

components were calculated, are presented in Appendix C. The CGROUP program estimates distributions

of scale scores by combining information from item responses of individuals and information from linear

regression of scale score on conditioning variables. For each individual, five plausible values are randomly

drawn from their predictive conditional distribution.

The proportion of variance of each original conditioning variable accounted for by the principal

components included in the conditioning model is listed in Appendix F. The estimated conditioning effects

for the principal components of the three samples defined by the three grade groups are also given in

Appendix C. The values of the conditioning effects are expressed in the metrics of the original calibration

scale. Definitions of derived conditioning variables are given in Appendix B.
285

12.5.7 The Transformation of the Mathematics Calibration Scale for Reporting

and the Formation of the Composite Scale

Like all IRT scales, the mathematics content strand scales have a linear indeterminacy that may be

resolved by an arbitrary choice of the origin and unit-size in each given content strand. In 1990 the NAEP

mathematics data were scaled across grades separately for each scale. The linear indeterminacies among the

scales were resolved by transforming the scale means and variances of three grade samples combined

together to the 250.5 4

, 50.0 metric using a transformation of the form,

θtarget = A • θcalibrated + B,

where A and B are linear transformation constants.

As a result, all of the scales that spanned all three grade samples were on a common scale. By

contrast, the 1992 and 1996 data were scaled within grade. It was necessary, therefore, to transform data

from both assessments to the 1990 cross-grade scale. This was accomplished first in the 1992 assessment,

when the 1992 data were linked to the 1990 scale in a two-stage process. In the next assessment, the 1996

data were linked to the 1992 transformed scale, which in effect put the 1996 data on the 1990 cross-grade

scale. The procedure for transforming the 1996 data will be described below. The similar procedure for

transforming the 1992 data was presented in The NAEP 1992 Technical Report (Johnson and Carlson,

1994).

The 1996 data were put on the 1992 reporting metric by using a linear transformation that converts

the 1996 thetas to the 1992 reporting scale. This linear transformation was created by the following

procedure. The 1992 thetas were reconditioned using CGROUP with the 1996 item parameters. This

analysis resulted in 1992 scores which were in the 1996 theta metric. We then transformed the new 1992

thetas (in the 1996 metric) to the 1992 reporting metric (in the 1992 metric) by a linear transformation

which created scores having the same mean and variance as the 1992 reporting scale. This is a common

population equating procedure. The linear constants of this transformation were then used to transform the

1996 thetas to the 1992 reporting metric. The transformation constants used for the five content strand

scales and for the estimation scale are given in Table 12-19.

While scores in five content strands provide useful insights into the relationships among

subpopulations, a single index to summarize overall performance is a useful tool for a compact overview of

subpopulation trends. For that reason, a composite score was defined as a weighted average of the five

mathematics content strands. The weight given to each content strand is a direct reflection of the relative

testing time intended for that content strand in the assessment, as defined in the Mathematics Framework

for the 1996 National Assessment of Educational Progress (National Assessment Governing Board, 1994).

Since the emphasis given to each content strand was different across grades, the weights assigned to each

strand in the composite also differed across grades.

 

4

 The function RPs = 250 + 50(θs) would have been preferable. Holland and Zwick (1986) have noted that the values actually

used correspond to the bi

 varying from –5.00 to +4.98 in steps of .02 instead of –4.99 to +4.99 as intended. The result is that the

RP scores are a half-point higher than appropriate for the hypothetical test.
286

Table 12-19

Coefficients of the Linear Transformations that Transform the Five Content Strand Scales

from the 1996 Calibration Metric to the 1992 Reporting Metric

Grade Coefficient

Number Sense,

Properties,

and

Operations Measurement

Geometry

and Spatial

Sense

Data Analysis,

Statistics, and

Probability

Algebra and

Functions

4 B

A

220.32

33.13

226.02

 32.22

224.70

 28.04

223.38

 30.86

223.63

 29.47

8 B

A

273.44

 35.12

269.33

 43.65

266.83

 33.54

270.97

 41.00

271.13

 35.52

12 B

A

300.43

 32.48

300.85

 35.71

304.67

 36.05

301.03

 34.58

303.42

 35.95

The definition of weights for the composite in each grade is given in Table 12-20. The mean and

standard deviations of the composite scales for all three grades is given in Table 12-21. Note that this

composite can be compared with the 1990 and 1992 composite scores since all three scales are on the 1990

cross-grade scale (as defined in the beginning of this section).

Table 12-20

Weights for the Mathematics Composite by Grade

Scale Grade 4 Grade 8 Grade 12

Number Sense, Properties, and Operations

Measurement

Geometry and Spatial Sense

Data Analysis, Statistics, and Probability

Algebra and Functions

.40

.20

.15

.10

.15

.25

.15

.20

.15

.25

.20

.15

.20

.20

.25

Table 12-21

Means and Standard Deviations on the Mathematics Composite Scale

All Five Plausible Values

Grade Mean S. D.

4 223.9 31.2

8 272.0 36.4

12 304.0 32.2
287

12.5.8 Partitioning of the Estimation Error Variance

For each scale within each grade, the error variance of the reporting scale means was partitioned

according to the procedure described in Chapter 11. The variance is partitioned into two parts; the

proportion of error variance due to sampling students (sampling variance) and the proportion of error

variance due to the fact that scale score, θ, is a latent variable that is estimated rather than observed. Table

12-22 contains estimates of the total error variance, the proportion of error variance due to sampling

students and the proportion of error variance due to the latent nature of θ (for stability, the estimates of the

between-imputation variance, B, in Equation 11.9). More detailed information by gender and race/ethnicity

is presented in Appendix E.

Table 12-22

Estimation Error Variance and Related Coefficients for the Mathematics Main Assessment

Total

Proportion of

Variance Due to...

Grade Scale

Estimation

Error Variance

Student

Sampling

Latency

of θ

4 Number Sense, Properties, and Operations

Measurement

Geometry and Spatial Sense

Data Analysis, Statistics, and Probability

Algebra and Functions

Composite

.88

1.25

.77

1.28

.99

.78

.91

.84

.82

.87

.83

.95

.09

.16

.18

.13

.17

.05

8 Number Sense, Properties, and Operations

Measurement

Geometry and Spatial Sense

Data Analysis, Statistics, and Probability

Algebra and Functions

Composite

1.18

2.06

1.10

2.31

1.19

1.13

.91

.87

.91

.92

.89

.96

.09

.13

.09

.08

.11

.04

12 Number Sense, Properties, and Operations

Measurement

Geometry and Spatial Sense

Data Analysis, Statistics, and Probability

Algebra and Functions

Composite

1.27

1.25

1.12

.99

1.33

.99

.93

.84

.88

.91

.93

.98

.07

.16

.12

.09

.07

.02

12.5.9 Mathematics Teacher Questionnaire

Teachers of fourth- and eighth-grade students assessed in mathematics were surveyed. Variables

derived from the questionnaire were used in the conditioning models for the grade 4 and the grade 8

samples, along with a variable that indicated whether a student record had been matched with a teacher

record, so that means for subgroups defined by these variables could be compared with no bias. Of the 6,612

fourth-grade students in the main sample, 6,105 (92%) were matched with both parts of the teacher

questionnaire and 99 (1.5%) were matched with only the first part of the questionnaire. Of the 7,146 eighthgrade students in the main sample, 6,144 (86%) were matched with both parts of the teacher questionnaire
288

and 49 (less than 1%) were matched with only the first part of the questionnaire. Thus, 92 percent of the

fourth graders and 86 percent of the eighth graders were matched with at least the background information

about their mathematics teachers.

12.5.10 Analysis of Dimensionality

Plausible values are drawn from the set of five correlated content strands (see Section 12.5.6). For this

reason, it is useful to inspect the correlations among the content strands for evidence of multidimensionality.

Tables 12-23 and 12-24 give conditional and marginal correlations for the five scales for the three grades.

Conditional correlations are analogous to pooled-within groups correlations when the groups are the grouping

variables used to condition the data with CGROUP. They are obtained from the error correlations of a

CGROUP analysis. The conditional correlations are quite high, averaging .85 for grade 4, .91 for grade 8, and

.85 for grade 12. The marginal correlations are the average correlations of the five plausible values of each

scale. In this case they average .86 for grade 4, .88 for grade 8, and .87 for grade 12. Although it is of

substantive interest to analyze the scales separately, the correlations indicate that they are highly redundant.

Table 12-23

Conditional Correlations from Conditioning (CGROUP)

Grade Scale

Number Sense,

Properties,

and

Operations Measurement

Geometry

and

Spatial

Sense

Data Analysis,

Statistics,

and

Probability

Algebra

and

Functions

4 Number Sense, Properties, and

Operations

Measurement

Geometry and Spatial Sense

Data Analysis, Statistics, and

Probability

Algebra and Functions

1.00

 .89

 .80

 .

96

 .94

1.00

 .78

.90

 .85

1.00

.75

 .78

1.00

 .89 1.00

8 Number Sense, Properties, and

Operations

Measurement

Geometry and Spatial Sense

Data Analysis, Statistics, and

Probability

Algebra and Functions

1.00

 .92

 .79

.96

 .97

1.00

 .92

.95

 .95

1.00

.86

 .84

1.00

 .96 1.00

12 Number Sense, Properties, and

Operations

Measurement

Geometry and Spatial Sense

Data Analysis, Statistics, and

Probability

Algebra and Functions

1.00

 .96

 .90

.84

 .91

1.00

 .94

.80

 .89

1.00

.67

 .88

1.00

 .66 1.00
289

Table 12-24

Marginal Correlations of Science Scales1

Grade Scale

Number Sense,

Properties,

and

Operations Measurement

Geometry

and

Spatial

Sense

Data Analysis,

Statistics,

and

Probability

Algebra

and

Functions

4 Number Sense, Properties, and

Operations

Measurement

Geometry and Spatial Sense

Data Analysis, Statistics, and

Probability

Algebra and Functions

1.00

 .89

 .84

 .91

 .92

1.00

 .83

.86

 .86

1.00

.81

 .82

1.00

 .86 1.00

8 Number Sense, Properties, and

Operations

Measurement

Geometry and Spatial Sense

Data Analysis, Statistics, and

Probability

Algebra and Functions

1.00

 .89

 .84

.91

.92

1.00

 .89

.89

 .89

1.00

.85

 .86

1.00

 .90 1.00

12 Number Sense, Properties, and

Operations

Measurement

Geometry and Spatial Sense

Data Analysis, Statistics, and

Probability

Algebra and Functions

1.00

 .90

 .90

.88

 .90

1.00

 .90

.86

 .87

1.00

.81

 .90

1.00

 .80 1.00

1

 Tabled values were obtained by computing a separate Pearson correlation coefficient for each plausible value, computing Fisher’s z-transformation

for each value, computing the average of the transformed values, and computing the inverse transformation of the average.



	





	

	


		



			


 
!
"#
	
!

	
	 

	



	


	

			



	
	

	




		
	

	
	$ %&&'

!
!()*
+,

	*
"$* 
!
"

	 !


 "#$$
 %&
		
	
	


	


	

	'!(



		
	
	

	

)*

 
		
	
				
(

	

	
	$ 

%&&'!
!()*
	
	
	'!





	


		


+	
 

	

			



+	


, 	-./0



	
*	


*






	

	 
1

 

		







	
	
(

		



*



		
(

	
	
-./0

	

	
	
		*

	



	
	

	
*



2(+	
2	
			
			




	

	


	
	

$ %&&'!
!
("



 !!"

#

$

!%

!!

####% 
&!'
#&()

$

####%

34

#5 67(62% 0	 8
3 929(2969:	
& % 27;

<4

#5 67(62% 0	 8
< 929(2969:	
& % %%3

64

#5 67(62% 0	 8
6 929(2969:	
& % ;2%

64(.
5 62<(637 0	 8
6 929(2969:	
& 6 32



,

*

.(; 

*+

0	 0	
		

# #

	

.
 .

	*	


	





=>
*	
 


 	 
	 		




)
*	

=?+	
		


	
*


	

	

 '


	

@:		'

-			
			

	*

>

,
 A	
0
 -.-



B
	
	


*

	

 




		


	
		


	
	**

	



	
	

	 



 

		




	





64(.
5
	

	64

#5


.
	




	*






C
	

-./0
	64(.
5

	
	2(2




*

$
	





	


		
			
		*


3 
< 


6	
		D	
		
D

-./0

	 
(

	*





	-./0
	
	*



 *	
		
< 

<6 %< 

	
 
 2 %
	
	
	
	

	

*



	*	
=	#*	

		*



.&.


+	
 	
	
		



		


	
		6


+	
	



		
	

	


	


		



	&

B
	

	

		 



	
 		
(

	
 
	 






	

	

	
	


*


B	
		



	 
*



	

	E

 
		
	
	
 >',

	23&F

6 	


	'!&
	2;&F

2 
		  
 
 	
	 

  	 G




H
	
	2&F

3 		

	
		
	
	


	


 
	2%&F

; 
		

	


	2%6&


	2<


	
				

 2
			
	



		
		
 27



1	*
	


			
1	*	*	


	
	
	

 





*	

		
			




	
, -


 .


-./0



	

	(	
	


	


		


	 	
	
	 	
B
	
		

	
	

	 

			

'	

	7-./0



		



I

	



 
C			(	
	


	*	

	
-
	
			
		7



	





	


			
*



	
		

	

	



		

		
(



	
(
C
	



	
B
	
	
	

	


*



	
*


!
!,-*%&&'
"

$* 
!
"	-	.

	8
I 2&		




	
*2 7 < 

	
 
3 < 6-	
		
	




/		
*
	
	




E!
!F(.
!"

!
!F"
*!
!
	




			
	


	



2(6*	

	
*		
	







	




2(6*	
	
			
 	
	

B	
	*







	2;6 

*&


	
,


*
!"
!
!/
$.,
"*!
!

*%

-/#&0

0&(0



0&(0

&'

0&(0 
!

3 0
(

0	(

3;

32

;2

3

3%

33

3;

2

< 0
(

0	(

2

6

;

2



;

3

7

6 0
(

0	(

7

;

3

6



;

7

<

,

 		
*


	;	
1
 
	
	

.	


3 	
	*


67	
	
	


F	
<6


C


27	
.


+	
6 	
*



	

		
1




+	
3
	
		


	&/	
	



		
	


	
	
,	;
*

(	*

	
	*



	
C
	
		
		*
C
	

			
(

&
			
		
(	*

*

	


			 	
	*
((



	

*

	





*


	
	

	 
		



	-	
	





	
	

/	

*
*	
(	

((



B	
/
((



		
		


2

3&	
,
	
 		

	
 	
	
	

	 


2(2	2(;

+

	







	
	
	










2(6	2(<
		

+	
6

+	
6	

		


	
		
		
	
	




	



%&&'$ !
!0"!)(

.#.(12$*
0*!"
	

!(#$0%#!(##0!%-!/!!$#/

 !0"

$&

!&0#

,

0)!/




0)!/

1

0)!/

2

0)!/

$#

#


!

#

2 7   7 7 7 %

3  3   7 7 %

; 6 7 3   7 <

 7 7 2 6 7 7 ;

% 6 7 %  7 7 7

<  7 % 7  7 

 6 7   7 7 

7  7  7 7 7 6

   3 7 7 7 

6  7 <  7 7 ;

2  7 3  7 7 

3 ; 7 ; 7 7 7 7

; 2 7 ;  7 7 

67  7 6 2 7 7 

6 ; 7 3   7 


! 2	 3 4, 	
 
 5 	12


	
1

%&&'$ !
!0"!)(

.#.(13$*
0*!"
	

!(#$0%#!(##0!%-!/!!$#/

 !0"

$&

!&0#

,

0)!/




0)!/

1

0)!/

2

0)!/

$#

#


!

#

2 7 7 3  7  

3 2 7 3 2 7 7 7

; 7 7 < 7 7 7 <

 7 7 ; 6 7 7 %

% 6 6 < 7 7 7 6

< ; 7 ; 7 7 7 7

 2 7   7 7 2

7 <    7 7 

 < 7 %  7 7 

6 <  ; 6 7 7 

2 < 7 %  7 7 

3 % 7 < 7 7  

; %     7 

67 < 7  6 7 7 

6 % 7 % 6 7 7 


! 41 2 62 	4 	 , 	61



	
2

%&&'$ !
!0"!)(

.#.(1%4$*
0*!"
	

!(#$0%#!(##0!%-!/!!$#/

 !0"

$&

!&0#

,

0)!/




0)!/

1

0)!/

2

0)!/

$#

#


!

#

2 7 7 3   7 

3 7 7   6 7 3

; 7  % 7 7 7 <

 7  ;   7 <

% ; 7 % 2 7 7 ;

<  7 % 7 7 7 2

 3 7 <   7 3

7 < 7   7 7 <

 <  3 2 7 7 

6 <  ; 6 7 7 

2 < 7 %  7 7 

3 < 7    7 

; 7  ; 6 7 7 <

67 % 7  2 7 7 

6 < 7 3 3 7 7 


! 45 2 72 ,1 3 5 	65


	

*
	7J&



	
	
	,
	
 



2(	2(<*		

	


		


	

	(
		
(

	

	'		

*		


	
 		
*

	
'		
(

	
*


	
	


	
	


*&

 
		





					


 			
*

*			



G
H&.	


	
	
	

*



	


		



		F	

*



			


.

	 *		



		'	 	





	
*





*


			

	
	


			

 





	2;

,
3 
	
((

	

	
		
(



	


	

	


		
(

	

7(2


*	
*


7(3
(	
	

	



		
(

	
		*	
(
	


,
< 
	
((

	

		
		
(



	
/		

	


		
(

	

7(2



	
*
7(3
(	
	
1	
	

		
(

	

	

	
	

	
(
	


,
6 
	
((

	


		
		
(



	

	

	


		
(

	

7(2


*	
*


7(3
(	
	
	
	

		
(

	
-
	

	
	
(
	




	
3

%&&'$ !
!0"!)(

.#.(12$*
$*!"
	5

!(#$0%#!(##0!%-!/!!$#/

 !0"

$&

!&0#

,

0)!/




0)!/

1

0)!/

2

0)!/

$#

#


!

#

2 7  3 7 7  

3  ;  7 7 7 %

; 6 7 3 7  7 %

 7 7 2  7 7 3

% 6 2 3  7 7 7

<  7 % 7  7 

  7 % 7 7 7 <

7  7 3 7 7  

  6 2 7 7 7 

6  7 ; 7 7 7 

2   2  7 7 

3 ; 7 ; 7 7 7 7

; 2 7 ;  7 7 

67  7 6 2 7 7 

6 ;  2   7 


! 25 	
 35 7 
 , 	
3

K+	

		
		*






	
4

%&&'$ !
!0"!)(

.#.(13$*
$*!"
	5

!(#$0%#!(##0!%-!/!!$#/

 !0"

$&

!&0#

,

0)!/




0)!/

1

0)!/

2

0)!/

$#

#


!

#

2 7  2  7  

3 6  3  7  

; 7  % 7 7 7 <

 7  3  7 7 

% 6 3  7 7 7 6

< ;  3 7 7 7 7

 2 6 %  7 7 2

7 <  % 7 7 7 

 <    7 7 

6 <  ; 6 7 7 

2 <  3  7  ;

3 % 7 < 7 7  

;      7 ;

67 < 6 3 6 7 7 

6 % 7 % 6 7 7 


! 4, 	7 7, 	
 	 1 	65

K+	

		
		*







	
7

%&&'$ !
!0"!)(

.#.(1%4$*
$*!"
	5

!(#$0%#!(##0!%-!/!!$#/

 !0"

$&

!&0#

,

0)!/




0)!/

1

0)!/

2

0)!/

$#

#


!

#

2 7  3  7 7 

3 7 7 6 6 7 7 3

; 7 6  7 7 7 <

 7 3 6   7 <

% ; 2 ; 6 7 7 ;

<  6 3 7 7 7 6

 3 2 ;   7 3

7 < 7   7 7 <

 <  3 2 7 7 

6 <  ; 6 7 7 

2 < 7 ;  7  ;

3 < 7  6 7 7 

; 7 6 3 6 7 7 <

67  7  6 7 7 3

6 < 7 3 3 7 7 


! 36 	6 4	 ,1 , 	 	73

K+	

		
		*





.		
(

	
*




	


 


+	


;'	 

G	
	
H*


.	
	

	

	
	
-./0

 		*		
&		

	
	
	
		 

	 	 





	

:
"A
 <% 	


	
1


		
	
	
&
	
*

	
	
	
 

			
*



		





'	
 
	
	*	



			*
((





	(	'	 	

		




C
		

		


 
C
	


		
			

					
	
 




(
	

C
		



		





	



	
C
		
	
	

	


	

2%

		
:	
			
 

	*

G
H	
	


C

	
	
		
	
C
	 	
		
	
	

	
	@	,

+	
3&.
		
	

	 









*

		
	
/
	
 

	

 
	
&*	

	
C
	
	
	

	

 -


.		

 	
*
			-./0

	

			(	
	


	*	
	 
	



	64(.
5


2(	
1
	

	*
	(
	


	
	**

	



	
	*





1	
	* 
	
	


	

 	


 
<	
&	





	
&



	

	
/





	
	E



 
	 




*
	


	

	 		
	
*

*



	


	

	
		
	
	

	


2(

'				

 	
	

		


*	

	

	E



	
	

+&

6



I& 

2		
L&


	
6

0"!
!
*%46!
7$!8(!
"
.0"

 !!" !)(&&8 !0"#

# &# 0!(% 
&% !$

62<  ! M 0

62  M 0 !

637  0 ! M


	*			

	
	


	

*




-+/
	
+	
	


		

C	


 	
64(.
5
*	


	

		


	
1 


	
1	 !(8(&!((%
#(/##


		
	

		
		




	

	*
*	
			
	
	

	


>		
	
(
6(	
		
(

&

*


	*0		
*

	
	9*
	*




*		


	



	




*





2(7 2( 2(6*	

	
 

*
	
	

 



*
	
	
(	(		

	


& *
	




	

	

	



	

		




	

	
	**

	

	
	*
	


	


			
	
	
	*
	*


		
			
1
	

	
$
0
	

	
*	
*


	





F*

 	
	*


			
			
			
	
	

	



	


!
(


!*0"!.



#0"9""

9!!
!*!
!/
("12$*
$*!"
	



 

    
     

-


	
 7  <     7 D*
	

$


6

.@@

<6%3;

;;%

<73<66

6

%3%%3

3

27;%<

67<

7<;;

672

7;2

676

7%

662

%<6%<

;<7

:
	


	



6

.@@

3%33%

3%3%3%

2<2%2<

33%3%

;6;6;6

3<3%3<

;;;6;2

26272

:
	


	

6

.@@



;%;6;;

;<;

%%%

3%

%2%%3

63

;733

:
	


(


6

.@@

;;;

;%;%;%

32

;%;%;%

36

63

;2;%;;

;2;3;3

:
	
		
			
			


6

.@@

<%<<

2%3<

%<;<6

<;<%<

<<<%<<

<%27

<%67

%<3<6




	

!
(


!*0"!.



#0"9""

9!!
!*!
!/
("13$*
$*!"
	



 

    
     

-


	
 6 7 2    ;  D*
	

$


6

.@@

<22<6<



<2<<3

<%

<27<32

%2

27

637

36

6%7

2<23

6%6

637

6

<2<3

<6

:
	


	



6

.@@

3;3233

;%;3;;

33%3<

2;2223

323632

222222

32<37

363636

:
	


(


6

.@@

%%;%;

;3;

<<<

32

%%%

%7%2%

;

%6%3%2

:
	


	

6

.@@

;;;

<%

;<;%;<

3%;73

;6;6;6

;6;3;2

3;33

;3;;;

:
	
		
			
			


6

.@@

<<<

%%%

2;

67

<3<2<3

<6<6<6

6<7


	

!
(


!*0"!.



#0"9""

9!!
!*!
!/
("1%4$*
$*!"
	



 

    
     

-


	
 ; 6 3 <   ;  D*
	

$


6

.@@

<2%<2

;

<2;<36

%%

%;7<7

;;

276

6;

;;%

666

727

66

6;

6%

<27<2<

<

:
	


	



6

.@@

333

;;;

33332

2%2;2

3%3;3

3<33%

;7;;7

373737

:
	


	

6

.@@

%%<%%

;2;

%<7<7

%%%%

%%

%%%<%%

3%

%;%%;

:
	


(


6

.@@

2;3

32

6;3

;%;<;<

33%3%

;<;<;<

;3;%;;

;<;<;<

:
	
		
			
			


6

.@@

<7%<7

266

<6<<<;

<%<;<

6;2

<<7<

26

<;<%<



.


+	
 -./0

	
	'!(

& 		




	*



		


	
&



			




		
&		
	/		
	

			

		
		
		
'	
		*

	
*

	
	
	

	




	
		
1

 *
		
	
*




		

	
			
				

 
C
	 
			


			
&



1	






	
	


.

	

2(7	2(6 			

	
(	(		
	

	


*	
	*
1
	

	

*

	
	
	





			
	I
			
			
				
 



	






 
		
	
	<		
				
	



*
<3



			
	
			
	


,	
((

 
		
	


*

	




*	

		
	*	

	:	


	

	

	 	

	

	

	

	



		



	

*

	


*

	
			
				
				


	
F
	
			
	
			
*	


	
	'	


				
	

	
	

		

 

	
1




*			


		
&*

	
	6-./0


	

>
 3&.				
 	*
	
	
1
	

		
	




>
		

	
1
	

	 	
	




*

	


	

 
	 !"#

.	


2(3	2( 
		*(		

	
*



		
(

*(	
		
(

	
*


	9*

	



		
	



		*






	

	
.
1'

	


			
+<&A 



	
		
	
	
		
	$
 

		

,


36 67

	
*
		

	.	
< 				


.

		*



.

	
	


	*
		
	



	
	

'

 		


			$


*C	


+A

	


	
<	2
3 %;	
< 

%	;
6

+	
%
	
			
	


	

*	


	

			
(

	

		
	
 	


1	

		
(

	
*
	
	
	


	
		*


	


/1	

		
(

	
*



	;	

	


*

'

 	
	
	7	3!		
1	

		
(

	




.
1' *		
$
 

	


	 +A

	
1

 $
%%
"#&



.

		
	>',&	

	
*
	
		
	



	

$
*



	



	
	 :	
I	
	 

:	
)	
	
	*	
		
		


1







		

	
	
	



B
	(		

	





C
	
1	

	


>',	

				


	



	
			 *
		
	

		
	

,		
 	#	
()
$


	

)
<<&*


	
	>',	


+	
&#	


#	
 2&*



	
	>',		




?* >
 82&






	



,>',
 *
	*




	

 


+	


>',
*

	

	


,		
 	>',
1

	

	

#	
()
$


*
	
	
	
	

	

EG. HGI HG+HG.H	



1
	>', GIH	

1
	*
	>', G+H	

1
			>',

G+H	
*


1

>',		









2(2$
	
		>',
		
,




 I )

	

,)	..1*
!

$"& '(


) ( *+&# ,
+-. ,
+/
!


 +(

I(

.(

.N

IN

+N

2

2

6



6

7

7

3

6

6

6

7

7



22

62



7

 +(

I(

.(

.N

IN

+N

2

7

3

62

3

7

7



22

37

6

7

7

6

23

32

6

7

 +(

I(

.(

.N

IN

+N



2

2<

66

;

7

7

3

2

2;



7

7

6

22

3

2

7

0	

	
1	
	
		


	

	

 

I )	
	&		




:	
	
	&G.NHG.(H


	>', GINH
*
	>',	 GI(G


*
	>',	



G+NHG+(H
	

	>',

0	

	
	
		*



	

	

2(3$
	


		>',
		
. 


 I ) 

	

	
				
*

	

	#	
			


				
			>',	

		
			
 

		$






	*





*
			*






		$





		
		

	
	

	

	



+	
,		
 	$




	6;


	

*		#	
			&*

		>','	



*			$




	6;
	
	
1	

#	
()
$
			


	G+H	


+	

	&

	

,)	..1*".

$"& '(


) ( *+&# ,
+-. ,
+/
!


 ++(

II(

..(

..N

IIN

++N

;

3

2

6

3

6

6

6

2

2

6

6

7



37

3



7

 ++(

II(

..(

..N

IIN

++N

%

%

32

;%

2



7

;

3<

;%

3

3

6

2

;3

;3

2

6

 ++(

II(

..(

..N

IIN

++N

2

%

37

3



2

2

6

;3

3;

;

6

6

6

;3

3;

3

3



0	

	
1	
	
		


	

	

 

I )	
	&		




:	
	
	&G.NHG.(H


	>', GINH
*
	>',	 GI(G


*
	>',	



G+NHG+(H
	

	>',

,*		
	/ 	

	
*>',*



*



		

	
	
	



B
	(		

	.


+	
 

		



*	B
	
	*		

			

*
".


	
	


F		 *			


		




	

*-./0

	
		*


	
G+HG++H	
		



	/		





*	
1
	
	
'	*			

B
			
		
			*

	

	
		

			
	
B
	
	

		
				*

*			

	



.			


	

	)

	
	 			


			*





	).



*
			 		
*		
1


*
	*



	
*
	C
		
*


		

		


	



 "0* 0120/01 34" 5'6"2)

'2 	-	.

	8
I-.8I&
	

			
-./0



	




*	(

*:	(


			

					



	
	


	

	'	
			





	

+	
	*		
* -.8I

		





*	(
.	
		

	

	

	

	*





	


 		

	
		

		
	





 	!
!,-*%&&'
"$* 
!
"	

-	.

	8
I 2&

			



	






*	(
@
*
 '!*

*	(


:	

 	
*


		

		
	
			






E
	 
'!(



	
	




*






	



+	

$
	
	



	

	
	
	

		
(
 			
(

 
1	



		
(

&

 "##0
#


,




 	

	

		
*


	

	-./0

I'@890.!+.@/ *

#
I<6&I'@8#I

&0.!+.@/	


		

	
		
	


	
	
( 	*( 	

(
	
	
 	

$
	
	






#6&

+	
&
	*

	


1



	
		
	*
	*


	

.


+	
 	
(
	
*

	

7 &*




		

(
	
	
		


		
(
	
*

	
	


	
	 *			


			
	



		


.		
(

	
*		*	

*

	
*


		*(


	
	
*		*
		

	

		7		


		
	
*



	
	

	,
	 	
		*

	
*

	
	
	*

	

	


		
1

!

	
1	

		
(

	
		*

(	*

	
	


		




	


.
		
*	'!
			



+	


 
1

	




	




	:
		






	
	 



	 		

	
	


		


	

	
	'
	



*		
(	




 



		
	
*
	



1



	
	
	
	
	





		
	
 					
		
 *


	





	/1

				
			
	
*


	G$



	H	
		
	
/1


G	
H		
		*

	
	


	




	
		
	
.	
	
*


	

$
	
	




/		(	
		


	
*
	

$
	


	

	
	* 	

  
	

	
	
	
	
1

	



*




*

2(;



	

"*!"*".

 	( 	( 	( 7	(

; +
	


3 +
	
 /
	

2 +
	
 /
	 .
C	


6 +
	
 0	
	 0	
	 0	
	

 0	
	 D		 D		 D		

7 : (	 

		


: (	 

		


: (	 

		


: (	 

		


/I

		
	

	
*


	
	I'@890.!+.@/

0	
	*


	

	
*		*		
E	


47 65&F
(47 ;5&F		
	*(
	


	*	
	




	

		



		
*
		;7&

	
			
&		

	
	
	*

	
	



		

*	
		
		

	


		
*

	
	
			'	

	

		


	*


,	 	
B
	
		
	*
1
47 5&	

	*


	

	

		
		*

	
		


C
	
	

*	
B
	
		
	*


		

	*		

	



		
.	


		
 	
B
	
		
	*
	$
	



$
	
	
+
 
	

		
		
*





	$


 08
%*&


>

C
			

	

		 
				'!
*


	
		
		


	*

	
	
	

	

			
	


				
			




/	
	*




,	(
	
(


	*(	


	
 
	*
	


1	
		
	


1
	
	&

	
	

			
	*(
	
	

(


	
	

	

	
	

	
		
	
	
	




#
" <% 276&,		
	
	
(	
		
(

	
 

	*



	
	
	
	


+	
&

.*		

		

			


	 
	





B
		

	
	


	

	B	
	







*
	
	


			





B
	





((	
	G
(C
H


I'@8#
"I <6&,	 	
1		
		

*	
	


	*

	 

	
	#
	<%&					



	


		
		
-./0*

1





	




	
		
	.	
	
 #
 #$$
3&	
	
				


	



(C
	
		





*





	

	
	

	I'@890.!+.@/
C
	B	
	
)*

 

		
	
*	
		
	
	









	

B	
		+
C
	 
(C

((	





	

		

	

	





 
+	
 		
		
		 		
 	
(




	
	


1	8
	

$

-./0 	
*




		
*
	
		
B
						
			
	



B
		

*	
$
&

	
	

	



	
				
(





		
	


	'

	
 	**
		

			
*	B
	
		

	
	

		
			
	

'

	
1	
	
 

*	

	*





		
	 


			

		
	


		

	


	
 	

	 
	
*	
	
		
		






-./0
	'	
		
			
*

	
F*

 


	


	*	
	


	

	


,	
B			
 			
*
1	

,
2(


	
1
*			
	(
	
		
			

*	
	


'		 	(1	
	

	


	

	1(1	




&
*
		
		

&

	
	

			
	


	

C
	

	(


		
&$
	

			
		

				

	
				


*	
		
	
		


	

	


	

		
		

	
	






	
	
	
	
		
	
	

&		


	$	
	
	
		
*
		
&.*		
		


		

	

		
*
	(
&.

			 	O


(	(

	
	

1	





	*		
(

	


	

	


,
2(6

1
	3(	

1	

		
(

	



1
	
	@
				(
	
(
	
 		*

	*
		

	
	
	
	
 

			
	
		
	


	
	
*	
&
		
		
		
*
&


		

	
		
C
	
	
		.*	,
2( 		*


	
		


C	
*
 		
		

		(


		


	
	*	


*			
(


		
		

	.
	

,
2(6*			
	
1

		

		**
	



	

		



	*

		
1	

		
(

	

		

	
1

*


	

*	



C	
*
	

	


.


 
		

	
	







	,
2(2	(
	
(
	
&2(3
1	

		
(



	
&
	
1
	
-	
		,
2(3 	






		
	
	*
		
	
	*
		
 
	

*		
	


	
1		
+
	
	*	

*

	
	 
		

*B
				*	
	
	
	*	'

 




	

	*


	
		
	
*

	
			
		


		
1

#	*
		

	
	
*			


			
B
	
		
	









	
	

$
		
	
		
	

	
	
	*	



	

'	
A737%
6 
	


&		*		
	&

*		B	
			



		,
2(;&+	

6

*



	
7(		
 		

		,
2(&

&
	

"5)(
	 (

!"/"70 
*(,
!


*0
.!:/
"
("7)
!; <


	1/",


K.	
	

		
	

	

	
*			F	


	

		
	


		



&
	

"5)(
	 (

!"/"70 
*)	.)!

!)


*".".! <


	1/",


K.	
	

		
	

	

	
*		
(

F	


	

		
	


	
(





&
	

"5)(
	 (

!"/"70 
*(,
!


*0
.7!:/
"
("7)
!; <


	/"/
*


K.	
	

		
	

	

	
*			F	


	

		
	


		


&
	

"5)(
	 (

!"/"70 
*)	.)!

!)


*".".! <


	/"/
*


K.	
	

		
	

	

	
*		
(

F	


	

		
	


	
(




&
	

"5)(
	 (

!"/"70 
*)	.)!

!)


*".".!:=>22%>%; <


	/",


K.	
	

		
	

	

	
*		
(

F	


	

		
	


	
(




&
	7

"5)(
	 (

!"/"70 
*)	.)!

!)


*".".!:=>22%>%;$*)""(
	)	
4?

K.	
	

		
	

	

	
*			F	


	

		
	


		

+:	

	
	
		
(

	

C
$
 		
*


	*	



	*
 

2(			
		



	
	
		


'
		


	
		


		
		*



	

	
	
*
	
		*


1
	
.	
< 	






	
	
	
		


				
.

	
-		

	



	



	
	
	'!
	
		

	



	


	
.
1>


	7

*%&&'!
!$!

	(!
"$


)

2'0

"$

-.+"#

2#

&
%


 $
!

 

3 A727; 3(; 0





+
	

E7  6




7 7 

@,	

3 A727% 3(% 0





+
	

E7  6 2




7 7  6

@,	

3 A72672 ;(2 0





>
 >




3 < A7227 ( 0





>
 >




3 A723<76 2(6 0





+
	

7  6




7 7 

@,	

3 A72376 %(6 /	





+
	

E7  6




7 7 

@,	

3 A72373 %(3 /	





+
	

E7  6




7  

@,	

3 A7237% %(% /	





+
	

E7  6




7  

@,	

3 A723;7 6(7 /	





+
	

E7  6 2




7  6 6

?


,
C


3 A737;7 6( /	





+
	

E7  6




7 7 

@,	

3 A7277 2( @






>
		
	
 >




3 A72776 2(6 @






>
		
	
 >




3 A727; (; @






>
 @,	

3 A727% ( @






+
	

7  6 2




7  6 6

@,	

3 A726;7 7(. @






>
		
	
 >




3 A726;76 7(8 @






>
		
	
 >




3 A722;7 ( @






+
	

7  6




7 7 

@,	

	
&


	74
5

*%&&'!
!$!

	(!
"$


)

2'0

"$

-.+"#

2#

&
%


 $
!

 

3 A?237 6( /	





>
 P

3 A?2337 6( /	





>
 P

3 A?23;7 6(7 /	





>
 P

3 A?23;76 6( /	





>
 P

< A7377 2( 0





+
	

7  6




7 7 

@,	

< A737%76 3(2 0





+
	

E7  6 2




7   6

@,	

< A737%7; 3(3 0





+
	

E7  6




7  

?


,
C


< A7227 ( 0





+
	

E7  6




7  

?


,
C


< A73272 ( 0





+
	

E7  6




7  

?


,
C


< A737% 3(6 /	





>
		
	
 >




< A737%6 3(2 /	





>
		
	
 >




< A737<72 ;(6 /	





+
	

E7  6




7  

@,	

< A7377 %( /	





+
	

E7  6




7  

@,	

< A7377; %(; /	





+
	

E7  6




7 7 

@,	

< 6 A73372 2(; /	





>
		
	
 >




< 6 A73373 2( /	





>
		
	
 >




	
*

	

	


	
*
	


	
&


	74
5

*%&&'!
!$!

	(!
"$


)

2'0

"$

-.+"#

2#

&
%


 $
!

 

< A7337 67(; /	





+
	

E7  6




7  

@,	

< A73337 67(< /	





+
	

E7  6




7  

@,	

< A7327 <( @






+
	

E7  6




7  

@,	

< A7272 (2 @






+
	

E7  6




7 7 

@,	

< A72 ( @






+
	

E7  6




7  

?


,
C


< A73676 7(; @






+
	

E7  6 2




7   6

@,	

< A7327 2(6 @






+
	

E7  6




7  

?


,
C


< A72%77 ;( @






>
 @,	

6 A7327 <( @






>
 @,	

6 A73;76 2(3 0





+
	

E7  6 2 3




7 7  6 2

@,	

6 A73;72 2(; 0





+
	

E7  6




7  

@,	

6 A7376 3(6 0





+
	

E7  6 2 3




7 7  6 6

@,	

6 A7372 3(2 0





+
	

E7  6 2 3




7  6 2 2

@,	

6 A73%76 (6 0





+
	

E7  6




7  

@,	

6 A73%72 (3 0





+
	

E7  6




7 7 

@,	

6 A73%7 (% 0





+
	

E7  6




7  

@,	

6 A7;<67 67( 0





>
 >',

	
&


	74
5

*%&&'!
!$!

	(!
"$


)

2'0

"$

-.+"#

2#

&
%


 $
!

 

6 A737<7 ;(7 /	





+
	

E7  6




7 7 

@,	

6 A73<73 %(3 /	





+
	

E7  6




7  

@,	

6 A73<7 %(7 /	





+
	

E7  6 2




7  6 6

@,	

6 A7<< %( /	





+
	

E7  6




7  

@,	

6 A73<6 %(6 /	





+
	

E7  6




7 7 

@,	

6 A7372 (2 /	





+
	

E7  6




7  

@,	

6 A7373 (3 /	





+
	

E7  6




7  

@,	

6 A737% (% /	





+
	

E7  6




7  

@,	

6 A7;%67 67(6 /	





>
 @,	

6 A73;7 2( @






+
	

E7  6 2




7  6 6

?


,
C


6 A7337 <(< @






+
	

E7  6




7 7 

@,	

6 A73373 <( @






+
	

E7  6




7  

@,	

6 A7;27 3(2 @






+
	

E7  6 2 3




7  6 2 2

@,	

6 A7;377 ;( @






+
	

E7  6




7 7 

@,	
	

7 )020 '"121&6'9"-60:'690

7 

!#!42,00#5

#		



*



	
	
	

9

			


	+8!D0


3&



	
	*
	

0	7 

	



*


	)*

 7		

	 


		







	

.		




*


7(		 	$		
0
	

	
		*



	




1	


	
	

	


	
	
	
	7

		
	

	



!
	

	7		
.

			
			




	
	*

		
		
		

	
	


	#$$
 = I*
 ", 3&

2(%			




	
	 *
		
	

		






	

(
**
!
!.@
!$!!*.)


	/"

*!
!/
$

2#% 2#% !
%%

(:


)










!

#!

(





0




6
%




 3< 2% 3 ;% ;

  73 26 2 2 3

 <7< 67 % % %7



/1				


7 


4) 19#5

	

(
	

 



	*



	
 


	
.
1+-./0(+8!D0


+	
&
	
	


	
	



		


	





	

,
 






*		

	
	


		

	

1

	
		
	


>
	

	



.
1I

7 '(
%$
#

(

.
	


 	

			
*		

	
	



2(<

2(
	
			


		


	


	

		
		
*		




	


				*	+8!D0	
	


		

	+8!D0	
	



%
3 7
< <2
6
	
	

	



	






	

+8!D0
	

	
	
*	



 
		
	


		
	 
<3


3 
< 7
6.		
		
	

		$
	



	
 	
		
			

	*	
	

	

)

")"
*)


	:)19;

) &
%
 (

 0
 6
%


3 0



/	



@




77

7%

7%

P

77

7%<

P

P

77

< 0



/	



@




77

76

7<

P

77

7<<

P

P

77

6 0



/	



@




77

7<%

7<6

P

77

7<7

P

P

77

	

/	
")"
*!
!!"

) &
%
 (

 0
 6
%


3 0



/	



@




77

7<3

7<3

P

77

7<3

P

P

77

< 0



/	



@




77

72

7

P

77

77

P

P

77

6 0



/	



@




77

76

77

P

77

7<

P

P

77






*


	

	
	
0

	

	



 	,

$(		

 		

		

 		

			





 /0&"2'6 1&""023'60

 &
%


@
'!
 	






	
		







	
	($






	*





*
*I

	*		
	

		

	*		



-./0



	 		
	*
	
	
*

		

	



	

 	*

	
	

*

	-./0



	




	6


	&*




*	(



* *			




	$
	
	
	



	. 
7(	(;77
*
	
	



	




	



'2 	-	.

	8
I-.8I&
	

			
-./0



	




*	(

*

								





	
'	
			



	

+	
	

*		
* -.8I

		



*	(
.	
		

	

	



	

	*



	


 		



	
		

		
	

-./03D	




	*





*	(
.	
 	
*


		
<


	
3 
6
	
<
	*


	


7(	(;77
		
		


 	
	





	


	
	
*	
	
		

		



 	-./0



	P**





*	(


P
*
	
	*	

	

	
7(	(277


	

	


	,

 	




*
		;7

		
	*2;+		
	
		
	;72;

	
 	



 	

	
				
		

)*

 

*	(


	

	


	
	


	
	*


			

	 	
		

		


	*


		



,



 	

	
	*


		;772;7	

		E

WDUJHWQ. FDOLEUDWHGNI

*
WDUJHW
	

		

FDOLEUDWHG
	

	


	
I'@890.!+.@/		
G.HGIH




+	
 
	2;			
		







2(67

	


)**
!
*
#*
*,
"*!
!!"

*)"

	!"

*(
	*
!
!.

) &
%
 ' -

3 0

 23 ;%

/	

 237 ;7%

@


 2;7 ;7;

< 0

 2;<; ;762

/	

 23; ;7;<

@


 2;3 ;76;

6 0

 2%% 3;

/	

 2; 3%%

@


 2; ;7

 #!

%

(

'			




 	
		





*


	







	
**
	







	




	 
&*
		
*

		

		

	



	
	

	
	
	!
!

,-*
	



2(6.	


6(3+	
6 	*
	


	



				

		

	
			
'




		

 	*
	*


		









1

	
	

 	
			I'@890.!+.@/


&

	


	* !,
"*!
!!",!
!)(


&
%
 ) ) )

0

 22 27 22

/	

 22 27 22

@


 22 37 22

, 	

				 		

	

1

			




*			(	
	



 			
	



		
	

	

	
	

	


	

		 
		

		


				






.
	
		*


C
		
		
	


	
		0!

"
"
("

!
!
*

!
!!(
*
"*!
!!"

!(


!
!!(
*
A!
!(*
!
!

A(*
!
!




B!

*
	

 ' ""12"2)1&/00"*'"120 1 :' "'20

,

 	

	 	


*		





+	



		
		


		
	

		
			
		


		




2(66






		

	
	 	
		
	

	*

(		
 0 /C	

&.	
	 	




9
		 


	
.
1/

	

 

 @
!")**
!
*!
!/
$



0
#


!
%:


$

) &
%


0

:




#!
 6(%

 0



/	



@




+	




7%6

7<

73

7<6

7<;

7%<

7<

7<

7;

766

7

 0



/	



@




+	


7

7<

7%

7%<

76

7

7<

73

77<

77

73

77

 0



/	



@




+	


72

7

7<7

7%

76

73

7<

7

77<

77

7

773


 "0200'/0 ;90"122'" 0


	(
	(
	
	*




		
	


	
	
	
			
	*			



	M



	C
	
*


		
 *	



			
*		
	



	*		

 *	


		





		
1	

	*

		(		
	

	% 27;	(
	
		
 <7

	*

	*	
				


C
	
66

	*

	*			 	

 		C
	


,	
	(
	
	
 <63

	*

	*	
				
C
	


3

	*

	*					C
	
*
		

		

	C
	

	(
	
	*
						
<	
	*

	

							
		+


	*	



,
3 	
		

				
 
			
 3

	

		
<2<

		
	
*

	*		
		


	
		

	



 /0,0")/&"60

:
		
		
	*
		

	
*
		





	'		
		*
	
*

-+
	


	/

.*


	#!
!"(*%&&'$	
!
!.
 

*	 ' "?

 <& 			
.

	
*			*

 
	

6
&
	
6&	*
	
*

1

	


	

.
 *	 ' "?

 <

	&

'		



	 	
6&*


. 	
*



	

	
	*	
	
	
	*		
/
	

	



	 

*

	
	 



		
	*
		
	*
	

	
*
		
*		

!
	
*
	

	
		
B
	

			
		
		





	
323

Chapter 14

DATA ANALYSIS FOR THE LONG-TERM TREND

READING ASSESSMENT1

Jo-Lin Liang and Lois H. Worthington

Educational Testing Service

14.1 INTRODUCTION

This chapter describes the analyses performed on the responses to the cognitive and background

items in the 1996 long-term trend reading assessment. These analyses led to the results presented in the

NAEP 1996 Trends in Academic Progress: Achievement of U.S. Students in Science, 1969 to 1996;

Mathematics, 1973 to 1996; Reading, 1971 to 1996; and Writing, 1984 to 1996 (Campbell, Voelkl, &

Donahue, 1997). The emphasis of this chapter is on the methods and results of procedures used to

develop the IRT-based scale scores that formed the basis of this report. The theoretical underpinnings of

the IRT and plausible values methodology described in this chapter are given in Chapter 11, and several

of the statistics are described in Chapter 9.

The objectives of the reading long-term trend analysis were to prepare scale values and perform

all analyses necessary to produce a long-term trend report in reading. The reading long-term trend results

include the years 1971, 1975, 1980, 1984, 1988, 1990, 1992, 1994, and 1996. The major analysis

components are discussed in turn. Some aspects of the analysis, such as procedures for item analysis,

scoring of constructed-response items, and methods of scaling, are described in previous chapters and are

therefore not detailed here.

The student samples that were administered reading items in the 1996 long-term trend reading

assessment are shown in Table 14-1. See Chapters 1 and 3 for descriptions of the target populations and

the sample design used for the assessment.

The long-term trend results reported in Trends in Academic Progress are based on print

administrations and occur at all of the age levels. The samples involved in the analysis are shown in

Table 14-1 as 9[RW-LTTrend], 13[RW-LTTrend], and 17[RW-LTTrend]. The long-term trend booklets

for these samples contained blocks of reading and writing items administered in print form. All students

received a block of common background questions, distinct for each age, and subject-area background

questions that were presented in the cognitive blocks. The booklets are identical to those used for longterm trend assessments in 1984, 1988, 1990, 1992, and 1994. The booklets and the blocks within those

booklets are listed in Chapter 4. Additional information about all of the items in these blocks also appears

in that chapter. This chapter includes specific information about the long-term trend items that were

scaled. Both age- and grade-selected students contributed to the long-term trend scaling. However, only

students in the “age-only” portion of the reading long-term trend samples contributed to the results

presented in Trends in Academic Progress.

 

1

 Jo-Lin Liang was the primary person responsible for the planning, specification, and coordination of the reading long-term

trend analyses, advised by Eiji Muraki, and Nancy L. Allen. Data analyses and scaling were performed by Lois H. Worthington,

advised by David S. Freund. Others contributing to the analysis of data were Bruce A. Kaplan, and Norma A. Norris. John R.

Donoghue provided consultation.
324

Table 14-1

NAEP 1996 Long-Term Trend Reading Student Samples

Sample

Booklet

IDs Mode

Cohort

Assessed

Time of

Testing

Age

Definition

Modal

Grade

Number

Assessed

 9 [RW-LTTrend]

13 [RW-LTTrend]

17 [RW-LTTrend]

51-56

51-56

51-56

Print

Print

Print

Age 9/Grade 4

Age 13/Grade 8

Age 17/Grade 11

1/3/96 - 3/8/96 (Winter)

10/9/95 - 12/22/95 (Fall)

3/11/96 - 5/10/96 (Spring)

CY

CY

Not CY

 4

 8

11

5,019

5,493

4,669

LEGEND

RW Reading and writing

LTTrend Long-term trend assessment

Print Print administration

CY Calendar year: birthdates in 1986, and 1982 for ages 9, and 13

Not CY Age 17 only: birthdates between October 1, 1978, and September 30, 1979

Table 14-2 clarifies the relationships between the 1996 long-term trend samples and samples

from previous years. For ages 9, 13, and 17, the [RW-LTTrend] samples allow direct comparisons with

1994, 1992, 1990, 1988, and 1984 samples. The long-term trend scale, established in 1984, was linked to

the 1971, 1975, and 1980 assessments using a complex equating strategy described in Implementing the

New Design: The NAEP 1983-84 Technical Report (Beaton, 1987). At each age, several intact booklets

were retained from the 1984 assessment, forming the basis of the reading long-term trend assessment in

1988, 1990, 1992, 1994, and 1996. Information about the 1988 assessment is available in Focusing the

New Design: The NAEP 1988 Technical Report (Johnson & Zwick, 1990); information about the 1990

assessment is given in The NAEP 1990 Technical Report (Johnson & Allen, 1992); information about the

1992 assessment is given in The NAEP 1992 Technical Report (Johnson & Carlson, 1994); and

information about the 1994 assessment is given in The NAEP 1994 Technical Report (Allen, Kline, &

Zelenak, 1996).

The 1996 long-term trend included, at each age level, six of the assessment booklets

administered in 1984. These booklets (51-56) contained both reading and writing blocks, as well as

background items. Although these long-term trend booklets represented only about one-tenth of the

reading booklets administered in the complex 1984 BIB design,2

 they contained 10 of the 12 reading

blocks that were scaled at each age/grade level in 1984. The samples of students who received these

long-term trend booklets are described in Table 14-1 and in Chapter 3. The purpose of the reading longterm trend analysis was to add to the reading trend results that extended from 1971 to 1994 for ages 9,

13, and 17. The numbers of scaled items for each age are presented in Table 14-3. Each age was scaled

separately. The numbers of items scaled in 1996 that were common across assessment years are given in

Table 14-4. As was the case for previous long-term trend analyses, the long-term trend scale is univariate.

Dimensionality analyses conducted following the 1984 assessment showed that the reading items were

well summarized by a unidimensional scale (Zwick, 1987).

 

2

 The long-term trend assessment included 1984 Booklets 16, 17, 27, 34, 55, and 60 at age 9 and Booklets 13, 16, 17, 21, 34, and

57 at ages 13 and 17 (see J. R. Johnson, 1987, pp. 120-121). The 1984 main assessment focused-BIB design included 57

booklets that contained at least one scaled reading block at age 9 and 56 such booklets at ages 13 and 17.
325

Table 14-2

NAEP Reading Samples Contributing to 1996 Long-Term Trend Results, 1971-1996

Cohort Year Sample Subjects

Time of

Testing

Mode of

Administration

Age

Definition

Modal

Grade

Age 9 1971

1975

1980

1984

1984

1988

1990

1992

1994

1996

Main

Main

Main

Main

T-84

LTTrend1

LTTrend1

LTTrend1

LTTrend1

LTTrend1

RL

RA

RA

RW

RW

RW

RW

RW

RW

RW

Winter

Winter

Winter

Winter, Spring

Winter

Winter

Winter

Winter

Winter

Winter

Tape

Tape

Tape

Print

Tape

Print

Print

Print

Print

Print

CY

CY

CY

CY

CY

CY

CY

CY

CY

CY

4

4

4

4

4

4

4

4

4

4

Age 13 1971

1975

1980

1984

1984

1988

1990

1992

1994

1996

Main

Main

Main

Main

T-84

LTTrend1

LTTrend1

LTTrend1

LTTrend1

LTTrend1

RL

RA

RA

RW

RW

RW

RW

RW

RW

RW

Fall

Fall

Fall

Winter, Spring

Fall

Fall

Fall

Fall

Fall

Fall

Tape

Tape

Tape

Print

Tape

Print

Print

Print

Print

Print

CY

CY

CY

CY

CY

CY

CY

CY

CY

CY

8

8

8

8

8

8

8

8

8

8

Age 17 1971

1975

1980

1984

1984

1988

1990

1992

1994

1996

Main

Main

Main

Main

T-84

LTTrend1

LTTrend1

LTTrend1

LTTrend1

LTTrend1

RL

RABS

RA

RW

RW

RW

RW

RW

RW

RW

Spring

Spring

Spring

Winter, Spring

Spring

Spring

Spring

Spring

Spring

Spring

Tape

Tape

Tape

Print

Tape

Print

Print

Print

Print

Print

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

11

11

11

11

11

11

11

11

11

11

1

Note: Within a cohort, these samples received common booklets.

LEGEND

RL

RA

Reading and literature

Reading and art

LTTrend Long-term trend (these samples received

common booklets within an age group)

RABS Reading, art, index of basic skills Print Print administration

RW Reading and writing Tape Audiotape administration

Main

T-84

Main assessment

Special sample in the 1984 assessment that was

used to establish links to previous assessments

(1971-1980) for the purposes of long-term trend

CY

Not CY

Calendar year: birthdates (1996 sample) in 1986

and 1982 for ages 9 and 13

Age 17 only (1996 sample): birthdates between

October 1 and September 30 of the appropriate

years
326

Table 14-3

Numbers of Scaled Reading Long-Term Trend

Items Common Across Ages

Age Number of Items

9 only

13 only

17 only

9 and 13 only

9 and 17 only

13 and 17 only

9, 13, and 17

611

221

23

13

2

42

261

Total 1891

1

 These figures have been updated since their

publication in the 1992 and 1994 NAEP technical

reports (Table 12-3 and Table 15-3, respectively).

Table 14-4

Numbers of Scaled Reading Long-Term Trend Items Common Across Assessments

Number of Items

Assessment Year Age 9 Age 13 Age 17

 1984, 1992, 1994, 1996

 1984, 1990, 1992, 1994, 1996

 1984, 1988, 1990, 1992, 1994, 1996

 1980, 1984, 1988, 1990, 1992, 1994, 1996

 1971, 1975, 1980, 1984, 1988, 1990, 1992, 1994, 1996

102

101

 98

 67

 36

103

101

 98

 71

 45

931

921

87 

52 

37 

1

 These figures have been updated since their publication in the NAEP 1992, and 1994 Technical Reports

(Table 12-4, and Table 15-4, respectively).

The steps in the reading long-term trend analysis are documented in the following sections. As is

usual in NAEP analyses, the first step was to gather item and block information. The trend items were

then calibrated according to the IRT model. Plausible values were generated after conditioning on

available background variables. Finally, the scale values were placed on the final reading long-term trend

scale used in previous trend assessments.

14.2 ITEM ANALYSIS FOR THE READING LONG-TERM TREND ASSESSMENT

Conventional item analyses did not identify any difficulties with the long-term trend data. The

results displayed in Table 14-5 contain the number of items, size of the unweighted sample administered

the block, average weighted proportion correct, average weighted r-biserial, and average weighted alpha

as a measure of reliability for each block. Because the blocks were presented in self-paced, printadministered form, the weighted proportion of students attempting the last item is included in the table to

give an indication of the speededness of each block. Common labeling of these blocks across ages does

not denote common items. Booklet information is detailed in Chapter 4. Student weights were used for
327

all statistics except for the sample sizes. The average values reflect only the items in the block that were

scaled. Overall, the 1996 item-level statistics were not very different from those for the 1994 assessment.

Table 14-5

Descriptive Statistics for Item Blocks in the Reading Long-Term Trend Samples

Blocks

Statistics B8 B10 B11 B12 B13 B14 B15 B16 B17 B18 B22

Age 9

Number of scaled items

Number of scaled

 constructed-response items

Unweighted sample size

Average weighted proportion

 correct

Average weighted r-biserial

Weighted alpha reliability

Weighted proportion of students

 attempting last item

10

1

630

.62

.75

.75

.92

8

0

610

.43

.62

.67

.82

11

0

610

.43

.62

.71

.77

7

1

603

.50

.82

.76

.72

11

1

624

.41

.61

.71

.66

12

1

624

.60

.75

.84

.66

11

0

595

.51

.60

.64

.86

—1

—1

—1

—1

—1

—1

—1

11

0

603

.56

.74

.81

.86

12

0

1222

.47

.64

.75

.82

9

3

592

.60

.73

.74

.97

Age 13

Number of scaled items

Number of scaled

 constructed-response items

Unweighted sample size

Average weighted proportion

 correct

Average weighted r-biserial

Weighted alpha reliability

Weighted proportion of students

 attempting last item

12

1

629

.64

.71

.71

.94

9

0

642

.59

.59

.60

.83

8

0

630

.63

.73

.68

.98

5

0

673

.64

.80

.57

.95

11

1

630

.58

.65

.66

.91

12

1

615

.66

.68

.78

.81

10

1

658

.66

.64

.55

.84

9

1

642

.73

.77

.67

.92

16

0

673

.59

.56

.70

.78

11

0

629

.67

.74

.77

.97

—2

—2

—2

—2

—2

—2

—2

Age 17

Number of scaled items

Number of scaled

 constructed-response items

Unweighted sample size

Average weighted proportion

 correct

Average weighted r-biserial

Weighted alpha reliability

Weighted proportion of students

 attempting last item

12

1

605

.72

.75

.72

.95

4

1

638

.72

.78

.43

.97

8

0

585

.79

.83

.67

1.00

6

1

643

.72

.81

.41

.96

11

1

585

.68

.76

.68

.96

12

1

588

.84

.78

.80

.89

13

1

622

.65

.62

.73

.69

10

1

638

.74

.71

.74

.82

10

0

643

.53

.63

.70

.72

7

0

605

.68

.79

.70

.98

—2

—2

—2

—2

—2

—2

—2

1

Block B16 was not administered at age class 9.

2 Block B22 was not administered at age class 13 or 17.
328

14.3 TREATMENT OF CONSTRUCTED-RESPONSE ITEMS

Data for constructed-response items in the long-term trend analysis were used for the 1984, 1990,

1992, 1994, and 1996 assessments only. Constructed-response items were not included in the original

scoring of the 1988 reading assessment because a previous study (Zwick, 1988) had shown that scoring

inconsistencies (drops in interrater reliability and/or scorer drift—that is, scorers showed evidence of

rating items more strictly or more leniently than did the original 1984 scorers) had affected these items.

Rater reliability within year was computed for the 1996 constructed-response items at each age.

Between year reliability was also studied with the 1994, and the 1984 responses. In general, the 1996

scoring did not show irregularities. All of the 1996 constructed-response items were used in the trend

analysis except the items that were excluded from calibration in the previous assessment. The deleted

items are listed in Table 14-6. The remaining constructed-response items were dichotomized according to

criteria developed by subject-area experts. The dichotomized versions of the constructed-response items

were included in the calibration.

Table 14-6

Items Deleted from the Reading Long-Term Trend Analysis

Age Block Item Reason for Exclusion

9 B10

B13

B10

N001801

N003003

N008905

Excluded in previous assessment

Excluded in previous assessment

Excluded in previous assessment (constructed-response item)

13 B10

B10

B11

B12

B17

N001801

N001904

N002302

N002804

N005001

Excluded in previous assessment

Excluded in previous assessment (constructed-response item)

Excluded in previous assessment

Excluded in previous assessment (constructed-response item)

Excluded in previous assessment

17 B10

B11

B17

N001702

N002302

N015905

Excluded in previous assessment

Excluded in previous assessment

Excluded in previous assessment (constructed-response item)

14.4 IRT SCALING FOR THE READING LONG-TERM TREND ASSESSMENT

14.4.1 Item Parameter Estimation

The first step in the scaling process was the estimation of item parameters for the long-term trend

items. This item calibration was performed using the BILOG/PARSCALE program described in

Chapter 11. Items were calibrated separately for each of the three age/grade groups. Item parameters

were estimated using combined data from the assessment years 1994 and 1996, treating each assessment

as a sample from a separate subpopulation. Student weights were used for the calibration. To ensure that

each assessment year had a similar influence on the calibration, student weights for 1994 examinees were

multiplied by a constant, to adjust them to have the same sum as the sum of the weights for the 1996

examinees. Approximately 600-700 examinee responses for each item were present in each assessment

year.
329

Starting values for item parameters were based on the final item parameter values from the

analysis of the 1994 long-term trend assessment. As described in Chapter 9, BILOG/PARSCALE

calibrations were completed in two stages. At stage one, the proficiency distribution of each assessment

year was constrained to be normal, although the means and variances differed across assessment years.

The values of the item parameters from this normal solution were then used as starting values for a

second-stage estimation run in which the proficiency distribution (modeled as a separate multinomial

distribution for each assessment year) was estimated concurrently with item parameters. Calibration was

concluded when changes in item parameters became negligibly small (i.e., less than .005).

14.4.2 Evaluation of Model Fit

During and subsequent to item parameter estimation, evaluations of the fit of the IRT models

were carried out for each of the items. These evaluations were based primarily on graphical analysis.

First, model fit was evaluated by examining plots of nonmodel-based estimates of the expected

proportion correct (conditional on proficiency) versus the proportion correct predicted by the estimated

item response function (see Chapter 9, and Mislevy & Sheehan, 1987, p. 302). In making decisions about

excluding items from the final scales, a balance was sought between being too stringent, hence deleting

too many items and possibly damaging the content representativeness of the pool of scaled items, and

being too lenient, hence including items with model fit poor enough to endanger the types of modelbased inferences made from NAEP results. A certain degree of misfit was tolerated for a number of items

included in the final scales.

Most of the items fit the model well. Items excluded from the analysis of the 1996 assessment

were the same items that were deleted from the 1994 reading long-term trend analysis. Table 14-6 lists

items that were excluded from the analysis of the 1996 long-term trend assessment.

The adequacy of the assumption of a common item response function across assessment years

was also evaluated by comparing the nonmodel-based expected proportions for each assessment year to

the single, model-based item response function fit by BILOG/PARSCALE. Items that showed clear

evidence of functioning differently across assessments were treated as separate items for each assessment

year--that is, separate item response functions were estimated for each assessment. As was the case with

deleting items, in making decisions about scaling items separately by assessment year, a balance was

sought between being too stringent, hence splitting too many items and possibly damaging the common

item link between the assessment years, and being too lenient, hence including items with model fit poor

enough to endanger the model-based trend inferences. These separately scaled items will be reexamined

in future long-term trend assessments.

At age 9, one constructed-response item was calibrated separately for each assessment year.

Examination of residual plots identified the item as functioning differently across assessments. Figure

14-1 shows item N014502 from the analysis for grade 4/age 9. Data are presented for 1994 (squares), and

for 1996 (asterisks)3

. For middle proficiency values, the two sets of symbols diverge, and the discrepancy

of the item characteristic curves of the two years is substantial. The top (1994 data), and the bottom

(1996 data) of Figure 14-2 show the plots for the item treated separately by assessment year. The 1996

data showed poorer fit. In order to maintain the link for the trend, this item was kept in the analysis but

with the 1994 data calibrated separately and the 1996 data excluded from the final calibration. The

remaining misfit is relatively small. Overall, one long-term trend reading item was calibrated separately

by assessment year. Table 14-7 lists the item that was calibrated separately across assessment years.

 

3

 The size of the symbols are proportional to the estimated number of students at a particular scale score level. The symbols are

ordinarily larger in the middle of the theta scale, where most students’ scale scores fall.
330

Figure 14-1

Example Long-Term Trend Item (N014502, Age 9)

Demonstrating Differential Item Functioning Across Assessment Years 1994 and 19961

1

This plot compares empirical and model-based estimates of the item response function (IRF). The smooth curve

represents the model-based estimate at each provisional proficiency level. The squares represent 1994 data;

asterisks represent 1996 data.

Note: When the number of alternatives of a constructed-response item equaled zero, the item was scored in only

two categories.
331

Figure 14-2

Example Long-Term Trend Item (N014502, Age 9)

Fitting Separate Item Response Functions for Each Assessment Year1

1

The plot compares empirical and model-based estimates of the item response function (IRF). The smooth curve

represents the model-based estimate at each provisional proficiency level. The squares represent 1994 data;

asterisks represent 1996 data.

Note: When the number of alternatives of a constructed-response item equaled zero, the item was scored in only

two categories.
332

Table 14-7

Item Calibrated Separately by Assessment Year in the Reading Long-Term Trend Analysis

Age Block Item Reason for Separate Calibration

 9 B22 N014502 Poor fit across assessments to common item response function

At age 17, two items (N002201 and N002202) caused difficulty in scaling. In preliminary

calibrations, both items did not fit the model well and had large slope-parameter values (3.9 and 5.0,

respectively). The item response function of N002202 also demonstrated an elevated tail. Further

examination of the items indicated that this might be due to local dependence of the items, although

neither item had been problematic at this age group in the 1994 assessment. The approach of fixing the

slope-parameter was taken to obtain stable item parameter estimates. At calibration stage-two, after the

estimation of the proficiency distribution was constrained to be normally distributed and calibrated to

convergence, the slope-parameter of N002201 was fixed at the value, and all items were calibrated to

convergence. Parameters estimates from this run served as the final estimates for age 17.

A list of the items scaled for each of the ages, along with their item parameter estimates, appears

in Appendix D.

14.5 GENERATION OF PLAUSIBLE VALUES

The generation of plausible values was conducted independently for each age/grade level for

each of the assessment years. The item parameters from BILOG/PARSCALE, final student weights, item

responses, and selected background variables were used with the computer program BGROUP (described

in Chapter 11) to generate the values for each age. The background variables included student

demographic characteristics (i.e., race/ethnicity of the student, highest level of education attained by

parents), students’ perceptions about reading, and student behavior both in and out of school (e.g.,

amount of television watched daily, amount of homework done each day). Appendix C gives the codings

for the conditioning variables for the three age groups. Table 14-8 contains a list of the number of

background contrasts included in conditioning, as well as the proportion of variance accounted for by the

conditioning model for each age/grade.

Table 14-8

Proportion of Proficiency Variance Accounted for by the Conditioning Model

for the Reading Long-Term Trend Assessment

Age/Grade

Number of Conditioning

Contrasts1

Proportion of Proficiency

Variance

9/4 49 .32

13/8 49 .35

17/11 47 .34

1

 Excluding the constant term.
333

14.6 THE FINAL READING LONG-TERM TREND SCALE

The linear indeterminacy of the long-term trend scale was resolved by linking the 1996 long-term

trend scales to previous long-term trend scales. For each age, the item parameters from the joint

calibration based on data from both 1994 and 1996 were used with the 1994 data to reestimate plausible

values for the 1994 data. The mean and standard deviation of the new 1994 estimates were calculated and

matched to the mean and standard deviation of the old 1994 plausible values that were reported

previously. The linear constants of this transformation were then applied to transform the 1996 scales to

the 1994 proficiency metric. The transformation equations (described in Chapter 9) that resulted from

this matching of the first two moments for the 1994 data are

Age 9: target = 52.46 • calibrated + 206.38,

Age 13: target = 39.73 • calibrated + 256.65, and

Age 17: target = 44.13 • calibrated + 283.89,

where target denotes values on the final transformed scale, and calibrated denotes values on the calibration

scale. Overall summary statistics for the long-term trend samples are given in Table 14-9.

As in the past, interpretation of the long-term trend results was facilitated through the provision

of scale anchoring information. In 1984, five NAEP reading scale levels were selected as anchor points.

These points (described in Trends in Academic Progress) are:

 150 = simple, discrete reading tasks;

200 = partially developed skills and understanding;

250 = interrelation of ideas and generalizations;

300 = understanding complicated information; and

350 = learning from specialized reading materials.

Detailed descriptions of the skills required to read at each level were derived and benchmark

exercises were selected to exemplify each level. These same anchor points were used in the 1988, 1990,

1992, 1994, and 1996 reading long-term trend reports. The estimated proportion of students in each

reporting category who are at or above each anchor point was examined in Trends in Academic Progress.

14.7 PARTITIONING OF THE ESTIMATION ERROR VARIANCE

For each age, the error variance of the final, transformed proficiency mean was partitioned into

two parts as described in Chapter 11. This analysis yielded estimates of the proportion of error variance

due to sampling students, and the proportion of error variance due to the latent nature of . These

estimates are given in Table 14-10 (for stability, the estimates of the between-imputation variance, B, in

Equation 11.9 are based on 100 imputations). More detailed information is available for gender and

race/ethnicity subgroups in Appendix E.
334

Table 14-9

Means and Standard Deviations on the Reading Long-Term Trend Scale

Assessment All Five Plausible Values

Age Year Mean S.D.

9 1984

1988

1990

1992

1994

1996

211.0

211.8

209.2

210.5

211.0

212.4

41.1

41.2

44.7

40.4

40.5

39.0

13 1984

1988

1990

1992

1994

1996

257.1

257.5

256.8

259.8

257.9

257.9

35.5

34.7

36.0

39.4

39.8

39.1

17 1984

1988

1990

1992

1994

1996

288.8

290.1

290.2

289.7

288.1

287.6

40.3

37.1

41.3

43.0

44.4

42.2

Table 14-10

Estimation Error Variance and Related Coefficients

for the Reading Long-Term Trend Assessment

Proportion of Variance Due to ...

Age

Total Estimation of

Error Variance Student Sampling Latency of 

9 1.01 0.86 0.14

13 0.93 0.85 0.15

17 1.08 0.82 0.18
335

Chapter 15

DATA ANALYSIS FOR THE LONG-TERM TREND

MATHEMATICS ASSESSMENT1

Jiahe Qian and Norma A. Norris

Educational Testing Service

15.1 INTRODUCTION

This chapter describes the analyses performed on the responses to the cognitive and background

items in the 1996 long-term trend assessment of mathematics. The emphasis of this chapter is on the

methods and results of procedures used to develop the IRT-based scale scores; however, some attention

is given to the analysis of constructed-response items. The theoretical underpinnings of the IRT and

plausible values methodology described in this chapter are given in Chapter 11.

The objectives of the mathematics analyses were to prepare scale values and perform all analyses

necessary to produce a long-term trend report in mathematics. The mathematics long-term trend results

include the years 1973, 1978, 1982, 1986, 1990, 1992, 1994, and 1996. The results of 1996 long-term

trend assessment of mathematics are presented in the NAEP 1996 Trends in Academic Progress:

Achievement of U.S. Students in Science, 1969 to 1996; Mathematics, 1973 to 1996; Reading, 1971 to

1996; and Writing, 1984 to 1996 (Campbell, Voelkl, & Donahue, 1997).

The student samples that were administered mathematics items in the 1996 long-term trend

assessment are shown in Table 15-1. See Chapters 1 and 3 for descriptions of the target populations and

the sample design used for the assessment.

Table 15-1

NAEP 1996 Mathematics Long-Term Trend Student Samples

Sample Booklet IDs Mode

Cohort

Assessed

Time of

Testing

Age

Definition

Modal

Grade

Number

Assessed

9 [MS-LTTrend]

13 [MS-LTTrend]

17 [MS-LTTrend]

91-93

91-93

84-85

Tape

Tape

Tape

Age 9

Age 13

Age 17

1/3/96 - 3/8/96 (Winter)

10/9/95 - 12/22/95 (Fall)

3/11/96 - 5/10/96 (Spring)

CY

CY

Not CY

 4

 8

11

5,414

5,658

3,539

LEGEND

MS Mathematics and science

LTTrend Long-term trend assessment: booklets are identical to 1986 long-term trend assessments

Tape Audiotape administration

CY Calendar year: birthdates (1996 sample) in 1986 and 1982 for ages 9 and 13

Not CY Age 17 only (1996 sample): birthdates between October 1 and September 30 of the appropriate years

 

1 Jiahe Qian was the primary person responsible for the planning, specification, and coordination of the mathematics long-term

trend analyses. Computer activities for all long-term trend mathematics scaling and data analyses were performed by Norma

Norris. Nancy Allen and Eiji Muraki provided consultation.
336

Data from the 1996 long-term trend samples that contributed to the trends in mathematics achievement

were scaled separately from the 1996 mathematics main samples. Accordingly, the long-term trend

analysis and main analysis are presented in separate chapters. This chapter pertains to the scaling of the

long-term trend data; information about the scaling of the data from the mathematics main assessment

samples is presented in Chapter 12.

The long-term trend results reported in the NAEP 1996 Trends in Academic Progress are based

on paced-tape administrations and occur at all age levels. The samples involved in the analysis are shown

as 9[MS-LTTrend], 13[MS-LTTrend], and 17[MS-LTTrend] in Table 15-1. For ages 9 and 13, the longterm trend booklets contained blocks of reading, mathematics and science items. In the assessments, the

mathematics and science blocks were administered by audiotape to pace the students through blocks (the

reading blocks were only presented in print form). The age 17 long-term trend booklets contained only

mathematics and science blocks, both administered by paced-tape recordings as well. All students

received a block of common background questions, yet distinct for each age. Subject-area background

questions were presented in the cognitive blocks. The booklets for the age 9 and age 13 samples

(Booklets 91-93) are the same as those used for long-term trend assessments in 1986, 1988, 1990, 1992,

and 1994. The booklets for the age 17 sample (Booklets 84-85) are the same as those used for the 1986,

1990, 1992, and 1994 long-term trend assessments. The booklets and the blocks within those booklets are

listed in Tables 4-20 through 4-22 in chapter 4.

Table 15-2 clarifies the relationships among the 1996 long-term trend samples and samples from

previous years. For ages 9, 13, and 17, the paced-tape bridge to the 1986 samples allows direct

comparisons between the samples from the long-term assessments after 1990 and the 1986 long-term

trend samples. There was also a paced-tape administration in 1988, at ages 9 and 13, that was comparable

to the other years. However, a paced-tape administration was not conducted at age 17 in 1988. Instead, a

noncomparable paper-based assessment was administered. Hence, 1988 is not included as a point in the

long-term trend reporting. In 1986, the mathematics long-term trend items were scaled with common

items from the 1978 and 1982 assessments. Because the 1973 assessment had few items in common with

the current assessment, data from that assessment was not scaled using the IRT model but was linked to

the trend line by a linear transformation involving the mean proportion correct for common items (See

Expanding the New Design: The NAEP 1985-86 Technical Report (Beaton, 1988)). When comparisons

were made including the 1970 and 1973 assessment results, z-tests rather than t-tests were used to test

statistical significance (See Section 18.5.1). The 1996 long-term trend assessment was linked to the

1973, 1978, and 1982 assessments through the 1986 assessment. Information about previous assessment

years is available in Expanding the New Design: The NAEP 1985-86 Technical Report (Beaton, 1988),

The NAEP 1992 Technical Report (Johnson & Carlson, 1994), and The NAEP 1994 Technical Report

(Allen, Kline, & Zelenak, 1996).

Table 15-3 indicates the number of items in common across different age combinations. Table

15-4 shows the number of items scaled in 1996 that were common across assessment years. The 1986,

1990, 1992, 1994, and 1996 assessments had all items in common. For age 9, the number of items

common across assessment years 1978 to 1996 was only 35; for age 13, it was 56; and for age 17, it was

54. Item parameters were estimated assuming a univariate scale, since the number of items presented to

each student was small and there were too few items to estimate several content area scales separately.
337

Table 15-2

NAEP Mathematics Samples Contributing to 1996 Long-Term Trend Results, 1973-1996

Cohort

Assessed Year Sample Subjects

Time of

Testing

Mode of

Administration

Age

Definition

Modal

Grade

Age 9 1973

1978

1982

1986

1990

1992

1994

1996

Main

Main

Main

LTTrend1

LTTrend1

LTTrend1

LTTrend1

LTTrend1

MS

M

MCS

MS

MS

MS

MS

MS

Winter

Winter

Winter

Winter

Winter

Winter

Winter

Winter

Tape

Tape

Tape

Tape2

Tape2

Tape2

Tape2

Tape2

CY

CY

CY

CY

CY

CY

CY

CY

4

4

4

4

4

4

4

4

Age 13 1973

1978

1982

1986

1990

1992

1994

1996

Main

Main

Main

LTTrend1

LTTrend1

LTTrend1

LTTrend1

LTTrend1

MS

M

MCS

MS

MS

MS

MS

MS

Fall

Fall

Fall

Fall

Fall

Fall

Fall

Fall

Tape

Tape

Tape

Tape2

Tape2

Tape2

Tape2

Tape2

CY

CY

CY

CY

CY

CY

CY

CY

8

8

8

8

8

8

8

8

Age 17 1973

1978

1982

1986

1990

1992

1994

1996

Main

Main

Main

LTTrend1

LTTrend1

LTTrend1

LTTrend1

LTTrend1

MS

M

MCS

MS

MS

MS

MS

MS

Spring

Spring

Spring

Spring

Spring

Spring

Spring

Spring

Tape

Tape

Tape

Tape

Tape

Tape

Tape

Tape

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

11

11

11

11

11

11

11

11

1

 Within an age group, these samples received common booklets. 2

 Mathematics and science administered by audiotape, reading administered by print.

LEGEND

M

MS

MCS

LTTrend

Mathematics

Mathematics and science

Mathematics, civics, and science

Long-term trend: booklets are

identical to the long-term trend

assessment of 1986.

Main

Tape

CY

Not CY

Main assessment

Audiotape administration

Calendar year: birthdates (1996 sample) in

1986 and 1982 for ages 9 and 13

Age 17 only (1996 sample): birthdates

between October 1 and September 30 of the

appropriate years

The steps in the mathematics long-term trend analysis are documented in the following sections.

Consistent with the procedures in earlier NAEP analyses, the first step was to calculate standard item

statistics. The results served as a check for data entry errors and as a reasonableness check against results

from previous assessments. 

The second step was to fit an IRT model to the data from the 1996 and 1994 assessments for each

age separately. This procedure puts item parameters and ability estimates on the same scale across years.

The same item may have different item parameters for different age groups.
338

Table 15-3

Numbers of Scaled Mathematics Long-Term Trend

Items Common Across Ages

Age Booklet Numbers Number of Items

 Total 153

 9 only

 13 only

 17 only

 9 and 13 only

 9 and 17 only

 13 and 17 only

 9, 13, and 17

91-93

91-93

84-85

91-93, 91-93

91-93, 84-85

91-93, 84-85

91-93, 91-93, 84-85

 32

 30

 41

 20

 0

 27

 3

Table 15-4

Numbers of Scaled Mathematics Long-Term Trend Items

Common Across Assessments

Number of Items

Assessment Year Age 9 Age 13 Age 17

 1986, 1990, 1992, 1994, 1996

 1982, 1986, 1990, 1992, 1994, 1996

 1978, 1986, 1990, 1992, 1994, 1996

 1978, 1982, 1986, 1990, 1992, 1994, 1996

55

53

35

35

80

79

56

56

71

65

54

54

Next, the analysis for an age group was completed by the creation of plausible values through a

multiple imputation estimation procedure in which item parameter estimates, student responses, and

student background information were combined to produce the most precise possible estimates of student

subgroup ability. Plausible values were used to calculate proficiency means for the entire sample and for

the selected subgroups.

Finally, the scales of the 1996 trend assessment were transformed to proficiency scale used in

previous mathematics trend assessments. These proficiency means constitute the last point in the

mathematics long-term trend from 1973 to 1996. The only available estimates of the proficiency means

for 1973 were linked via extrapolation to the IRT scale, but the data from that year was never scaled

using an IRT model.

15.2 ITEM ANALYSIS FOR THE MATHEMATICS LONG-TERM TREND 

ASSESSMENT

No problems in coding, formats, or data were detected. The conventional item analysis, with

results displayed in Table 15-5, was performed at the block level on the paced-tape long-term trend data.

Table 15-5 contains the number of items, size of the sample administered to the block, mean

weighted proportion correct, mean weighted r-biserial, and mean weighted alpha as a measure of
339

reliability for each block. The average values were calculated using examinee weights and the items in

the block that were scaled. The 1996 item-level statistics were not very different from those for the 1994

assessment. Similar statistics for the 1994 assessment were reported in Table 16-5 of The NAEP 1994

Technical Report.

Table 15-5

Descriptive Statistics for Item Blocks in the

Mathematics Long-Term Trend Samples (1996)

Block

Statistic M1 M2 M31

Age 9

Number of scaled items

Number of scaled constructed response items

Unweighted sample size

Average weighted proportion correct

Average weighted r-biserial

Weighted alpha reliability

 24

 9

1,852

.62

.61

.82

 26

 9

1,841

.64

.65

.86

 5

 0

1,721

.68

.83

.50

Age 13

Number of scaled items

Number of scaled constructed response items

Unweighted sample size

Average weighted proportion correct

Average weighted r-biserial

Weighted alpha reliability

 36

 9

1,928

.68

.59

.87

 36

 8

1,864

.62

.55

.85

 8

 0

1,866

.66

.68

.61

Age 17

Number of scaled items

Number of scaled constructed response items

Unweighted sample size

Average weighted proportion correct

Average weighted r-biserial

Weighted alpha reliability

 33

 10

1,848

.66

.69

.90

 33

 5

1,848

.67

.63

.87

 5

 1

1,691

.56

.76

.54

1

 This block is mostly calculator items, which were not analyzed. For the item analysis, students who

did not respond to any items in the block were omitted; however, such students were assigned

proficiencies in the final database.

In the 1996 mathematics long-term trend assessment, 20 percent of the samples of the

constructed-response items were used to check the interrater reliability—the score agreement between

first and second raters. The percent of exact agreement ranged from 96.3 to 100 percent; and the

intraclass correlation ranged from .902 to 1.00—except .886 for item N269201 in the age 13 sample. In

general, the interrater reliability was very high in the 1996 mathematics long-term trend assessment.

The correspondence between blocks, booklets, and samples is given for the mathematics longterm trend assessment in Tables 4-20 through 4-22 in Chapter 4. Common labeling of these blocks across

ages does not denote common items.
340

15.3 IRT SCALING FOR THE MATHEMATICS LONG-TERM TREND 

ASSESSMENT

15.3.1 Item Parameter Estimation

The scaling process began with the estimation of item parameters. IRT parameters were

estimated using the NAEP version of the BILOG/PARSCALE program (Mislevy & Bock, 1982; Muraki

& Bock, 1991) described in Chapter 11. Item calibration was performed separately for each of the three

age groups, using the total combined data from the 1994 and 1996 assessments. Including the 1994

assessment data assures that item parameters will be similar for adjacent assessments so that year-to-year

trends will not be distorted by abrupt changes in calibration. The calibration was performed on the entire

sample of students, resulting in a range of about 1,700 to 1,900 examinee responses to each item in each

assessment year. The calibration was based on student weights that were rescaled for the 1996 data so

that the sum of the weights equaled the unweighted sample size. Also, weights for the 1996 data were

restandardized to give equal weight to the two assessment years included in the scaling. As with the

previous assessment, calculator items were excluded from the analysis. Because calculators have changed

greatly since the start of the long-term trend assessment, it was judged that calculator questions are no

longer comparable across time. These items were kept in the assessment, since excluding them would

have changed the testing context.

Since parameters for items in blocks M1, M2, and M3 were estimated separately for ages 9, 13,

and 17, items administered at more than one age have multiple sets of item parameter estimates. Items

were examined for lack of fit with the data. Those that exhibited extreme violation of IRT assumptions

(i.e., did not have monotonically increasing item characteristic curves) were deleted from the analysis, as

they were in previous assessments. Other items were deleted because they were calculator items, which

were not considered part of the regular assessment. These excluded items appear in Tables 15-6, 15-7,

and 15-8. As a result of these deletions, 55 items were scaled for age 9, 80 items were scaled for age 13,

and 71 items were scaled for age 17. Of the 153 noncalculator items that were part of the assessment,

seven items (5%) were excluded due to poor fit with the data. A list of the items scaled for each of the

ages, along with their item parameter estimates, appears in Appendix D.

15.4 DERIVED BACKGROUND VARIABLES

In the long-term trend analysis, all derived background variables were used to define subgroups of

students for reporting. For this reason, these variables were also used in conditioning. Derived reporting

variables are described in Appendix B.
341

Table 15-6

Items Deleted from the Age 9 Mathematics Long-Term Trend Analysis

Booklet

IDs Block Item Reason for Exclusion

91 M1 N252601

N262502

 Was deleted in prior assessment

 Was deleted in prior assessment

92 M3 N268221

N276021

N276022

N276821

N276822

N276823

N277621

N277622

N277623

N284021

N284022

 Calculator item1

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

1 All calculator items were deleted from the analysis.

Table 15-7

Items Deleted from the Age 13 Mathematics Long-Term Trend Analysis

Booklet

IDs Block Item Reason for Exclusion

91 M1 N262502 Was deleted in prior assessment

93 M2 N261601 Was deleted in prior assessment

92 M3 N264521

N259921

N276821

N276822

N276823

N278921

N278922

N278923

N278924

N278925

N280621

N280622

N280623

N280624

N280625

N280626

 Calculator item1

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

1 All calculator items were deleted from the analysis. 
342

Table 15-8

Items Deleted from the Age 17 Mathematics Long-Term Trend Analysis

Booklet

IDs Block Item Reason for Exclusion

84 M1 N282801

N285701

 Was deleted in prior assessment

 Was deleted in prior assessment

84 M2 N266801

N255301

 Was deleted in prior assessment

 Was deleted in prior assessment

85 M3 N259921

N264321

N264521

N267921

N276821

N276822

N276823

N278921

N278922

N278923

N278924

N278925

N280621

N280622

N280623

N280624

N280625

N280626

N285321

 Calculator item1

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

 Calculator item

1

All calculator items were deleted from the analysis. 

15.5 GENERATION OF PLAUSIBLE VALUES

Plausible values were calculated separately for each age group. In this phase of analysis, student

background information was used to condition item responses in order to more accurately estimate

average subgroup abilities. The conditioning program BGROUP was used to combine NAEP

BILOG/PARSCALE item parameters with weighted item responses and background variables to produce

posterior ability estimates called plausible values. As defined in Chapter 11, BGROUP is an enhanced

version of the original conditioning program, MGROUP. Plausible values are not test scores in the usual

sense, but can be used to provide consistent estimates of population characteristics. There were 53

contrasts in the conditioning model at age 9, 56 at age 13, and 63 at age 17. Appendix C gives the

codings for the conditioning variables for the three age groups. A check was made on the distributions of

the plausible values for each age, including inspection of the whole group and subgroup means and

standard deviations. Table 15-9 contains a list of the number of background contrasts included in

conditioning, as well as the proportion of variance accounted for by the conditioning model for each

age/grade.
343

Table 15-9

Proportion of Proficiency Variance Accounted for by the Conditioning Model

for the Mathematics Long-Term Trend Assessment

Age/Grade

Number of

Conditioning Contrasts1

Proportion of

Proficiency Variance

9/4 53 .37

13/8 56 .35

17/12 63 .57

1 Including the constant term.

15.6 THE FINAL MATHEMATICS LONG-TERM TREND SCALE

Since the plausible value (theta) scales have a linear indeterminacy, comparisons with previous

assessments will be sensible only if the scale is linearly transformed to a meaningful metric. This

indeterminacy was resolved by linking the 1996 scales to previous long-term trend scales. The 1996 data

had to be transformed to compensate for linear changes in the scale due to employing newly estimated

item parameters and new BGROUP conditioning parameters in 1996. The transformation was

accomplished by first reestimating the 1994 student abilities using 1996 item parameters and 1996

BGROUP parameters. (For score metric transformation, see Chapter 9.) The new 1994 ability estimates

were then equated to the old 1994 ability estimates by matching the first two moments (i.e., the mean and

standard deviation). The constants for this transformation were then applied to the 1996 data. The

transformation equations that resulted are

Age 9: target = 34.04 • calibrated + 230.46

Age 13: target = 33.08 • calibrated + 273.91

Age 17: target = 30.46 • calibrated + 306.57,

where target denotes an individual’s value on the final transformed scale of the 1996 data and calibrated

denotes an individual’s value on the original 1996 theta scale. Overall summary statistics for the

long-term trend samples are given in Table 15-10. For the descriptions of the results of the mathematics

long-term trend study, see NAEP 1996 Trends in Academic Progress (Campbell, Voelkl, & Donahue,

1997).

To provide a context for interpreting the overall mathematics long-term trend results, the NAEP

mathematics results were “anchored” at five NAEP mathematic scale levels. These points (described in

the NAEP 1996 Trends in Academic Progress) are:

 150 = simple arithmetic facts;

 200 = beginning skills and understanding;

 250 = numerical operations and beginning problem solving;

 300 = moderately complex procedures and reasoning; and

 350 = multi-step problem solving and algebra.

These same anchor points were used in the 1978, 1982, 1986, 1990, 1992, and 1994 mathematics longterm trend reports.
344

Table 15-10

Means and Standard Deviations on the

Mathematics Long-Term Trend Proficiency Scale

All Five Plausible Values

Age Assessment Mean S. D.

9 1978

1982

1986

1990

1992

1994

1996

218.6

219.0

221.7

229.6

229.6

231.1

231.0

36.0

34.8

34.0

32.9

33.1

33.2

33.8

13 1978

1982

1986

1990

1992

1994

1996

264.1

268.6

269.0

270.4

273.1

274.3

274.3

39.0

33.4

30.8

31.3

30.9

32.4

31.6

17 1978

1982

1986

1990

1992

1994

1996

300.4

298.5

302.0

304.6

306.7

306.2

307.2

34.9

32.4

31.0

31.3

30.1

30.2

30.2

15.7 PARTITIONING OF THE ESTIMATION ERROR VARIANCE

For each age’s scale, the error variance of the final transformed proficiency mean was partitioned

as described in Chapter 11. The partition of error variance consists of two parts: the proportion of error

variance due to sampling students (sampling variance) and the proportion of error variance due to the fact

that proficiency, , is a latent variable that is estimated rather than observed. Table 15-11 contains

estimates of the total error variance, the proportion of error variance due to sampling students, and the

proportion of error variance due to the latent nature of  (for stability, the estimates of the betweenimputation variance, B, in Equation 11.9 are based on 100 imputations.).

Table 15-11

Estimation Error Variance and Related Coefficients

for the Mathematics Long-Term Trend Assessment

Proportion of Variance Due to . . .

Age Total Estimation Error Variance Student Sampling Latency of 

 9 0.66 0.85 0.15

13 0.66 0.89 0.11

17 1.24 0.93 0.07

More detailed information is available for gender and race/ethnicity subgroups in Appendix E.
345

Chapter 16

DATA ANALYSIS FOR THE LONG-TERM TREND

SCIENCE ASSESSMENT1

 Jinming Zhang and Norma A. Norris

Educational Testing Service

16.1 INTRODUCTION

This chapter describes the analyses performed on the responses to the cognitive and background

items in the 1996 long-term trend assessment of science. The objectives of the science analyses are to

prepare scale values and perform all analyses necessary to produce a long-term trend report in science.

The results obtained from these analyses includes the years 1969-1970, 1973, 1977, 1982, 1986, 1990,

1992, 1994, and 1996, and are presented in the NAEP 1996 Trends in Academic Progress: Achievement

of U.S. Students in Science, 1969 to 1996; Mathematics, 1973 to 1996; Reading, 1971 to 1996; and

Writing, 1984 to 1996 (Campbell, Voelkl, & Donahue, 1997). The theoretical underpinnings of the IRT

and the plausible values methodology used in this chapter are described in Chapter 9 and Chapter 11, and

are therefore not detailed here. 

The student samples that were administered science items in the 1996 long-term trend assessment

are shown in Table 16-1 as 9[MS-LTTrend], 13[MS-LTTrend], and 17[MS-LTTrend]. (See Chapters 1

and 3 for descriptions of the target populations and the sample design used for the assessment.) Data

from the long-term trend samples that contributed to the trends in science achievement were scaled

separately from the 1996 science main focused-BIB samples. Accordingly, the long-term trend and main

analyses are presented in separate chapters. Information about the scaling of the data from the science

main focused-BIB samples is presented in Chapter 13.

The science long-term trend results reported in the NAEP 1996 Trends in Academic Progress are

based on paced-tape administrations at all three age levels. For ages 9 and 13, the long-term trend

booklets contain one mathematics block, one reading block, and one science block. The science and

mathematics blocks were paced by tape-recordings (i.e., tape-recordings were used to be sure that the

items were read in a consistent manner in every session and pace students through the blocks) and the

reading block was presented in print form only and were not paced by tape-recordings. The age 17 longterm trend booklets contain only mathematics and science blocks, both paced by tape-recordings. All

students received a block of common background questions, distinct for each age. Subject-area

background questions were presented in the cognitive blocks. The booklets for the age 9 and age 13

samples (Booklets 91-93) and the booklets for the age 17 samples (Booklets 84-85) are the same as those

used for long-term trend assessments in 1986, 1990, 1992, and 1994. The booklets and the blocks within

those booklets are listed in Chapter 4. Additional information about all of the items in these blocks is also

found in that chapter. This chapter includes specific information about the long-term trend items that

were scaled.

 

1 Jinming Zhang was the primary person responsible for the planning, specification, and coordination of the science long-term

trend analyses. Computer activities for all long-term trend science scaling and data analyses were performed by Norma Norris.

Nancy Allen, Eiji Muraki, and John Donoghue provided consultation.
346

Table 16-1

NAEP 1996 Long-Term Trend Science Student Samples

Sample

Booklet

IDs Mode

Cohort

Assessed

Time of

Testing

Age

Definition

Modal

Grade

Number

Assessed

9 [MS-LTTrend]

13 [MS-LTTrend]

17 [MS-LTTrend]

 91-93

 91-93

 84-85

Tape

Tape

Tape

Age 9

Age 13

Age 17

1/3/96 – 3/8/96 (Winter)

10/9/95 – 12/22/95 (Fall)

3/11/96 – 5/10/96 (Spring)

CY

CY

Not CY

 4

 8

11

5,414

5,658

3,539

LEGEND

MS Mathematics and science

LTTrend Long-term trend assessment: booklets are identical to 1986 long-term trend assessments

Tape Audiotape administration

CY Calendar year: birthdates in 1986 and 1982 for ages 9 and 13, respectively

Not CY Age 17 only: birthdates between October 1, 1978, and September 30, 1979

Table 16-2 clarifies the relationships among the 1996 long-term trend samples and samples from

previous years. For all ages, the 1996 science long-term trend samples allow direct comparisons with

1986, 1990, 1992, and 1994 long-term trend samples because the same booklets were used in these

assessments. There was also a tape administration in 1988 at ages 9 and 13 that was comparable to the

other years. However, a tape administration was not conducted at age 17 in 1988. Instead, a

noncomparable paper-based assessment was conducted. Hence, 1988 is not included as a point in the

long-term trend reporting. In 1986, the science long-term trend items were scaled with common items

from the 1977 and 1982 assessments. Because of the small number of items in common with those in the

1969-70 and 1973 assessments, data from the 1969-70 and 1973 assessments were not scaled using the

IRT model, but were linked to the long-term trend line by a linear transformation involving the logit of

mean proportion correct for common items. When comparisons were made including the 1969-70 and

1973 assessment results, z-tests rather than t-tests were used to test statistical significance (See Section

18.5.1). From 1990, each new long-term trend assessment was linked to the previous assessments

through the last assessment. For instance, the 1996 long-term trend assessment was linked to the previous

assessments through the 1994 long-term trend assessment. Information about previous assessment years,

including 1969-70 and 1973, is available in Chapter 11 of Expanding the New Design: The NAEP 1985-

86 Technical Report (Yamamoto, 1988), Chapter 14 of The NAEP 1990 Technical Report (Allen, 1992),

Chapter 14 of The NAEP 1992 Technical Report (Allen & Isham, 1994), and Chapter 17 of The NAEP

1994 Technical Report (Swinton, Allen, Isham & Chen, 1996).

The numbers of scaled items in common across different ages are presented in Table 16-3. As

was done with previous long-term trend analyses, each age was scaled separately and the long-term trend

scales are univariate. Derivation of scales for specific content areas was not feasible given the limited

number of items presented to students in the long-term trend samples. The number of items scaled in

1996 that were common across assessment years is presented in Table 16-4.

The steps in the science long-term trend analysis are documented in the following sections. As is

usual in NAEP analyses, the first step was to gather item-level and block-level information. Then, the

cognitive items were calibrated according to the IRT model. Next, derived background variables were

calculated, and plausible values were generated after conditioning on available background variables and

selected two-way interactions. Finally, the scale values were placed on the final science long-term trend

scale used in previous trend assessments.
347

Table 16-2

NAEP Science Samples Contributing to 1996 Long-Term Trend Results, 1970-1996

Cohort

Assessed Year Sample Subjects

Time of

Testing

Mode of

Administration

Age

Definition

Modal

Grade

Age 9 1970

1973

1977

1982

1986

1990

1992

1994

1996

Main

Main

Main

Main

LTTrend1

LTTrend1

LTTrend1

LTTrend1

LTTrend1

SWC

MS

SCI

MSC

MS

MS

MS

MS

MS

Winter

Winter

Winter

Winter

Winter

Winter

Winter

Winter

Winter

Tape

Tape

Tape

Tape

Tape2

Tape2

Tape2

Tape2

Tape2

CY

CY

CY

CY

CY

CY

CY

CY

CY

4

4

4

4

4

4

4

4

4

Age 13 1970

1973

1977

1982

1986

1990

1992

1994

1996

Main

Main

Main

Main

LTTrend1

LTTrend1

LTTrend1

LTTrend1

LTTrend1

SWC

MS

SCI

MSC

MS

MS

MS

MS

MS

Fall

Fall

Fall

Fall

Fall

Fall

Fall

Fall

Fall

Tape

Tape

Tape

Tape

Tape2

Tape2

Tape2

Tape2

Tape2

CY

CY

CY

CY

CY

CY

CY

CY

CY

8

8

8

8

8

8

8

8

8

Age 17 1969

1973

1977

1982

1986

1990

1992

1994

1996

Main

Main

Main

Main

LTTrend1

LTTrend1

LTTrend1

LTTrend1

LTTrend1

SWC

MS

SCI

MSC

MS

MS

MS

MS

MS

Spring

Spring

Spring

Spring

Spring

Spring

Spring

Spring

Spring

Tape

Tape

Tape

Tape

Tape

Tape

Tape

Tape

Tape

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

Not CY

11

11

11

11

11

11

11

11

11

1

 Within an age group, these samples received common booklets. 2 Mathematics and science administered by audiotape, reading administered by print.

LEGEND

SCI

MS

Science

Mathematics and science

LTTrend Long-term trend: booklets are identical to the longterm trend assessment of 1986

MSC Mathematics, science, and citizenship Tape Audiotape administration

SWC Science, writing, and citizenship CY Calendar year: birthdates in 1986 and 1992 for ages

9 and 13 in the 1996 assessment

Main Main assessment Not CY Age 17 only: birthdates between October 1 and

September 30 of the appropriate years
348

Table 16-3

Numbers of Scaled Science Long-Term Trend

Items Common Across Ages

Age Booklet Numbers Number of Items

 9 only

 13 only

 17 only

 9 and 13 only

 9 and 17 only

 13 and 17 only

 9, 13, and 17

91-93

91-93

84-85

91-93, 91-93

91-93, 84-85

91-93, 84-85

91-93, 91-93, 84-85

55

30

32

0

0

451

1

 Total 163

1 One of these items (N406303) was treated as a different item from 1990 in the

scaling of the 1992 assessment, but only for age 13. It was treated as an item

common to 1992, 1994 and 1996 for all ages in the 1994 and 1996 assessments.

Table 16-4

Numbers of Scaled Science Long-Term Trend

Items Common Across Assessments

Number of Items

Assessment Years Age 9 Age 13 Age 17

 1986, 1990, 1992, 1994,1996

 1982, 1986, 1990, 1992, 1994,1996

 1977, 1986, 1990, 1992, 1994,1996

 1977, 1982, 1986, 1990, 1992, 1994, 1996

56

 101

56

 101

76

58

76

582

78

47

76

45

1 Twenty-four items common to years 1977 and 1982, but not later years, were included in the 1986 scaling of these

items to stabilize the estimation of the item parameters. See Expanding the New Design: The NAEP 1985-86

Technical Report for more information. 2 One of these items (N406303) was treated as a different item from 1990 in the scaling of the 1992 assessment, but

only for age 13. It was treated as an item common to 1992, 1994 and 1996 in the 1994 and 1996 assessments for all

ages.

16.2 ITEM ANALYSIS FOR THE SCIENCE LONG-TERM TREND ASSESSMENT

Conventional item analyses did not identify any difficulties with the 1996 long-term trend data

for the 1996 samples that bridge to 1986. Table 16-5 contains information about the science long-term

trend blocks. These blocks were presented to samples 9[MS-LTTrend], 13[MS-LTTrend], and 17[MSLTTrend]. At all ages, the blocks labeled S1, S2, and S3 were presented intact to students in the 1986,

1990, 1992, 1994, and 1996 long-term trend samples. The age 9 and age 13 blocks appeared in Booklets

91 through 93. For age 17, Block S3 was in Booklet 84, and Blocks S1 and S2 were in Booklet 85. The

correspondence between blocks, booklets, and samples is given for the long-term trend assessment in

Tables 4-14 through 4-16 in Chapter 4. Common labeling of these blocks across ages does not denote

common items.

Table 16-5 contains the number of scaled items, size of the sample administered to the block,

mean weighted proportion correct, mean weighted r-biserial, and mean weighted alpha as a measure of

reliability for each block. The average values were calculated using examinee sampling weights and the
349

responses to the items in the block that were scaled. On average, the 1996 item-level statistics were not

very different from those for the 1994 assessments. The percent of examinees not reaching items in the

science long-term trend blocks was almost always zero because the items were administered with a taperecording to pace response time.

Table 16-5

Descriptive Statistics for Item Blocks in the Science Long-Term Trend Samples (1996)

Blocks

Statistic S1 S2 S3

Age 9

 Number of scaled items

 Number of scaled constructed-response items

 Unweighted sample size

 Average weighted proportion correct

 Average weighted r-biserial

 Weighted alpha reliability

17

0

1,852

0.62

0.57

0.71

20

0

1,721

0.58

0.48

0.64

19

0

1,841

0.71

0.58

0.73

Age 13

 Number of scaled items

 Number of scaled constructed-response items

 Unweighted sample size

 Average weighted proportion correct

 Average weighted r-biserial

 Weighted alpha reliability

23

0

1,928

0.53

0.53

0.74

30

0

1,866

0.56

0.50

0.79

23

0

1,864

0.61

0.52

0.72

Age 17

 Number of scaled items

 Number of scaled constructed-response items

 Unweighted sample size

 Average weighted proportion correct

 Average weighted r-biserial

 Weighted alpha reliability

24

0

1,691

0.65

0.49

0.68

31

0

1,691

0.65

0.54

0.79

23

0

1,848

0.61

0.64

0.82

16.3 IRT SCALING FOR THE SCIENCE LONG-TERM TREND ASSESSMENT

16.3.1 Item Parameter Estimation

The first step in the scaling process was the estimation of item parameters for the long-term trend

items. This item calibration was performed using the NAEP version of the BILOG/PARSCALE program,

which combines Mislevy and Bock’s (1982) BILOG and Muraki and Bock’s (1991) PARSCALE

computer programs. Items were calibrated separately for each of the three age groups, using combined

data from the 1994 and 1996 assessment years and treating each assessment sample as a sample from a

separate subpopulation. In several previous long-term trend analyses, combined data from the last

assessment and the current assessment were used for item parameter estimation. The purposes for

including the last long-term trend assessment data are to assure that item parameter estimates will be

similar for adjacent assessments so that year-to-year trends will not be distorted by abrupt changes in

calibration, and to make it possible to link the current long-term trend assessment to the previous

assessments through the last assessment. Student weights were used for the calibration. To ensure that
350

each assessment year had a similar influence on the calibration, student weights for each 1994 age group

were multiplied by a constant, to adjust them to have the same sum as the sum of the student weights for

the corresponding 1996 age group.

Although other items were examined for irregularities, only items that were deleted from the

previous scaling of the paced-tape long-term trend data were excluded in the 1996 analysis. Eight percent

of the items (18 items) administered to the long-term trend sample were excluded from analyses of

previous assessments. The deleted items appear in Table 16-6. As a result of these deletions, 56 items

were scaled for age 9, 76 items were scaled for age 13, and 78 items were scaled for age 17. A list of the

items scaled for each of the ages, along with their item parameter estimates, appears in Appendix D.

Table 16-6

Items Deleted from the Paced-Tape Science Long-Term Trend Analysis

Age

Booklet

IDs Block Item Reason for Exclusion

 9 91

92

92

92

92

92

93

S1

S2

S2

S2

S2

S2

S3

N400201

N401701

N402003

N402004

N402601

N402603

N403802

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

13 91

91

92

93

93

93

93

S1

S1

S2

S3

S3

S3

S3

N404902

N404903

N407501

N409401

N409402

N409403

N409801

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

17 85

85

85

85

S1

S1

S1

S2

N410001

N410002

N410301

N407402

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

 Excluded in previous assessment

16.3.2 Derived Background Variables

In the long-term trend analysis, all variables derived from background questions were used both

in generating plausible values and in reporting (to define subgroups). Derived conditioning and reporting

variables are described in Appendix B.
351

16.4 GENERATION OF PLAUSIBLE VALUES

The generation of plausible values was conducted independently for each age group. The item

parameters from NAEP-BILOG/PARSCALE, final student weights, item responses and selected

background variables (conditioning variables) were used with the computer program BGROUP

(described in Chapter 11) in order to generate the plausible values for each student. There were 49

contrasts in the conditioning model (11.8) at age 9, excluding an overall constant, 52 at age 13, and 58 at

age 17. Appendix C gives the codings for the conditioning variables for the three age groups. A check on

the distributions of the plausible values for each age was made. The generation of plausible values is

described in more detail in Chapters 9 and 11. Table 16-7 contains a list of the number of background

contrasts included in conditioning, as well as the proportion of variance accounted for by the

conditioning model for each age. This proportion is the ratio of the difference between the total variance

and the BGROUP residual variance, divided by the total variance. The total variance is the mean of the

five theta-scale variances obtained by their respective plausible values.

Table 16-7

Proportion of Proficiency Variance Accounted for by the Conditioning Model

for the Science Long-Term Trend Assessment

Age

Number of

Conditioning Contrasts1

Proportion of

Proficiency Variance

9 49 0.33

13 52 0.37

17 58 0.46

1 Excluding the constant term.

16.5 THE FINAL SCIENCE LONG-TERM TREND SCALE

The linear indeterminacy of the long-term trend scale was resolved by linking the 1996 long-term

trend scales to the previous long-term trend scales using the following procedure. For each age, the item

parameters based on combined data from 1994 and 1996 were used with the 1994 data to find plausible

values for the 1994 data. The mean and standard deviation of all of the plausible values (theta scale) were

calculated and matched to the mean and standard deviation of all of the science long-term trend scale

scores (final reporting scale) based on the 1994 item parameters and 1994 data as reported in the NAEP

1994 Technical Report. The transformations that resulted from this matching of the first two moments for

the 1994 data are

Age 9: target = 38.57  calibrated + 232.56,

Age 13: target = 40.11  calibrated + 255.54, and

Age 17: target = 48.28  calibrated + 293.82,

where target denotes values on the final reporting scale of the 1996 data and calibrated denotes values on the

original 1996 calibration (theta) scale. Overall summary statistics for the long-term trend scales are given
352

in Table 16-8. The detailed science long-term trend results from the analyses described in this chapter are

reported in NAEP 1996 Trends in Academic Progress.

Table 16-8

Means and Standard Deviations on the Science Long-Term Trend Scale

All Five Plausible Values

Age Assessment Mean S. D.

9 1977

1982

1986

1990

1992

1994

1996

219.9

220.8

224.3

228.7

230.6

231.0

229.7

44.9

40.9

41.6

40.2

39.9

40.9

42.2

13 1977

1982

1986

1990

1992

1994

1996

247.4

250.1

251.4

255.2

258.0

256.8

256.0

43.5

38.6

36.6

37.6

36.9

37.2

38.4

17 1977

1982

1986

1990

1992

1994

1996

289.5

283.3

288.5

290.4

294.1

294.0

295.7

45.0

46.7

44.4

46.2

44.7

45.6

45.1

As in the past, interpretation of the science long-term trend results was facilitated through the

provision of scale anchoring information. In 1986, five science scale levels were selected as anchor

points, using the process described in Expanding the New Design: The 1985-86 Technical Report

(Beaton, 1988). Because the 1996 science long-term trend scale was tied to the 1986 long-term trend

scale through the 1990, 1992, and 1994 data, the distribution of proficiency scores derived from the longterm trend samples can be described in terms of scale anchors. The five levels of science proficiency are

150 = Knows everyday science facts;

200 = Understands simple scientific principles;

250 = Applies basic scientific information;

300 = Analyzes scientific procedures and data; and

350 = Integrates specialized scientific information.
353

16.6 PARTITIONING OF THE ESTIMATION ERROR VARIANCE

For each age, the error variance of the final reporting scale mean was partitioned into two parts

as described in Chapter 11. One part of the error variance is due to the sampling of students (sampling

variance) and the other is due to the fact that proficiency  is a latent variable that is estimated rather than

observed. These estimates are given in Table 16-9 (for stability, the estimates of the between-imputation

variance, B, in Equation 11.9 are calculated based on 100 imputations). More detailed information for

gender and race/ethnicity subgroups is available in Appendix E.

Table 16-9

Estimation Error Variance and Related Coefficients

for the Science Long-Term Trend Assessment

Proportion of Variance Due To . . .

Age

Total Estimation

Error Variance Student Sampling Latency of 

 9 1.13 0.81 0.19

13 1.04 0.87 0.13

17 1.40 0.86 0.14

355

Chapter 17

DATA ANALYSIS FOR THE LONG-TERM TREND

WRITING ASSESSMENT

[This chapter is intended to provide information about the 1996 long-term

trend assessment in writing; however, the data from this assessment are

currently under review. After additional examination and analyses, this

chapter will be included in a revised web version of the complete report.]

367

Chapter 18

CONVENTIONS USED IN HYPOTHESIS TESTING

AND REPORTING NAEP RESULTS1

Spencer S. Swinton, David S. Freund, and Nancy L. Allen

Educational Testing Service

18.1 OVERVIEW

Results for the 1996 NAEP Assessments were disseminated in several different reports: the

NAEP 1996 Mathematics Report Card for the Nation and the States, the NAEP 1996 Science Report

Card for the Nation and the States, NAEP 1996 Trends in Academic Progress, Cross-State Data

Compendium from the NAEP 1996 Mathematics Assessment, Cross-State Compendium from the NAEP

1996 Science Assessment, and, distributed only in electronic form, six sections of summary data tables

for each report. These reports are published on the NCES/NAEP website http://nces.ed.gov/naep. Several

other reports based on 1996 NAEP data will be forthcoming.

 The NAEP 1996 Mathematics Report Card for the Nation and the States and the NAEP 1996

Science Report Card for the Nation and the States highlight key assessment results for the nation and

summarize results across the jurisdictions participating in the assessments. These reports contain

composite scale score results (scale score means, etc.) for the nation, for each of the four regions of the

country, and for public-school students within each jurisdiction participating in the State Assessment2

,

both overall and by primary reporting variables. The seven key reporting variables (referred to here as

primary reporting variables) are gender, race/ethnicity, level of parents’ education, Title I participation,

eligibility for free or reduced cost school lunch, type of location, and type of school (public, Catholic

schools, other religious schools, and other private schools). For public-school students, scale score means

were reported for a variety of other subpopulations defined by responses to items from the student,

teacher, and school questionnaires and by school and location demographic variables provided by

Westat, Inc.3

 Upcoming reports will include estimates of scale score means and selected percentiles for

specific subgroups of students of interest in each report.

The report NAEP 1996 Trends in Academic Progress provides a look at NAEP results for

Science, Mathematics, Reading, and Writing since the first NAEP assessments of those subjects in 1969-

70. This report includes scale score results for the nation overall and by gender, race/ethnicity, gender

and race/ethnicity, region, level of parents’ education and type of school (public and nonpublic). It also

provides percentages of students in categories defined by subject specific background variables (such as

students who reported having experimented with living plants), along with their average scale scores. The

report contains trends in average scale scores by quartile and percentages of students performing at or

above selected performance levels. An additional report gives data for the mechanics of writing longterm trend.

 

1

 Spencer S. Swinton played a role in making decisions about hypothesis testing methods and procedures and worked with David

S. Freund who implemented many of the methods and procedures in computer programs. Nancy L. Allen contributed to the

current version of this chapter.

2

 Further technical documentation for the State Assessments appears in the Technical Report of the NAEP 1996 State Assessment

Program in Mathematics and the Technical Report of the NAEP 1996 State Assessment Program in Science.

3

 Some of these variables were used by Westat, Inc., in developing the sampling frame for the assessment and in drawing the

sample of participating schools.
368

The third type of report consists of a number of data compendia. Two of these are entitled the

Cross-State Data Compendium from the NAEP 1996 Mathematics Assessment and the Cross-State Data

Compendium from the NAEP 1996 Science Assessment. Like the Report Cards, the Compendia report

results for the nation and for all of the jurisdictions participating in the State Assessment. The

Compendia contain most of the tables included in the Report Cards plus tables that provide composite

scale results for a large number of secondary reporting variables (e.g., amount of homework, teacher

preparation).

The fourth type of summary report is an electronically-delivered collection of summary data

tables that contain detailed breakdowns of the science scale score data for each sample according to the

responses to the student, teacher, and school questionnaires for the public-school, nonpublic-school, and

combined populations as a whole and for important subgroups of the public-school population, as defined

by the primary reporting variables. There are six sections in each collection of summary data tables:

The Distribution Data Section provides selected composite-scale and science subscale

percentiles for the public-school, nonpublic-school, and total populations and for the

major demographic subgroups of the national school population.

The Student Questionnaire Section breaks down the composite scale score data according

to the students’ responses to questions in the three student questionnaires (common core,

subject-specific background, and motivational section) included in the assessment

booklets.

The Teacher Questionnaire Section breaks down the composite scale score data

according to the teachers’ responses to questions in teacher questionnaires, where they

are available.

The School Questionnaire Section breaks down the composite scale score data according

to the principals’ (or other administrators’) responses to questions in the school

characteristics and policies questionnaire.

The Scale Section breaks down the scale score data for the mathematics content strands

or the fields of science according to selected items (such as the amount of science

homework done per day) from the questionnaires.

The Item Section provides the response data (percent of students choosing each option)

for each cognitive item in the assessment.

The production of these reports required many decisions about a variety of data analysis and

statistical issues. For example, certain categories of the reporting variables contained limited numbers of

examinees. A decision was needed as to what constituted a sufficient sample size to permit the reliable

reporting of subgroup results, and which, if any, estimates were sufficiently unreliable to need to be

“flagged” as a caution to readers. As a second example, the performance for subgroups of students were

compared. A number of inferential rules, based on logical and statistical considerations, had to be

developed to ensure that conclusions are adequately supported by the data from the assessment. Practical

comparison procedures were required to control for Type I errors without paying too large a penalty with

respect to the statistical power for detecting real and substantively interesting differences. For most tests,

the number of related tests was not so large that the Bonferroni test (Hochberg, 1988) exacted too large a

penalty in power in exchange for protection from Type I error. For sets of comparisons with very large

numbers of related tests, such as comparing a state to all other states, a new multiple comparison
369

criterion, False Discovery Rate or FDR (Benjamini & Hochberg, 1994), was implemented. FDR controls

the rate of false rejections (e.g., five false rejections per 100 rejections), rather than controlling the

probability of one such error (Familywise Error Rate, or FWE), as the Bonferroni procedure does.

The purpose of this chapter is to document the major conventions and statistical procedures used

in generating the Report Cards, NAEP 1996 Trends, the Data Compendia, and the summary data tables.

Additional details about procedures relevant to the Report Card and Cross-State Data Compendia can be

found in the text and technical appendices of those reports.

18.2 MINIMUM SCHOOL AND STUDENT SAMPLE SIZES FOR REPORTING

SUBGROUP RESULTS

In all of the reports, estimates of quantities such as composite and scale score means and

percentages of students indicating particular levels of background variables (as measured in the student,

teacher, and school questionnaires) are reported for the population of students in each grade. These

estimates are also reported for certain key subgroups of interest as defined by primary NAEP reporting

variables. Where possible, NAEP reports results for gender, for five racial/ethnic subgroups (White,

Black, Hispanic, Asian American/Pacific Islander, and American Indian/Alaskan Native), three types of

locations (central cities, urban fringes/large towns, rural/small town areas), four levels of parents’

education (did not finish high school, high school graduate, some college, college graduate), Title 1

participation, eligibility for the free or reduced-cost school lunch component of the National School

Lunch Program, and type of school. However, for some regions of the country and sometimes for the

nation as a whole, school and/or student sample sizes were too small for one or more of the categories of

these variables to permit accurate reporting.

A consideration in deciding whether to report an estimated quantity is whether the sampling error

is too large to permit effective use of the estimates. A second, and equally important, consideration is

whether the standard error estimate that accompanies a statistic is itself sufficiently accurate to inform

potential readers about the reliability of the statistic. The precision of a sample estimate (be it sample

mean or standard error estimate) for a population subgroup from a three-stage sample design (the one

used to select samples for the national assessments) is a function of the sample size of the subgroup and

of the distribution of that sample across first-stage sampling units (i.e., PSUs in the case of the national

assessments). Hence, both of these factors were used in establishing minimum sample sizes for reporting.

Here a decision was reached to report subgroup results only if the student sample size exceeded

61.5

 A design effect of two was assumed for this decision, implying a sample design-based variance twice

that of simple random sampling. This assumption is consistent with previous NAEP experience (Johnson

& Rust, 1992). In carrying out the statistical power calculations when comparing a subgroup to the total

group, it was assumed that the total population sample size is large enough to contribute negligibly to

standard errors. Furthermore, it was required that the students within a subgroup be adequately

distributed across PSUs to allow for reasonably accurate estimation of standard errors. In consultation

with Westat, a decision was reached to publish only those statistics that had standard error estimates

based on five or more degrees of freedom. The same minimum student and PSU sample size restrictions

were applied to proportions and to comparisons of percentages or proportions as well as average scale

scores and comparisons of average scale scores.

 

5

 This number was obtained by determining the sample size necessary to detect an effect size of 0.5 with a probability of 0.8 or

greater.
370

18.3 IDENTIFYING ESTIMATES OF STANDARD ERRORS WITH LARGE MEAN

SQUARED ERRORS

As noted above, standard errors of average scale scores, proportions, and percentiles play an

important role in interpreting subgroup results and in comparing the performances of two or more

subgroups. The jackknife standard errors reported by NAEP are statistics whose quality depends on

certain features of the sample from which the estimate is obtained. In certain cases, the mean squared

error6

 associated with the estimated standard errors may be quite large. This result typically occurred

when the number of students upon which the standard error is based is small or when this group of

students comes from a small number of participating PSUs. The minimum PSU and student sample sizes

that were imposed in most instances suppressed statistics where such problems existed. However, the

possibility remained that some statistics based on sample sizes that exceed the minimum requirements

had standard errors that were not well estimated. Therefore, in the reports, estimated standard errors for

published statistics that are themselves subject to large mean squared errors are followed by the symbol

“!”.

The magnitude of the mean squared error associated with an estimated standard error for the

mean or proportion of a group depends on the coefficient of variation (CV) of the estimated size of the

population group, denoted as N (Cochran, 1977, Section 6.3). The coefficient of variation is estimated

by:

where N is a point estimate of N and SE(N) is the jackknife standard error (described in Chapter 10 of

this report) of N.

Experience with previous NAEP assessments suggests that when this coefficient exceeds 0.2, the

mean squared error of the estimated standard errors of means and proportions based on samples of this

size may be quite large. (Further discussion of this issue can be found in Johnson & Rust, 1992.)

Therefore, the standard errors of means and proportions for all subgroups for which the coefficient of

variation of the population size exceeds 0.2 are marked as described above. In the Report Cards, NAEP

Trends, the Data Compendia, and the summary data tables, statistical tests involving one or more

quantities that have standard errors, confidence intervals, or significance tests so flagged should be

interpreted with caution.

18.4 TREATMENT OF MISSING DATA FROM THE STUDENT, TEACHER, AND

SCHOOL QUESTIONNAIRES

As previously described, responses to the student, teacher, and school questionnaires played a

prominent role in all reports. Although the return rate on all three types of questionnaire was high,7

 there

were missing data for each type of questionnaire.

 

6

 The mean squared error of the estimated standard error is defined as [ S - ] ,

2  σ where 

S is the estimated standard error,σ is

the “true” standard error, and  is the expectation, or expected value operator.

7

 Information about survey participation rates (both school and student), as well as proportions of students excluded by each

jurisdiction from the assessment, is given in Appendix B. Sampling adjustments intended to account for school and student

nonresponse are described in Chapter 7.

CV(N) = SE(N)

N

 


371

The reported estimated percentages of students in the various categories of background

variables, and the estimates of the average scale score of such groups, were based on only those students

for whom data on the background variable were available. In the terminology of Little and Rubin (1987),

the analyses pertaining to a particular background variable presented in the reports are contingent on the

assumption that the data are missing completely at random.8

The estimates of proportions and proficiencies based on “missing-completely-at-random”

assumptions are subject to potential nonresponse bias if, as may be the case, the assumptions are not

correct. The amount of missing data was small (usually, less than 2%) for most of the variables obtained

from the student, school, and teacher questionnaires. For analyses based on these variables, reported

results are subject to little, if any, nonresponse bias. However, for particular background items from the

student, school, and teacher questionnaires, the level of nonresponse was somewhat higher. As a result,

the potential for nonresponse bias in the results of analyses based on this latter set of background items is

also somewhat greater. Background items for which more than 10 percent of the returned questionnaires

were missing are identified in the questionnaire sections (as specified at the beginning of this chapter) of

the summary data tables. Again, results for analyses involving these background variables should be

interpreted with caution.

To analyze the relationships among teachers’ questionnaire responses and their students’

achievement, each teacher’s questionnaire had to be matched to the students who were taught by that

teacher. If a student could not be matched to a teacher, all teacher questionnaire responses are missing for

that student. The percentages of students that were matched to teacher questionnaires in each sample for

which a teacher questionnaire was administered are reported in the subject area Chapters 12 and 13.

Lower percentages of students with teacher questionnaire data indicate that there is less certainty about

results for variables from the teacher questionnaire. Note that these match rates do not reflect the

additional missing data due to item-level nonresponse. The amount of additional item-level nonresponse

in the returned teacher questionnaires can be found in the summary data tables.

18.5 HYPOTHESIS TESTING CONVENTIONS

18.5.1 Comparing Means and Proportions for Different Groups of Students

Many of the group comparisons explicitly commented on in the reports involved mutually

exclusive sets of students. Examples include comparisons of the average scale score for male and female

students, White and Hispanic students, students attending schools in central city and urban fringe/large

town locations, students who reported watching six or more hours of television each night and students

who report watching less than one hour each night.

The text in the reports indicate that means or proportions from two groups were different only

when the difference in the point estimates for the groups being compared was statistically significant at

an approximate simultaneous level of 0.05. An approximate procedure was used for determining

statistical significance NAEP staff judged to be statistically defensible, as well as being computationally

tractable. Although all pairs of levels within a variable were tested and reported in the summary data

tables, some text within the reports was developed for only a subset of these comparisons although the

family size was maintained at that of the original tests. For example, text was included in the reports to

compare the majority ethnic group and each minority group, but text for all possible comparisons of

groups may not have been included.

 

8

 The mechanism generating the missing data is independent of both the response to the particular background items and the

scale score.
372

The procedure used to make statistical tests is described in the following paragraphs. This

procedure was used in all cases except when comparisons were made with students assessed in

assessment years for which average scale scores were extrapolated as part of the long-term trend

analyses. In those cases, z-tests comparing the test statistics to the appropriate value from the standard

normal distribution was used.

Let Ai be the statistic in question (e.g., a mean for group i) and let SAi

 be the jackknife standard

error of the statistic. The text in the reports identified the means or proportions for groups i and j as being

different if:

where Ta is the (1 - ) percentile of the t distribution with degrees of freedom, df, as estimated below,

and c is the number of related comparisons being tested. See the following section (Section 18.5.2) for a

more specific description of multiple comparisons. In cases where group comparisons were treated as

individual units, the value of c was taken as 1, and the test statistic was approximately equivalent to a standard

two-tailed t-test for the difference between group means or proportions from large independent samples with

the level set at 0.05. When c is not 1, this test is based on the Bonferroni procedure described in Hochberg

(1988). The degrees of freedom of this t-test is defined by a Satterthwaite (Johnson & Rust, 1992)

approximation as follows:

where N is the number of subgroups involved, and dfAk is as follows:

where m is the number of replicates, tj is the j

th replicated estimate for the mean of a subgroup, and t is the

estimate of the subgroup mean using the overall weights and the first plausible value.

The procedures in this section assume that the data being compared are from independent

samples. Because of the sampling design in which PSUs, schools, and students within school are

randomly sampled, the data from mutually exclusive sets of students may not be strictly independent.

Therefore, the significance tests employed are, in many cases, only approximate. As described in Section

| A - A |

S A + S A

 T i j

2

Ai i A j j

.05

2c     2 ≥

Ak

j k

j

m

j k

j

m df = 

m 

t t

t t

k

k

316 2 77

2

1

2

4

1

. . − 



 

	


 − 



 

	




−





























=

=

∑

∑

 

 

df = 

( S )

S

df

k=1

N

A

2 2

k=1

N

A

4

A

k

k

k

∑

∑
373

10.4, another procedure, one that does not assume independence, could have been conducted. However,

that procedure is computationally burdensome. A comparison of the standard errors using the

independence assumption and the correlated group assumption was made using NAEP data. The

estimated standard error of the difference based on independence assumptions was approximately ten

percent larger than the more complicated estimate based on correlated groups. In almost every case, the

correlation of NAEP data across groups was positive. Because, in NAEP, significance tests based on

assumptions of independent samples are only somewhat conservative, the approximate procedure was

used for most comparisons.

The procedures described above were used for testing differences of both means and nonextreme

percentages. The approximation for the test for percentages works best when sample sizes are large, and

the percentages being tested have magnitude relatively close to 50 percent. Statements about group

differences should be interpreted with caution if at least one of the groups being compared is small in

size and/or if “extreme” percentages are being compared. Differences in percentages were treated as

involving “extreme” percentages if for either percentage, P:

P P

NEFF

< =

+

lim

200

2

,where the effective sample size is N P P

SE EFF

JK

= ( ) −

( ) , 100

2 and SEJK is the jackknife

standard error of P. Similarly, at the other end of the 0 - 100 scale, a percentage is deemed extreme if 100

- P < Plim. In either extreme case, the normal approximation to the distribution is a poor approximation,

and the value of P was reported, but no standard error was estimated and hence no significance tests were

conducted.

18.5.2 Multiple Comparison Procedures

Frequently, groups (or families) of comparisons were made and were presented as a single set.

The appropriate text, usually a set of sentences or a paragraph, was selected for inclusion in a report

based on the results for the entire set of comparisons. For example, some reports contain a section that

compared average scale scores for a predetermined group, generally the majority group (in the case of

race/ethnicity, for example, White students) to those obtained by other minority groups. The entire set of

tests was presented in the summary data tables. For families of comparisons like these, a Bonferroni

procedure (Miller, 1966), controlling the Familywise Error Rate (FWE), was used. This procedure

defines the value of Ta, as in the previous section, where c is the number of contrasts in the set. In the

race/ethnicity example, c was taken to be the number of minority groups meeting minimum sample size

requirements, and each statistical test was consequently carried out at an level of 0.05/c.

However, in an attempt to gain greater power, two separate definitions of family size were

employed for comparisons in two-way tables. For n levels of a control variable (e.g., ethnicity) and m

levels of a comparison variable (e.g., number of hours of homework), the standard Bonferroni family size

of n  m  (m-1)/2 was used. In addition, when the m  (m-1)/2 marginal tests yielded a significant

difference for a pair of categories of the comparison variable, the n levels of the control variable

corresponding to that pair of categories were tested with a family size of n. Significance was reported if

either definition of family size met the criterion.

Further, in the Report Card and summary data tables, two-way interactions were tested directly

for some variables. The tests for an m  n table were t-tests using a family size n  (n-1)  m  (m-1)/4. In

these cases, a modification due to Hochberg of the standard Bonferroni procedure was employed, in

which probabilities associated with outcomes are ordered, and α is divided by an integer which increases
374

from 1 to the family size as successively smaller probabilities are tested. More formally, the Hochberg

Stagewise Procedure (Hochberg, 1988) is defined as follows:

Let q be the number of significance tests made (the family size) and let P1 ≤ P2 ≤ ... ≤ Pm be the

ordered significance levels for the q tests. Let  be the combined significance level. The Hochberg

procedure compares Pq with α, Pq-1 with α /2, ..., Pj with α/(q−j+1), stopping comparisons with the first j

such that Pj < α/(q−j+1). All tests associated with P1, ..., Pj are declared significant; all tests associated

with Pj+1, ..., Pq are declared nonsignificant.

To compare a jurisdiction in a State Assessment with the nation and all other participating

jurisdictions, as many as 46 different comparisons need to be computed. This is done in the comparisons

of overall scale score maps in the State Assessment reports and in the comparisons of short-term trends in

mathematics achievement in the Mathematics Report Card. A potentially more powerful multiple

comparison procedure was used to judge significance in this case. The procedure, described by

Benjamini and Hochberg (1994), was the procedure chosen. Unlike the Bonferroni procedure that

controls the FWE, the procedure described by Benjamini and Hochberg (1994) controls the expected

proportion of falsely rejected hypotheses among all rejections (FDR). For example, at the 0.05 level, for

every 100 rejections of the null hypothesis, the procedure ensures that no more than five will be expected

to be false. Note that control of the FDR is a less conservative type of error control than that of the FWE.

Simulations have shown that “the FDR is controlled at level  for the dependent tests involved in

pairwise comparisons as well as for independent tests” (Shaffer, 1994).

The Benjamini and Hochberg application of the False Discovery Rate (FDR) criterion can be

described as follows. Let q be the number of significance tests made and let

P1 ≤ P2 ≤ . . . ≤ Pq be the ordered significance levels of the q tests, from lowest to highest probability. Let

 be the combined significance level desired, usually 0.05. The procedure will compare Pq with ,

Pq-1 with (q-1)/q, . . ., Pj with j/q, stopping the comparisons with the first j such that Pj ≤ j/q. All tests

associated with P1, . . ., Pj are declared significant; all tests associated with Pj+1, . . ., Pq are declared

nonsignificant.

18.5.3 Linear and Quadratic Tests of Trends

Tests of significance designed to identify consistent patterns of trend data are available and,

although they are more complex, they provide more power to identify those specific patterns than a series

of t- or z-tests would provide.

One such set of tests of significance is the test of linear and test of quadratic trends applied to the

long-term trend data for the nation and selected subpopulations. The purpose of this first set of general

tests was to determine whether the results of the series of assessments in a given subject could be

generally characterized as increasing or decreasing, and whether the results have steadily increased (or

decreased) over the time period of interest. Simple curvilinear (i.e., quadratic) relationships capture more

complex patterns. For example, one possible pattern is to have initial score declines over part of the time

period followed by score increases in more recent assessments. Another possible pattern is to have a

sequence of several assessments in which scores increased followed by a period of relative stable

performance. These examples are two, but not all, of the simple curvilinear relationships that were tested.

The linear and quadratic components of the trend in average scale scores for a given subject area

and age group were estimated by applying two sets of contrasts to the set of average scale scores by year.

The linear component of the trend was estimated by the sum b1=cjxj, where the xj are the average scale
375

scores by year and the cj are defined such that b1 corresponds to the slope of an unweighted regression of

the average scale scores on the assessment year. In other words,

c

y N y

y N y

j

j i

i

k i

k i

=

−

−

∑

∑ ∑

1

1 2 ( )

where yj represents an assessment year. The quadratic component was estimated by the sum b2 = ∑djxj in

which the dj are formally orthogonal to the cj and are defined such that b2 is the quadratic term in the

unweighted regression of the average scale scores on the assessment year and the square of the

assessment year. In other words,

d h

h j

j

i

i

=

∑ 2

where

h y N y cy N y N j j i k y

k

k i

i

j i i i

= − ∑ − − y





 



 ( ) ( )( ) 2 ∑ ∑ • − ∑ 2 2 2 1 1 1

Both cj and dj match expected linear quadratic contrasts in common texts when the years are equally

spaced through time (Winer, 1962/1971). The statistical significance of b1 and b2 was evaluated by

comparing each estimate to its estimated standard error. The standard error of b1 was estimated as the

square root of the sum c SE j j

2 2

∑ , in which SEj is the estimated standard error of xj. The estimated

standard error of the b2 was analogously defined.

The linear and quadratic trend tests allow statements to be made about results across assessment

years in a more powerful way than is possible if results for each year had been compared to those of

every other year, using a multiple-comparison procedure such as the Bonferroni method. These tests do

not control the overall Type I error rate when they are applied to several related subgroups, such as the

students in each region of the country. For this reason, the Bonferroni method for controlling Type I error

was used when the trends for related subgroups were tested. For example, when tests were conducted for

linear trend for the separate race/ethnicity groups (i.e., White, Black, and Hispanic), these tests were

treated as a single family of comparisons of size 3. The significance level for each of the separate tests

was adjusted by the Bonferroni procedure to yield a family-wise error rate of .05.

18.5.4 Comparing Proportions Within a Group

Certain analyses involved the comparison of proportions. One example was the comparison of

the proportion of students who reported that a parent graduated from college to the proportion of students

who indicated that their parents did not finish high school to determine which proportion was larger.

There are other such proportions of interest in this example, such as the proportion of students with at

least one parent graduating from high school but neither parent graduating from college. For these types

of analyses, NAEP staff determined that the dependencies in the data could not be ignored.
376

Unlike the case for analyses of the type described in Section 18.5.1, the correlation between the

proportion of students reporting a parent graduated from college and the proportion reporting that their

parents did not finish high school is likely to be negative and large. For a particular sample of students, it

is likely that the higher the proportion of students reporting “at least one parent graduated from college”

is, the lower the proportion of students reporting “neither parent graduated from high school” will be. A

negative dependence will result in underestimates of the standard error if the estimation is based on

independence assumptions (as is the case for the procedures described in Section 18.5.1). Such

underestimation can result in an unacceptably large number of “nonsignificant” differences being

identified as significant.

The procedures of Section 18.5.1 were modified for analyses that involved comparisons of

proportions within a group. The modification involved using a jackknife method for obtaining the

standard error of the difference in dependent proportions. The standard error of the difference in

proportions was obtained by first obtaining a separate estimate of the difference in question for each

jackknife replicate, using the first plausible value only, then taking the standard deviation of the set of

replicate estimates as the estimate. The procedures used for proportions within a group differed from the

procedures of Section 18.5.1 only with respect to estimating the standard error of the difference; all other

aspects of the procedures were identical.
377

Chapter 19

STATISTICAL SUMMARY OF THE 1996 NAEP SAMPLES1

Bruce A. Kaplan

Educational Testing Service

19.1 INTRODUCTION

The analysis of the 1996 NAEP data has resulted in the production of thousands of tables

presenting estimates of the proficiency of students, and various subgroups of students, in American

schools. This chapter provides a statistical summary of the 1996 NAEP national samples. The chapter

assumes a general familiarity with the structure of NAEP as summarized in the Introduction and in the

overviews presented in Chapters 1 and 9. Similar results for the state samples appear in the data

compendia for the state mathematics and science assessments.

Two of the many types of NAEP results are presented here:

1. the results of the instrument development process, including the sizes of

the item pools and numbers of booklets; and

2. the results of the sampling process, including the numbers of students in

each sample by selected subgroups.

19.2 MEASUREMENT INSTRUMENTS

For the 1996 assessment, 79 different assessment booklets and questionnaires were printed for

age class 9, 80 for age class 13, and 81 for age class 17. These instruments are shown by age level and

type in Table 19-1.

The item pool contributing to all main and long-term trend booklets is described in Table 19-2. In

general, there are two types of items, cognitive and noncognitive. The cognitive items are developed to

measure proficiency in particular subject areas, such as reading and mathematics. Cognitive items may be

constructed-response or multiple-choice. The noncognitive items are usually questions about the

student’s or teacher’s backgrounds and perceptions but may also probe other areas, such as school

policies or teaching methods. Because many items were used at more than one age class, the total number

of items in an item pool is not the sum of the item pools used for the three age classes. However, results

for cognitive items that were common across two or three age classes were not compared, due to a NAEP

policy of within grade scaling.

The SD/LEP Student Questionnaires, Teacher Questionnaires, and School Characteristics and

Policies Questionnaires contained only noncognitive questions. The number of items in the noncognitive

 

1 Bruce A. Kaplan was responsible for the text, specifying the tables, and coordinating table production. Shuyi Hua produced

most of the tables in this chapter. David Freund’s advice was invaluable in the production of this chapter.
378

pools is the same as the number of items in the questionnaires. More information about the instruments

that were developed is provided in Chapters 2 and 4.

19.3 SAMPLE CHARACTERISTICS

In this section, the characteristics of the final reporting NAEP samples are described. The

process by which the samples were selected is discussed in Chapter 3.

In the 1996 main assessment, NAEP contacted 2,267 schools (2,263 original and 4

replacements), of which 1,791 contributed data to the assessment. The disposition of these schools is

shown in Table 19-3. Some of the schools were unwilling to cooperate; others were believed to be

eligible from the sampling frame, but were not. The cooperation rate is calculated as the sum of

cooperating schools and the schools that were found to have no eligible students divided by the same sum

plus the schools that refused or were from districts that refused to cooperate.

Table 19-3 also shows the number of schools in several categories: region of the country

(Northeast, Southeast, Central, West), school type (public, nonpublic, Catholic, Bureau of Indian Affairs,

Department of Defense Education Activity), type of location, number of teachers, and number of

students.

For the 1996 long-term trend studies, NAEP contacted 856 schools (844 original and 12

replacements), of which 681 contributed data to the various trend assessments. Table 19-4 supplies the

same information for the schools assessed for the long-term trend studies that Table 19-3 supplies for the

main assessment schools.

The numbers of respondents to the teacher questionnaires are summarized in Table 19-5. The

first column in this table includes the number of teachers who responded, by grade and subject area. The

second column is the number of students who were not linked to teachers. The third column is the

number of students linked to teachers, but not specific classes of these teachers (for eighth grade) or

teachers who did not answer classroom information (for fourth grade). The last column is the number of

students linked to their teachers and their specific classes.

NAEP is administered in units called assessment sessions. If the number of students attending an

assessment session is fewer than a predetermined number, the students missing from the session are

assigned to a makeup session and then assessed. Table 19-6 shows the number of regular and makeup

sessions in 1996 NAEP by age class for the main and long-term trend samples. Altogether, 103,814

assessed and excluded students were involved in the 1996 NAEP.

Tables 19-7 through 19-9 display the distribution of the students assessed in the cross-sectional

NAEP assessment in several basic categories: gender, racial/ethnic grouping, region of the country,

parental education, type of location, school type, and modal age. These data are presented for assessed

students in the mathematics main and estimation samples in Table 19-7, the mathematics theme and

advanced samples in Table 19-8, and the science main and advanced samples in Table 19-9. Tables 19-

10, 19-11, and 19-12 provide equivalent information, respectively, for excluded students.

Tables 19-13, 19-14, and 19-15 display the distribution of students assessed in the long-term

trend reading and writing assessment for several basic categories: gender, racial/ethnic grouping, region

of the country, parental education, type of location, and school type.
379

There is one table for each age/grade. The tables have four columns:

• eligible by age, which means that the students were in an appropriate age group;

• eligible by grade, which means that the students were in an appropriate grade;

• eligible by age and by grade, which means that the students were of both an

appropriate age and appropriate grade; and

• eligible by age or by grade, which is the total number of students for whom data

were collected.

Tables 19-16, 19-17, and 19-18 provide similar information for the long-term trend science and

mathematics assessment. Note that since these are age-only samples, the number of students who are ageeligible only will be the same as the number of students who are age- or grade-eligible. Likewise, the

number of students who are grade-eligible only will be the same as the number of students who are both

age- and grade-eligible. Tables 19-19 through 19-24 enumerate the excluded students across the various

long-term trend samples.

19.4 POPULATION ESTIMATES

The 1996 NAEP samples were designed for estimating the size and attributes of a number of

different populations of students. The estimation procedures use sampling weights, developed by Westat,

Inc., that are associated with the members of the sample (see Chapter 3). In this chapter, all estimates of

population parameters are calculated using these sampling weights. Note these estimates are for the

reporting samples (see Chapter 3 for an explanation of the reporting and modular samples).

The sum of the initial weights for a given sample is an estimate of the number of students who

are in the population represented by the sample. In other words, the sum of the initial weights is taken as

the estimated population size. In analyses, however, this sum of weights was rescaled to sum to the

sample size. For example, in Table 19-25, the estimated number of fourth graders in the nation is

3,711,786, as estimated from the main mathematics sample, as opposed to the 6,627 students in the

sample given in Table 19-7.

Due to design considerations the main assessment was divided into subsamples, and were

administered, and therefore weighted, independently, so that the sum of the initial weights for each

subsample estimates the population size. The subsamples for mathematics were main, estimation, theme

and advanced; for science, the subsamples were main and advanced.

Note that the samples for the main (cross-sectional) assessment are grade-only samples, while

reading and writing long-term trend are grade and age samples. The samples for the mathematics and

science long-term trend are age-only samples. The sum of the initial weights of the excluded students

estimates the number of ineligible students at the respective age/grade levels.

In most cases, the number of students in an age/grade combination is not of interest; a researcher

will be interested in estimating the number of students at either a grade or an age level. For the samples

that contain both grade- and age-eligible students, an estimate of the total number of students at an age

level can be made by summing the initial weights of only the age-eligible students and adding the
380

corresponding sample of age-eligible excluded students’ initial weights. An estimate of the total number

of students in a grade sample can be made by summing the initial weights of grade-eligible students plus

the initial weights of grade-eligible students from the appropriate excluded student sample.

Tables 19-25 to 19-42 show the sizes of the estimated populations of assessable students and the

weighted percentages for the NAEP reporting categories of gender, race/ethnicity, region of the country,

parents’ education level, type of location, school type and modal age. The estimated subpopulation

percentages for the cross-sectional samples are shown in Tables 19-25 through 19-30. Tables 19-31 to 19-

36 show the same information for the long-term trend samples. In a similar manner, Tables 19-37 to 19-42

show the estimated total population of excluded students and the weighted percentages by demographic

subgroups (data about parents’ education level is not collected for excluded students and therefore not

reported; data about reasons for exclusion are included instead).

In previous years, this chapter also provided several tables showing selected proficiency results for

assessed students, as an aid to readers who are interested in the estimates of proficiency that led to the

interpretive results provided in the NAEP subject area reports. These tables are no longer included in this

report. Instead, readers are encouraged to take advantage of the electronic version of these results, in the

form of thousands of summary data tables computed to analyze the 1996 data. The summary data tables are

available both on CD-ROM and via the World-Wide Web at http://nces.ed.gov/naep.

Table 19-1

Measurement Instruments Developed for 1996 NAEP

Age Class

Student Assessment Booklets 9 13 17

Total Number of Cross-Sectional (MAIN) 661 671 70

Mathematics 291 301 30

Main 261 261 26

Estimation 1 1 1

Theme 2 2 2

Advanced —2 1 1

Science 37 37 40

Main 37 37 37

Advanced —3 —3 3

Total Number of Long-Term Trend 9 9 8

Reading and Writing 6 6 6

Mathematics and Science 3 3 2

Total Number of Questionnaires 4 4 3

Excluded Students (Long-Term Trend only) 1 1 1

SD/LEP (Cross-Sectional (main) only) 1 1 1

Teacher 1 1 0

School 1 1 1

Total Number of Assessment Instruments 791 801 81

1

A bilingual book was also used, but not counted as a separate book for this table.

2 No advanced mathematics booklets were administered to age/class 9.

3 No advanced science booklets were administered to age/class 9 or 13.
381

Table 19-2

Number of Items Administered, by Sample and Age Class

———— Age Class ————

9 13 17

Distinct

Items

COMMON BACKGROUND

Cross-Sectional (Main Math) 24 26 36 42

Cross-Sectional (Main Science) 24 26 36 45

Reading and Writing Long-Term Trend 37 34 48 48

Math and Science Long-Term Trend 28 30 48 58

MATH MAIN

Background 25 31 44 56

Cognitive — Main 144 178 183 358

Cognitive — Estimation 31 32 38 76

Cognitive — Theme 14 22 18 45

Cognitive — Advanced 0 22 22 44

Motivation 5 5 5 5

SCIENCE MAIN

Background 39 42 53 68

Cognitive — Main 141 194 190 439

Cognitive — Advanced 0 0 66 66

Motivation 5 5 5 5

LONG-TERM TREND

Reading Background 40 42 78 81

Reading Cognitive 105 108 96 193

Writing Background 53 65 65 77

Writing Cognitive 6 6 6 12

Mathematics Background 3 29 39 49

Mathematics Cognitive 68 96 94 184

Science Background 16 29 29 45

Science Cognitive 63 83 82 180

SD/LEP STUDENT QUESTIONNAIRE 114 114 114 58

MATH TEACHER QUESTIONNAIRE

Teacher Background 77 0 59 79

Math Background 17 17 22 36

Math Classroom 40 49 57 96

SCIENCE TEACHER QUESTIONNAIRE

Teacher Background 77 59 0 79

Science Background 14 13 0 14

Science Classroom 59 59 0 59

SCHOOL QUESTIONNAIRE 100 105 127 196
382

Table 19-3

School Characteristics in Main Samples (All Samples)

Grade 4 Grade 8 Grade 12 Total

TOTAL ORIGINAL SAMPLE 723 761 779 2,263

Cooperating 604 592 591 1,787

No Eligibles Enrolled 20 42 28 90

School Refused 99 127 160 386

COOPERATION RATE 86 83 79 83

COOPERATING REPLACEMENTS

FOR REFUSALS

1 1 24

TOTALS

Cooperating Schools 605 5931 5931 1,791

Completed Questionnaires 605 5941 5951 1,794

REGION

Northeast 130 126 123 379

Southeast 134 133 151 418

Central 165 163 145 473

West 176 170 174 520

1

 Occasionally schools with a completed questionnaire had no eligible students, so they were not included as participating

cooperating schools.
383

Table 19-3 (continued)

School Characteristics in Main Samples (All Samples)

Grade 4 Grade 8 Grade 12 Total

SCHOOL TYPE

Public 387 335 428 1,150

Nonpublic 200 243 151 594

Private 81 91 90 262

Catholic 119 152 61 332

BIA 0 0 1 1

DoDea 0 0 0

NUMBER OF TEACHERS

Unclassified 0 0 0 0

1-4 15 9 4 28

5-9 84 61 18 163

10-19 141 149 52 342

20-49 311 244 202 757

50-74 30 89 122 241

75-99 5 20 69 94

100+ 1 8 113 122

Missing 18 14 15 47

NUMBER OF STUDENTS

Unclassified 0 0 0 0

1-99 34 18 22 74

100-299 176 184 103 463

300-499 173 113 69 355

500-749 133 105 89 327

750-999 53 86 55 194

1000-1499 12 51 89 152

1500+ 6 23 153 182

Missing 18 14 15 47
384

Table 19-4

School Characteristics in Long-Term Trend Samples

———— Age Class ————

9 13 17 TOTAL

TOTAL ORIGINAL SAMPLE 291 316 237 844

Cooperating 240 238 191 669

No Eligibles Enrolled 8 27 2 37

School Refused 43 51 44 138

COOPERATION RATE 85 84 81 81

COOPERATING REPLACEMENTS

FOR REFUSALS 8 4 0 12

TOTALS

 Cooperating Schools 248 242 191 681

 Completed Questionnaires 248 242 191 681

 REGION

Northeast 51 54 36 141

 Southeast 62 64 56 182

 Central 59 54 42 155

 West 76 70 57 203
385

Table 19-5

Numbers of Responses to Teacher Questionnaires and Students Matched with Teacher Data

Number of —— Number of Students with ——

Teachers

Responding

No

Match

Partial

Match

Complete

Match

MATH

GRADE 4

Main 752 408 99 6,105

Estimation 320 154 11 1,841

Theme 608 351 34 3,405

Advanced 0 0 0 0

GRADE 8

Main 607 953 49 6,144

Estimation 242 274 8 1,901

Theme 437 603 51 3,373

Advanced 330 343 9 1,985

GRADE 12

Main 0 0 0 0

Estimation 0 0 0 0

Theme 0 0 0 0

Advanced 404 393 241 2,331

SCIENCE

GRADE 4

Main 535 646 159 6,500

 Advanced 0 0 0 0

GRADE 8

Main 371 1,258 112 6,404

 Advanced 0 0 0 0

GRADE 12

Main 0 0 0 0

 Advanced 0 0 0 0
386

Table 19-6

Number of Students Assessed and Excluded by Sample and Age Class

———— Age Class ————

9 13 17

ASSESSED STUDENTS 30,178 34,618 33,629

Cross-Sectional 19,745 23,467 25,421

Math 12440 15,693 15,453

Main 6,627 7,146 6,904

Estimation 2,023 2,183 1,849

Theme 3,790 4,027 3,735

Advanced 0 2,337 2,965

Science 7305 7,774 9,968

Main 7305 7,774 7,537

Advanced 0 0 2,431

Long-Term Trend 10,433 11,151 8,208

Reading and Writing 5,019 5,493 4,669

Math and Science 5,414 5,658 3,539

EXCLUDED STUDENTS 2,256 1,698 1,435

Cross-Sectional 1,139 765 722

Math 383 339 297

Main 204 166 116

Estimation 43 56 75

Theme 136 113 99

Advanced 0 4 7

Science 756 426 425

Main 756 426 425

Advanced 0 0 0

Long-Term Trend 1,117 933 713

Reading and Writing 532 481 412

Math and Science 585 452 301
387

Table 19-7

Number of Students in the Mathematics Main and Estimation Samples

by Subgroup Classification, Grades 4, 8, and 12

MAIN ESTIMATION

Grade 4 Grade 8 Grade 12 Grade 4 Grade 8 Grade 12

TOTAL 6,627 7,146 6,904 2,023 2,183 1,849 

GENDER

Male 3,290 3,597 3,244 994 1,052 898

Female 3,337 3,549 3,660 1,029 1,131 951

RACE/ETHNICITY

White 4,125 4,501 4,596 1,193 1,407 1,258

Black 1,106 1,193 1,106 348 370 273

Hispanic 974 911 732 328 247 229

Asian American 250 408 339 98 124 72

American Indian 149 110 115 53 26 10

Unclassified 23 23 16 3 9 7

REGION

Northeast 1,414 1,312 1,414 471 489 297

Southeast 1,669 1,883 1,924 540 520 509

Central 1,606 1,726 1,675 396 549 470

West 1,938 2,225 1,891 616 625 573

PARENT’S EDUCATION

Less Than High School 219 466 462 77 113 125

High School 837 1,503 1,300 227 438 305

Greater Than High School 462 1,310 1,741 146 361 390

Graduated College 2,804 3,112 3,177 852 994 985

Unknown 2,232 736 200 681 247 39

TYPE OF LOCATION

 Central City 2,380 3,218 2,555 859 988 823

Urban Fringe/Large Town 2,794 2,186 2,428 721 698 618

Rural/Small Town 1,453 1,742 1,921 443 497 408

SCHOOL TYPE

Public 5,215 5,590 5,398 1,528 1,707 1,340

Nonpublic 1,412 1,556 1,455 495 476 509

Private 458 576 521 164 117 200

Catholic 954 980 934 331 359 309

BIA 0 0 51 0 0 0

DoDEA 0 0 0 0 0 0

MODAL AGE

Younger 35 48 92 12 4 21

At Modal Age 4,197 4,380 4,441 1,335 1,333 1,194

Older 2,395 2,718 2,371 676 846 634
388

Table 19-8

Number of Students in the Mathematics Theme and Advanced Samples

by Subgroup Classification, Grades 4, 8 and 12

THEME ADVANCED

Grade 4 Grade 8 Grade 12 Grade 4* Grade 8 Grade 12

TOTAL 3,790 4,027 3,735 0 2,337 2,971

GENDER

Male 1,905 2,030 1,797 0 1,130 1,532

Female 1,885 1,997 1,938 0 1,207 1,439

RACE/ETHNICITY

White 2,206 2,440 2,279 0 1,650 2,001 

Black 655 731 695 0 280 319

Hispanic 672 641 497 0 216 327 

Asian American 169 140 228 0 149 306

American Indian 84 65 26 0 33 12 

Unclassified 4 10 10 0 9 6

REGION

Northeast 723 608 851 0 413 638

Southeast 1,037 1,125 1,025 0 552 800

Central 887 937 742 0 626 666

West 1,143 1,357 1,117 0 746 867

PARENT’S EDUCATION

Less Than High School 162 320 296 0 84 129

High School 498 922 746 0 308 398

Greater Than High School 277 732 980 0 445 664

Graduated College 1,494 1,648 1,578 0 1,352 1,709

Unknown 1,354 396 125 0 121 54 

TYPE OF LOCATION

Central City 1,721 1,495 1,452 0 968 1,106 

Urban Fringe/Large Town 1,289 1,798 1,339 0 902 1,106

Rural/Small Town 780 734 944 0 467 759

SCHOOL TYPE

Public 3,034 3,438 3,075 0 1,661 2,130 

Nonpublic 756 589 660 0 676 841

Private 299 219 235 0 234 346 

Catholic 457 370 425 0 442 495

BIA 0 0 0 0 0 0 

DoDEA 0 0 0 0 0 0

MODAL AGE

Younger 16 21 63 0 18 51

At Modal Age 2,467 2,417 2,427 0 1,594 2,113

Older 1,307 1,589 1,245 0 725 807

*Advanced students not sampled for Grade 4.
389

Table 19-9

Number of Students in the Science Main and Advanced

Samples by Subgroup Classification, Grades 4, 8, and 12

MAIN ADVANCED

Grade 4 Grade 8 Grade 12 Grade 4* Grade 8* Grade 12

TOTAL 7,305 7,774 7,537 0 0 2,431

GENDER

Male 3,651 3,872 3,547 0 0 1,167

Female 3,654 3,902 3,990 0 0 1,264

RACE/ETHNICITY

White 4,106 4,292 4,748 0 0 1,714

Black 1,251 1,492 1,225 0 0 293

Hispanic 1,352 1,426 1,015 0 0 197

Asian American 356 382 458 0 0 209

American Indian 223 149 70 0 0 12

Unclassified 17 33 21 0 0 6

REGION

Northeast 1,503 1,068 1,562 0 0 541

Southeast 1,843 2,246 2,148 0 0 695

Central 1,699 1,595 1,589 0 0 634

West 2,260 2,865 2,238 0 0 561

PARENT’S EDUCATION

Less Than High School 271 553 606 0 0 87

High School 938 1,471 1,414 0 0 272

Greater Than High School 544 1,428 1,879 0 0 526

Graduated College 2,994 3,400 3,308 0 0 1,476

Unknown 2,433 774 211 0 0 0

TYPE OF LOCATION

Central City 3,228 3,055 3,080 0 0 949

Urban Fringe/Large Town 2,769 2,963 2,488 0 0 895

Rural/Small Town 1,308 1,756 1,969 0 0 587

SCHOOL TYPE

Public 5,814 6,376 6,112 0 0 1,739

Nonpublic 1,491 1,398 1,425 0 0 692

Private 499 597 499 0 0 185

Catholic 992 801 926 0 0 507

BIA 0 0 0 0 0 0

DODEA 0 0 0 0 0 0

MODAL AGE

Younger 46 46 93 0 0 38

At Modal Age 4,739 4,553 4,802 0 0 1,720

Older 2,520 3,175 2,642 0 0 673

*Advanced students not sampled for Grade 4 and Grade 8.
390

Table 19-10

Number of Excluded Students in the Mathematics Main and Estimation

Samples by Subgroup Classification, Grades 4, 8, and 12

MAIN ESTIMATION

Grade 4 Grade 8 Grade 12 Grade 4 Grade 8 Grade 12

TOTAL 204 166 116 43 56 75

GENDER

Male 122 104 72 33 34 44 

Female 82 62 44 10 22 31

RACE/ETHNICITY

White 92 100 65 24 33 32

Black 34 30 22 7 8 16

Hispanic 66 18 25 9 12 23

Asian American 8 10 3 2 3 4

American Indian 3 3 1 1 0 0

Unclassified 1 5 0 0 0 0

REGION

Northeast 21 45 22 5 19 8

Southeast 49 34 27 9 11 9

Central 29 36 18 14 11 9

West 105 51 49 15 15 49

PARENT’S EDUCATION

Less Than High School 0 0 0 0 0 0

High School 0 0 0 0 0 0

Greater Than High School 0 0 0 0 0 0

Graduated College 0 0 0 0 0 0

Unknown 0 0 0 0 0 0

TYPE OF LOCATION

Central City 82 64 40 10 29 36

Urban Fringe/Large Town 61 50 44 21 15 13 

Rural/Small Town 61 52 32 12 12 26

SCHOOL TYPE

Public 197 162 115 43 56 75

Nonpublic 7 4 1 0 0 0

Private 1 0 0 0 0 0

Catholic 6 4 1 0 0 0

BIA 0 0 0 0 0 0

DoDEA 0 0 0 0 0 0

MODAL AGE

Younger 0 3 1 0 0 0

At Modal Age 106 48 31 22 21 17

Older 98 115 84 21 35 58
391

Table 19-11

Number of Excluded Students in the Mathematics Theme and Advanced

Sample by Subgroup Classification, Grades 4, 8, and 12

THEME ADVANCED

Grade 4 Grade 8 Grade 12 Grade 4* Grade 8 Grade 12

TOTAL 136 113 99 0 4 7 

GENDER

Male 71 59 55 0 4 6

Female 65 54 44 0 0 1

RACE/ETHNICITY

White 37 34 55 0 2 2

Black 26 30 17 0 0 0

Hispanic 60 44 19 0 0 0

Asian American 9 3 7 0 2 5

American Indian 3 2 1 0 0 0

Unclassified 1 0 0 0 0 0

REGION

Northeast 18 15 30 0 0 2

Southeast 23 37 25 0 2 0

Central 29 15 8 0 0 2

West 66 46 36 0 2 3

PARENT’S EDUCATION

Less Than High School 0 0 0 0 0 0

High School 0 0 0 0 0 0

Greater Than High School 0 0 0 0 0 0

Graduated College 0 0 0 0 0 0 

Unknown 0 0 0 0 0 0

TYPE OF LOCATION

Central City 91 45 52 0 2 4

Urban Fringe/Large Town 35 47 31 0 0 1

Rural/Small Town 10 21 16 0 2 2

SCHOOL TYPE

Public 133 112 91 0 4 7

Nonpublic 3 1 8 0 0 0

Private 1 0 8 0 0 0

Catholic 2 1 0 0 0 0

BIA 0 0 0 0 0 0

DoDEA 0 0 0 0 0 0

MODAL AGE

Younger 2 3 1 0 0 0

At Modal Age 77 46 21 0 2 3

Older 57 64 77 0 2 4

*Advanced students not sampled for Grade 4.
392

Table 19-12

Number of Excluded Students in the Science Main and Advanced Samples

by Subgroup Classification, Grades 4 ,8, and 12

MAIN ADVANCED

Grade 4 Grade 8 Grade 12 Grade 4 Grade 8 Grade 12

TOTAL 756 426 425 0 0 0

SEX

Male 457 265 259 0 0 0

Female 299 161 166 0 0 0

RACE/ETHNICITY

White 239 145 185 0 0 0

Black 124 98 103 0 0 0

Hispanic 317 159 99 0 0 0

Asian American 65 15 33 0 0 0

American Indian 5 7 2 0 0 0

Unclassified 6 2 3 0 0 0

REGION

Northeast 91 38 84 0 0 0

Southeast 170 119 112 0 0 0

Central 132 40 57 0 0 0

West 363 229 172 0 0 0

PARENT’S EDUCATION

Less Than High School 0 0 0 0 0 0

High School 0 1 0 0 0 0

Greater Than High School 0 0 0 0 0 0

Graduated College 0 0 0 0 0 0

Unknown 0 0 0 0 0 0

TYPE OF LOCATION

Central City 482 187 212 0 0 0

Urban Fringe/Large Town 178 136 127 0 0 0

Rural/Small Town 96 103 86 0 0 0

SCHOOL TYPE

Public 752 424 419 0 0 0

Nonpublic 4 2 6 0 0 0

Private 2 0 5 0 0 0

Catholic 2 2 1 0 0 0

BIA 0 0 0 0 0 0

DoDEA 0 0 0 0 0 0

MODAL AGE

<Modal Age 8 5 6 0 0 0

=Modal Age 363 144 113 0 0 0

>Modal Age 385 277 306 0 0 0
393

Table 19-13

Number of Students in the Reading and Writing Long-Term Trend

Sample by Type of Eligibility and Subgroup Classification, Age 9/Grade 4

Age Grade Age and Grade Age or Grade

TOTAL 3,654 3,789 2,424 5,019

GENDER

Male 1,808 1,838 1,128 2,518

Female 1,846 1,951 1,296 2,501

RACE/ETHNICITY

White 2,067 2,183 1,356 2,894

Black 598 634 421 811

Hispanic 741 727 462 1,006

Asian American 139 151 126 164

American Indian 99 83 50 132

Unclassified 10 11 9 12

REGION

Northeast 795 861 630 1,026

Southeast 1,027 1,019 622 1,424

Central 707 747 413 1,041

West 1,125 1,162 759 1,528

PARENT’S EDUCATION

Less Than High School 158 176 99 235

High School 574 615 367 822

Greater Than High School 187 181 117 251

Graduated College 1,430 1,562 1,016 1,976

Unknown 1,274 1,212 806 1,680

TYPE OF LOCATION

Central City 1,513 1,635 1,019 2,129

Urban Fringe/Large Town 1,271 1,297 893 1,675

Rural/Small Town 870 857 512 1,215

SCHOOL TYPE

Public 3,237 3,342 2,116 4,463

Nonpublic 417 447 308 556

Private 174 169 116 227

Catholic 243 278 192 329

BIA 0 0 0 0

DoDEA 0 0 0 0
394

Table 19-14

Number of Students in the Reading and Writing Long-Term Trend

Sample by Type of Eligibility and Subgroup Classification, Age 13/Grade 8

Age Grade Age and Grade Age or Grade

TOTAL 3,847 4,150 2,504 5,493

GENDER

Male 1,870 2,060 1,124 2,806

Female 1,977 2,090 1,380 2,687

RACE/ETHNICITY

White 2,389 2,550 1,518 3,421

Black 540 593 348 785

Hispanic 565 635 373 827

Asian American 222 226 178 270

American Indian 125 141 84 182

Unclassified 6 5 3 8

REGION

Northeast 811 894 582 1,123

Southeast 1,088 1,159 662 1,585

 Central 771 854 478 1,147

 West 1,177 1,243 782 1,638

PARENT’S EDUCATION

 Less Than High School 217 301 143 375

 High School 1,089 1,172 690 1,571

 Greater Than High School 406 441 281 566

 Graduated College 1,725 1,846 1,164 2,407

 Unknown 394 375 218 551

TYPE OF LOCATION

 Central City 1,441 1,560 937 2,064

 Urban Fringe/Large Town 1,412 1,455 952 1,915

 Rural/Small Town 994 1,135 615 1,514

SCHOOL TYPE

 Public 3,421 3,720 2,217 4,924

 Nonpublic 410 396 274 532

 Private 190 190 138 242

 Catholic 220 206 136 290

 BIA 16 34 13 37

 DODEA 0 0 0 0
395

Table 19-15

Number of Students in the Reading and Writing Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 17/Grade 11

Age Grade Age and Grade Age or Grade

TOTAL 3,681 3,737 2,749 4,669

GENDER

Male 1,874 1,943 1,356 2,461

Female 1,807 1,794 1,393 2,208

RACE/ETHNICITY

White 2,528 2,573 1,986 3,115

Black 449 440 279 610

Hispanic 465 468 302 631

Asian American 163 178 123 218

American Indian 69 67 52 84

Unclassified 7 11 7 11

REGION

Northeast 682 721 523 880

Southeast 1,063 1,038 748 1,353

Central 871 900 686 1,085

West 1,065 1,078 792 1,351

PARENT’S EDUCATION

Less Than High School 251 254 151 354

High School 945 947 675 1,217

Greater Than High School 672 692 528 836

Graduated College 1,673 1,717 1,316 2074

Unknown 103 96 61 138

TYPE OF LOCATION

Central City 1,111 1,101 780 1,432

Urban Fringe/Large Town 1,537 1,577 1,196 1,918

Rural/Small Town 1,033 1,059 773 1,319

SCHOOL TYPE

Public 3,384 3,411 2,511 4,284

Nonpublic 289 318 230 377

Private 140 145 105 180

Catholic 149 173 125 197

BIA 8 8 8 8

DODEA 0 0 0 0
396

Table 19-16

Number of Students in the Mathematics and Science Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 91

Age Grade Age and Grade Age or Grade

TOTAL 5414 3,665 3,665 5,414

GENDER

Male 2,709 1,766 1,766 2,709

Female 2,705 1,899 1,899 2,705

RACE/ETHNICITY

White 3,204 2,146 2,146 3,204

Black 801 578 578 801

Hispanic 1,075 687 687 1,075

Asian American 188 156 156 188

American Indian 134 88 88 134

Unclassified 12 10 10 12

REGION

Northeast 1,142 918 918 1,142

Southeast 1,436 906 906 1,436

Central 1,188 698 698 1,188

West 1,648 1,143 1,143 1,648

PARENT’S EDUCATION

Less Than High School 230 146 146 230

High School 713 466 466 713

Greater Than High School 386 289 289 386

Graduated College 2,274 1,604 1,604 2,274

Unknown 1,792 1,150 1,150 1,792

TYPE OF LOCATION

Central City 2,485 1,721 1,721 2,485

Urban Fringe/Large Town 1,670 1,198 1,198 1,670

Rural/Small Town 1,259 746 746 1,259

SCHOOL TYPE

Public 4,790 3,231 3,231 4,790

Nonpublic 609 422 422 609

Private 162 104 104 162

Catholic 447 318 318 447

BIA 15 12 12 15

DODEA 0 0 0 0

1 Note: Since this is an age-only sample, the number of students who are age-eligible only will be the same as the number of

students who are age- or grade-eligible. Likewise, the number of students who are grade-eligible only will be the same as the

number of students who are both age- and grade-eligible.
397

Table 19-17

Number of Students in the Mathematics and Science Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 131

Age Grade Age and Grade Age or Grade

TOTAL 5,658 3,662 3,662 5,658

GENDER

 Male 2,736 1,652 1,652 2,736

 Female 2,922 2,010 2,010 2,922

RACE/ETHNICITY

 White 3,528 2,272 2,272 3,528

 Black 776 509 509 776

 Hispanic 943 565 565 943

 Asian American 293 234 234 293

 American Indian 112 76 76 112

 Unclassified 6 6 6 6

REGION

 Northeast 1,221 900 900 1,221

 Southeast 1,589 937 937 1,589

 Central 1,129 693 693 1,129

 West 1,719 1,132 1,132 1,719

PARENT’S EDUCATION

 Less Than High School 353 188 188 353

 High School 1,295 815 815 1,295

 Greater Than High School 943 672 672 943

 Graduated College 2,458 1,655 1,655 2,458

 Unknown 587 320 320 587

TYPE OF LOCATION

 Central City 2,063 1,357 1,357 2,063

 Urban Fringe/Large Town 2,047 1,386 1,386 2,047

 Rural/Small Town 1,548 919 919 1,548

SCHOOL TYPE

 Public 5,096 3,260 3,260 5,096

Nonpublic 562 402 402 562

 Private 224 181 181 224

 Catholic 338 221 221 338

 BIA 0 0 0 0

 DODEA 0 0 0 0

1 Note: Since this is an age-only sample, the number of students who are age-eligible only will be the same as the number of

students who are age- or grade-eligible. Likewise, the number of students who are grade-eligible only will be the same as the

number of students who are both age- and grade-eligible.
398

Table 19-18

Number of Students in the Mathematics and Science Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 171

Age Grade Age and Grade Age or Grade

TOTAL 3,539 2,532 2,532 3,539

GENDER

 Male 1,755 1,196 1,196 1,755

 Female 1,784 1,336 1,336 1,784

RACE/ETHNICITY

 White 2,401 1,836 1,836 2,401

 Black 531 329 329 531

 Hispanic 401 244 244 401

 Asian American 155 94 94 155

 American Indian 43 23 23 43

 Unclassified 8 6 6 8

REGION

 Northeast 712 519 519 712

 Southeast 1,122 803 803 1,122

 Central 733 529 529 733

 West 972 681 681 972

PARENT’S EDUCATION

 Less Than High School 236 122 122 236

 High School 757 506 506 757

 Greater Than High School 835 616 616 835

 Graduated College 1,619 1,238 1,238 1,619

 Unknown 71 37 37 71

TYPE OF LOCATION

 Central City 1,311 896 896 1,311

 Urban Fringe/Large Town 1,189 883 883 1,189

 Rural/Small Town 1,039 753 753 1,039

SCHOOL TYPE

 Public 3,257 2,309 2,309 3,257

 Nonpublic 282 223 223 282

 Private 124 99 99 124

 Catholic 158 124 124 158

 BIA 0 0 0 0

 DODEA 0 0 0 0

1 Note: Since this is an age-only sample, the number of students who are age-eligible only will be the same as the number of

students who are age- or grade-eligible. Likewise, the number of students who are grade-eligible only will be the same as the

number of students who are both age- and grade-eligible.
399

Table 19-19

Number of Excluded Students in the Reading and Writing Long-Term Trend Sample by Type of Eligibility

and Subgroup Classification, Age 9/Grade 4

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

TOTAL 345 404 217 532

SEX

Male 207 243 124 326

Female 138 161 93 206

RACE/ETHNICITY

White 133 161 66 228

Black 54 66 30 90

Hispanic 122 134 90 166

Asian American 30 35 26 39

American Indian 3 6 3 6

Unclassified 3 2 2 3

REGION

Northeast 30 46 19 57

Southeast 96 130 52 174

Central 49 47 18 78

West 170 181 128 223

TYPE OF LOCATION

Central City 189 211 130 270

Urban Fringe/Large Town 106 117 69 154

Rural/Small Town 50 76 18 108

SCHOOL TYPE

Public 340 402 215 527

Nonpublic 4 2 2 4

Private 0 0 0 0

Catholic 4 2 2 4

BIA 0 0 0 0

DoDea 0 0 0 0
400

Table 19-20

Number of Excluded Students in the Reading and Writing Long-Term Trend Sample by Type of Eligibility

and Subgroup Classification, Age 13/Grade 8

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

TOTAL 265 303 87 481

SEX

Male 175 211 67 319

Female 90 92 20 162

RACE/ETHNICITY

White 153 187 47 293

Black 33 35 8 60

Hispanic 58 58 23 93

Asian American 10 11 16

American Indian 12 4 18

Unclassified 1 0 0 1

REGION

Northeast 45 46 17 74

Southeast 93 104 21 176

Central 41 74 14 101

West 86 79 35 130

TYPE OF LOCATION

Central City 118 121 35 204

Urban Fringe/Large Town 79 80 35 124

Rural/Small Town 68 102 17 153

SCHOOL TYPE

Public 257 291 84 464

Nonpublic 3 1 0 4

Private 2 1 0 3

Catholic 1 0 0 1

BIA 5 11 3 13

DoDea 0 0 0 0
401

Table 19-21

Number of Excluded Students in the Reading and Writing Long-Term Trend Sample by Type of Eligibility

and Subgroup Classification, Age 17/Grade 11

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

TOTAL 277 227 92 412

SEX

Male 171 139 43 267

Female 106 88 49 145

RACE/ETHNICITY

White 161 151 63 249

Black 65 33 18 80

Hispanic 35 26 7 54

Asian American 10 10 1 19

American Indian 4 7 3 8

Unclassified 2 0 0 2

REGION

Northeast 36 44 18 62

Southeast 117 75 32 160

Central 51 41 15 77

West 73 67 27 113

TYPE OF LOCATION

Central City 80 68 25 123

Urban Fringe/Large Town 107 105 46 166

Rural/Small Town 90 54 21 123

SCHOOL TYPE

Public 277 224 92 409

Nonpublic 0 1 0 1

Private 0 0 0 0

Catholic 0 1 0 1

BIA 0 2 0 2

DoDea 0 0 0 0
402

Table 19-22

Number of Excluded Students in the Mathematics and Science Long-Term Trend Sample by Type of

Eligibility and Subgroup Classification, Age 9/Grade 4

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

TOTAL 585 316 316 585

SEX

Male 360 192 192 360

Female

225

 124 124 225

RACE/ETHNICITY

White 220 103 103 220

Black 96 41 41 96

Hispanic 217 133 133 217

Asian American 45 35 35 45

American Indian 2 1 1 2

Unclassified

5

 3 3 5

REGION

Northeast 65 38 38 65

Southeast 202 72 72 202

Central 80 35 35 80

West 238 171 171 238

TYPE OF LOCATION

Central City 334 200 200 334

Urban Fringe/Large Town 148 91 91 148

Rural/Small Town 103 25 25 103

SCHOOL TYPE

Public 578 314 314 578

Nonpublic 7 2 2 7

Private 0 0 0 0

Catholic 7 2 2 7

BIA 0 0 0 0

DoDea 0 0 0 0
403

Table 19-23

Number of Excluded Students in the Mathematics and Science Long-Term Trend Sample by Type of

Eligibility and Subgroup Classification, Age13/Grade 8

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

TOTAL 452 150 150 452

SEX

Male 286 86 86 286

Female 166 64 64 166

RACE/ETHNICITY

White 239 75 75 239

Black 76 20 20 76

Hispanic 116 47 47 116

Asian American 16 7 7 16

American Indian 2 1 1 2

Unclassified 3 0 0 3

REGION

Northeast 87 34 34 87

Southeast 156 41 41 156

Central 89 25 25 89

West 120 50 50 120

TYPE OF LOCATION

Central City 187 57 57 187

Urban Fringe/Large Town 125 57 57 125

Rural/Small Town 140 36 36 140

SCHOOL TYPE

Public 450 150 150 450

Nonpublic 2 0 0 2

Private 2 0 0 2

Catholic 0 0 0 0

BIA 0 0 0 0

DoDea 0 0 0 0
404

Table 19-24

Number of Excluded Students in the Mathematics and Science Long-Term Trend Sample by Type of

Eligibility and Subgroup Classification, Age17/Grade 11

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

TOTAL 301 110 110 301

SEX

Male 202 67 67 202

Female 99 43 43 99

RACE/ETHNICITY

White 170 75 75 170

Black 55 17 17 55

Hispanic 55 14 14 55

Asian American 16 3 3 16

American Indian 5 1 1 5

Unclassified 0 0 0 0

REGION

Northeast 57 36 36 57

Southeast 103 24 24 103

Central 51 18 18 51

West 90 32 32 90

TYPE OF LOCATION

Central City 109 36 36 109

Urban Fringe/Large Town 107 50 50 107

Rural/Small Town 85 24 24 85

SCHOOL TYPE

Public 298 110 110 298

Nonpublic 3 0 0 3

Private 3 0 0 3

Catholic 0 0 0 0

BIA 0 0 0 0

DoDea 0 0 0 0
405

 Table 19-25

Weighted Percentage of Students in the Mathematics Main and Estimation Samples

by Subgroup Classification, Grades 4, 8, and 12

MAIN ESTIMATION

Grade 4 Grade 8 Grade 12 Grade 4 Grade 8 Grade 12

TOTAL 3,711,786 3,566,392 2,827,040 3,688,821 3,598,564 2,740,931

GENDER

Male 50.8 52.3 47.6 49.3 50.4 47.4

Female 49.2 47.7 52.4 50.7 49.6 52.6

RACE/ETHNICITY

White 67.8 69.1 69.6 68.0 69.6 70.3

Black 14.6 14.2 14.1 14.6 14.1 13.8

Hispanic 12.9 12.3 11.2 13.0 11.9 11.4

Asian American 2.7 3.3 3.6 2.5 3.3 3.9

American Indian 1.7 1.1 1.3 1.9 0.9 0.4

Unclassified 0.2 0.1 0.2 0.1 0.2 0.2

REGION

 Northeast 21.9 20.3 21.8 21.6 20.7 23.5

 Southeast 21.0 23.3 21.6 22.1 21.3 20.6

 Central 24.8 24.3 24.0 24.6 24.4 24.5

West 32.3 32.1 32.6 31.7 33.7 31.5

PARENT’S EDUCATION

Less Than High School 3.8 6.9 6.4 3.8 6.0 6.6

High School 12.5 21.8 18.8 11.6 22.0 17.2

Greater Than High School 7.0 18.5 25.4 7.5 16.3 21.0

Graduated College 39.4 41.7 46.4 40.0 42.5 52.9

Unknown 35.6 10.7 2.7 34.9 11.5 2.1

TYPE OF LOCATION

Central City 30.1 33.3 31.7 37.3 37.1 39.3

Urban Fringe/Large Town 46.2 36.2 39.8 36.5 36.8 34.1

Rural/Small Town 23.7 30.5 28.5 26.2 26.1 26.6

SCHOOL TYPE

Public 89.1 89.3 87.3 85.4 88.6 83.1

Non Public 10.9 10.7 12.0 14.6 11.4 16.9

Private 3.7 4.5 4.2 5.7 4.0 6.9

Catholic 7.2 6.3 7.9 8.9 7.4 10.0

BIA 0.0 0.0 0.6 0.0 0.0 0.0

DoDEA 0.0 0.0 0.0 0.0 0.0 0.0

 MODAL AGE

Younger 0.6 0.6 1.3 0.6 0.1 1.1

At Modal Age 59.6 56.2 64.9 60.8 55.6 65.2

Older 39.8 43.1 33.8 38.6 44.3 33.7
406

Table 19-26

Weighted Percentage of Students in the Mathematics Theme and Advanced Mathematics

Samples by Subgroup Classification, Grades 4, 8, and 12

THEME ADVANCED

Grade 4 Grade 8 Grade 12 Grade 4* Grade 8 Grade 12

TOTAL 3,690,245 3,566,103 2,845,023 0 809,085 696,805

GENDER

Male 51.8 52.6 49.2 0.0 48.3 50.6

Female 48.2 47.4 50.8 0.0 51.7 49.4

RACE/ETHNICITY

White 68.5 70.5 69.3 0.0 71.0 74.1

Black 14.5 13.7 14.2 0.0 14.3 7.2

Hispanic 12.5 11.6 11.6 0.0 6.5 7.9

Asian American 2.9 2.4 4.2 0.0 5.6 10.3

American Indian 1.5 1.7 0.5 0.0 2.2 0.3

Unclassified 0.1 0.1 0.2 0.0 0.4 0.2

REGION

Northeast 21.0 20.7 23.2 0.0 26.9 24.8

Southeast 22.9 22.5 21.2 0.0 16.9 21.0

Central 25.7 23.0 23.4 0.0 29.9 28.6

West 30.4 33.8 32.1 0.0 26.3 25.6

PARENT’S EDUCATION

Less Than High School 4.5 7.9 6.9 0.0 3.0 3.8

High School 12.6 23.8 19.1 0.0 13.7 13.3

Greater Than High School 7.4 18.1 26.5 0.0 20.2 22.6

Graduated College 40.1 40.5 44.2 0.0 57.4 58.5

Unknown 35.3 9.4 3.2 0.0 4.7 1.5

TYPE OF LOCATION

Central City 39.9 28.1 31.6 0.0 36.1 32.0

Urban Fringe/Large Town 37.7 49.2 41.3 0.0 38.7 38.5

Rural/Small Town 22.4 22.7 27.1 0.0 25.2 29.4

SCHOOL TYPE

Public 87.6 90.5 88.7 0.0 83.5 81.5

Nonpublic 12.4 9.5 11.3 0.0 16.5 18.5

Private 5.0 3.9 3.9 0.0 5.3 7.9

Catholic 7.4 5.5 7.4 0.0 11.3 10.6

BIA 0.0 0.0 0.0 0.0 0.0 0.0

DoDEA 0.0 0.0 0.0 0.0 0.0 0.0

MODAL AGE

Younger 0.4 0.6 1.5 0.0 0.7 1.5

At Modal Age 59.0 55.5 64.4 0.0 68.7 72.2

Older 40.5 43.9 34.1 0.0 30.6 26.3

*Advanced students not sampled for Grade 4.
407

Table 19-27

Weighted Percentage of Students in the Science Main and Advanced

Samples by Subgroup Classification, Grades 4, 8, and 12

MAIN ADVANCED

Grade 4 Grade 8 Grade 12 Grade 41 Grade 81 Grade 12

TOTAL 3,618,494 3,564,079 2,903,402 0 0 585,798

GENDER

Male 50.3 50.8 48.4 0.0 0.0 49.2

Female 49.7 49.2 51.6 0.0 0.0 50.8

RACE/ETHNICITY

White 68.8 69.6 69.9 0.0 0.0 74.1

Black 14.6 14.1 14.3 0.0 0.0 9.1

Hispanic 12.2 11.8 11.2 0.0 0.0 7.0

Asian American 2.6 2.6 3.8 0.0 0.0 9.1

American Indian 1.8 1.7 0.7 0.0 0.0 0.5

Unclassified 0.1 0.2 0.1 0.0 0.0 0.2

REGION

Northeast 21.8 22.1 21.6 0.0 0.0 24.4

Southeast 22.4 21.8 21.2 0.0 0.0 21.2

Central 25.7 23.8 24.2 0.0 0.0 30.4

West 30.0 32.3 33.1 0.0 0.0 24.0

PARENT’S EDUCATION

Less Than High School 4.2 6.2 6.6 0.0 0.0 2.7

High School 13.6 19.5 18.2 0.0 0.0 11.2

Greater Than High School 7.4 19.5 25.2 0.0 0.0 20.8

Graduated College 39.7 44.0 45.8 0.0 0.0 62.8

Unknown 33.2 9.1 2.5 0.0 0.0 0.0

TYPE OF LOCATION

Central City 36.9 26.6 35.0 0.0 0.0 31.2

Urban Fringe/Large Town 38.1 45.1 36.3 0.0 0.0 41.3

Rural/Small Town 25.0 28.3 28.7 0.0 0.0 27.4

SCHOOL TYPE

Public 87.5 88.8 87.7 0.0 0.0 83.0

Nonpublic 12.5 11.2 12.3 0.0 0.0 17.0

Private 4.8 4.7 4.4 0.0 0.0 5.4

Catholic 7.7 6.6 7.9 0.0 0.0 11.7

BIA 0.0 0.0 0.0 0.0 0.0 0.0

DODEA 0.0 0.0 0.0 0.0 0.0 0.0

MODAL AGE

Younger 0.7 0.6 1.1 0.0 0.0 1.4

At Modal Age 59.3 54.5 63.6 0.0 0.0 70.9

Older 40.0 44.9 35.3 0.0 0.0 27.7

1

Advanced students not sampled for Grade 4 or Grade 8.
408

Table 19-28

Weighted Percentage of Excluded Students in the Mathematics Main and Estimation

Samples by Subgroup Classification, Grades 4, 8, and 12

MAIN ESTIMATION

 Grade 4 Grade 8 Grade 12 Grade 4 Grade 8 Grade 12

TOTAL 229,564 162,944 88,046 180,417 142,276 110,197

GENDER

Male 62.0 62.1 67.9 77.2 56.8 56.3

Female 38.0 37.9 32.1 22.8 43.2 43.7

RACE/ETHNICITY

White 56.1 65.5 65.2 69.4 64.3 42.7

Black 16.4 18.3 15.9 11.8 12.6 24.4

Hispanic 22.9 11.2 17.6 14.2 19.2 29.3

Asian American 2.6 2.8 1.0 2.5 3.9 3.6

American Indian 1.6 1.4 0.3 2.1 0.0 0.0

Unclassified 0.3 0.8 0.0 0.0 0.0 0.0

REGION

Northeast 12.2 22.0 29.8 12.4 22.1 16.4

Southeast 25.0 20.5 19.7 19.0 20.8 13.3

Central 16.4 24.6 14.4 35.8 18.1 14.9

West 46.4 32.9 36.2 32.8 39.0 55.4

PARENT’S EDUCATION

Less Than High School 0.0 0.0 0.0 0.0 0.0 0.0

High School 0.0 0.0 0.0 0.0 0.0 0.0

Greater Than High School 0.0 0.0 0.0 0.0 0.0 0.0

Graduated College 0.0 0.0 0.0 0.0 0.0 0.0

Unknown 0.0 0.0 0.0 0.0 0.0 0.0

TYPE OF LOCATION

Central City 28.8 38.1 27.9 23.5 49.7 51.1

Urban Fringe/Large Town 35.6 35.6 51.4 46.9 24.5 18.3

Rural/Small Town 35.6 26.3 20.8 29.5 25.7 30.6

SCHOOL TYPE

Public 97.8 99.4 99.5 100.0 100.0 100.0

Non Public 2.2 0.6 0.5 0.0 0.0 0.0

Private 1.0 0.0 0.0 0.0 0.0 0.0

Catholic 1.1 0.6 0.5 0.0 0.0 0.0

BIA 0.0 0.0 0.0 0.0 0.0 0.0

DODEA 0.0 0.0 0.0 0.0 0.0 0.0

MODAL AGE

Younger 0.0 2.4 0.8 0.0 0.0 0.0

At Modal Age 41.0 23.1 36.2 48.8 33.8 20.3

Older 59.0 74.5 63.0 51.2 66.2 79.7
409

Table 19-29

Weighted Percentage of Excluded Students in the Mathematics Theme and Advanced

Samples by Subgroup Classification, Grades 4, 8, and 12

THEME ADVANCED

Grade 4 Grade 8 Grade 12 Grade 41 Grade 8 Grade 12

TOTAL 238,555 160,860 115,182 0 1,042 1,701

GENDER

Male 58.3 55.1 57.1 0.0 100.0 89.2

Female 41.7 44.9 42.9 0.0 0.0 10.8

RACE/ETHNICITY

White 42.0 46.0 67.7 0.0 0.0 36.1

Black 18.2 25.3 13.8 0.0 0.0 0.0

Hispanic 31.3 24.3 14.8 0.0 0.0 0.0

Asian American 4.9 1.7 3.1 0.0 100.0 63.9

American Indian 2.8 2.8 0.6 0.0 0.0 0.0

Unclassified 0.7 0.0 0.0 0.0 0.0 0.0

REGION

Northeast 12.7 12.9 28.5 0.0 0.0 41.3

Southeast 18.9 31.0 21.0 0.0 0.0 0.0

Central 25.7 21.1 16.3 0.0 0.0 26.5

West 42.6 35.0 34.1 0.0 100.0 32.3

PARENT’S EDUCATION

Less Than High School 0.0 0.0 0.0 0.0 0.0 0.0

High School 0.0 0.0 0.0 0.0 0.0 0.0

Greater Than High School 0.0 0.0 0.0 0.0 0.0 0.0

Graduated College 0.0 0.0 0.0 0.0 0.0 0.0

Unknown 0.0 0.0 0.0 0.0 0.0 0.0

TYPE OF LOCATION

Central City 54.8 31.3 42.3 0.0 100.0 41.0

Urban Fringe/Large Town 36.5 40.2 40.1 0.0 0.0 22.9

Rural/Small Town 8.7 28.5 17.6 0.0 0.0 36.1

SCHOOL TYPE

Public 98.6 99.2 95.2 0.0 100.0 100.0

Nonpublic 1.4 0.8 4.8 0.0 0.0 0.0

Private 0.3 0.0 4.8 0.0 0.0 0.0

Catholic 1.1 0.8 0.0 0.0 0.0 0.0

BIA 0.0 0.0 0.0 0.0 0.0 0.0

DODEA 0.0 0.0 0.0 0.0 0.0 0.0

MODAL AGE

Younger 2.7 1.5 0.4 0.0 0.0 0.0

At Modal Age 46.8 30.7 22.7 0.0 100.0 59.0

Older 50.6 67.8 76.8 0.0 0.0 41.0

1 Advanced students not sampled for Grade 4.
410

Figure 19-30

Weighted Percentage of Excluded Students in the Science Main and Advanced Samples by Subgroup

Classification, Grades 4, 8, and 12

MAIN ADVANCED

Grade 4 Grade 8 Grade 12 Grade 4 Grade 8 Grade 12

TOTAL 322,613 164,891 119,759 0 0 0

SEX

Male 63.2 63.6 61.9 0.0 0.0 0.0

Female 36.4 38.1 38.1 0.0 0.0 0.0

RACE/ETHNICITY

White 49.1 54.6 54.1 0.0 0.0 0.0

Black 16.6 19.4 22.4 0.0 0.0 0.0

Hispanic 27.6 21.6 18.1 0.0 0.0 0.0

Asian American 5.3 2.2 4.2 0.0 0.0 0.0

American Indian 0.8 2.0 0.5 0.0 0.0 0.0

Unclassified 0.5 0.2 0.7 0.0 0.0 0.0

REGION

Northeast 14.6 12.7 21.8 0.0 0.0 0.0

Southeast 23.6 23.3 22.6 0.0 0.0 0.0

Central 22.2 21.9 18.3 0.0 0.0 0.0

West 39.6 42.2 37.2 0.0 0.0 0.0

PARENT’S EDUCATION

Less Than High School 0.0 0.0 0.0 0.0 0.0 0.0

High School 0.0 0.5 0.0 0.0 0.0 0.0

Greater Than High School 0.0 0.0 0.0 0.0 0.0 0.0

Graduated College 0.0 0.0 0.0 0.0 0.0 0.0

Unknown 0.0 0.0 0.0 0.0 0.0 0.0

TYPE OF LOCATION

Central City 52.0 29.1 40.1 0.0 0.0 0.0

Urban Fringe/Large Town 24.8 33.6 32.8 0.0 0.0 0.0

Rural/Small Town 23.3 37.3 27.1 0.0 0.0 0.0

SCHOOL TYPE

Public 99.5 99.4 99.0 0.0 0.0 0.0

Nonpublic 0.5 0.6 1.0 0.0 0.0 0.0

Private 0.2 0.0 0.8 0.0 0.0 0.0

Catholic 0.3 0.6 0.2 0.0 0.0 0.0

BIA 0.0 0.0 0.0 0.0 0.0 0.0

DoDEA 0.0 0.0 0.0 0.0 0.0 0.0

MODAL AGE

<Modal Age 0.9 1.0 1.1 0.0 0.0 0.0

=Modal Age 37.7 25.7 25.2 0.0 0.0 0.0

>Modal Age 61.3 73.2 73.7 0.0 0.0 0.0
411

Table 19-31

Weighted Percentage of Students in the Reading and Writing Long-Term Trend

Sample by Type of Eligibility and Subgroup Classification, Age 9/Grade 4

Age Grade Age and Grade Age or Grade

TOTAL 3,170,010 3,579,694 2,119,331 4,630,373

GENDER

Male 49.5 49.3 46.3 50.8

Female 50.5 50.7 53.7 49.2

RACE/ETHNICITY

White 66.7 68.3 67.5 67.6

Black 15.4 14.6 15.1 15.0

Hispanic 12.9 12.7 12.4 13.0

Asian American 2.6 2.6 3.5 2.3

American Indian 2.1 1.5 1.3 2.0

Unclassified 0.2 0.2 0.3 0.2

REGION

Northeast 23.5 22.1 26.0 21.3

Southeast 23.9 23.8 22.2 24.6

Central 24.6 25.5 22.1 26.4

West 28.0 28.6 29.7 27.7

PARENT’S EDUCATION

Less Than High School 4.0 4.4 3.9 4.4

High School 15.3 16.0 14.5 16.2

Greater Than High School 5.0 4.7 4.6 4.9

Graduated College 39.5 41.6 42.1 39.9

Unknown 35.5 32.2 34.1 33.6

TYPE OF LOCATION

Central City 40.8 42.5 41.6 41.8

Urban Fringe/Large Town 34.3 33.4 36.6 32.6

Rural/Small Town 24.8 24.0 21.8 25.6

SCHOOL TYPE

Public 86.2 86.3 85.2 86.7

Nonpublic 13.8 13.7 14.8 13.3

Private 6.8 6.2 6.7 6.4

Catholic 7.1 7.4 8.1 6.9

BIA 0.0 0.0 0.0 0.0

DODEA 0.0 0.0 0.0 0.0
412

Table 19-32

Weighted Percentage of Students in the Reading and Writing Long-Term Trend Sample

 by Type of Eligibility and Subgroup Classification, Age 13/Grade 8

Age Grade Age and Grade Age or Grade

TOTAL 3,173,938 3,465,078 1,943,322 4,695,694

GENDER

Male 48.6 50.3 44.4 51.6

Female 51.4 49.7 55.6 48.4

RACE/ETHNICITY

White 66.6 69.1 68.9 67.4

Black 15.1 14.4 13.6 15.2

Hispanic 12.7 12.0 12.2 12.4

Asian American 3.5 2.8 3.8 2.9

American Indian 2.0 1.7 1.5 2.0

Unclassified 0.1 0.1 0.1 0.1

REGION

Northeast 23.1 22.3 24.7 21.8

Southeast 25.5 25.0 23.8 25.9

Central 21.8 23.4 21.1 23.3

West 29.6 29.3 30.4 29.0

PARENT’S EDUCATION

Less Than High School 5.3 7.0 5.3 6.5

High School 28.8 28.5 27.8 28.9

Greater Than High School 10.4 10.6 11.0 10.3

Graduated College 45.0 45.0 47.3 44.1

Unknown 10.1 8.5 8.4 9.7

TYPE OF LOCATION

Central City 36.2 36.0 35.4 36.4

Urban Fringe/Large Town 35.7 33.9 38.2 33.3

Rural/Small Town 28.1 30.1 26.4 30.3

SCHOOL TYPE

Public 88.5 89.4 87.9 89.4

Nonpublic 11.2 9.9 11.7 10.0

Private 4.8 4.6 5.7 4.3

Catholic 6.3 5.2 6.0 5.7

BIA 0.4 0.7 0.4 0.6

DODEA 0.0 0.0 0.0 0.0
413

Table 19-33

Weighted Percentage of Students in the Reading and Writing Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 17/Grade 11

Age Grade Age and Grade Age or Grade

TOTAL 3,224,505 3,160,512 1,977,398 4,407,619

GENDER

Male 51.4 52.4 48.9 53.3

Female 48.6 47.6 51.1 46.7

RACE/ETHNICITY

White 69.3 68.3 74.2 66.4

Black 14.4 15.1 11.7 16.1

Hispanic 11.7 12.4 9.9 13.0

Asian American 3.1 2.9 2.6 3.2

American Indian 1.4 1.2 1.3 1.3

Unclassified 0.1 0.2 0.2 0.1

REGION

Northeast 22.3 24.9 23.9 23.5

Southeast 24.2 22.7 21.4 24.4

Central 24.5 24.5 26.0 23.8

West 29.0 27.9 28.8 28.3

PARENT’S EDUCATION

Less Than High School 6.8 6.7 4.9 7.6

High School 26.7 26.0 24.7 27.1

Greater Than High School 18.0 18.7 19.8 17.7

Graduated College 44.7 45.2 47.9 43.6

Unknown 2.8 2.7 2.1 3.0

TYPE OF LOCATION

Central City 34.7 34.2 32.3 35.4

Urban Fringe/Large Town 39.2 39.5 41.4 38.5

Rural/Small Town 26.1 26.3 26.4 26.1

SCHOOL TYPE

Public 91.7 90.8 90.7 91.5

Nonpublic 8.1 9.0 9.0 8.3

Private 3.5 3.4 3.6 3.4

Catholic 4.6 5.6 5.4 4.9

BIA 0.2 0.2 0.4 0.2

DODEA 0.0 0.0 0.0 0.0
414

Table 19-34

Weighted Percentage of Students in the Mathematics and Science Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 91

Age Grade Age and Grade Age or Grade

TOTAL 3,320,984 2,207,888 2,207,888 3,320,984

GENDER

Male 49.6 48.0 48.0 49.6

Female 50.4 52.0 52.0 50.4

RACE/ETHNICITY

White 69.0 68.1 68.1 69.0

Black 14.3 15.5 15.5 14.3

Hispanic 12.4 11.6 11.6 12.4

Asian American 2.3 2.9 2.9 2.3

American Indian 1.9 1.8 1.8 1.9

Unclassified 0.2 0.2 0.2 0.2

REGION

Northeast 21.7 25.7 25.7 21.7

Southeast 23.4 22.5 22.5 23.4

Central 25.0 21.4 21.4 25.0

West 30.0 30.4 30.4 30.0

PARENT’S EDUCATION

Less Than High School 3.9 3.8 3.8 3.9

High School 12.5 12.2 12.2 12.5

Greater Than High School 7.1 7.9 7.9 7.1

Graduated College 42.7 45.0 45.0 42.7

Unknown 33.1 30.6 30.6 33.1

Type Of Location

Central City 42.3 43.0 43.0 42.3

Urban Fringe/Large Town 32.4 34.3 34.3 32.4

Rural/Small Town 25.4 22.7 22.7 25.4

SCHOOL TYPE

Public 86.8 86.3 86.3 86.8

Nonpublic 12.9 13.4 13.4 12.9

Private 4.0 3.8 3.8 4.0

Catholic 8.9 9.5 9.5 8.9

BIA 0.3 0.3 0.3 0.3

DODEA 0.0 0.0 0.0 0.0

1

 Note: Since this is an age-only sample, the number of students who are age-eligible only will be the same as the number of

students who are age- or grade-eligible. Likewise, the number of students who are grade-eligible only will be the same as the

number of students who are both age- and grade-eligible.
415

Table 19-35

Weighted Percentage of Students in the Mathematics and Science Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 131

Age Grade Age and Grade Age or Grade

TOTAL 3,360,572 2,128,872 2,128,872 3,360,572

 GENDER

 Male 48.5 45.2 45.2 48.5

 Female 51.5 54.8 54.8 51.5

 RACE/ETHNICITY

 White 68.6 68.0 68.0 68.6

 Black 14.3 14.7 14.7 14.3

 Hispanic 12.2 11.3 11.3 12.2

 Asian American 3.6 4.4 4.4 3.6

 American Indian 1.3 1.4 1.4 1.3

 Unclassified 0.1 0.1 0.1 0.1

 REGION

 Northeast 22.0 25.0 25.0 22.0

 Southeast 24.9 22.9 22.9 24.9

 Central 23.1 21.9 21.9 23.1

 West 30.0 30.2 30.2 30.0

 PARENT’S EDUCATION

 Less Than High School 5.5 4.3 4.3 5.5

 High School 22.7 22.1 22.1 22.7

 Greater Than High School 16.7 18.7 18.7 16.7

 Graduated College 45.0 46.7 46.7 45.0

 Unknown 9.7 7.9 7.9 9.7

 TYPE OF LOCATION

 Central City 35.7 36.2 36.2 35.7

 Urban Fringe/Large Town 35.9 38.1 38.1 35.9

 Rural/Small Town 28.3 25.6 25.6 28.3

 SCHOOL TYPE

 Public 88.8 88.2 88.2 88.8

 Nonpublic 11.2 11.8 11.8 11.2

 Private 4.5 5.1 5.1 4.5

 Catholic 6.7 6.7 6.7 6.7

 BIA 0.0 0.0 0.0 0.0

 DODEA 0.0 0.0 0.0 0.0

1 Note: Since this is an age-only sample, the number of students who are age-eligible only will be the same as the number of

students who are age- or grade-eligible. Likewise, the number of students who are grade-eligible only will be the same as the

number of students who are both age- and grade-eligible.
416

Table 19-36

Weighted Percentage of Students in the Mathematics and Science Long-Term Trend Sample

by Type of Eligibility and Subgroup Classification, Age 171

Age Grade Age and Grade Age or Grade

TOTAL 3,185,309 2,250,256 2,250,256 3,185,309

GENDER

 Male 49.5 47.0 47.0 49.5

 Female 50.5 53.0 53.0 50.5

RACE/ETHNICITY

 White 69.3 73.8 73.8 69.3

 Black 14.5 12.6 12.6 14.5

 Hispanic 11.6 9.7 9.7 11.6

 Asian American 3.5 3.0 3.0 3.5

 American Indian 0.8 0.7 0.7 0.8

 Unclassified 0.2 0.2 0.2 0.2

REGION

 Northeast 23.4 24.1 24.1 23.4

 Southeast 22.4 22.1 22.1 22.4

 Central 24.5 24.2 24.2 24.5

 West 29.7 29.6 29.6 29.7

PARENT’S EDUCATION

 Less Than High School 6.4 4.4 4.4 6.4

 High School 20.9 19.6 19.6 20.9

 Greater Than High School 23.8 24.4 24.4 23.8

 Graduated College 46.0 49.2 49.2 46.0

 Unknown 2.0 1.5 1.5 2.0

TYPE OF LOCATION

 Central City 36.9 35.3 35.3 36.9

 Urban Fringe/Large Town 37.9 39.4 39.4 37.9

 Rural/Small Town 25.1 25.3 25.3 25.1

SCHOOL TYPE

 Public 91.4 90.7 90.7 91.4

 Nonpublic 8.6 9.3 9.3 8.6

 Private 3.5 3.7 3.7 3.5

 Catholic 5.1 5.6 5.6 5.1

 BIA 0.0 0.0 0.0 0.0

 DODEA 0.0 0.0 0.0 0.0

1

 Note: Since this is an age-only sample, the number of students who are age-eligible only will be the same as the number of

students who are age- or grade-eligible. Likewise, the number of students who are grade-eligible only will be the same as the

number of students who are both age- and grade-eligible.
417

Table 19-37

Weighted Percentage of Excluded Students in the Reading and Writing Long-Term Trend Sample by Type

of Eligibility and Subgroup Classification, Age 9/Grade 4

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

SEX

Male 61.2 63.6 57.7 64.0

Female 38.8 36.4 42.3 36.0

RACE/ETHNICITY

White 48.3 57.6 42.7 57.5

Black 18.3 16.9 14.9 17.8

Hispanic 24.3 19.0 30.1 18.5

Asian American 7.6 4.7 10.4 4.5

American Indian 0.8 1.6 1.3 1.4

Unclassified 0.7 0.1 0.6 0.2

REGION

Northeast 9.4 13.1 7.8 12.9

Southeast 26.9 36.4 23.5 35.8

Central 17.9 18.5 11.7 19.7

West 45.7 32.0 57.0 31.5

TYPE OF LOCATION

Central City 50.2 42.8 55.0 42.8

Urban Fringe/Large Town 34.5 28.7 36.7 29.1

Rural/Small Town 15.3 28.5 8.3 28.1

SCHOOL TYPE

Public 98.4 99.7 98.7 99.4

Nonpublic 1.3 0.3 1.3 0.4

Private 0.0 0.0 0.0 0.0

Catholic 1.3 0.3 1.3 0.4

BIA 0.0 0.0 0.0 0.0

DoDea 0.0 0.0 0.0 0.0

ESTIMATED TOTAL

POPULATION 106,503 266,020 64,398 308,125
418

 Table 19-38

Weighted Percentage of Excluded Students in the Reading and Writing Long-Term Trend Sample by Type

of Eligibility and Subgroup Classification, Age 13/Grade 8

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

SEX

Male 68.4 67.3 77.6 66.8

Female 31.6 32.7 22.4 33.2

RACE/ETHNICITY

White 57.5 69.5 60.5 66.6

Black 18.4 13.5 9.6 15.4

Hispanic 17.8 13.1 23.4 13.7

Asian American 3.4 1.2 3.7 1.6

American Indian 2.4 2.6 2.8 2.5

Unclassified 0.4 0.0 0.0 0.1

REGION

Northeast 20.3 17.4 21.8 17.9

Southeast 34.1 32.9 23.1 34.1

Central 15.1 28.6 15.4 25.6

West 30.4 21.1 39.7 22.4

TYPE OF LOCATION

Central City 45.4 40.0 41.3 41.5

Urban Fringe/Large Town 26.7 21.2 39.4 21.4

Rural/Small Town 27.9 38.8 19.4 37.1

SCHOOL TYPE

Public 96.8 96.9 97.4 96.8

Nonpublic 1.3 0.5 0.0 0.8

Private 0.9 0.5 0.0 0.7

Catholic 0.4 0.0 0.0 0.1

BIA 1.9 2.6 2.6 2.4

DoDea 0.0 0.0 0.0 0.0

ESTIMATED TOTAL

POPULATION 87,533 223,797 24,049 287,281
419

Table 19-39

Weighted Percentage of Excluded Students in the Reading and Writing Long-Term Trend Sample by Type

of Eligibility and Subgroup Classification, Age 17/Grade 11

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

SEX

Male 64.4 66.7 47.6 67.9

Female 35.6 33.3 52.4 32.1

RACE/ETHNICITY

White 61.1 67.6 71.3 64.1

Black 23.1 14.0 19.4 17.6

Hispanic 11.7 12.3 6.4 12.7

Asian American 2.7 3.2 0.6 3.3

American Indian 0.9 2.9 2.3 2.0

Unclassified 0.5 0.0 0.0 0.2

REGION

Northeast 15.0 25.0 23.7 20.5

Southeast 36.5 23.1 26.4 29.0

Central 19.9 19.5 18.1 19.9

West 28.6 32.3 31.8 30.6

TYPE OF LOCATION

Central City 29.6 30.9 26.3 30.8

Urban Fringe/Large Town 38.3 46.3 52.2 41.8

Rural/Small Town 32.2 22.9 21.6 27.4

SCHOOL TYPE

Public 100.0 97.8 100.0 98.6

Nonpublic 0.0 0.6 0.0 0.4

Private 0.0 0.0 0.0 0.0

Catholic 0.0 0.6 0.0 0.4

BIA 0.0 1.7 0.0 1.1

DoDea 0.0 0.0 0.0 0.0

ESTIMATED TOTAL

POPULATION 121,771 167,734 29,838 259,667
420

Table 19-40

Weighted Percentage of Excluded Students in the Mathematics and Science Long-Term Trend Sample by

Type of Eligibility and Subgroup Classification, Age 9

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

SEX

Male 62.0 61.0 61.0 62.0

Female 38.0 39.0 39.0 38.0

RACE/ETHNICITY

White 47.5 46.4 46.4 47.5

Black 19.5 13.2 13.2 19.5

Hispanic 25.4 30.7 30.7 25.4

Asian American 6.5 8.8 8.8 6.5

American Indian 0.3 0.3 0.3 0.3

Unclassified 0.8 0.6 0.6 0.8

REGION

Northeast 12.0 11.7 11.7 12.0

Southeast 33.2 22.6 22.6 33.2

Central 18.0 15.6 15.6 18.0

West 36.8 50.0 50.0 36.8

TYPE OF LOCATION

Central City 54.8 61.0 61.0 54.8

Urban Fringe/Large Town 27.2 30.0 30.0 27.2

Rural/Small Town 18.0 9.0 9.0 18.0

SCHOOL TYPE

Public 98.5 99.2 99.2 98.5

Nonpublic 1.5 0.8 0.8 1.5

Private 0.0 0.0 0.0 0.0

Catholic 1.5 0.8 0.8 1.5

BIA 0.0 0.0 0.0 0.0

DoDea 0.0 0.0 0.0 0.0

ESTIMATED TOTAL

POPULATION 173,491 91,294 91,294 173,491
421

Table 19-41

Weighted Percentage of Excluded Students in the Mathematics and Science Long-Term Trend Sample by

Type of Eligibility and Subgroup Classification, Age 13

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

SEX

Male 64.6 59.0 59.0 64.6

Female 35.4 41.0 41.0 35.4

RACE/ETHNICITY

White 54.4 55.9 55.9 54.4

Black 20.1 14.1 14.1 20.1

Hispanic 21.2 26.9 26.9 21.2

Asian American 3.4 2.8 2.8 3.4

American Indian 0.3 0.3 0.3 0.3

Unclassified 0.5 0.0 0.0 0.5

REGION

Northeast 19.9 23.1 23.1 19.9

Southeast 31.5 24.1 24.1 31.5

Central 21.0 19.0 19.0 21.0

West 27.6 33.8 33.8 27.6

TYPE OF LOCATION

Central City 44.2 36.8 36.8 44.2

Urban Fringe/Large Town 25.9 38.3 38.3 25.9

Rural/Small Town 30.0 24.9 24.9 30.0

SCHOOL TYPE

Public 99.6 100.0 100.0 99.6

Nonpublic 0.4 0.0 0.0 0.4

Private 0.4 0.0 0.0 0.4

Catholic 0.0 0.0 0.0 0.0

BIA 0.0 0.0 0.0 0.0

DoDea 0.0 0.0 0.0 0.0

ESTIMATED TOTAL

POPULATION 146,608 43,419 43,419 146,608
422

Table 19-42

Weighted Percentage of Excluded Students in the Mathematics and Science Long-Term Trend Sample by

Type of Eligibility and Subgroup Classification, Age 17

————— Eligible by —————

Age Grade

Age &

Grade

Age or

Grade

SEX

Male 67.4 62.0 62.0 67.4

Female 32.6 38.0 38.0 32.6

RACE/ETHNICITY

White 61.0 71.0 71.0 61.0

Black 18.6 16.1 16.1 18.6

Hispanic 15.4 10.8 10.8 15.4

Asian American 4.0 1.7 1.7 4.0

American Indian 1.0 0.4 0.4 1.0

Unclassified 0.0 0.0 0.0 0.0

REGION

Northeast 22.1 36.4 36.4 22.1

Southeast 30.5 16.8 16.8 30.5

Central 17.9 17.6 17.6 17.9

West 29.6 29.2 29.2 29.6

TYPE OF LOCATION

Central City 37.3 33.4 33.4 37.3

Urban Fringe/Large Town 36.3 47.7 47.7 36.3

Rural/Small Town 26.4 18.9 18.9 26.4

SCHOOL TYPE

Public 98.6 100.0 100.0 98.6

Nonpublic 1.4 0.0 0.0 1.4

Private 1.4 0.0 0.0 1.4

Catholic 0.0 0.0 0.0 0.0

BIA 0.0 0.0 0.0 0.0

DoDea 0.0 0.0 0.0 0.0

ESTIMATED TOTAL

POPULATION 131,897 38,085 38,085 131,897
423

Appendix A

PARTICIPANTS IN THE OBJECTIVES AND ITEM

DEVELOPMENT PROCESS

The National Assessment of Educational Progress extends its deep appreciation to all those

individuals who participated in the development of the framework, objectives, and items for the 1996

national assessment.

Mathematics Project Steering Committee

Frank Betts Director, Curriculum Technology Center, Association for Supervision and

Curriculum Development, Alexandria, Virginia

Ed Esty Senior Staff Officer, Mathematical Sciences Education Board, Washington, DC

James Gates Executive Director, National Council of Teachers of Mathematics, Reston,

Virginia

Alice Gill Coordinator, Thinking Mathematics Dissemination, American Federation of

Teachers, Cleveland, Ohio

Joyce McCray Executive Director, Council for American Private Education, Washington, DC

Henry Pollack Visiting Professor, Teachers College, Columbia (formerly with Bell Labs), New

York, New York

John Porter Director, Education and University Relations, IBM, Armonk, New York

Rodney Riffel Senior Professional Associate, National Education Association, Washington, DC

Ramsay Selden Director, State Education Assessment Center, Council of Chief State School

Officers, Washington, DC

Charles Watson President, Association of State Supervisors of Mathematics, Arkansas Director

of Mathematics, Little Rock, Arkansas

Michael Webb National Education Director, National Urban League, New York, New York

Gene Wilhoit Executive Director, National Association of State Boards of Education,

Alexandria, Virginia
424

Science Project Steering Committee

William O. Baker Retired, AT&T Bell Laboratories, Murray Hill, New Jersey

Mary Louise Bellamy Education Director, National Association of Biology Teachers, Reston, Virginia

Frank Betts Director, Curriculum Technology Center, Association for Supervision and

Curriculum Development, Alexandria, Virginia

William B. Campbell Executive Director, National Industry Council for Science Education, College

Park, Maryland

Glenn A. Crosby Professor, Chemistry Department, Washington State University, Pullman,

Washington

Gerald Difford Executive Director, Colorado Association of Schools, Englewood, Colorado

Janice Earle Director, Center on Educational Equity, National Association of State Boards of

Education, Alexandria, Virginia

John Fowler Director, Triangle Coalition for Science and Technology, College Park,

Maryland

Johnnie Hamilton Principal, Franklin Intermediate School, Chantilly, Virginia

Elam Hertzler Retired, Secretary’s Commission on Achieving Necessary Skills, Washington, DC

Ann Kahn Past President, Parent Teachers Association, Fairfax, Virginia

Douglas Lapp Director, National Science Resources Center, Smithsonian Institution,

Washington, DC

John Layman Director, Science Teaching Center, University of Maryland, College Park,

Maryland

Harold Pratt Executive Director, Science and Technology Management, Jefferson County

Schools, Golden, Colorado

Judith Torney-Purta Department of Human Development, University of Maryland, College Park,

Maryland

Douglas Reynolds Chief, Bureau of Science Education, State Department of Education, Albany,

New York

Bella Rosenberg Assistant to the President, American Federation of Teachers, Washington, DC

Jane Sisk Biology Teacher, Calloway County High School, Murray, Kentucky
425

Mathematics Project Planning Committee

Martha Baca Supervisor, K-12 Public Schools, Roosevelt School District #66, Phoenix,

Arizona

Glen Blume Associate Professor, Penn State University, University Park, Pennsylvania

Dave Heckman Private School Teacher and Presidential Award Recipient, 1987, Monmouth,

Maine

Audrey Jackson Middle School Mathematics Supervisor and Teacher, St. Louis, Missouri

Pat Kenney Researcher, Learning Research and Development Center, University of

Pittsburgh, Pennsylvania

Donna Long Indiana State Supervisor of Mathematics, Indianapolis, Indiana

Larry Wiley New Jersey State Supervisor of Mathematics, Trenton, New Jersey
426

Science Project Planning Committee

Andrew Ahlgren Associate Director, Project 2061, American Association for the Advancement of

Science, Washington, DC

Bill Aldridge Executive Director, National Science Teachers’ Association, Washington, DC

J. Myron Atkin Professor, School of Education, Stanford University, Stanford, California

Joan Boykoff Baron Assessment Coordinator, Connecticut Common Core of Learning, Bureau of

Evaluation and Student Assessment, Connecticut Department of Education,

Hartford, Connecticut

Audrey Champagne Professor, School of Education, State University of New York, Albany, New

York

Sally Crissman Lowers School Head/Science Teacher, Shady Hill School, Cambridge,

Massachusetts

Edmund W. Gordon Professor, Department of Psychology, Yale University, New Haven, Connecticut

Henry Heikkinen Director, M.A.S.T. Center, University of Northern Colorado, Greeley, Colorado

George Hein Professor, Lesley College, Cambridge, Massachusetts

Joseph L. Premo Science Consultant, Minneapolis Public Schools, Minneapolis, Minnesota

Senta A. Raizen Director, National Center for Improving Science Education, Washington, DC

James Robinson Retired, Curriculum and Evaluation, Boulder Valley Schools, Boulder, Colorado

Thomas P. Sachse Manager, Math and Science Unit, California Department of Education,

Sacramento, California

Gary E. Skaggs Test Development Analyst, Office of Research and Evaluation, Fairfax County

Public Schools, Falls Church, Virginia
427

Mathematics Project Staff

The College Board

Robert Orrill Executive Director, Office of Academic Affairs

Valarie French Director of Curriculum and Instructional Development, Office of Academic

Affairs

Bruce Kennedy Office Manager, Office of Academic Affairs

Consultants to the College Board

John Dossey Project Director, Distinguished University Professor of Mathematics, Illinois

State University, Normal, Illinois

Cathy Seeley Project Coordinator, Mathematics Consultant, Former Texas Director of

Mathematics, Austin, Texas
428

Science Project Staff

Ramsay W. Selden Director, State Education Assessment Center, Council of Chief State School

Officers

Richard C. Clark Consensus Coordinator, Council of Chief State School Officers

Senta A. Raizen Director, National Center for Improving Science Education

Julia H. Mitchell Associate Research Scientist, American Institutes for Research

Bonnie L. Verrico Administrative Assistant, Council of Chief State School Officers
429

Science Achievement Levels Panel

Andrew Ahlgren Associate Director, Project 2061, American Association for the Advancement of

Science, Washington, DC

Audrey Champagne Professor, School of Education, State University of New York, Albany, New

York

Richard C. Clark Consensus Coordinator and Science Coordinator, Minnesota Department of

Education, St. Paul, Minnesota

Yvonne Curbeam Science Department Chair, Dunbar High School, Baltimore, Maryland

Joseph Premo Science Consultant, New Hope, Minnesota

Thomas Preston Science Teacher, Frick International Studies Academy, Pittsburgh, Pennsylvania

Senta Raizen Director, National Center for Improving Science Education, Washington, DC

Dwight Sieggreen Science Teacher, Cooke Middle School, Northville, Michigan

William Spooner Chief Consultant, Science Education, North Carolina Department of Public

Instruction, Raleigh, North Carolina

Douglas Wagner Science Teacher, Emmanuel Lutheran School, St. Charles, Missouri

Sylvia Ware Director, Education Department, American Chemical Society, Washington, DC

431

Appendix B

REPORTING SUBGROUPS

FOR THE 1996 NAEP ASSESSMENT

Results for the 1996 assessment were reported for student subgroups defined by gender,

race/ethnicity, type of location, parents’ level of education, participation in the National School Lunch

Program, eligibility of Title I funding, and geographical region. The following explains how each of

these subgroups was derived.

DSEX (Gender)

The variable SEX is the gender of the student being assessed, as taken from school records. For a

few students, data for this variable was missing and was imputed by ETS after the assessment. The

resulting variable DSEX contains a value for every student and is used for gender comparisons among

students.

DRACE (Race/Ethnicity for Main and Long-Term Trend Assessments)

The variable DRACE is an imputed definition of race/ethnicity, derived from up to three sources

of information. This variable is used for race/ethnicity subgroup comparisons in the 1996 main

assessments (science and mathematics), in the 1996 long-term trend samples (reading, mathematics, and

science), and in the 1996 writing trend samples (see also “observed race/ethnicity” below). Two items

from the student demographics questionnaire were used in the determination of derived race/ethnicity:

Demographic Item Number 2:

2. If you are Hispanic, what is your Hispanic background?

 I am not Hispanic.

 Mexican, Mexican American, or Chicano

 Puerto Rican

 Cuban

 Other Spanish or Hispanic background
432

Students who responded to Item Number 2 by filling in the second, third, fourth, or fifth oval

were considered Hispanic. For students who filled in the first oval, did not respond to the item, or

provided information that was illegible or could not be classified, responses to item number 1 were

examined in an effort to determine race/ethnicity. Item Number 1 read as follows:

Demographic Item Number 1:

1. Which best describes you?

 White (not Hispanic)

 Black (not Hispanic)

 Hispanic (“Hispanic” means someone who is Mexican, Mexican

American, Chicano, Puerto Rican, Cuban, or from some other

Spanish or Hispanic background.)

 Asian or Pacific Islander (“Asian or Pacific Islander” means

someone who is Chinese, Japanese, Korean, Filipino, Vietnamese,

or from some other Asian or Pacific Island background.)

 American Indian or Alaskan Native (“American Indian or Alaskan

Native” means someone who is from one of the American Indian

tribes, or one of the original people of Alaska.)

 Other (What?) _________

Students’ race/ethnicity was then assigned to correspond with their selection. For students who

filled in the sixth oval (Other), provided illegible information or information that could not be classified,

or did not respond at all, race/ethnicity as provided from school records was used.

Derived race/ethnicity could not be determined for students who did not respond to background items 1

or 2 and for whom race/ethnicity was not provided by the school.

RACE (Observed Race/Ethnicity)

The variable RACE is the race/ethnicity of the student being assessed as observed and recorded

by the exercise administrator. Observed race/ethnicity was used in NAEP assessments before 1984 (see

also “self-identified race/ethnicity” above). Observed race/ethnicity is used for all race/ethnicity

subgroup trend comparisons for which the starting long-term trend point comes from pre-1984

assessment data.
433

TOL8 (Type of Location)

TOL5

TOL3

The variable TOL8 is used by NAEP to provide information about the type of location in which

schools are located. The variable is defined using population size information from the 1990 Census and

the definitions of Metropolitan Statistical Areas (MSAs) as of June 1995. There are eight categories for

TOL8:

1 Large Central City a central city of an MSA with a population greater than or equal

to 400,000, or a population density greater than or equal to 6,000

persons per square mile

2 Midsize Central City a central city of an MSA but not designated as a large city

3 Urban Fringe of a place within an MSA of a large central city and defined as

Large City urban by the U.S. Bureau of Census

4 Urban Fringe of a a place within an MSA of a midsize central city and defined

Midsize City as urban by the U.S. Bureau of Census

5 Large Town a place not within an MSA, but with a population greater than or

equal to 25,000 and defined as urban by the U.S. Bureau of Census

6 Small Town a place not within an MSA, with a population less than 25,000, but

greater than or equal to 2,500 and defined as urban by the U.S.

Bureau of Census

7 Rural MSA a place within an MSA with a population of less than 2,500 and

defined as rural by the U.S. Bureau of Census

8 Rural NonMSA a place not within an MSA with a population of less than 2,500 and

defined as rural by the U.S. Bureau of Census

These categories are designed to be exhaustive and mutually exclusive. Every place in the 50

United States and the District of Columbia is classified as belonging to exactly one of these categories.

The definitions of MSAs and PMSAs, and their central cities, are carried out by the Office of

Management and Budget (OMB). OMB Bulletin No. 93-17 states that “all agencies that conduct

statistical activities to collect and publish data for Metropolitan Areas should use the most recent

definitions of Metropolitan Areas established by OMB.” The definitions used (as of June 1995) were

those current at the time of the 1996 assessment. The definitions of places and their populations are

obtained from the published results of the 1990 Population Census, as are the definitions of Urbanized

Areas.

Further details about the creation of the eight-category type of location variable are provided in

The NAEP 1994 Sampling and Weighting Report (Wallace & Rust, 1996).
434

The variable TOL5 was created by collapsing the information provided in the variable TOL8 to

five levels:

1 Large Central City

2 Midsize Central City

3 Urban Fringe of Large City, Urban Fringe of Midsize City, and Large Town

4 Small Town

5 Rural MSA and Rural NonMSA

The variable TOL3 is used extensively in the NAEP reports. TOL3 collapses TOL8 to three

levels:

1 Central City (Large Central City and Midsize Central City) This category

includes central cities of all MSAs. Central City is a geographic

term and is not synonymous with “inner city.”

2 Urban Fringe/Large Town (Urban Fringe of Large City, Urban Fringe of Midsize City, and

Large Town) An Urban Fringe includes all densely settled

places and areas within MSAs that are classified as urban by the

Bureau of Census. A Large Town is defined as a place outside

MSAs with a population greater than or equal to 25,000.

3 Rural/Small Town (Small Town, Rural MSA, and Rural NonMSA) Rural includes

all places and areas with a population of less than 2,500 that are

classified as rural by the Bureau of Census. A Small Town is

defined as a place outside MSAs with a population of less than

25,000 but greater than or equal to 2,500.

PARED (Student’s Report of Parents’ Education Level)

The variable PARED is derived from responses to two questions, B003501 and B003601, in the

student demographic questionnaire. Each student was asked to indicate the extent of his or her mother’s

education (B003501—How far in high school did your mother go?) by choosing one of the following:

 She did not finish high school.

 She graduated from high school.

 She had some education after high school.

 She graduated from college.

 I don’t know.

Each student was asked to provide the same information about the extent of his or her father’s

education (B003601—How far in high school did your father go?) by choosing one of the following:

 He did not finish high school.

 He graduated from high school.

 He had some education after high school.

 He graduated from college.

 I don’t know.
435

The information was combined into one parental education reporting category (PARED) as

follows: If a student indicated the extent of education for only one parent, that level was included in the

data. If a student indicated the extent of education for both parents, the higher of the two levels was

included in the data. For students who did not know the level of education for both parents or did not

know the level of education for one parent and did not respond for the other, the parental education level

was classified as unknown. If the student did not respond for both parents, the student was recorded as

having provided no response.

REGION (Region of the Country)

Jurisdictions were grouped into four geographical regionsNortheast, Southeast, Central, and

Westas shown in Table B-1. All 50 states and the District of Columbia are listed. The part of Virginia

that is included in the Washington, DC, metropolitan statistical area is included in the Northeast region;

the remainder of the state is included in the Southeast region.

Table B-1

NAEP Geographic Regions

NORTHEAST SOUTHEAST CENTRAL WEST

Connecticut

Delaware

District of

Columbia

Maine

Maryland

Massachusetts

New Hampshire

New Jersey

New York

Pennsylvania

Rhode Island

Vermont

Virginia

Alabama

Arkansas

Florida

Georgia

Kentucky

Louisiana

Mississippi

North Carolina

South Carolina

Tennessee

Virginia

West Virginia

Illinois

Indiana

Iowa

Kansas

Michigan

Minnesota

Missouri

Nebraska

North Dakota

Ohio

South Dakota

Wisconsin

Alaska

Arizona

California

Colorado

Hawaii

Idaho

Montana

Nevada

New Mexico

Oklahoma

Oregon

Texas

Utah

Washington

Wyoming

MODAGE (Modal Age)

The modal age (the age of most of the students in the grade sample) for the fourth-grade students

is age 9 (age 13 for grade 8 and age 17 for grade 12). A value of 1 for MODAGE indicates that the

student is younger than the modal age; a value of 2 indicates that the student is of the modal age; a value

of 3 indicates that the student is older than the modal age.
436

IEP (Individualized Education Program)1

The variable IEP comes from the student booklet cover. A value of 1 indicates that a student has

an individualized education program, while a value of 2 indicates no individualized education program.

LEP (Limited English Proficiency)

The variable LEP comes from the student booklet cover. A value of 1 indicates that a student is

considered to have limited English proficiency while a value of 2 indicates that the student does not have

limited English proficiency.

TITLE1

The variable TITLE1 comes from the student booklet cover. A value of 1 indicates that a student

is eligible for Title 1 funding and a value of 2 indicates that the student is not eligible for Title 1 funding.

SLUNCH

SLUNCH1

The variable SLUNCH is provided by Westat, Inc. and is used to determine if a student

participates in the National School Lunch Program. The values for this variable are as follow:

1 not eligible

2 eligible for reduced price lunch

3 eligible for free lunch

4 no information available

5 school refused to provide information

The variable SLUNCH1 collapses the information provided in the variable SLUNCH to three

levels:

1 eligible for free or reduced price lunch

2 not eligible for free or reduced price lunch

3 no information available

 

1

 A student identified on the Administration Schedule as a student with a disability (SD) or an equivalent classification may be

excluded from the assessment if: 1) the student is mainstreamed less than 50% of the time in academic subjects and is judged

incapable of participating meaningfully in the assessment, or 2) the Individualized Education Program (IEP) team or equivalent

group has determined that the student is incapable of participating meaningfully in the assessment. SD/LEP students meeting the

above criteria should be assessed if, in the judgment of school staff, they are capable of taking the assessment.
437

SCHTYPE

The variable SCHTYPE is provided by Westat, Inc., and is used to determine the type of school

that a student attended. The values for this variable are as follow:

1 Public

2 Private

3 Catholic

4 Bureau of Indian Affairs (BIA)

5 Department of Defense Education Activity (DoDEA) schools

439

VARIABLES DERIVED FROM

THE STUDENT AND TEACHER QUESTIONNAIRES

Several variables were formed from the systematic combination of response values for one or

more items from either the student demographic questionnaire, the student background questionnaire, or

the teacher questionnaire.

HOMEEN2 (Home EnvironmentArticles [of 4] in the Home) - Science

For the science sample the variable HOMEEN2 was created from the responses to student

demographic items B000901 (Does your family get a newspaper regularly?), B000903 (Is there an

encyclopedia in your home?), B000904 (Are there more than 25 books in your home?), and B000905

(Does your family get any magazines regularly?). The values for this variable were derived as follows:

1 0-2 types The student responded to at least two items and answered “yes” to two or

fewer.

2 3 types The student answered “yes” to three items.

3 4 types The student answered “yes” to four items.

8 Omitted The student answered fewer than two items.

HOMEEN3 (Home Environment—Articles [of 4] in the Home) - Mathematics

For the mathematics samples the variable HOMEEN3 was created from the responses to student

demographic items B000901 (Does your family get a newspaper regularly?), B000903 (Is there an

encyclopedia in your home?), B008801 (How many books are in your home?), collapsed to indicate

whether or not there are more than 25 books in the home), and B000905 (Does your family get any

magazines regularly?). The values for this variable were derived as follows:

1 0-2 types The student responded to at least two items and answered “yes” to two or 

fewer.

2 3 types The student answered “yes” to three items.

3 4 types The student answered “yes” to four items.

8 Omitted The student answered fewer than two items.
440

NCOMP (Number of Computer Science Courses Taken)

For age class 17 (long-term trend mathematics and science), NCOMP was created from

responses to items B005312 and B005313 concerning the student’s coursework in computer science. The

values for NCOMP were derived as follows:

1 0 The student answered HAVE NOT to both courses.

2 1 The student answered HAVE to one course.

3 2 The student answered “yes” to both courses.

8 No response The student did not respond to one or both items.

9 Mult. & out-of-range The student filled in more than one oval for both items.

NMATH (Highest Level of Mathematics Courses Taken)

For age class 17, long-term trend mathematics and science) NMATH was created from responses

to items B005301 through B005307 concerning the student’s coursework in mathematics. The values for

NMATH were derived as follows:

1 Gen. mathematics or The student answered HAVE NOT to all items or

pre-algebra HAVE to B005301 or B005302 and HAVE NOT to

all others.

2 Algebra The student answered HAVE to B005303 and HAVE

NOT to B005304, B005305, B005306, and B005307.

3 Geometry The student answered HAVE to both B005303 and

B005305 and HAVE NOT to B005304, B005306, and

B005307.

4 Algebra 2 The student answered HAVE to B005304 or B005306

but HAVE NOT to B005307.

5 Calculus The student answered HAVE to B005307.

6 Something else Any other response combination

8 No response The student did not respond to any item.
441

NSCI (Highest Level of Science Courses Taken)

For age class 17, (long-term trend mathematics and science) NSCI was created from responses to

items B005308 through B005311, which concerned the student’s coursework in science. The values for

NSCI were derived as follows:

1 No biology The student answered HAVE NOT to all items or HAVE to

B005308 and other than HAVE to all other items.

2 Biology The student answered HAVE to B005309 and other than

HAVE to both B005310 and B005311.

3 Chemistry The student answered HAVE to both B005309 and B005310

and other than HAVE to B005311.

4 Physics The student answered HAVE to B005309, B005310, and

B005311.

5 Something else Any other response combination

8 No response The student answered none of the items.

443

VARIABLES DERIVED FROM COGNITIVE ITEMS

BKSCOR (Booklet-Level Score)

The booklet-level score is a student-level score based on the sum of the number correct for

dichotomous items plus the sum of the scores on the polytomous items, where the score for a polytomous

item starts from 0 for the unacceptable category. Thus, for a 4-point extended constructed-response item,

scores of “no response”, “off-task”, and “unsatisfactory” are assigned an item score of 0. Scores of

“partial,” “essential,” and “extensive” are assigned item scores of 1, 2, and 3, respectively. The score is

computed based on all cognitive items in a student’s assessment booklet.

LOGIT (Logit Percent Correct Within Booklet)

In order to compute the LOGIT score, a percent correct within booklet was first computed. This

score was based on the ratio of the booklet score (BKSCOR) over the maximum booklet score. The

percent correct score was set to .0001 if no items were answered correctly; if BKSCOR equaled the

maximum booklet score, the percent correct score was set to .9999. A logit score, LOGIT, was calculate

for each student by the following formula:

LOGIT PCTCOR

PCTCOR = −





 



 ln

1

A logit score, LOGIT, was calculated within booklet for each student by the following formula:

LOGIT was then truncated to a value x, such that -3  x  3. After computing LOGIT for each student,

the mean and standard deviation were calculated for each booklet as the first step in standardizing the

logit scores. The standardized logit score, ZLOGIT, was then calculated for each student by the following

formula:

ZLOGIT =  −



 



 LOGIT mean logit

standard deviation
444

NORMIT (Normit Gaussian Score)

SCHNORM (School-Level Mean Gaussian Score)

The normit score is a student-level Gaussian score based on the inverse normal transformation of

the mid-percentile rank of a student’s number-correct booklet score within that booklet. The normit

scores were used to decide collapsing of variables, finalize conditioning coding, and check the results of

scaling.

The number correct is based on the number of dichotomous items answered correctly plus the

score obtained on extended constructed-response items. The mid-percentile rank is based on the formula:

CF i CF i

N

  + −  1

2

where CF(i) is the cumulative frequency at i items correct and N is the total sample size. If i = 0 then

CF CF

N

0 1

2

2

    +

A school-level normit, SCHNORM, was also created; this was the mean normit across all reading

booklets administered in a school. These school-level mean normit scores were used in conditioning

procedures to take into account differences in school proficiency. For each school, the weighted mean of

the logits for the students in that school was calculated. Each student was then assigned that mean as his

or her school-level mean logit score value.
445

VARIABLES RELATED TO PROFICIENCY SCALING

Proficiency Score Variables

Item response theory (IRT) was used to estimate average proficiency for the nation and for

various subpopulations, based on students’ performance on the set of cognitive items they received. IRT

provides a common scale on which performance can be reported for the nation and subpopulations, even

when all students do not answer the same set of questions. This common scale makes it possible to report

on relationships between students’ characteristics (based on their responses to the background questions)

and their overall performance in the assessment.

A scale ranging from 1 to 500 or from 1 to 300 was created to report performance for each

content area or strand. A composite scale was created based on a weighted average of scales, where the

weight for each content area or strand was proportional to the relative importance assigned to the content

area as specified in the mathematics and science objectives.

Scale proficiency estimates were obtained for all students. The NAEP methods use random

draws (plausible values) from estimated proficiency distributions to compute population statistics.

Plausible values are not optimal estimates of individual proficiency; instead, they serve as intermediate

values to be used in estimating population characteristics. Chapter 11 provides further details on the

computation and use of plausible values. Chapters 12-18 provide additional information as appropriate to

each sample/subject area.

The proficiency score (plausible value) variables are provided on the student data files for each of the

scales and are named as shown in Table B-2.

Table B-2

Scaling Variables for the 1996 National Assessment Samples

Sample Scale Data Variables

Mathematics Main Number Sense, Properties, and

Operations

MRPS11 to MRPS15

Measurement MRPS21 to MRPS25

Geometry and Spatial Sense MRPS31 to MRPS35

Data Analysis, Statistics, and

Probability

MRPS41 to MRPS45

Algebra and Functions MRPS51 to MRPS55

Composite MRPCM1 to MRPCM5

Science Main Physical Science SRPS11 to SRPS15

Earth Science SRPS21 to SRPS25

Life Science SRPS31 to SRPS35

Composite SRPCM1 to SRPCM5

Reading Long-Term Trend Univariate REDVAL1 to REDVAL5

Writing Long-Term Trend Univariate WRPSCT1 to WRPSCT5

Mathematics Long-Term Trend Univariate MRPSCT1 to MRPSCT5

Science Long-Term Trend Univariate SRPSCT1 to SRPSCT5
446

SMEANx, SMNx1 (School Mean Score Using First Plausible Value)

SRANKx, SRNKx1 (School Rank Using First Plausible Value)

SRNK3x, SRK3x1 (Top, Middle, Bottom Third Using First Plausible Value

where x = M or S for Science or Mathematics)

A mean composite score (SMEANx on the student files, SMNx1 on the school files) was

calculated using the first composite plausible value for each school within each grade/subject area. The

mean composite score was based on the values from the scaling variable xRPCM1 and was calculated

using the students’ sampling weights. The schools were then ordered from highest to lowest mean score

(SRANKx on the student files, SRNKx1 on the school files) within a sample using school-level weights

—the school with the highest mean score was given a ranking of 1, and the school with the lowest mean

score was given a ranking equal to the number of schools in the sample.

These variables were then used in partitioning the schools within the national grade sample into

three groups (top third, middle third, and bottom third) based on their ranking (SRNK3x on the student

files, SRK3x1 on the school files).

SMNxn (School Mean Score Using Plausible Values 2 Through 5)

SRNKxn (School Rank Using Plausible Values 2 Through 5)

SRK3xn (Top, Middle, Bottom Third Using Plausible Values 2 Through 5)

SMNRPn (School Mean Score Using Plausible Values 2 Through 5, Public Schools

Only)

School ranking results presented in the 1996 NAEP reports are based on the first plausible value.

However, since there are four additional estimates of proficiency (plausible values) for each student,

school ranking data were also created for those estimates. These school rank values were created using

the same procedures described above, substituting proficiency variables xRPCM2 through xRPCM5 to

compute the results. In the variable names, n denotes the plausible value 2, 3, 4, or 5.
447

QUALITY EDUCATION DATA VARIABLES (QED)

The data files contain several variables obtained from information supplied by Quality Education

Data, Inc. (QED). QED maintains and annually updates lists of schools showing grade span, total

enrollment, instructional dollars per pupil, and other information for each school. These data variables

are retained on both the school and student files and are identified in the data layouts by “(QED)” in the

SHORT LABEL field.

Most of the QED variables are defined sufficiently in the data codebooks. Explanations of others

are provided below.

ORSHPT is the Orshansky Percentile, an indicator of relative wealth that specifies the

percentage of school-age children in a district who fall below the poverty line.

IDP represents, at the school district level, dollars per student spent for textbooks and

supplemental materials. The range code for instructional dollars spent per pupil excluding teacher

salaries are:

0 = Unclassified

1 = Under $10

2 = $10-49

3 = $50-99

4 = $100-149

 5 = $150-299

6 = $300-399

7 = $400-499

8 = $500-999

9 = $1000 +

ADULTED indicates whether or not adult education courses are offered at the school site.

URBAN defines the school’s urbanization: urban (central city); suburban (area surrounding

central city, but still located within the counties constituting the metropolitan statistical area); or rural

(area outside any metropolitan statistical area).

449

Appendix C

CONDITIONING VARIABLES

AND CONTRAST CODINGS

This appendix contains information about the conditioning variables used in scaling/plausible

value estimation for the 1996 NAEP assessment. The initial step in construction of conditioning variables

involves forming primary student-based vectors of response data from answers to student, teacher, and

school questionnaires, demographic and background data such as supplied by Westat, Inc., and other

student information known prior to scaling. The initial conditioning vectors concatenate this student

background information into a series of identifying “contrasts” comprising:

1. Categorical variables derived by expanding the response options of a

questionnaire variable into a binary series of one-degree-of-freedom

“dummy” variables or contrasts, (these form the majority of each student

conditioning vector);

2. Questionnaire or demographic variables that possess ordinal response

options, such as number of hours spent watching television, which are

included as linear and/or quadratic multi-degree-of-freedom contrasts;

3. Continuous variables, such as student logit scores based on percent correct

values, included as contrasts in their original form or a transformation of

their original form, and;

4. Interactions of two or more categorical variables forming a set of orthogonal

one-degree-of-freedom dummy variables or contrasts.

This appendix gives the specifications used for constructing the conditioning variables. Table C1 defines the information provided for each main sample variable. Table C-2 provides a summary of the

mathematics conditioning variables specifications that are contained in the remainder of this appendix.

Table C-3 provides a summary of the science conditioning variables specifications that are contained in

the remainder of this appendix. Conditioning variable data specific to each subject area is shown for

main mathematics in Table C-4 and for science in Table C-5. Similar information for long-term trend

samples is given in Tables C-6, C-7, C-8, and C-9 respectively for reading, mathematics, science, and

writing.

As described in Chapter 9, the linear conditioning model employed for the estimation of

plausible values did not directly use the conditioning variable specifications listed in this appendix. To

eliminate inherent instabilities in estimation encountered when using a large number of correlated

variables, a principal component transformation of the correlation matrix obtained from the conditioning

variable contrasts derived according to these primary specifications was performed. The principal

components scores based on this transformation were used as the predictor variables in estimating the

linear conditioning model. The proportions of variance of the conditioning contrast accounted for by the

principal components are given for each age/grade level in Tables C-10, C-11, and C-12 for mathematics,

and Tables C-13, C-14, and C-15 for science.

451

Table C-1

Description of Specifications Provided for Each Conditioning Variable

Title Description

CONDITIONING ID A unique eight-character ID assigned to identify each conditioning

variable corresponding to a particular background or subject area

question within the entire pool of conditioning variables. The first

four characters identify the origin of the variable: BACK (background

questionnaire), READ (student reading questionnaire), SCHL (school

questionnaire), TCHR (background part of teacher questionnaire), and

TSUB (subject classroom part of teacher questionnaire). The second

four digits represent the sequential position within each origin group.

DESCRIPTION A short description of the conditioning variable.

GRADES/ASSESSMENTS Three characters identifying assessment (“S” for state, “N” for

national) and grade (04, 08, and 12) in which the conditioning

variable was used.

CONDITIONING VAR

LABEL

A descriptive eight-character label identifying the conditioning

variable.

NAEP ID The seven-character NAEP database identification for the

conditioning variable.

TYPE OF CONTRAST The type of conditioning variable. “CLASS” identifies a categorical

conditioning variable and “SCALE” identifies continuous or quasicontinuous conditioning variables. “INTERACTION” identifies a set

of orthogonal contrasts formed from two or more “CLASS” variables.

“OTHER” conditioning variables do not fall into any of the above

types.

TOTAL NUMBER OF

SPECIFIED CONTRASTS

Each conditioning variable forms a set of one or more contrasts. For

each valid response value of conditioning variable a contrast must be

defined. One or more response values may be collapsed together to

form one contrast. The number of response value “sets” of a

conditioning variable forming a unique contrast is the value given in

this field.

NUMBER OF INDEPENDENT

CONTRASTS

The number of degree of freedom in a set of contrasts formed from a

conditioning variable. For a categorical conditioning variable this

number would be the number of response options minus one if each

response option formed its own unique contrast.
Table C-2

Summary Table of the 1996 Mathematics Assessment Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

452BACK0001 BKSER GRAND MEAN BACK0002 DSEX DERIVED SEX BACK0003 DRACE DERIVED RACE/ETHNICITY BACK0004 B003101 TB003101 IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND? BACK0005 TOL8 MSA/NON-MSA BACK0006 TOL5 TYPE OF LOCALE (5 CATEGORIES) BACK0007 DOC DESCRIPTION OF COMMUNITY BACK0008 PARED PARENTS’ HIGHEST LEVEL OF EDUCATION BACK0009 REGION REGION OF THE COUNTRY BACK0010 SCHTYPE SCHOOL TYPE (PQ) BACK0011 IEP INDIVIDUALIZED EDUCATION PROGRAM BACK0012 LEP LIMITED ENGLISH PROFICIENCY BACK0013 TITLE1 TITLE 1 (BOOK COVER) BACK0014 SLUNCH DO YOU RECEIVE A FREE OR REDUCED-PRICE LUNCH? BACK0015 B003201 TB003201 HOW OFTEN DO THE PEOPLE IN YOUR HOME SPEAK A LANGUAGE OTHER THAN ENGLISH? BACK0016 B008901 HE002489 DO YOU HAVE YOUR OWN STUDY DESK OR TABLE AT HOME? BACK0017 B001801 TB001801 HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (LINEAR) BACK0018 B001801 TB001801 HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (QUADRATIC) BACK0019 B006601 TB006601 HOMEWORK ASSIGNED?: BASED ON TIME SPENT ON HOMEWORK EACH DAY. BACK0020 B006601 TB006601 HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY? (LINEAR) BACK0021 B006601 TB006601 HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY (QUADRATIC) BACK0022 HOMEEN2 NUMBER OF ITEMS IN THE HOME (NEWSPAPER, > 25 BOOKS, ENCYCLOPEDIA, MAGAZINES) (DEBACK0023 B005601 TB005601 DOES MOTHER OR STEPMOTHER LIVE AT HOME WITH YOU? BACK0024 B005701 TB005701 DOES FATHER OR STEPFATHER LIVE AT HOME WITH YOU? BACK0025 S004001 TS004001 HOW MANY DAYS OF SCHOOL MISSED LAST MONTH? BACK0026 B008001 LC000006 HOW LONG LIVED IN THE UNITED STATES? BACK0027 B007601 JL001015 HOW MANY GRADES IN THIS STATE? (4TH GRADE) SCHL0001 SCHNORM SCHOOL LEVEL AVERAGE MATH NORMIT (MISSING VS NON-MISSING) SCHL0002 SCHNORM SCHOOL LEVEL AVERAGE MATH NORMIT BACK0028 B008301 LC000168 HOW MANY GRADES IN THIS STATE (12TH GRADE) BACK0029 B007301 HE000712 HOW MANY TIMES HAVE YOU CHANGED SCHOOLS IN PAST TWO YEARS BECAUSE YOU MOVED? BACK0030 B007401 HE000717 HOW OFTEN DO YOU DISCUSS THINGS STUDIED IN SCHOOL WITH SOMEONE AT HOME? BACK0031 B001101 TB001101 ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK? BACK0032 B001101 TB001101 ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK? BACK0033 B008501 HE002549 WHICH BEST DESCRIBES YOUR HIGH-SCHOOL PROGRAM? BACK0034 B007101 HE000333 SEMESTERS ENGLISH/LITERATURE/WRITING (MISSING VS NON-MISSING) BACK0035 B007101 HE000333 NUMBER OF SEMESTERS ENGLISH/LITERATURE/WRITING (LINEAR) BACK0036 B007102 HE000334 SEMESTERS MATHEMATICS (MISSING VS NON-MISSING) BACK0037 B007102 HE000334 NUMBER OF SEMESTERS MATHEMATICS (LINEAR) BACK0038 B007103 HE000335 SEMESTERS SCIENCE (MISSING VS NON-MISSING) BACK0039 B007103 HE000335 NUMBER OF SEMESTERS SCIENCE (LINEAR) BACK0040 B007104 HE000336 SEMESTERS HISTORY/SOCIAL STUDIES/GEOGRAPHY (MISSING VS NON-MISSING) BACK0041 B007104 HE000336 NUMBER OF SEMESTERS HISTORY/SOCIAL STUDIES/GEOGRAPHY (LINEAR) SUBJ0001 MATTAKE WHAT KIND OF MATH CLASS ARE YOU TAKING THIS YEAR? BACK0042 B007105 HE000337 SEMESTERS FOREIGN LANGUAGES (MISSING VS NON-MISSING) BACK0043 B007105 HE000337 NUMBER OF SEMESTERS FOREIGN LANGUAGES (LINEAR) BACK0044 B007106 HE000338 SEMESTERS VOCATIONAL/TECHNICAL/BUSINESS EDUCATION (MISSING VS NON-MISSING) BACK0045 B007106 HE000338 NUMBER OF SEMESTERS VOCATIONAL/TECHNICAL/BUSINESS EDUCATION (LINEAR) 
Table C-2 (continued)

Summary Table of the 1996 Mathematics Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

BACK0046 B007107 TB007100 SEMESTERS ART/MUSIC (MISSING VS NON-MISSING) BACK0047 B007107 TB007100 NUMBER OF SEMESTERS ART/MUSIC (LINEAR) BACK0048 INTERACT INTERACTION: GENDER BY RACE/ETHNICITY BACK0049 INTERACT INTERACTION: GENDER BY TYPE OF LOCALE (5 CATEGORIES) BACK0050 INTERACT INTERACTION: GENDER BY PARENTS’ EDUCATION BACK0051 INTERACT INTERACTION: GENDER BY SCHOOL TYPE BACK0052 INTERACT INTERACTION: RACE/ETHNICITY BY TYPE OF LOCALE (5 CATEGORIES) BACK0053 INTERACT INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION BACK0054 INTERACT INTERACTION: RACE/ETHNICITY BY SCHOOL TYPE BACK0055 INTERACT INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY PARENT’S EDUCATION BACK0056 INTERACT INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY SCHOOL TYPE BACK0057 INTERACT INTERACTION: PARENTS’ EDUCATION BY SCHOOL TYPE BACK0058 INTERACT INTERACTION: GENDER BY MATH COURSES TAKING THIS YEAR BACK0059 INTERACT INTERACTION: GENDER BY NUMBER OF SEMESTERS MATH BACK0060 INTERACT INTERACTION: RACE/ETHNICITY BY MATH COURSES TAKING THIS YEAR BACK0061 INTERACT INTERACTION: RACE/ETHNICITY BY NUMBER OF SEMESTERS MATH BACK0062 INTERACT INTERACTION: PARENTS’ EDUCATION BY MATH COURSES TAKING THIS YEAR BACK0063 INTERACT INTERACTION: PARENTS’ EDUCATION BY NUMBER OF SEMESTERS MATH BACK0064 INTERACT INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY MATH COURSES TAKING THIS YEAR BACK0065 INTERACT INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY NUMBER OF SEMESTERS MATH BACK0066 INTERACT INTERACTION: SCHOOL TYPE BY MATH COURSES TAKING THIS YEAR BACK0067 INTERACT INTERACTION: SCHOOL TYPE BY NUMBER OF SEMESTERS MATH BACK0068 MA93FLG MSA/NON-MSA BACK0069 MONSTUD STATE ADMINISTRATION MONITORED/UNMONITORED SESSION BACK0070 INTERACT INTERACTION: SCHOOL TYPE BY MONITORED/UNMONITORED SESSION BACK0071 SUBSAMP SAMPLE TYPE BACK0072 INTERACT INTERACTION: SAMPLE BY RACE/ETHNICITY BACK0073 RPTSAMP REPORTING SAMPLE BACK0074 B009301 HE002795 HOW OFTEN USE A HOME COMPUTER FOR SCHOOLWORK? BACK0075 B009401 HE002541 HOW SAFE DO YOU FEEL AT SCHOOL? BACK0076 B009501 HE003221 DOES YOUR STEP/MOTHER WORK AT A JOB FOR PAY? BACK0077 B009502 HE003222 DOES YOUR STEP/FATHER WORK AT A JOB FOR PAY? SUBJ0002 M812701 TM810101 HOW OFTEN DO YOU DO MATH PROBLEMS FROM TEXTBOOK? SUBJ0003 M812702 TM810102 HOW OFTEN DO YOU DO MATH PROBLEMS ON WORK SHEETS? SUBJ0004 M812703 HE002478 HOW OFTEN SOLVE MATH PROBLEMS WITH PARTNER/GROUP? SUBJ0005 M812704 TM810112 HOW OFTEN IN MATH WORK W/ RULERS, GEOM SHAPES? SUBJ0006 M812705 TM810109 HOW OFTEN WRITE ABOUT SOLVING A MATH PROBLEM? SUBJ0007 M812706 TM810107 HOW OFTEN DO YOU TAKE MATHEMATICS TESTS? SUBJ0008 M812707 HE002479 HOW OFTEN DO YOU TALK TO THE CLASS RE MATH WORK? SUBJ0009 M812708 HE002480 HOW OFTEN DO 10 OR MORE MATH PROBLEMS BY YOURSELF? SUBJ0010 M812709 HE002481 HOW OFTEN DISCUSS MATH PROBLEMS W/ OTHER STUDENTS? SUBJ0011 M812710 TM810106 HOW OFTEN DO YOU USE A COMPUTER FOR MATH? SUBJ0012 M812711 TM810105 HOW OFTEN DO YOU USE A CALCULATOR FOR MATH? SUBJ0013 M811201 HE000378 DO YOU HAVE CALCULATOR TO USE WITH MATH HOMEWORK? SUBJ0014 M812001 LC000519 HOW OFTEN USE A CALCULATOR FOR MATH CLASSWORK? SUBJ0015 M812002 LC000520 HOW OFTEN USE A CALCULATOR FOR MATH HOMEWORK? SUBJ0016 M812003 LC000521 HOW OFTEN USE A CALCULATOR FOR MATH TESTS/QUIZZES? SUBJ0017 M812101 TM810601 HOW MUCH TIME SPEND DAILY ON MATH HOMEWORK? 
Table C-2 (continued)

Summary Table of the 1996 Mathematics Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

454SUBJ0018 M811401 HE000705 GET MATH HELP FROM SPECIAL TEACHER/AIDE/TUTOR? SUBJ0019 M811101 TM811101 AGREE/DISAGREE: I LIKE MATH SUBJ0020 M811103 TM811103 AGREE/DISAGREE: I AM GOOD AT MATH SUBJ0021 M811106 HE000651 HOW MUCH AGREE-UNDERSTAND MOST OF MATH CLASS? SUBJ0022 M811109 HE002482 HOW MUCH AGREE-ONLY 1 CORRECT WAY TO SOLVE PROB? SUBJ0023 M811107 HE000654 HOW MUCH AGREE-LEARNING MATH IS MEMORIZING FACTS? SUBJ0024 M811105 TM811105 AGREE/DISAGREE: MATH USED FOR SOLVING PROBLEMS SUBJ0025 M811108 HE000367 HOW MUCH AGREE-IF CHOICE I WOULDN’T STUDY MORE? SUBJ0026 M811110 LC000022 HOW MUCH AGREE-EVERYONE CAN DO WELL IF TRY? SUBJ0027 MM00101 HE000782 ABOUT HOW MANY QUESTIONS RIGHT ON TEST? SUBJ0028 MM00201 HE000783 HOW HARD TEST COMPARED TO THOSE IN SCHOOL? SUBJ0029 MM00301 JL001008 HOW HARD DID YOU TRY ON TEST COMPARED TO OTHERS? SUBJ0030 MM00401 JL001009 HOW IMPORTANT WAS IT YOU DO WELL ON THIS TEST? SUBJ0031 MM00501 HE000784 HOW OFTEN HAD TO WRITE LONG ANSWERS TO QSTS? BACK0078 B009701 HE002543 DESCRIBE YOUR OVERALL GRADES SINCE 6TH GRADE BACK0079 B009801 HE002544 HOW FAR IN SCHOOL DO YOU THINK YOU WILL GO? BACK0080 B009601 HE003223 DOES YOUR STEP/MOTHER WORK AT A JOB FOR PAY? BACK0081 B009602 HE003224 DOES YOUR STEP/FATHER WORK AT A JOB FOR PAY? SUBJ0032 M812712 TM810104 WORK W/ MEAS. INSTRUMENTS/GEOM SOLIDS FOR MATH? SUBJ0033 M812713 TM810108 HOW OFTEN WRITE REPORTS OR DO MATH PROJECTS? SUBJ0034 M812201 HE002483 IS THERE A PORTFOLIO W/ YOUR MATH WORK IN IT? SUBJ0035 M812301 HE002484 DO YOU USE A SCIENTIFIC CALCULATOR FOR MATH WORK? SUBJ0036 M812401 HE002485 DO YOU USE A GRAPHING CALCULATOR FOR MATH WOR SUBJ0037 MATEXP WHAT KIND OF MATH CLASS WILL YOU TAKE IN 9TH GR? SUBJ0038 M810701 TM810701 DO YOU AGREE: I LIKE MATH SUBJ0039 M810703 LD001476 DO YOU AGREE: I AM GOOD IN MATH SUBJ0040 M810707 HE000369 AGREE/DISAGREE: UNDERSTAND MOST OF MATH CLASS SUBJ0041 M810709 LD001473 AGREE/DISAGREE: ONLY ONE WAY TO SOLVE MATH PROBLEM SUBJ0042 M810708 HE000631 AGREE/DISAGREE: MATH IS MOSTLY MEMORIZING FACTS SUBJ0043 M810710 LD001474 AGREE/DISAGREE: CONCEPTS ARE AS IMPORTANT AS OPER SUBJ0044 M810705 TM810705 DO YOU AGREE: MATH USEFUL/SOLVING EVERYDAY PROBLEM SUBJ0045 M810706 HE000374 AGREE/DISAGREE: WOULD NOT STUDY MORE MATH SUBJ0046 M810711 LD001475 AGREE/DISAGREE: ALL CAN DO WELL IN MATH IF BACK0082 B008301 LC000168 SINCE KDG, GRADES ATTENDED IN THIS STATE? BACK0083 B009901 HE002548 DESCRIBE YOUR OVERALL GRADES SINCE 9TH GRADE BACK0084 B008501 HE002549 WHICH BEST DESCRIBES YOUR HIGH SCHOOL PROGRAM BACK0085 B005501 TB005501 WHAT WILL TAKE LARGEST AMT. OF TIME AFTER HIGH-SCH BACK0086 MOTHOCC KIND OF WORK DONE BY YOUR MOTHER/FEMALE GUARDIAN? BACK0087 B011A01 HE003220 KIND OF WORK DONE BY YOUR MOTHER/FEMALE GUARDIAN? BACK0088 FATHOCC KIND OF WORK DONE BY YOUR FATHER/MALE GUARDIAN? BACK0089 B012A02 BO001994 KIND OF WORK DONE BY YOUR FATHER/MALE GUARDIAN? SUBJ0047 M810901 TM810901 ARE YOU TAKING MATH CLASSES THIS YEAR SUBJ0048 M811801 JL001014 WHAT GRADE DID YOU TAKE 1ST YR ALGEBRA? SUBJ0049 M811001 TM811001 HOW LONG HAVE YOU TAKEN GENERAL MATH (9-12) SUBJ0050 M811002 TM811002 HOW LONG HAVE YOU TAKEN BUSINESS OR CONSUMER MATH SUBJ0051 M811014 HE002487 HOW LONG TAKEN APPLIED MATH? SUBJ0052 M811003 TM811003 HOW LONG HAVE YOU TAKEN INTRO (PRE) - ALGEBRA SUBJ0053 M811004 TM811004 HOW LONG HAVE YOU STUDIED FIRST-YEAR ALGEBRA 
Table C-2 (continued)

Summary Table of the 1996 Mathematics Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

SUBJ0054 M811005 TM811005 HOW LONG HAVE YOU TAKEN GEOMETRY SUBJ0055 M811006 TM811006 HOW LONG HAVE YOU TAKEN SECOND-YEAR ALGEBRA SUBJ0056 M811007 TM811007 HOW LONG HAVE YOU TAKEN TRIGONOMETRY SUBJ0057 M811008 TM811008 HOW LONG HAVE YOU TAKEN PRE-CALC (3RD YR ALGEBRA) SUBJ0058 M811012 LC000030 HOW LONG TAKEN UNIFIED/INTEGRTD/SEQ MATH? SUBJ0059 M811009 TM811009 HOW LONG HAVE YOU TAKEN PROBABILITY OR STATISTICS SUBJ0060 M811011 TM811011 HOW LONG HAVE YOU TAKEN CALCULUS SUBJ0061 M812801 WP000071 IN/HAVE TAKEN MATH ADVANCED PLACEMENT CLASS? SCHL0003 C030901 HE000839 BEST DESCRIBES HOW 4TH GR ARE ORGANIZED? SCHL0004 C037101 HE000840 4TH GRADERS ASSIGNED BY ABILITY/ACHIEVEMENT LEVEL? SCHL0005 C031212 HE002000 HOW OFTEN IS 4TH-GRADER INSTRUCTED IN MATH? SCHL0006 C031205 HE002002 HOW OFTEN IS 4TH-GRADER INSTRUCTED IN SCIENCE? SCHL0007 C031213 LD001554 HOW OFTEN IS 4TH-GRADER INSTRUCTED IN READING? SCHL0008 C031214 LD001555 HOW OFTEN IS 4TH-GRADER INSTRUCTED IN ARTS? SCHL0009 C031603 HE000861 HAS MATH BEEN IDENTIFIED AS A PRIORITY? SCHL0010 C031607 LC000469 HAS SCIENCE BEEN IDENTIFIED AS A PRIORITY? SCHL0011 C031601 HE000859 HAS READING BEEN IDENTIFIED AS A PRIORITY? SCHL0012 C031610 LD001556 HAS ARTS BEEN IDENTIFIED AS A PRIORITY? SCHL0013 C031606 HE000958 HAS SUBJECT INTEGRATION BEEN A PRIORITY? SCHL0014 C035701 HE000843 COMPUTERS AVAILABLE ALL THE TIME IN CLASSROOM? SCHL0015 C035702 HE000864 COMPUTERS GROUPED IN SEPARATE LAB AND AVAILABLE? SCHL0016 C035703 HE000866 COMPUTERS AVAILABLE TO BRING TO ROOM WHEN NEEDED? SCHL0017 C037201 HE002006 SCHOOL W/ SPECIAL FOCUS ON MATH? SCHL0018 C037202 HE002008 SCHOOL W/ SPECIAL FOCUS ON SCIENCE? SCHL0019 C037207 LD001557 SCHOOL W/ SPECIAL FOCUS ON MATH? SCHL0020 C037204 LD001558 SCHOOL W/ SPECIAL FOCUS ON ARTS? SCHL0021 C037205 HE002011 SCHOOL W/ SPECIAL FOCUS ON OTHER? SCHL0022 C037206 HE002012 SCHOOL NOT A SPECIAL FOCUS SCHOOL? SCHL0023 C037301 HE002014 SCHOOL FOLLOW DISTRICT/STATE MATH CURRICULUM? SCHL0024 C037302 HE002016 SCHOOL FOLLOW DISTRICT/STATE SCIENCE CURRICULUM? SCHL0025 C037303 LD001559 SCHOOL FOLLOW DISTRICT/STATE READING CURRICULUM? SCHL0026 C037304 LD001560 SCHOOL FOLLOW DISTRICT/STATE ARTS CURRICULUM? SCHL0027 C037305 HE002019 SCHOOL FOLLOW DISTRICT/STATE FOR NONE OF ABOVE? SCHL0028 C037401 HE002021 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR MATH? SCHL0029 C037402 HE002023 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR SCIENCE? SCHL0030 C037403 LD001561 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR READING? SCHL0031 C037404 LD001562 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR ARTS? SCHL0032 C037405 HE002026 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR OTHER? SCHL0033 C037406 HE002027 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR NONE ABOVE? SCHL0034 C037501 HE002029 4TH GRADERS IN EXTRACURR ACTS FOR MATH? SCHL0035 C037502 HE002031 4TH GRADERS IN EXTRACURR ACTS FOR SCIENCE? SCHL0036 C037503 LD001563 4TH GRADERS IN EXTRACURR ACTS FOR READING? SCHL0037 C037504 LD001564 4TH GRADERS IN EXTRACURR ACTS FOR ARTS? SCHL0038 C037505 HE002034 4TH GRADERS IN EXTRACURR ACTS FOR NONE OF ABOVE? SCHL0039 C037601 HE002036 4TH GRADERS IN SUMMER PROGRAMS IN MATH? SCHL0040 C037602 HE002038 4TH GRADERS IN SUMMER PROGRAMS IN SCIENCE? SCHL0041 C037603 LD001565 4TH GRADERS IN SUMMER PROGRAMS IN READING? SCHL0042 C037604 LD001566 4TH GRADERS IN SUMMER PROGRAMS IN ARTS? 
Table C-2 (continued)

Summary Table of the 1996 Mathematics Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

456SCHL0043 C037605 HE002041 4TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE? SCHL0044 C036601 LC000502 WHICH BEST DESCRIBES PRIMARY WAY LIBRARY STAFFED? SCHL0045 C032207 HE000875 INVOLVE PARENTS AS AIDES IN CLASSROOM? SCHL0046 C032209 LC000482 HAVE PARENTS REVIEW/SIGN HOMEWORK? SCHL0047 C032210 LC000484 ASSIGN HOMEWORK STUDENTS DO WITH PARENTS? SCHL0048 C032211 LC000486 HAVE A PARENT VOLUNTEER PROGRAM? SCHL0049 C037701 HE002142 WHAT % OF PARENTS IN PARENT-TEACHER ORGS? SCHL0050 C037702 HE002108 WHAT % OF PARENTS IN OPEN HOUSE/BACK SCHOOL NIGHT? SCHL0051 C037703 HE002109 WHAT % OF PARENTS IN PARENT-TEACHER COMFERENCES? SCHL0052 C037704 HE002110 WHAT % PARENTS INVOLVED MAKING CURRICULUM DECISION SCHL0053 C037705 HE002111 WHAT % OF PARENTS IN VOLUNTEER PROGRAMS? SCHL0054 C032402 HE000888 IS STUDENT ABSENTEEISM A PROBLEM IN YOUR SCHOOL? SCHL0055 C032401 HE000887 IS STUDENT TARDINESS A PROBLEM IN YOUR SCHOOL? SCHL0056 C032404 HE000890 ARE PHYSICAL CONFLICTS A PROBLEM IN YOUR SCHOOL? SCHL0057 C032406 HE000892 IS TEACHER ABSENTEEISM A PROBLEM IN YOUR SCHOOL? SCHL0058 C032407 HE000893 ARE RACE/CULT. CONFLICTS A PROBLEM IN YOUR SCHOOL? SCHL0059 C032408 HE000894 IS STUDENT HEALTH A PROBLEM IN YOUR SCHOOL? SCHL0060 C032409 HE002121 IS LACK OF PARENT INVLMT A PROBLEM IN YOUR SCHOOL? SCHL0061 C032410 HE002122 IS STUD USE OF ALCOHOL A PROBLEM IN YOUR SCHOOL? SCHL0062 C032411 HE002123 IS STUDENT TOBACCO USE A PROBLEM IN YOUR SCHOOL? SCHL0063 C032412 HE002124 IS STUDENT DRUG USE A PROBLEM IN YOUR SCHOOL? SCHL0064 C032413 HE002125 ARE GANG ACTIVITIES A PROBLEM IN YOUR SCHOOL? SCHL0065 C032414 HE002126 IS STUDENT MISBEHAVIOR A PROBLEM IN YOUR SCHOOL? SCHL0066 C032415 HE002127 IS STUDENT CHEATING A PROBLEM IN YOUR SCHOOL? SCHL0067 C032502 HE000897 IS TEACHER MORALE POS. OR NEG.? SCHL0068 C032503 HE000898 ARE STUDENT ATTITUDES TO ACADEMICS POS. OR NEG.? SCHL0069 C032505 HE000900 IS PARENT SUPPORT FOR ACHIEVEMENT POS. OR NEG.? SCHL0070 C032506 HE000901 IS REGARD FOR SCHOOL PROPERTY POS. OR NEG.? SCHL0071 C033601 HE000917 % ABSENT ON AVERAGE DAY? SCHL0072 C036501 LC000488 WHAT % OF TEACHERS ABSENT ON GIVEN DAY? SCHL0073 C037801 HE000918 % OF STUDS EROLLED AT START OF YR EROLLED AT END? SCHL0074 C037901 HE002112 % OF 4TH GRADERS HELD BACK & REPEATING 4TH GRADE? SCHL0075 C038001 HE000920 % OF FULL TIME TEACHERS LEFT BEFORE END OF YR? SCHL0076 C038301 HE002094 IS SCHOOL IN NATIONAL SCHOOL LUNCH PROGRAM? SCHL0077 C038801 WP000069 SCHOOL RECEIVE CHAP 1/TITLE 1 FUNDING? SCHL0078 C034101 HE002143 DID PRINCIPAL FILL OUT THIS QUESTIONNAIRE SCHL0079 C034102 HE002113 DID HEADMASTER/HEADMISTRESS FILL OUT QUESTIONNAIRE SCHL0080 C034103 HE002114 DID HEAD TEACHER FILL OUT THIS QUESTIONNAIRE SCHL0081 C034104 HE002115 DID VICE PRINCIPAL FILL OUT THIS QUESTIONNAIRE SCHL0082 C034105 HE002116 DID COUNSELOR FILL OUT THIS QUESTIONNAIRE SCHL0083 C034106 HE002117 DID CURRICULUM COORD FILL OUT THIS QUESTIONNAIRE SCHL0084 C034107 HE002118 DID TEACHER FILL OUT THIS QUESTIONNAIRE SCHL0085 C034108 HE002119 DID SECRETARY FILL OUT THIS QUESTIONNAIRE SCHL0086 C034109 HE002120 DID OTHER PERSON FILL OUT THIS QUESTIONNAIRE TCHR0001 T055901 HE001004 WHAT IS YOUR GENDER? TCHR0002 T056001 LD001610 WHICH BEST DESRIBES YOU? TCHR0003 T040301 HE001007 YEARS TAUGHT TCHR0004 T056101 LD001500 HOW MANY YRS TOTAL YOU TAUGHT MATH? 
Table C-2 (continued)

Summary Table of the 1996 Mathematics Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

TCHR0005 T056102 LD001501 HOW MANY YRS TOTAL YOU TAUGHT SCIENCE? TCHR0006 T056201 HE002551 TYPE TCHNG CERT IN THIS ST IN MAIN FIELD? TCHR0007 T040501 HE001010 CERTIFICATION, ELEMENTARY OR MIDDLE/JUNIOR HS ED? TCHR0008 T040506 HE002552 DO YOU HAVE CERTIFICATION IN ELEMENTARY MATH? TCHR0009 T040504 HE001082 DO YOU HAVE CERTIFICATION IN JR HIGH/SEC MATH? TCHR0010 T040507 HE002553 CERTIFICATION, ELEMENTARY SCIENCE? TCHR0011 T040508 HE002554 CERTIFICATION, MIDDLE/JUNIOR SCIENCE TCHR0012 T040505 HE002555 CERTIFICATION, OTHER TCHR0013 T056301 HE001012 HIGHEST ACADEMIC DEGREE YOU HOLD? TCHR0014 T040701 HE002556 EDUCATION UNDERGRAD MAJOR TCHR0015 T040706 HE002557 ELMENT ED UNDERGRAD MAJOR TCHR0016 T040707 HE002558 SEC ED UNDERGRAD MAJOR TCHR0017 T040703 HE002559 WAS YOUR UNDERGRADUATE MAJOR MATH? TCHR0018 T040704 HE002560 WAS YOUR UNDERGRADUATE MAJOR MATH ED? TCHR0019 T040710 HE002561 SCIENCE ED UNDERGRAD MAJOR TCHR0020 T040711 HE002562 LIFE SCIENCE UNDERGRAD MAJOR? TCHR0021 T040712 HE002563 PHYSICAL SCIENCE UNDERGRAD MAJOR? TCHR0022 T040713 HE002564 EARTH SCIENCE UNDERGRAD MAJOR? TCHR0023 T040708 HE002565 SPECIAL EDUCATION UNDERGRAD MAJOR TCHR0024 T040709 HE002566 BILINGUAL ED/ESL UNDERGRAD MAJOR TCHR0025 T040705 HE002567 OTHER UNDERGRAD MAJOR TCHR0026 T040801 HE002568 EDUCATION GRAD MAJOR TCHR0027 T040807 HE002569 ELEMENTARY ED GRAD MAJOR TCHR0028 T040808 HE002570 SECONDARY ED GRAD MAJOR TCHR0029 T040803 HE002571 WAS YOUR GRADUATE MAJOR MATHEMATICS? TCHR0030 T040804 HE002572 WAS YOUR GRADUATE MAJOR MATH ED? TCHR0031 T040814 HE002573 SCIENCE ED GRAD MAJOR? TCHR0032 T040815 HE002574 LIFE SCIENCE GRAD MAJOR? TCHR0033 T040816 HE002575 PHYSICAL SCIENCE GRAD MAJOR? TCHR0034 T040817 HE002576 EARTH SCIENCE GRAD MAJOR? TCHR0035 T040809 HE002577 SPECIAL ED GRAD MAJOR TCHR0036 T040810 HE002578 BILINGUAL GRAD MAJOR TCHR0037 T040811 HE002579 ADMIN/SUPERVISION GRAD MAJOR TCHR0038 T040812 HE002580 CURRICULUM/INSTRUCTION GRAD MAJOR? TCHR0039 T040813 LD001506 COUNSELING GRAD MAJOR? TCHR0040 T040805 HE002581 OTHER GRAD MAJOR TCHR0041 T040806 HE002582 NO GRADUATE STUDY TCHR0042 T056401 HE002584 UNDERGRAD/GRAD MINOR STUDY-EDUCATION TCHR0043 T056402 HE002585 UNDERGRAD/GRAD MINOR STUDY-ELEMENTARY ED TCHR0044 T056403 HE002586 UNDERGRAD/GRAD MINOR STUDY-SECONDARY ED TCHR0045 T056404 HE002587 UNDERGRAD/GRAD MINOR STUDY-MATHEMATICS TCHR0046 T056405 HE002588 UNDERGRAD/GRAD MINOR STUDY-MATHEMATICS ED TCHR0047 T056413 HE002589 UNDERGRAD/GRAD MINOR STUDY-SCIENCE ED TCHR0048 T056414 HE002590 UNDERGRAD/GRAD MINOR STUDY-LIFE SCIENCE TCHR0049 T056415 HE002591 UNDERGRAD/GRAD MINOR STUDY-PHYSICAL SCIENCE TCHR0050 T056416 HE002592 UNDERGRAD/GRAD MINOR STUDY-EARTH SCIENCE TCHR0051 T056406 HE002593 UNDERGRAD/GRAD MINOR STUDY-SPECIAL ED TCHR0052 T056407 HE002594 UNDERGRAD/GRAD MINOR STUDY-BILINGUAL ED 
Table C-2 (continued)

Summary Table of the 1996 Mathematics Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

458TCHR0053 T056408 HE002595 UNDERGRAD/GRAD MINOR STUDY-ADMIN & SUPERVISION TCHR0054 T056409 HE002596 UNDERGRAD/GRAD MINOR STUDY-CURRICULUM & INSTRUC TCHR0055 T056410 LD001509 UNDERGRAD/GRAD MINOR STUDY-COUNSELING TCHR0056 T056411 HE002597 UNDERGRAD/GRAD MINOR STUDY-OTHER TCHR0057 T056412 HE002598 UNDERGRAD/GRAD MINOR STUDY-NONE TCHR0058 T056501 HE002599 LAST YR, HOW MUCH TIME IN MATH/MATH ED SEM/WRKSHP? TCHR0059 T058101 he002600 LAST YR, HOW MUCH TIME IN SCI/SCI ED SEM/WRKSHPS? TCHR0060 T056601 HE002601 LAST 2 YRS, HOW MANY MATH/MATH ED UNIV COURSES? TCHR0061 T058201 HE002602 LAST 2 YRS, HOW MANY SCI/SCI ED UNIV COURSES? TCHR0062 T056701 HE002604 PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-TELECOMM USE TCHR0063 T056702 HE002605 PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-TECH USE TCHR0064 T056703 HE002606 PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-COOP INSTRCT TCHR0065 T056704 HE002607 PAST 5 YRS, COURSES/IN PRO DEVLP-INTERDISP INSTRCT TCHR0066 T056705 HE002608 PAST 5 YRS, COURSES/IN PRO DEVLP-PORTFOLIO ASSMNT TCHR0067 T056706 HE002609 PAST 5 YRS, COURSES/IN PRO DEVLP-PERF BASED ASSMNT TCHR0068 T056707 HE002610 PAST 5 YRS, COURSES/PRO DEVLP-TEACH HIGHORDER THKG TCHR0069 T056708 HE002611 PAST 5 YRS, COURSES/PRO DEVLP-TEACH DIFF CULT BKGD TCHR0070 T056709 HE002612 PAST 5 YRS, COURSES/PRO DEVLP-TEACH LEP STUDENTS TCHR0071 T056710 HE002613 PAST 5 YRS, COURSES/PRO DEVLP-TEACH SPEC NEED STDS TCHR0072 T056711 HE002614 PAST 5 YRS, COURSES/PRO DEVLP-CLASSRM MNGMT/ORG TCHR0073 T056712 HE002615 PAST 5 YRS, COURSES/PRO DEVLP-OTHER PROF ISSUES TCHR0074 T056713 HE002616 PAST 5 YRS, COURSES/PRO DEVLP-NONE OF ABOVE TCHR0075 T041201 HE001022 AVAILABILITY OF RESOURCES TCHR0076 T041302 LD001512 ARE CURRICULUM SPECIALISTS AVAILABLE FOR MATH? TCHR0077 T041303 LD001513 SCIENCE CURRICULUM SPECIALIST TCHR0078 T056801 HE001251 HOW MANY SCHOOL HOURS ARE PREP TIME PER WEEK? TCHR0079 T056901 HE003110 METHODS OF TEACHING ELEM MATH- 1+COLLEGE COURSE TCHR0080 T0569A1 HE003110 METHODS OF TEACHING ELEM MATH-PART COLLEGE COURSE TCHR0081 T0569B1 HE003110 METHODS OF TEACHING ELEM MATH-SEMINAR TCHR0082 T0569C1 HE003110 METHODS OF TEACHING ELEM MATH-LITTLE NO EXPOSURE TCHR0083 T056902 HE003111 NUMBER SYSTEMS & NUMERATION-1+ COLLEGE COURSE TCHR0084 T0569A2 HE003111 NUMBER SYSTEMS & NUMERATION-PART COLLEGE COURSE TCHR0085 T0569B2 HE003111 NUMBER SYSTEMS & NUMERATION-SEMINAR TCHR0086 T0569C2 HE003111 NUMBER SYSTEMS & NUMERATION-LITTLE/NO EXPOSURE TCHR0087 T056903 HE003112 MEASUREMENT IN MATH- 1+COLLEGE COURSE TCHR0088 T0569A3 HE003112 MEASUREMENT IN MATH- PART COLLEGE COURSE TCHR0089 T0569B3 HE003112 MEASUREMENT IN MATH- -SEMINAR TCHR0090 T0569C3 HE003112 MEASUREMENT IN MATH- -LITTLE NO EXPOSURE TCHR0091 T056904 HE003113 GEOMETRY- 1+COLLEGE COURSE TCHR0092 T0569A4 HE003113 GEOMETRY-PART COLLEGE COURSE TCHR0093 T0569B4 HE003113 GEOMETRY-SEMINAR TCHR0094 T0569C4 HE003113 GEOMETRY-LITTLE NO EXPOSURE TCHR0095 T056905 HE003114 PROBABILITY/STATISTICS- 1+COLLEGE COURSE TCHR0096 T0569A5 HE003114 PROBABILITY/STATISTICS-PART COLLEGE COURSE TCHR0097 T0569B5 HE003114 PROBABILITY/STATISTICS-SEMINAR TCHR0098 T0569C5 HE003114 PROBABILITY/STATISTICS-LITTLE NO EXPOSURE TCHR0099 T056906 WO001011 COLLEGE ALGEBRA- 1+COLLEGE COURSE TCHR0100 T0569A6 WO001011 COLLEGE ALGEBRA-PART COLLEGE COURSE 
Table C-2 (continued)

Summary Table of the 1996 Mathematics Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

TCHR0101 T0569B6 WO001011 COLLEGE ALGEBRA-SEMINAR TCHR0102 T0569C6 WO001011 COLLEGE ALGEBRA-LITTLE NO EXPOSURE TCHR0103 T056907 HE003116 CALCULUS- 1+COLLEGE COURSE TCHR0104 T0569A7 HE003116 CALCULUS-PART COLLEGE COURSE TCHR0105 T0569B7 HE003116 CALCULUS-SEMINAR TCHR0106 T0569C7 HE003116 CALCULUS-LITTLE NO EXPOSURE TCHR0107 T056908 HE003117 ABSTRACT/LINEAR ALGEBRA- 1+COLLEGE COURSE TCHR0108 T0569A8 HE003117 ABSTRACT/LINEAR ALGEBRA-PART COLLEGE COURSE TCHR0109 T0569B8 HE003117 ABSTRACT/LINEAR ALGEBRA-SEMINAR TCHR0110 T0569C8 HE003117 ABSTRACT/LINEAR ALGEBRA-LITTLE NO EXPOSURE TCHR0111 T057001 HE003118 EVER STUDIED ESTIMATION? TCHR0112 T057002 HE003119 EVER STUDIED PROBLEM SOLVING IN MATH? TCHR0113 T057003 HE003120 EVER STUDIED USE OF MANIPULATIVES? TCHR0114 T057004 HE003121 EVER STUDIED USE OF CALCULATORS IN MATH INSTRC? TCHR0115 T057005 HE003122 EVER STUDIED UNDERSTANDING STUDS MATH THINKING? TCHR0116 T057006 HE003123 EVER STUDIED GENDER ISSUES IN TEACHING MATH? TCHR0117 T057007 HE003124 EVER STUDIED TEACHING STUDS OF DIFF CULTURES? TCHR0118 T057101 WO001012 KNOWLEDGE OF NCTM CURR & EVAL STANDARDS FOR MATH? TCHR0119 T057201 WO001013 PRO ACTVTS-STRATEGIES LOCAL WORKSHOPS TCHR0120 T057211 WO001013 PRO ACTVTS-STRATEGIES REGIONAL NCTM MEETING TCHR0121 T057221 WO001013 PRO ACTVTS-STRATEGIES NATIONAL NCTM MEETING TCHR0122 T057231 WO001013 PRO ACTVTS-STRATEGIES OTHER TCHR0123 T057241 WO001013 PRO ACTVTS-STRATEGIES NO TSUB0001 T057301 HE002379 IMPORTANCE W/ STUDS-APPLYING MATH IDEAS? TSUB0002 T057302 HE002380 IMPORTANCE W/ STUDS-PROB SOLVING=GOAL & CONCEPT? TSUB0003 T057303 HE002381 IMPORTANCE W/ STUDS-? TECHS PROMOTE STUD TALK? TSUB0004 T057304 HE002382 IMPORTANCE W/ STUDS-USE RESULTS TO INFORM DECISION TSUB0005 T057401 HE001131 TO ACCESS PROGRESS HOW OFTEN USE MULT CHOICE TESTS TSUB0006 T057402 HE001132 TO ACCESS PROGRESS HOW OFTEN USE PROBLEM SETS TSUB0007 T057403 HE001133 TO ACCESS PROGRESS HOW OFTEN USE SHORT WRITTEN RSP TSUB0008 T057404 HE001134 TO ACCESS PROGRESS HOW OFTEN USE INDV/GROUP PRJCTS TSUB0009 T057405 HE002396 TO ACCESS PROGRESS HOW OFTEN USE PORTFOLIOS TSUB0010 T057501 HE002401 BEST DESCRIPTION OF COMPUTER AVAILABLITY IN MATH TSUB0011 T057601 HE002402 PRIMARY USE OF COMPUTERS FOR MATH INSTRUCTION? TSUB0012 T044002 HE002412 ARE STUDENTS ASSIGNED TO THIS CLASS BY ABILITY? TSUB0013 T057701 HE002383 IF ASSIGNED BY ABILITY, WHAT BASIS ASSIGNED? TSUB0014 T057801 HE002384 IF ASSIGNED BY ABILITY, WHAT IS MATH ABILITY? TSUB0015 T044201 HE001104 CREATE GROUPS IN CLASS FOR MATH ON ABILITY BASIS? TSUB0016 T044301 HE001105 TIME/WEEK ON MATH INSTRUTION? TSUB0017 T057901 HE002385 HOW MUCH TIME PER WEEK STUDS DO MATH W/ PARTNER? TSUB0018 T044401 HE001106 AMOUNT MATH HOMEWORK ASSIGN/DAY? TSUB0019 T044501 HE001108 HOW OFTEN DO STUDENTS DO MATH FROM TEXTBOOKS? TSUB0020 T044502 HE001109 HOW OFTEN DO STUDENTS DO MATH ON WORKSHEETS? TSUB0021 T044512 HE001110 HOW OFTEN DO STUDENTS SOLVE PROBS W/ OTHER STUDS? TSUB0022 T044513 HE002397 HOW OFTEN DO STUDENTS WORK W/ OBJECTS LIKE RULERS? TSUB0023 T044514 HE002398 HOW OFTEN WORK W/ COUNTING BLOCKS.GEOMETRIC SHAPES TSUB0024 T044505 HE001112 HOW OFTEN DO STUDENTS USE A CALCULATOR? TSUB0025 T044515 HE002399 HOW OFTEN DO STUDENTS TAKE MATH TESTS? 
Table C-2 (continued)

Summary Table of the 1996 Mathematics Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

460TSUB0026 T044507 HE001114 HOW OFTEN DO STUDENTS WRITE ABOUT PROBLEM-SOLVING? TSUB0027 T044516 HE002400 HOW OFTEN DO STUDENTS TALK ABOUT MATH WORK? TSUB0028 T044508 HE001115 HOW OFTEN DO STUDENTS WRITE REPORTS/DO PROJECTS? TSUB0029 T044509 HE001116 HOW OFTEN DO STUDENTS DISCUSS MATH W/OTHER STDNTS? TSUB0030 T044510 HE001117 HOW OFTEN DO STUDENTS WORK REAL-LIFE MATH PRBLMS? TSUB0031 T044506 HE001113 HOW OFTEN DO STUDENTS USE A COMPUTER? TSUB0032 T058001 HE002387 IN MATH CLASS HOW OFTEN ADDRESS-NUMBERS & OPS? TSUB0033 T058002 HE002388 IN MATH CLASS HOW OFTEN ADDRESS-MEASUREMENT? TSUB0034 T058003 HE002389 IN MATH CLASS HOW OFTEN ADDRESS-GEOMETRY? TSUB0035 T058004 HE002390 IN MATH CLASS HOW OFTEN ADDRESS-DATA ANALYSIS? TSUB0036 T058005 HE002391 IN MATH CLASS HOW OFTEN ADDRESS-ALGEBRA & FUNCT? TSUB0037 T058006 HE002392 IN MATH HOW OFTEN ADDRESS-LRN MATH FACTS/CONCEPTS? TSUB0038 T058007 HE002393 IN MATH HOW OFTEN ADDRESS-LRN SKILLS/PROCEDURES? TSUB0039 T058008 HE002394 IN MATH HOW OFTEN ADDRESS-DEVELOP REASONING ABLTY? TSUB0040 T058009 HE002395 IN MATH HOW OFTEN ADDRESS-LRN TO COMMUNICATE MATH? TSUB0041 T045401 HE001135 DO YOU PERMIT UNRESTRICTED USE OF CALCULATORS? TSUB0042 T044801 HE001136 DO YOU PERMIT USE OF CALCULATORS ON TESTS? TSUB0043 T045001 HE001183 STUDENTS HAVE ACCESS TO SCHL-OWNED CALCULATORS? TSUB0044 T044901 HE001279 DO YOU PROVIDE INSTRUCTION IN USE OF CALCULATORS? TSUB0045 T045304 HE001276 HOW PREPARED TO TEACH MATH CONCEPTS? TSUB0046 T045305 HE002403 HOW PREPARED TO TEACH MATH PROCEDURES? TSUB0047 T045302 HE001145 HOW PREPARED TO TEACH USE OF COMPUTERS? TSUB0048 T045303 HE001146 HOW PREPARED TO TEACH USE OF CALCULATORS? TSUB0049 T044000 WP000051 NUMBER OF STUDENTS IN CLASS? SCHL0087 C034201 HE000926 BEST DESCRIBES HOW 8TH GRADES ARE ORGANIZED? SCHL0088 C034402 HE002232 ARE 8TH-GRADERS ASSIGNED TO MATH BY ABILITY? SCHL0089 C034403 HE002234 ARE 8TH-GRADERS ASSIGNED TO SCIENCE BY ABILITY? SCHL0090 C034401 LD001571 ARE 8TH-GRADERS ASSIGNED TO ENGLISH BY ABILITY? SCHL0091 C034406 LD001572 ARE 8TH-GRADERS ASSIGNED TO ARTS BY ABILITY? SCHL0092 C034510 HE002146 HOW OFTEN 8TH GRDS RECEIVE COMP SCI INSTRUCTION? SCHL0093 C034511 HE002148 HOW OFTEN 8TH GRDS RECEIVE MATH INSTRUCTION? SCHL0094 C034512 HE002149 HOW OFTEN 8TH GRDS RECEIVE SCIENCE INSTRUCTION? SCHL0095 C034513 LD001573 HOW OFTEN 8TH GRDS RECEIVE ENGLISH INSTRUCTION? SCHL0096 C034514 LD001574 HOW OFTEN 8TH GRDS RECEIVE ARTS INSTRUCTION? SCHL0097 C031611 LD001575 HAS ENGLISH BEEN IDENTIFIED AS A PRIORITY? SCHL0098 C034601 HE000935 SCHOOL OFFER 8TH GR STUDS ALGEBRA FOR HS CREDIT? SCHL0099 C037203 LD001557 SCHOOL W/ SPECIAL FOCUS ON ENGLISH? SCHL0100 C037306 LD001577 SCHOOL FOLLOW DISTRICT/STATE ENGLISH CURRICULUM? SCHL0101 C039401 HE002155 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR MATH? SCHL0102 C039402 HE002157 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR SCIENCE? SCHL0103 C039403 LD001578 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR READING? SCHL0104 C039404 LD001579 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR ARTS? SCHL0105 C039405 HE002160 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR OTHER? SCHL0106 C039406 HE002161 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR NONE ABOVE? SCHL0107 C039501 HE002164 8TH GRADERS IN EXTRACURR ACTS FOR MATH? SCHL0108 C039502 HE002166 8TH GRADERS IN EXTRACURR ACTS FOR SCIENCE? SCHL0109 C039503 LD001580 8TH GRADERS IN EXTRACURR ACTS FOR ENG/LANG ARTS? SCHL0110 C039504 LD001581 8TH GRADERS IN EXTRACURR ACTS FOR ARTS? 
Table C-2 (continued)

Summary Table of the 1996 Mathematics Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

SCHL0111 C039505 HE002169 8TH GRADERS IN EXTRACURR ACTS FOR NONE OF ABOVE? SCHL0112 C039601 HE002172 8TH GRADERS IN SUMMER PROGRAMS IN MATH? SCHL0113 C039602 HE002174 8TH GRADERS IN SUMMER PROGRAMS IN SCIENCE? SCHL0114 C039603 LD001582 8TH GRADERS IN SUMMER PROGRAMS IN ENG/LANG ARTS? SCHL0115 C039604 LD001583 8TH GRADERS IN SUMMER PROGRAMS IN ARTS? SCHL0116 C039605 HE002177 8TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE? SCHL0117 C041901 HE002230 WHAT % OF 8TH GRDS HELD BACK/REPEAT 8TH GRADE? TCHR0124 T063001 HE001081 YRS TOTAL TAUGHT MATH TCHR0125 T058301 HE002617 CURRICULUM SPECIALIST TO HELP/ADVISE IN MATH? SCHL0118 C035002 HE002247 ARE 12TH-GRADERS ASSIGNED TO MATH BY ABILITY? SCHL0119 C035003 HE002249 ARE 12TH-GRADERS ASSIGNED TO SCIENCE BY ABILITY? SCHL0120 C035006 LD001588 ARE 12TH-GRADERS ASSIGNED TO ENGLISH BY ABILITY? SCHL0121 C035007 LD001589 ARE 12TH-GRADERS ASSIGNED TO ARTS BY ABILITY? SCHL0122 C040201 HE002253 FROM 9TH ON HOW MANY YRS REQUIRED FOR MATH? SCHL0123 C040202 HE002254 FROM 9TH ON HOW MANY YRS REQUIRED FOR SCIENCE? SCHL0124 C040203 LD001590 FROM 9TH ON HOW MANY YRS REQUIRED FOR ENG/LIT? SCHL0125 C040204 HE002255 FROM 9TH ON HOW MANY YRS REQUIRED FINE/PERF ARTS? SCHL0126 C040301 HE002256 COURSES 1 OR > SEMESTERS TAUGHT IN ADVANCED BIO? SCHL0127 C040302 HE002257 COURSES 1 OR > SEMESTERS TAUGHT IN ADVANCED CHEM? SCHL0128 C040303 HE002258 COURSES 1 OR > SEMESTERS TAUGHT IN ADV PHYSICS? SCHL0129 C040304 HE002259 NO ADVANCED SCIENCE COURSES TAUGHT SCHL0130 C040305 WP000094 COURSES 1 OR > SEMESTERS TAUGHT IN COMP SCI? SCHL0131 C040306 LC000512 COURSES 1 OR > SEMESTERS TAUGHT IN CALCULUS? SCHL0132 C040307 WP000095 COURSES 1 OR > SEMESTERS TAUGHT IN TRIGONOMETRY? SCHL0133 C040308 WP000096 COURSES 1 OR > SEMESTERS TAUGHT IN PRECALCULUS? SCHL0134 C040309 LC000511 COURSES 1 OR > SEMESTERS TAUGHT IN PROB/STAT? SCHL0135 C040310 WP000098 COURSES 1 OR > SEMESTERS TAUGHT IN UNI/INTEG MATH? SCHL0136 C040311 HE002260 NO ADVANCED MATH COURSES TAUGHT? SCHL0137 C040401 HE002272 STUDS REQUIRED TO PASS STATE TEST IN MATH? SCHL0138 C040402 HE002273 STUDS REQUIRED TO PASS STATE TEST IN SCIENCE? SCHL0139 C040403 LD001591 STUDS REQUIRED TO PASS STATE TEST IN ENG/LANG ARTS SCHL0140 C040404 HE002274 STUDS REQUIRED TO PASS STATE TEST IN FINE/PERF ART SCHL0141 C040501 HE002277 SCHOOL SPONSER 12TH GR FIELD TRIPS FOR MATH? SCHL0142 C040502 HE002279 SCHOOL SPONSER 12TH GR FIELD TRIPS FOR SCIENCE? SCHL0143 C040503 LD001592 SCHOOL SPONSER 12TH GR FIELD TRIPS IN ENG/LANG ART SCHL0144 C040504 LD001593 SCHOOL SPONSER 12TH GR FIELD TRIPS IN ARTS? SCHL0145 C040505 HE002286 SCHOOL SPONSER 12TH GR FIELD TRIPS IN OTHER? SCHL0146 C040506 HE002288 SCHOOL SPONSER 12TH GR FIELD TRIPS IN NONE ABOVE? SCHL0147 C040601 HE002296 12TH GRADERS IN EXTRACURR ACTS IN MATH? SCHL0148 C040602 HE002288 12TH GRADERS IN EXTRACURR ACTS IN SCIENCE? SCHL0149 C040603 LD001594 12TH GRADERS IN EXTRACURR ACTS IN ENG/LANG ARTS? SCHL0150 C040604 LD001595 12TH GRADERS IN EXTRACURR ACTS IN ARTS? SCHL0151 C040605 HE002291 12TH GRADERS IN EXTRACURR ACTS IN NONE OF ABOVE? SCHL0152 C040701 HE002294 12TH GRADERS IN SUMMER PROGRAMS IN MATH? SCHL0153 C040702 HE002296 12TH GRADERS IN SUMMER PROGRAMS IN SCIENCE? SCHL0154 C040703 LD001596 12TH GRADERS IN SUMMER PORGRAMS IN ENG/LANG ARTS? SCHL0155 C040704 LD001597 12TH GRADERS IN SUMMER PROGRAMS IN ARTS? SCHL0156 C040705 HE002299 12TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE? 
Table C-2 (continued)

Summary Table of the 1996 Mathematics Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

462SCHL0157 C040801 HE002346 # STUDS ENROLLED IN AP SCIENCE COURSES? SCHL0158 C040802 HE002347 # STUDS ENROLLED IN AP CALCULUS COURSES? SCHL0159 C040803 HE002350 # STUDS ENROLLED IN AP COMP SCI COURSES? SCHL0160 C040804 LD001598 # STUDS ENROLLED IN AP ENGLISH COURSES? SCHL0161 C040901 HE002353 ANY 12TH GRDS TAKING COLLEGE COURSES IN MATH? SCHL0162 C040902 HE002355 ANY 12TH GRDS TAKING COLLEGE COURSES IN SCIENCE? SCHL0163 C040903 LD001599 ANY 12TH GRDS TAKING COLLEGE COURSES ENG/LANG ARTS SCHL0164 C040904 LD001600 ANY 12TH GRDS TAKING COLLEGE COURSES IN ARTS? SCHL0165 C040905 HE002358 ANY 12TH GRDS TAKING COLLEGE COURSES IN NONE ABOVE SCHL0166 C041001 HE002359 WHAT % 12TH GRDS HELD BACK AND REPEAT 12TH GRADE? SCHL0167 C041101 HE003193 LAST YR WHAT % OF 12TH GRDS GRADUATED? SCHL0168 C036001 HE001002 WHAT % OF GRADUATING CLASS NOW IN 2-YR COLLEGE? SCHL0169 C036002 HE001003 WHAT % OF GRADUATING CLASS NOW IN 4-YR COLLEGE? SCHL0170 C036003 HE002360 WHAT % OF GRADUATING CLASS NOW IN VO-TEC SCHOOL? SCHL0171 C036004 HE002361 WHAT % OF GRAD CLASS NOW IN EMPLOYER TRAINING? SCHL0172 C036005 HE002362 % OF GRADUATING CLASS NOW IN MILITARY SERVICE? SCHL0173 NTLUNSC PERCENT OF STUDENTS WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0174 NTLUNSC PERCENT OF STUDENTS WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0175 REMRDSC PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL READING SCHL0176 REMRDSC PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL READING SCHL0177 REMMHSC PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL MATH SCHL0178 REMMHSC PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL MATH SCHL0179 NTLUNGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0180 NTLUNGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0181 REMRDGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL READING SCHL0182 REMRDGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL READING SCHL0183 REMMHGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL MATH SCHL0184 REMMHGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL MATH SCHL0185 NTLUNGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0186 NTLUNGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0187 REMRDGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL READING SCHL0188 REMRDGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL READING SCHL0189 REMMHGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL MATH SCHL0190 REMMHGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL MATH SCHL0191 NTLUNGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRASCHL0192 NTLUNGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRASCHL0193 REMRDGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL READING SCHL0194 REMRDGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL READING SCHL0195 REMMHGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL MATH SCHL0196 REMMHGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL MATH BACK0091 BOOK BOOK NUMBER FOR SCREENING 
Table C-3

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

BACK0001 BKSER GRAND MEAN BACK0002 DSEX DERIVED SEX BACK0003 DRACE DERIVED RACE/ETHNICITY BACK0004 B003101 TB003101 IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND? BACK0005 TOL8 MSA/NON-MSA BACK0006 TOL5 TYPE OF LOCALE (5 CATEGORIES) BACK0007 DOC DESCRIPTION OF COMMUNITY BACK0008 PARED PARENTS’ HIGHEST LEVEL OF EDUCATION BACK0009 REGION REGION OF THE COUNTRY BACK0010 SCHTYPE SCHOOL TYPE (PQ) BACK0011 IEP INDIVIDUALIZED EDUCATION PROGRAM BACK0012 LEP LIMITED ENGLISH PROFICIENCY BACK0013 TITLE1 TITLE 1 (BOOK COVER) BACK0014 SLUNCH DO YOU RECEIVE A FREE OR REDUCED-PRICE LUNCH? BACK0015 B003201 TB003201 HOW OFTEN DO THE PEOPLE IN YOUR HOME SPEAK A LANGUAGE OTHER THAN ENGLISH? BACK0016 B009001 LD001715 HOW MUCH TV/VIDEO DO YOU USUALLY WATCH EACH DAY? (LINEAR) BACK0017 B009001 LD001715 HOW MUCH TV/VIDEO DO YOU USUALLY WATCH EACH DAY? (QUADRATIC) BACK0018 B006601 TB006601 HOMEWORK ASSIGNED?: BASED ON TIME SPENT ON HOMEWORK EACH DAY. BACK0019 B006601 TB006601 HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY? (LINEAR) BACK0020 B006601 TB006601 HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY (QUADRATIC) BACK0021 B009101 LD001716 HOURS EXTRA READING/WK, NOT CONNECTED W/ SCHOOL? BACK0022 HOMEEN3 NUMBER OF ITEMS IN THE HOME (NEWSPAPER, > 25 BOOKS, ENCYCLOPEDIA, MAGAZINES) (DEBACK0023 S004001 TS004001 HOW MANY DAYS OF SCHOOL MISSED LAST MONTH? BACK0024 B007602 TM810801 HOW MANY GRADES IN THIS STATE (4) BACK0025 B010101 HE002545 SINCE 1ST GR, NOT PROMOTION, HOW OFTEN DIFF SCHLS? BACK0026 B008001 LC000006 HOW LONG LIVED IN THE UNITED STATES? BACK0027 B007601 JL001015 HOW MANY GRADES IN THIS STATE? (4TH GRADE) SCHL0001 SCHNORM SCHOOL LEVEL AVERAGE SCIENCE NORMIT (MISSING VS NON-MISSING) SCHL0002 SCHNORM SCHOOL LEVEL AVERAGE SCIENCE NORMIT BACK0028 B009201 HE002537 TIMES CHANGED SCHLS SINCE 1ST GR; NOT PROMOTIONS? BACK0029 B007302 HE000712 TIMES CHANGED SCHOOL SINCE FIRST GRADE BACK0030 B008301 LC000168 HOW MANY GRADES IN THIS STATE (12TH GRADE) BACK0031 B007401 HE000717 HOW OFTEN DO YOU DISCUSS THINGS STUDIED IN SCHOOL WITH SOMEONE AT HOME? BACK0032 B009301 HE002795 HOW OFTEN USE A HOME COMPUTER FOR SCHOOLWORK? BACK0033 B001101 TB001101 ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK? BACK0034 B001101 TB001101 ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK? BACK0035 B008501 HE002549 WHICH BEST DESCRIBES YOUR HIGH-SCHOOL PROGRAM? BACK0036 B007101 HE000333 SEMESTERS ENGLISH/LITERATURE/WRITING (MISSING VS NON-MISSING) BACK0037 B007101 HE000333 NUMBER OF SEMESTERS ENGLISH/LITERATURE/WRITING (LINEAR) BACK0038 B007102 HE000334 SEMESTERS MATHEMATICS (MISSING VS NON-MISSING) BACK0039 B007102 HE000334 NUMBER OF SEMESTERS MATHEMATICS (LINEAR) BACK0040 B007103 HE000335 SEMESTERS SCIENCE (MISSING VS NON-MISSING) BACK0041 B007103 HE000335 NUMBER OF SEMESTERS SCIENCE (LINEAR) BACK0042 B007104 HE000336 SEMESTERS HISTORY/SOCIAL STUDIES/GEOGRAPHY (MISSING VS NON-MISSING) BACK0043 B007104 HE000336 NUMBER OF SEMESTERS HISTORY/SOCIAL STUDIES/GEOGRAPHY (LINEAR) BACK0044 B007105 HE000337 SEMESTERS FOREIGN LANGUAGES (MISSING VS NON-MISSING) BACK0045 B007105 HE000337 NUMBER OF SEMESTERS FOREIGN LANGUAGES (LINEAR) BACK0046 B007106 HE000338 SEMESTERS VOCATIONAL/TECHNICAL/BUSINESS EDUCATION (MISSING VS NON-MISSING) 
Table C-3 (continued)

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

464BACK0047 B007106 HE000338 NUMBER OF SEMESTERS VOCATIONAL/TECHNICAL/BUSINESS EDUCATION (LINEAR) BACK0048 B007107 TB007100 SEMESTERS ART/MUSIC (MISSING VS NON-MISSING) BACK0049 B007107 TB007100 NUMBER OF SEMESTERS ART/MUSIC (LINEAR) SUBJ0001 SCITAKE WHAT KIND OF SCI CLASS ARE YOU TAKING THIS YEAR? BACK0050 INTERACT INTERACTION: GENDER BY RACE/ETHNICITY BACK0051 INTERACT INTERACTION: GENDER BY TYPE OF LOCALE (5 CATEGORIES) BACK0052 INTERACT INTERACTION: GENDER BY PARENTS’ EDUCATION BACK0053 INTERACT INTERACTION: GENDER BY SCHOOL TYPE BACK0054 INTERACT INTERACTION: RACE/ETHNICITY BY TYPE OF LOCALE (5 CATEGORIES) BACK0055 INTERACT INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION BACK0056 INTERACT INTERACTION: RACE/ETHNICITY BY SCHOOL TYPE BACK0057 INTERACT INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY PARENT’S EDUCATION BACK0058 INTERACT INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY SCHOOL TYPE BACK0059 INTERACT INTERACTION: PARENTS’ EDUCATION BY SCHOOL TYPE BACK0060 INTERACT INTERACTION: GENDER BY SCIENCE COURSES TAKING THIS YEAR BACK0061 INTERACT INTERACTION: GENDER BY NUMBER OF SEMESTERS SCIENCE BACK0062 INTERACT INTERACTION: RACE/ETHNICITY BY SCIENCE COURSES TAKING THIS YEAR BACK0063 INTERACT INTERACTION: RACE/ETHNICITY BY NUMBER OF SEMESTERS SCIENCE BACK0064 INTERACT INTERACTION: PARENTS’ EDUCATION BY SCIENCE COURSES TAKING THIS YEAR BACK0065 INTERACT INTERACTION: PARENTS’ EDUCATION BY NUMBER OF SEMESTERS SCIENCE BACK0066 INTERACT INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY SCIENCE COURSES TAKING THIS YEAR BACK0067 INTERACT INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY NUMBER OF SEMESTERS SCIENCE BACK0068 INTERACT INTERACTION: SCHOOL TYPE BY SCIENCE COURSES TAKING THIS YEAR BACK0069 INTERACT INTERACTION: SCHOOL TYPE BY NUMBER OF SEMESTERS SCIENCE BACK0073 SUBSAMP SAMPLE TYPE BACK0074 INTERACT INTERACTION: SAMPLE BY RACE/ETHNICITY BACK0075 B008901 HE002489 DO YOU HAVE YOUR OWN STUDY DESK OR TABLE AT HOME? BACK0076 B009401 HE002541 HOW SAFE DO YOU FEEL AT SCHOOL? BACK0077 B005601 TB005601 DOES MOTHER OR STEPMOTHER LIVE AT HOME WITH YOU? BACK0078 B005701 TB005701 DOES FATHER OR STEPFATHER LIVE AT HOME WITH YOU? SUBJ0002 K811001 HE002997 AGREE/DISAGREE: I LIKE SCIENCE SUBJ0003 K811002 HE002998 AGREE/DISAGREE: I AM GOOD AT SCIENCE SUBJ0004 K811003 HE002999 AGREE/DISAGREE: LEARNING SCI MOSTLY MEMORIZATION SUBJ0005 K811004 HE003000 AGREE/DISAGREE: SCI USEFUL FOR EVERYDAY PROBLEMS SUBJ0006 K811005 HE003001 AGREE/DISAGREE: IF CHOICE, WOULD NOT STUDY SCIENCE SUBJ0007 K811006 HE003002 AGREE/DISAGREE: ALL CAN DO WELL IN SCI IF THEY TRY SUBJ0008 K811007 HE003003 AGREE/DISAGREE: SCIENCE IS BORING SUBJ0009 K811008 HE003004 AGREE/DISAGREE: SCIENCE IS A HARD SUBJECT SUBJ0010 K811101 HE003006 EVER DONE HANDS-ON PROJECT WITH LIVING THINGS? SUBJ0011 K811102 HE003007 EVER DONE HANDS-ON PROJECT WITH ELECTRICITY? SUBJ0012 K811103 HE003008 EVER DONE HANDS-ON PROJECT WITH CHEMICALS? SUBJ0013 K811104 HE003009 EVER DONE HANDS-ON PROJECT WITH ROCKS OR MINERALS? SUBJ0014 K811105 HE003010 DONE HANDS-ON PROJ W/ MAGNIFYING GLASS/MICROSCOPE? SUBJ0015 K811106 HE003011 DONE HANDS-ON PROJ W/ THERMOMETER OR BAROMETER? SUBJ0016 K811107 HE003012 EVER DONE HANDS-ON PROJECT WITH SIMPLE MACHINES? SUBJ0017 K811108 HE003013 HAVE DONE HANDS-ON PROJECT WITH NONE OF THE ABOVE? SUBJ0018 K811201 LC000144 HOW OFTEN DO YOU STUDY SCIENCE IN SCHOOL? SUBJ0019 K811301 LC000146 HOW MUCH TIME PER WEEK DOING SCIENCE HOMEWORK? 
Table C-3 (continued)

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

SUBJ0020 K811401 LC000147 DO SCI PROJECTS IN SCHOOL THAT TAKE 1 OR MORE WKS? SUBJ0021 K811501 WO001032 LAST 2 YRS, BEEN IN SCI FAIR, FESTIVAL, SCI DAY? SUBJ0022 K811601 HE003015 FOR SCI IN SCHOOL, HOW OFTEN DO YOU READ TEXTBOOK? SUBJ0023 K811602 HE003016 FOR SCI IN SCHOOL, HOW OFTEN DO YOU READ MAGS/BKS? SUBJ0024 K811603 HE003017 FOR SCI IN SCHOOL, HOW OFTEN DISCUSS SCIENCE NEWS? SUBJ0025 K811604 WO001033 FOR SCI IN SCHOOL, HOW OFTEN WORK WITH OTHERS? SUBJ0026 K811605 WO001034 FOR SCI IN SCHOOL, HOW OFTEN GIVE ORAL REPORT? SUBJ0027 K811606 WO001035 FOR SCI IN SCHOOL, HOW OFTEN GIVE WRITTEN REPORT? SUBJ0028 K811607 WO001036 FOR SCI IN SCHOOL, HOW OFTEN DO HANDS-ON PROJECT? SUBJ0029 K811608 WO001037 FOR SCI IN SCHOOL, HOW OFTEN DISCUSS RESULTS? SUBJ0030 K811609 WO001038 FOR SCI IN SCHOOL, HOW OFTEN DO YOU USE COMPUTER? SUBJ0031 K811610 WO001039 FOR SCI IN SCHOOL, HOW OFTEN TAKE TEST OR QUIZ? SUBJ0032 K811611 WO001040 FOR SCI IN SCHOOL, HOW OFTEN DO YOU USE LIBRARY? SUBJ0033 K811612 WO001041 FOR SCI IN SCHOOL, HOW OFTEN OBSERVE/MEAS OUTSIDE? SUBJ0034 K811701 WO001043 HOW OFTEN DOES SCIENCE TEACHER TALK TO CLASS? SUBJ0035 K811702 WO001044 HOW OFTEN DOES SCIENCE TEACHER DO DEMONSTRATION? SUBJ0036 K811703 WO001045 HOW OFTEN DOES SCIENCE TEACHER SHOW VIDEO OR TV? SUBJ0037 K811704 WO001046 HOW OFTEN DOES SCIENCE TEACHER USE COMPUTER? SUBJ0038 K811705 WO001047 HOW OFTEN DOES SCI TEACHER USE CD’S/LASER DISCS? SUBJ0039 K811801 WO001048 HOW OFTEN DOES SCI CLASS GO ON A FIELD TRIP? SUBJ0040 K811901 WO001049 HOW OFTEN DOES GUEST SPEAKER COME TO SCI CLASS? SUBJ0041 SM00101 LC000075 ABOUT HOW MANY QUESTIONS RIGHT ON TEST? SUBJ0042 SM00201 LC000076 HOW HARD TEST COMPARED TO THOSE IN SCHOOL? SUBJ0043 SM00301 LC000077 HOW HARD DID YOU TRY ON TEST COMPARED TO OTHERS? SUBJ0044 SM00401 LD000078 HOW IMPORTANT WAS IT YOU DO WELL ON THIS TEST? SUBJ0045 SM00501 LC000079 HOW OFTEN HAD TO WRITE LONG ANSWERS TO QSTS? BACK0079 B009701 HE002543 DESCRIBE YOUR OVERALL GRADES SINCE 6TH GRADE BACK0080 B009801 HE002544 HOW FAR IN SCHOOL DO YOU THINK YOU WILL GO? BACK0081 B009501 HE003221 DOES YOUR STEP/MOTHER WORK AT A JOB FOR PAY? BACK0082 B009502 HE003222 DOES YOUR STEP/FATHER WORK AT A JOB FOR PAY? BACK0083 B009601 HE003223 DOES YOUR STEP/MOTHER WORK AT A JOB FOR PAY? BACK0084 B009602 HE003224 DOES YOUR STEP/FATHER WORK AT A JOB FOR PAY? SUBJ0046 K812101 LC000145 DO YOU/TEACHER SAVE YOUR SCI WORK IN A PORTFOLIO? SUBJ0047 K812201 WO001050 HOW MUCH TIME WEEKLY SPENT ON SCIENCE HOMEWORK? SUBJ0048 K811613 LD001451 FOR SCI IN SCHOOL, HOW OFTEN HANDS-ON ACTIVITIES? SUBJ0049 K811614 LD001477 FOR SCI, HOW OFTEN DISCUSS HANDS-ON RESULTS? SUBJ0050 K811615 WO001051 FOR SCI, DESIGN & CARRY OUT OWN INVESTIGATION? BACK0085 B008301 LC000168 SINCE KDG, GRADES ATTENDED IN THIS STATE? BACK0086 B009901 HE002548 DESCRIBE YOUR OVERALL GRADES SINCE 9TH GRADE BACK0087 B008501 HE002549 WHICH BEST DESCRIBES YOUR HIGH SCHOOL PROGRAM BACK0088 B005501 TB005501 WHAT WILL TAKE LARGEST AMT. OF TIME AFTER HIGH-SCH BACK0089 MOTHOCC KIND OF WORK DONE BY YOUR MOTHER/FEMALE GUARDIAN? BACK0090 B011A01 HE003220 KIND OF WORK DONE BY YOUR MOTHER/FEMALE GUARDIAN? BACK0091 FATHOCC KIND OF WORK DONE BY YOUR FATHER/MALE GUARDIAN? BACK0092 B012A02 BO001994 KIND OF WORK DONE BY YOUR FATHER/MALE GUARDIAN? SUBJ0051 K812301 LD001446 DONE SCHOOL SCI INVEST/PROJECTS W/ LIVING THINGS? SUBJ0052 K812302 LD001448 DONE SCHOOL SCI INVEST/PROJECTS W/ ELECTRICITY? SUBJ0053 K812303 LD001449 DONE SCHOOL SCI INVEST/PROJECTS W/ CHEMICALS? 
Table C-3 (continued)

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

466SUBJ0054 K812304 LD001450 DONE SCHOOL SCI INVEST/PROJECTS W/ ROCKS/MINERALS? SUBJ0055 K812305 LD001452 DONE SCI INVEST/PROJECTS W/ MAG. GLASS/MICROSCOPE? SUBJ0056 K812306 LD001453 DONE SCI INVEST/PROJECTS W/ THERMOMETER/BAROMETER? SUBJ0057 K812307 LD001454 DONE SCI INVEST/PROJECTS W/ SIMPLE MACHINES? SUBJ0058 K812308 LD001455 DONE SCI PROJECTS W/ INSTRUMENTS MEAS. SPEED? SUBJ0059 K812309 LD001456 DONE SCHOOL SCI INVEST/PROJECTS W/ NONE OF ABOVE? SUBJ0060 K812401 LC000172 ARE YOU TAKING A SCIENCE COURSE THIS YEAR? SUBJ0061 K812501 LC000175 SINCE 9TH GRADE, HOW MUCH GENERAL SCIENCE? SUBJ0062 K812502 LC000181 SINCE 9TH GRADE, HOW MUCH EARTH & SPACE SCIENCE? SUBJ0063 K812503 LC000176 SINCE 9TH GRADE, HOW MUCH BIOLOGY? SUBJ0064 K812504 LC000177 SINCE 9TH GRADE, HOW MUCH LIFE SCIENCE (NOT BIO)? SUBJ0065 K812505 LC000178 SINCE 9TH GRADE, HOW MUCH CHEMISTRY? SUBJ0066 K812506 LC000179 SINCE 9TH GRADE, HOW MUCH PHYSICS? SUBJ0067 K812507 LC000180 SINCE 9TH GRADE, HOW MUCH OTHER PHYSICAL SCIENCE? SUBJ0068 K812508 WO001053 SINCE 9TH GRADE, HOW MUCH INTEGRATED SCIENCE? SUBJ0069 K812509 WO001054 SINCE 9TH GRADE, HOW MUCH SCIENCE AND TECHNOLOGY? SUBJ0070 K812510 WO001055 SINCE 9TH GRADE, HOW MANY OTHER SCIENCE COURSES? SUBJ0071 K812601 LC000182 ENROLLED IN SCIENCE ADVANCED PLACEMENT COURSE? SUBJ0072 K812701 LC000186 DONE SCI INVEST/PROJECTS IN SCHOOL 1 WK OR MORE? SUBJ0073 K811616 WO001058 HOW OFTEN ANALYZE OWN SCI DATA, FORM CONCLUSIONS? SCHL0003 C030901 HE000839 BEST DESCRIBES HOW 4TH GR ARE ORGANIZED? SCHL0004 C037101 HE000840 4TH GRADERS ASSIGNED BY ABILITY/ACHIEVEMENT LEVEL? SCHL0005 C031212 HE002000 HOW OFTEN IS 4TH-GRADER INSTRUCTED IN MATH? SCHL0006 C031205 HE002002 HOW OFTEN IS 4TH-GRADER INSTRUCTED IN SCIENCE? SCHL0007 C031213 LD001554 HOW OFTEN IS 4TH-GRADER INSTRUCTED IN READING? SCHL0008 C031214 LD001555 HOW OFTEN IS 4TH-GRADER INSTRUCTED IN ARTS? SCHL0009 C031603 HE000861 HAS MATH BEEN IDENTIFIED AS A PRIORITY? SCHL0010 C031607 LC000469 HAS SCIENCE BEEN IDENTIFIED AS A PRIORITY? SCHL0011 C031601 HE000859 HAS READING BEEN IDENTIFIED AS A PRIORITY? SCHL0012 C031610 LD001556 HAS ARTS BEEN IDENTIFIED AS A PRIORITY? SCHL0013 C031606 HE000958 HAS SUBJECT INTEGRATION BEEN A PRIORITY? SCHL0014 C035701 HE000843 COMPUTERS AVAILABLE ALL THE TIME IN CLASSROOM? SCHL0015 C035702 HE000864 COMPUTERS GROUPED IN SEPARATE LAB AND AVAILABLE? SCHL0016 C035703 HE000866 COMPUTERS AVAILABLE TO BRING TO ROOM WHEN NEEDED? SCHL0017 C037201 HE002006 SCHOOL W/ SPECIAL FOCUS ON MATH? SCHL0018 C037202 HE002008 SCHOOL W/ SPECIAL FOCUS ON SCIENCE? SCHL0019 C037207 LD001557 SCHOOL W/ SPECIAL FOCUS ON MATH? SCHL0020 C037204 LD001558 SCHOOL W/ SPECIAL FOCUS ON ARTS? SCHL0021 C037205 HE002011 SCHOOL W/ SPECIAL FOCUS ON OTHER? SCHL0022 C037206 HE002012 SCHOOL NOT A SPECIAL FOCUS SCHOOL? SCHL0023 C037301 HE002014 SCHOOL FOLLOW DISTRICT/STATE MATH CURRICULUM? SCHL0024 C037302 HE002016 SCHOOL FOLLOW DISTRICT/STATE SCIENCE CURRICULUM? SCHL0025 C037303 LD001559 SCHOOL FOLLOW DISTRICT/STATE READING CURRICULUM? SCHL0026 C037304 LD001560 SCHOOL FOLLOW DISTRICT/STATE ARTS CURRICULUM? SCHL0027 C037305 HE002019 SCHOOL FOLLOW DISTRICT/STATE FOR NONE OF ABOVE? SCHL0028 C037401 HE002021 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR MATH? SCHL0029 C037402 HE002023 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR SCIENCE? SCHL0030 C037403 LD001561 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR READING? 
Table C-3 (continued)

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

SCHL0031 C037404 LD001562 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR ARTS? SCHL0032 C037405 HE002026 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR OTHER? SCHL0033 C037406 HE002027 SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR NONE ABOVE? SCHL0034 C037501 HE002029 4TH GRADERS IN EXTRACURR ACTS FOR MATH? SCHL0035 C037502 HE002031 4TH GRADERS IN EXTRACURR ACTS FOR SCIENCE? SCHL0036 C037503 LD001563 4TH GRADERS IN EXTRACURR ACTS FOR READING? SCHL0037 C037504 LD001564 4TH GRADERS IN EXTRACURR ACTS FOR ARTS? SCHL0038 C037505 HE002034 4TH GRADERS IN EXTRACURR ACTS FOR NONE OF ABOVE? SCHL0039 C037601 HE002036 4TH GRADERS IN SUMMER PROGRAMS IN MATH? SCHL0040 C037602 HE002038 4TH GRADERS IN SUMMER PROGRAMS IN SCIENCE? SCHL0041 C037603 LD001565 4TH GRADERS IN SUMMER PROGRAMS IN READING? SCHL0042 C037604 LD001566 4TH GRADERS IN SUMMER PROGRAMS IN ARTS? SCHL0043 C037605 HE002041 4TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE? SCHL0044 C036601 LC000502 WHICH BEST DESCRIBES PRIMARY WAY LIBRARY STAFFED? SCHL0045 C032207 HE000875 INVOLVE PARENTS AS AIDES IN CLASSROOM? SCHL0046 C032209 LC000482 HAVE PARENTS REVIEW/SIGN HOMEWORK? SCHL0047 C032210 LC000484 ASSIGN HOMEWORK STUDENTS DO WITH PARENTS? SCHL0048 C032211 LC000486 HAVE A PARENT VOLUNTEER PROGRAM? SCHL0049 C037701 HE002142 WHAT % OF PARENTS IN PARENT-TEACHER ORGS? SCHL0050 C037702 HE002108 WHAT % OF PARENTS IN OPEN HOUSE/BACK SCHOOL NIGHT? SCHL0051 C037703 HE002109 WHAT % OF PARENTS IN PARENT-TEACHER COMFERENCES? SCHL0052 C037704 HE002110 WHAT % PARENTS INVOLVED MAKING CURRICULUM DECISION SCHL0053 C037705 HE002111 WHAT % OF PARENTS IN VOLUNTEER PROGRAMS? SCHL0054 C032402 HE000888 IS STUDENT ABSENTEEISM A PROBLEM IN YOUR SCHOOL? SCHL0055 C032401 HE000887 IS STUDENT TARDINESS A PROBLEM IN YOUR SCHOOL? SCHL0056 C032404 HE000890 ARE PHYSICAL CONFLICTS A PROBLEM IN YOUR SCHOOL? SCHL0057 C032406 HE000892 IS TEACHER ABSENTEEISM A PROBLEM IN YOUR SCHOOL? SCHL0058 C032407 HE000893 ARE RACE/CULT. CONFLICTS A PROBLEM IN YOUR SCHOOL? SCHL0059 C032408 HE000894 IS STUDENT HEALTH A PROBLEM IN YOUR SCHOOL? SCHL0060 C032409 HE002121 IS LACK OF PARENT INVLMT A PROBLEM IN YOUR SCHOOL? SCHL0061 C032410 HE002122 IS STUD USE OF ALCOHOL A PROBLEM IN YOUR SCHOOL? SCHL0062 C032411 HE002123 IS STUDENT TOBACCO USE A PROBLEM IN YOUR SCHOOL? SCHL0063 C032412 HE002124 IS STUDENT DRUG USE A PROBLEM IN YOUR SCHOOL? SCHL0064 C032413 HE002125 ARE GANG ACTIVITIES A PROBLEM IN YOUR SCHOOL? SCHL0065 C032414 HE002126 IS STUDENT MISBEHAVIOR A PROBLEM IN YOUR SCHOOL? SCHL0066 C032415 HE002127 IS STUDENT CHEATING A PROBLEM IN YOUR SCHOOL? SCHL0067 C032502 HE000897 IS TEACHER MORALE POS. OR NEG.? SCHL0068 C032503 HE000898 ARE STUDENT ATTITUDES TO ACADEMICS POS. OR NEG.? SCHL0069 C032505 HE000900 IS PARENT SUPPORT FOR ACHIEVEMENT POS. OR NEG.? SCHL0070 C032506 HE000901 IS REGARD FOR SCHOOL PROPERTY POS. OR NEG.? SCHL0071 C033601 HE000917 % ABSENT ON AVERAGE DAY? SCHL0072 C036501 LC000488 WHAT % OF TEACHERS ABSENT ON GIVEN DAY? SCHL0073 C037801 HE000918 % OF STUDS EROLLED AT START OF YR EROLLED AT END? SCHL0074 C037901 HE002112 % OF 4TH GRADERS HELD BACK & REPEATING 4TH GRADE? SCHL0075 C038001 HE000920 % OF FULL TIME TEACHERS LEFT BEFORE END OF YR? SCHL0076 C038301 HE002094 IS SCHOOL IN NATIONAL SCHOOL LUNCH PROGRAM? SCHL0077 C038801 WP000069 SCHOOL RECEIVE CHAP 1/TITLE 1 FUNDING? SCHL0078 C034101 HE002143 DID PRINCIPAL FILL OUT THIS QUESTIONNAIRE 
Table C-3 (continued)

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

468SCHL0079 C034102 HE002113 DID HEADMASTER/HEADMISTRESS FILL OUT QUESTIONNAIRE SCHL0080 C034103 HE002114 DID HEAD TEACHER FILL OUT THIS QUESTIONNAIRE SCHL0081 C034104 HE002115 DID VICE PRINCIPAL FILL OUT THIS QUESTIONNAIRE SCHL0082 C034105 HE002116 DID COUNSELOR FILL OUT THIS QUESTIONNAIRE SCHL0083 C034106 HE002117 DID CURRICULUM COORD FILL OUT THIS QUESTIONNAIRE SCHL0084 C034107 HE002118 DID TEACHER FILL OUT THIS QUESTIONNAIRE SCHL0085 C034108 HE002119 DID SECRETARY FILL OUT THIS QUESTIONNAIRE SCHL0086 C034109 HE002120 DID OTHER PERSON FILL OUT THIS QUESTIONNAIRE TCHR0001 T055901 HE001004 WHAT IS YOUR GENDER? TCHR0002 T056001 LD001610 WHICH BEST DESRIBES YOU? TCHR0003 T040301 HE001007 YEARS TAUGHT TCHR0004 T056101 LD001500 HOW MANY YRS TOTAL YOU TAUGHT MATH? TCHR0005 T056102 LD001501 HOW MANY YRS TOTAL YOU TAUGHT SCIENCE? TCHR0006 T056201 HE002551 TYPE TCHNG CERT IN THIS ST IN MAIN FIELD? TCHR0007 T040501 HE001010 CERTIFICATION, ELEMENTARY OR MIDDLE/JUNIOR HS ED? TCHR0008 T040506 HE002552 DO YOU HAVE CERTIFICATION IN ELEMENTARY MATH? TCHR0009 T040504 HE001082 DO YOU HAVE CERTIFICATION IN JR HIGH/SEC MATH? TCHR0010 T040507 HE002553 CERTIFICATION, ELEMENTARY SCIENCE? TCHR0011 T040508 HE002554 CERTIFICATION, MIDDLE/JUNIOR SCIENCE TCHR0012 T040505 HE002555 CERTIFICATION, OTHER TCHR0013 T056301 HE001012 HIGHEST ACADEMIC DEGREE YOU HOLD? TCHR0014 T040701 HE002556 EDUCATION UNDERGRAD MAJOR TCHR0015 T040706 HE002557 ELMENT ED UNDERGRAD MAJOR TCHR0016 T040707 HE002558 SEC ED UNDERGRAD MAJOR TCHR0017 T040703 HE002559 WAS YOUR UNDERGRADUATE MAJOR MATH? TCHR0018 T040704 HE002560 WAS YOUR UNDERGRADUATE MAJOR MATH ED? TCHR0019 T040710 HE002561 SCIENCE ED UNDERGRAD MAJOR TCHR0020 T040711 HE002562 LIFE SCIENCE UNDERGRAD MAJOR? TCHR0021 T040712 HE002563 PHYSICAL SCIENCE UNDERGRAD MAJOR? TCHR0022 T040713 HE002564 EARTH SCIENCE UNDERGRAD MAJOR? TCHR0023 T040708 HE002565 SPECIAL EDUCATION UNDERGRAD MAJOR TCHR0024 T040709 HE002566 BILINGUAL ED/ESL UNDERGRAD MAJOR TCHR0025 T040705 HE002567 OTHER UNDERGRAD MAJOR TCHR0026 T040801 HE002568 EDUCATION GRAD MAJOR TCHR0027 T040807 HE002569 ELEMENTARY ED GRAD MAJOR TCHR0028 T040808 HE002570 SECONDARY ED GRAD MAJOR TCHR0029 T040803 HE002571 WAS YOUR GRADUATE MAJOR MATHEMATICS? TCHR0030 T040804 HE002572 WAS YOUR GRADUATE MAJOR MATH ED? TCHR0031 T040814 HE002573 SCIENCE ED GRAD MAJOR? TCHR0032 T040815 HE002574 LIFE SCIENCE GRAD MAJOR? TCHR0033 T040816 HE002575 PHYSICAL SCIENCE GRAD MAJOR? TCHR0034 T040817 HE002576 EARTH SCIENCE GRAD MAJOR? TCHR0035 T040809 HE002577 SPECIAL ED GRAD MAJOR TCHR0036 T040810 HE002578 BILINGUAL GRAD MAJOR TCHR0037 T040811 HE002579 ADMIN/SUPERVISION GRAD MAJOR TCHR0038 T040812 HE002580 CURRICULUM/INSTRUCTION GRAD MAJOR? TCHR0039 T040813 LD001506 COUNSELING GRAD MAJOR? TCHR0040 T040805 HE002581 OTHER GRAD MAJOR 
Table C-3 (continued)

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

TCHR0041 T040806 HE002582 NO GRADUATE STUDY TCHR0042 T056401 HE002584 UNDERGRAD/GRAD MINOR STUDY-EDUCATION TCHR0043 T056402 HE002585 UNDERGRAD/GRAD MINOR STUDY-ELEMENTARY ED TCHR0044 T056403 HE002586 UNDERGRAD/GRAD MINOR STUDY-SECONDARY ED TCHR0045 T056404 HE002587 UNDERGRAD/GRAD MINOR STUDY-MATHEMATICS TCHR0046 T056405 HE002588 UNDERGRAD/GRAD MINOR STUDY-MATHEMATICS ED TCHR0047 T056413 HE002589 UNDERGRAD/GRAD MINOR STUDY-SCIENCE ED TCHR0048 T056414 HE002590 UNDERGRAD/GRAD MINOR STUDY-LIFE SCIENCE TCHR0049 T056415 HE002591 UNDERGRAD/GRAD MINOR STUDY-PHYSICAL SCIENCE TCHR0050 T056416 HE002592 UNDERGRAD/GRAD MINOR STUDY-EARTH SCIENCE TCHR0051 T056406 HE002593 UNDERGRAD/GRAD MINOR STUDY-SPECIAL ED TCHR0052 T056407 HE002594 UNDERGRAD/GRAD MINOR STUDY-BILINGUAL ED TCHR0053 T056408 HE002595 UNDERGRAD/GRAD MINOR STUDY-ADMIN & SUPERVISION TCHR0054 T056409 HE002596 UNDERGRAD/GRAD MINOR STUDY-CURRICULUM & INSTRUC TCHR0055 T056410 LD001509 UNDERGRAD/GRAD MINOR STUDY-COUNSELING TCHR0056 T056411 HE002597 UNDERGRAD/GRAD MINOR STUDY-OTHER TCHR0057 T056412 HE002598 UNDERGRAD/GRAD MINOR STUDY-NONE TCHR0058 T056501 HE002599 LAST YR, HOW MUCH TIME IN MATH/MATH ED SEM/WRKSHP? TCHR0059 T058101 he002600 LAST YR, HOW MUCH TIME IN SCI/SCI ED SEM/WRKSHPS? TCHR0060 T056601 HE002601 LAST 2 YRS, HOW MANY MATH/MATH ED UNIV COURSES? TCHR0061 T058201 HE002602 LAST 2 YRS, HOW MANY SCI/SCI ED UNIV COURSES? TCHR0062 T056701 HE002604 PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-TELECOMM USE TCHR0063 T056702 HE002605 PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-TECH USE TCHR0064 T056703 HE002606 PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-COOP INSTRCT TCHR0065 T056704 HE002607 PAST 5 YRS, COURSES/IN PRO DEVLP-INTERDISP INSTRCT TCHR0066 T056705 HE002608 PAST 5 YRS, COURSES/IN PRO DEVLP-PORTFOLIO ASSMNT TCHR0067 T056706 HE002609 PAST 5 YRS, COURSES/IN PRO DEVLP-PERF BASED ASSMNT TCHR0068 T056707 HE002610 PAST 5 YRS, COURSES/PRO DEVLP-TEACH HIGHORDER THKG TCHR0069 T056708 HE002611 PAST 5 YRS, COURSES/PRO DEVLP-TEACH DIFF CULT BKGD TCHR0070 T056709 HE002612 PAST 5 YRS, COURSES/PRO DEVLP-TEACH LEP STUDENTS TCHR0071 T056710 HE002613 PAST 5 YRS, COURSES/PRO DEVLP-TEACH SPEC NEED STDS TCHR0072 T056711 HE002614 PAST 5 YRS, COURSES/PRO DEVLP-CLASSRM MNGMT/ORG TCHR0073 T056712 HE002615 PAST 5 YRS, COURSES/PRO DEVLP-OTHER PROF ISSUES TCHR0074 T056713 HE002616 PAST 5 YRS, COURSES/PRO DEVLP-NONE OF ABOVE TCHR0075 T041201 HE001022 AVAILABILITY OF RESOURCES TCHR0076 T041302 LD001512 ARE CURRICULUM SPECIALISTS AVAILABLE FOR MATH? TCHR0077 T041303 LD001513 SCIENCE CURRICULUM SPECIALIST TCHR0078 T056801 HE001251 HOW MANY SCHOOL HOURS ARE PREP TIME PER WEEK? TCHR0079 T060201 WO001016 HOW MANY YRS TAUGHT SCI IN PUB/PRIV SCHOOLS? TCHR0080 T060301 WO001018 METHODS OF TCHING SCI? COLLEGE COURSE TCHR0081 T060311 WO001018 METHODS OF TCHING SCI?WRKSHP >1 WK TCHR0082 T060321 WO001018 METHODS OF TCHING SCI?WRKSHP <1 WK >1 DAY TCHR0083 T060331 WO001018 METHODS OF TCHING SCI?WRKSHP <= 1 DAY TCHR0084 T060341 WO001018 METHODS OF TCHING SCI?OTHER PROF. DEV TCHR0085 T060302 WO001019 UNIV COURSES IN-BIO/LIFE SCI? COLLEGE COURSE TCHR0086 T060312 WO001019 UNIV COURSES IN-BIO/LIFE SCI?WRKSHP >1 WK TCHR0087 T060322 WO001019 UNIV COURSES IN-BIO/LIFE SCI?WRKSHP <1 WK >1 DAY TCHR0088 T060332 WO001019 UNIV COURSES IN-BIO/LIFE SCI?WRKSHP <= 1 DAY 
Table C-3 (continued)

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

470TCHR0089 T060342 WO001019 UNIV COURSES IN-BIO/LIFE SCI?OTHER PROF. DEV TCHR0090 T060303 WO001020 UNIV COURSES IN-CHEMISTRY? COLLEGE COURSE TCHR0091 T060313 WO001020 UNIV COURSES IN-CHEMISTRY?WRKSHP >1 WK TCHR0092 T060323 WO001020 UNIV COURSES IN-CHEMISTRY?WRKSHP <1 WK >1 DAY TCHR0093 T060333 WO001020 UNIV COURSES IN-CHEMISTRY?WRKSHP <= 1 DAY TCHR0094 T060343 WO001020 UNIV COURSES IN-CHEMISTRY?OTHER PROF. DEV TCHR0095 T060304 WO001021 UNIV COURSES IN-PHYSICS? COLLEGE COURSE TCHR0096 T060314 WO001021 UNIV COURSES IN-PHYSICS?WRKSHP >1 WK TCHR0097 T060324 WO001021 UNIV COURSES IN-PHYSICS?WRKSHP <1 WK >1 DAY TCHR0098 T060334 WO001021 UNIV COURSES IN-PHYSICS?WRKSHP <= 1 DAY TCHR0099 T060344 WO001021 UNIV COURSES IN-PHYSICS?OTHER PROF. DEV TCHR0100 T060305 WO001022 UNIV COURSES IN-EARTH SCI? COLLEGE COURSE TCHR0101 T060315 WO001022 UNIV COURSES IN-EARTH SCI?WRKSHP >1 WK TCHR0102 T060325 WO001022 UNIV COURSES IN-EARTH SCI?WRKSHP <1 WK >1 DAY TCHR0103 T060335 WO001022 UNIV COURSES IN-EARTH SCI?WRKSHP <= 1 DAY TCHR0104 T060345 WO001022 UNIV COURSES IN-EARTH SCI?OTHER PROF. DEV TCHR0105 T060306 WO001023 UNIV COURSES-OTHER TYPES OF SCI? COLLEGE COURSE TCHR0106 T060316 WO001023 UNIV COURSES-OTHER TYPES OF SCI?WRKSHP >1 WK TCHR0107 T060326 WO001023 UNIV COURSES-OTHR TYPES OF SCI?WRKSHP <1 WK >1 DAY TCHR0108 T060336 WO001023 UNIV COURSES-OTHER TYPES OF SCI?WRKSHP <= 1 DAY TCHR0109 T060346 WO001023 UNIV COURSES-OTHER TYPES OF SCI?OTHER PROF. DEV TCHR0110 T060307 WO001024 UNIV COURSES IN-NONE OF ABOVE? COLLEGE COURSE TCHR0111 T060317 WO001024 UNIV COURSES IN-NONE OF ABOVE?WRKSHP >1 WK TCHR0112 T060327 WO001024 UNIV COURSES IN-NONE OF ABOVE?WRKSHP <1 WK >1 DAY TCHR0113 T060337 WO001024 UNIV COURSES IN-NONE OF ABOVE?WRKSHP <= 1 DAY TCHR0114 T060347 WO001024 UNIV COURSES IN-NONE OF ABOVE?OTHER PROF. DEV TCHR0115 T060401 WO001026 PAST 5 YRS, COURSES/ACTVTS IN-COMP USE TO GET DATA TCHR0116 T060402 WO001027 PAST 5 YRS, COURSES/ACTVTS IN-COMP DATA ANALYSIS? TCHR0117 T060403 WO001028 PAST 5 YRS, COURSES/ACTVTS IN-MULTIMEDIA SCI ED? TCHR0118 T060404 WO001029 PAST 5 YRS, COURSES/ACTVTS IN-LAB MNGMT/SAFETY? TCHR0119 T060405 WO001030 PAST 5 YRS, COURSES/ACTVTS IN-INTEGRATED SCI INST? TCHR0120 T060501 WO001031 YOU BELONG TO 1 OR > SCI RELATED SCI ORGS? TSUB0001 T060601 HE002415 HOW OFTEN STUDS READ SCI TEXTBOOK? TSUB0002 T060602 HE002416 HOW OFTEN STUDS READ BOOK/MAN ABOUT SCI? TSUB0003 T060603 HE002417 HOW OFTEN STUDS DISCUSS SCI IN THE NEWS? TSUB0004 T060604 HE002418 HOW OFTEN STUDS WORK W/ OTHER STUDS ON ACT/PROJCT? TSUB0005 T060605 HE002419 HOW OFTEN STUDS GIVE ORAL SCI REPORT? TSUB0006 T060606 HE002420 HOW OFTEN STUDS PREPARE A WRITTEN SCI REPORT? TSUB0007 T060607 HE002421 HOW OFTEN STUDS DO HANDS ON SCI ACTIVITIES IN SCI? TSUB0008 T060608 HE002422 HOW OFTEN STUDS TALK ABOUT MEASURES/RESULTS? TSUB0009 T060609 HE002423 HOW OFTEN STUDS TAKE SCI TEST OR QUIZ? TSUB0010 T060610 HE002424 HOW OFTEN STUDS USE LIBRARY RESOURCES FOR SCI? TSUB0011 T060611 HE002425 HOW OFTEN STUDS USE COMPUTERS FOR SCI? TSUB0012 T060701 HE002427 HOW OFTEN DO YOU TALK TO CLASS ABOUT SCI? TSUB0013 T060702 HE002428 HOW OFTEN DO YOU DO A SCI DEMONSTRATION? TSUB0014 T060703 HE002429 HOW OFTEN DO YOU SHOW A SCI VIDEOTAPE/TV PROGRAM? TSUB0015 T060704 HE002432 HOW OFTEN DO YOU USE COMPUTERS FOR SCI? TSUB0016 T060705 HE002430 HOW OFTEN DO YOU USE CDS OR LASER DISKS ON SCI? 
Table C-3 (continued)

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

TSUB0017 T060801 HE002431 HOW OFTEN YOUR SCI STUDS GO ON SCI FIELD TRIPS? TSUB0018 T060901 HE002433 HOW OFTEN DO YOU BRING GUEST SPEAKER FOR SCI STUDS TSUB0019 T061001 HE002434 SAVE STUDS SCI WORK IN PORTFOLIOS FOR ASSESSMENT? TSUB0020 T061101 HE002436 HOW MUCH EMPHASIS-KNOWING SCI FACTS/TERMS? TSUB0021 T061102 HE002437 HOW MUCH EMPHASIS-UNDERSTANDING KEY SCI CONCEPTS? TSUB0022 T061103 HE002438 HOW MUCH EMPHASIS-DEVELOP SCI PROB SOLVING SKILL? TSUB0023 T061104 HE002439 HOW MUCH EMPHASIS-SCI RELEVANCE TO SOCIETY/TECH? TSUB0024 T061105 HE002440 HOW MUCH EMPHASIS-COMMUNICATE IDEAS IN SCI? TSUB0025 T061106 HE002441 HOW MUCH EMPHASIS-DEVELOPING LAB SKILLS? TSUB0026 T061107 HE002442 HOW MUCH EMPHASIS-DEVELOPING STUDS SCI INTEREST? TSUB0027 T061108 HE002443 HOW MUCH EMPHASIS-DEVELOPING DATA ANALYSIS SKILLS TSUB0028 T061109 HE002444 HOW MUCH EMPHASIS-USING TECH AS SCI TOOL? TSUB0029 T061201 HE002445 EVER ASSIGN SOLO/GROUP SCI PROJECTS THAT TAKE >WK? TSUB0030 T061301 HE002447 HOW OFTEN USE MULT CHOICE TESTS TO ACCESS? TSUB0031 T061302 HE002448 HOW OFTEN USE SHOR/LONG WRITTEN RESPONSE TO ACCESS TSUB0032 T061303 HE002449 HOW OFTEN USE SOLO PROJECTS TO ACCESS? TSUB0033 T061304 HE002450 HOW OFTEN USE GROUP PROJECTS TO ACCESS? TSUB0034 T061305 HE002451 HOW OFTEN USE WORK PORTFOLIOS TO ACCESS? TSUB0035 T061306 HE002452 HOW OFTEN USE IN CLASS ESSAYS TO ACCESS? TSUB0036 T061307 HE002453 HOW OFTEN USE SELF/PEER EVAL TO ACCESS? TSUB0037 T061308 HE002454 HOW OFTEN USE LAB NOTEBOOKS/JOURNALS TO ACCESS? TSUB0038 T061309 HE002455 HOW OFTEN USE HOMEWORK TO ACCESS? TSUB0039 T061310 HE002456 HOW OFTEN USE HANDS ON ACTIVITIES TO ACCESS? TSUB0040 T061401 HE002457 PROPORTION OF EVAL IN SCI BASED ON HANDS ON ACTVS? TSUB0041 T061501 HE002458 BEST DESCRIPTION OF COMPUTER AVAILABILITY FOR SCI TSUB0042 T061601 HE002459 USE COMPUTERS FOR SCI INSTRUCTION: PLAYING SCI TSUB0043 T061611 HE002459 USE COMPUTERS FOR SCI INSTRUCTION: PLAYING SCI TSUB0044 T061621 HE002459 USE COMPUTERS FOR SCI INSTRUCTION: SIMULATIONS TSUB0045 T061631 HE002459 USE COMPUTERS FOR SCI INSTRUCTION: DATA ANALYSIS TSUB0046 T061641 HE002459 USE COMPUTERS FOR SCI INSTRUCTION: WORD PROCESS TSUB0047 T061651 HE002459 USE COMPUTERS FOR SCI INSTRUCTION: DO NOT USE TSUB0048 T061701 HE002460 STUDS ASSIGNED TO CLASS BY ABILITY/ACHVMNT LEVEL? TSUB0049 T061801 HE002461 IF ASSIGNED BY ABILITY WHICH BEST DESCRIBES LEVEL? TSUB0050 T061901 HE002462 COMPOSITION OF CLASS ACCORDING TO GENDER? TSUB0051 T062001 HE002463 HOW MUCH TIME CLASS SPEND ON LIFE SCIENCE? TSUB0052 T062002 HE002463 HOW MUCH TIME CLASS SPEND ON EARTH SCIENCE? TSUB0053 T062003 HE002463 HOW MUCH TIME CLASS SPEND ON PHYSICAL SCIENCE? TSUB0054 T062101 HE002464 WHICH BEST DESCRIBES SPACE WHERE CLASS TAUGHT? TSUB0055 T062201 HE002466 DO STUDS PRODUCE NOTEBOOKS/REPORTS OF LAB WORK? TSUB0056 T062202 HE002467 DO STUDS PRODUCE REPORTS OF EXTENDED SCI PROJECTS? TSUB0057 T062203 HE002468 DO STUDS PRODUCE REPORTS ON SPECIFIC TOPIC/ISSUE? TSUB0058 T062204 HE002469 DO STUDS PRODUCE REPORTS/RECORDS OF FIELD TRIPS? TSUB0059 T062205 HE002470 DO STUDS PRODUCE JOURNALS/DIARIES/LOGS OF IDEAS? TSUB0060 T062206 HE002471 DO STUDS PRODUCE PHOTO RECORDS OF PROJECTS? TSUB0061 T062207 HE002472 DO STUDS PRODUCE AUDIO/VIDEOTAPE RECORDS OF ACTVS? TSUB0062 T062208 HE002473 DO STUDS PRODUCE REPORTS OF PERSONAL INTERVIEWS? TSUB0063 T062209 HE002474 DO STUDS PRODUCE 3D SCI MODELS? TSUB0064 T062210 HE002475 DO STUDS PRODUCE COMP GENERATED MULTMEDIA PROJECTS 
Table C-3 (continued)

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

472TSUB0065 T062301 HE002476 TIME PER WEEK EXPECT STUD TO SPEND ON HOMEWORK? TSUB0066 T062401 LD001465 CLASS PERIOD AND # OF STUDS IN CLASS SCHL0087 C034201 HE000926 BEST DESCRIBES HOW 8TH GRADES ARE ORGANIZED? SCHL0088 C034402 HE002232 ARE 8TH-GRADERS ASSIGNED TO MATH BY ABILITY? SCHL0089 C034403 HE002234 ARE 8TH-GRADERS ASSIGNED TO SCIENCE BY ABILITY? SCHL0090 C034401 LD001571 ARE 8TH-GRADERS ASSIGNED TO ENGLISH BY ABILITY? SCHL0091 C034406 LD001572 ARE 8TH-GRADERS ASSIGNED TO ARTS BY ABILITY? SCHL0092 C034510 HE002146 HOW OFTEN 8TH GRDS RECEIVE COMP SCI INSTRUCTION? SCHL0093 C034511 HE002148 HOW OFTEN 8TH GRDS RECEIVE MATH INSTRUCTION? SCHL0094 C034512 HE002149 HOW OFTEN 8TH GRDS RECEIVE SCIENCE INSTRUCTION? SCHL0095 C034513 LD001573 HOW OFTEN 8TH GRDS RECEIVE ENGLISH INSTRUCTION? SCHL0096 C034514 LD001574 HOW OFTEN 8TH GRDS RECEIVE ARTS INSTRUCTION? SCHL0097 C031611 LD001575 HAS ENGLISH BEEN IDENTIFIED AS A PRIORITY? SCHL0098 C034601 HE000935 SCHOOL OFFER 8TH GR STUDS ALGEBRA FOR HS CREDIT? SCHL0099 C037203 LD001557 SCHOOL W/ SPECIAL FOCUS ON ENGLISH? SCHL0100 C037306 LD001577 SCHOOL FOLLOW DISTRICT/STATE ENGLISH CURRICULUM? SCHL0101 C039401 HE002155 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR MATH? SCHL0102 C039402 HE002157 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR SCIENCE? SCHL0103 C039403 LD001578 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR READING? SCHL0104 C039404 LD001579 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR ARTS? SCHL0105 C039405 HE002160 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR OTHER? SCHL0106 C039406 HE002161 SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR NONE ABOVE? SCHL0107 C039501 HE002164 8TH GRADERS IN EXTRACURR ACTS FOR MATH? SCHL0108 C039502 HE002166 8TH GRADERS IN EXTRACURR ACTS FOR SCIENCE? SCHL0109 C039503 LD001580 8TH GRADERS IN EXTRACURR ACTS FOR ENG/LANG ARTS? SCHL0110 C039504 LD001581 8TH GRADERS IN EXTRACURR ACTS FOR ARTS? SCHL0111 C039505 HE002169 8TH GRADERS IN EXTRACURR ACTS FOR NONE OF ABOVE? SCHL0112 C039601 HE002172 8TH GRADERS IN SUMMER PROGRAMS IN MATH? SCHL0113 C039602 HE002174 8TH GRADERS IN SUMMER PROGRAMS IN SCIENCE? SCHL0114 C039603 LD001582 8TH GRADERS IN SUMMER PROGRAMS IN ENG/LANG ARTS? SCHL0115 C039604 LD001583 8TH GRADERS IN SUMMER PROGRAMS IN ARTS? SCHL0116 C039605 HE002177 8TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE? SCHL0117 C041901 HE002230 WHAT % OF 8TH GRDS HELD BACK/REPEAT 8TH GRADE? TCHR0121 T062501 LC000419 COUNTING THIS YR, HOW MANY YRS TOTAL TAUGHT SCI? TCHR0122 T062601 HE002600 LAST YR, TIME IN PRO WORKSHOPS/SEMS IN SCI? TCHR0123 T062701 HE002602 LAST 2 YRS, # OF UNIV COURSES IN SCI/SCI ED? TCHR0124 T062801 HE002618 CURRICULUM SPECIALIST TO HELP/ADVISE IN SCI? SCHL0118 C035002 HE002247 ARE 12TH-GRADERS ASSIGNED TO MATH BY ABILITY? SCHL0119 C035003 HE002249 ARE 12TH-GRADERS ASSIGNED TO SCIENCE BY ABILITY? SCHL0120 C035006 LD001588 ARE 12TH-GRADERS ASSIGNED TO ENGLISH BY ABILITY? SCHL0121 C035007 LD001589 ARE 12TH-GRADERS ASSIGNED TO ARTS BY ABILITY? SCHL0122 C040201 HE002253 FROM 9TH ON HOW MANY YRS REQUIRED FOR MATH? SCHL0123 C040202 HE002254 FROM 9TH ON HOW MANY YRS REQUIRED FOR SCIENCE? SCHL0124 C040203 LD001590 FROM 9TH ON HOW MANY YRS REQUIRED FOR ENG/LIT? SCHL0125 C040204 HE002255 FROM 9TH ON HOW MANY YRS REQUIRED FINE/PERF ARTS? SCHL0126 C040301 HE002256 COURSES 1 OR > SEMESTERS TAUGHT IN ADVANCED BIO? SCHL0127 C040302 HE002257 COURSES 1 OR > SEMESTERS TAUGHT IN ADVANCED CHEM? SCHL0128 C040303 HE002258 COURSES 1 OR > SEMESTERS TAUGHT IN ADV PHYSICS? 
Table C-3 (continued)

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

SCHL0129 C040304 HE002259 NO ADVANCED SCIENCE COURSES TAUGHT SCHL0130 C040305 WP000094 COURSES 1 OR > SEMESTERS TAUGHT IN COMP SCI? SCHL0131 C040306 LC000512 COURSES 1 OR > SEMESTERS TAUGHT IN CALCULUS? SCHL0132 C040307 WP000095 COURSES 1 OR > SEMESTERS TAUGHT IN TRIGONOMETRY? SCHL0133 C040308 WP000096 COURSES 1 OR > SEMESTERS TAUGHT IN PRECALCULUS? SCHL0134 C040309 LC000511 COURSES 1 OR > SEMESTERS TAUGHT IN PROB/STAT? SCHL0135 C040310 WP000098 COURSES 1 OR > SEMESTERS TAUGHT IN UNI/INTEG MATH? SCHL0136 C040311 HE002260 NO ADVANCED MATH COURSES TAUGHT? SCHL0137 C040401 HE002272 STUDS REQUIRED TO PASS STATE TEST IN MATH? SCHL0138 C040402 HE002273 STUDS REQUIRED TO PASS STATE TEST IN SCIENCE? SCHL0139 C040403 LD001591 STUDS REQUIRED TO PASS STATE TEST IN ENG/LANG ARTS SCHL0140 C040404 HE002274 STUDS REQUIRED TO PASS STATE TEST IN FINE/PERF ART SCHL0141 C040501 HE002277 SCHOOL SPONSER 12TH GR FIELD TRIPS FOR MATH? SCHL0142 C040502 HE002279 SCHOOL SPONSER 12TH GR FIELD TRIPS FOR SCIENCE? SCHL0143 C040503 LD001592 SCHOOL SPONSER 12TH GR FIELD TRIPS IN ENG/LANG ART SCHL0144 C040504 LD001593 SCHOOL SPONSER 12TH GR FIELD TRIPS IN ARTS? SCHL0145 C040505 HE002286 SCHOOL SPONSER 12TH GR FIELD TRIPS IN OTHER? SCHL0146 C040506 HE002288 SCHOOL SPONSER 12TH GR FIELD TRIPS IN NONE ABOVE? SCHL0147 C040601 HE002296 12TH GRADERS IN EXTRACURR ACTS IN MATH? SCHL0148 C040602 HE002288 12TH GRADERS IN EXTRACURR ACTS IN SCIENCE? SCHL0149 C040603 LD001594 12TH GRADERS IN EXTRACURR ACTS IN ENG/LANG ARTS? SCHL0150 C040604 LD001595 12TH GRADERS IN EXTRACURR ACTS IN ARTS? SCHL0151 C040605 HE002291 12TH GRADERS IN EXTRACURR ACTS IN NONE OF ABOVE? SCHL0152 C040701 HE002294 12TH GRADERS IN SUMMER PROGRAMS IN MATH? SCHL0153 C040702 HE002296 12TH GRADERS IN SUMMER PROGRAMS IN SCIENCE? SCHL0154 C040703 LD001596 12TH GRADERS IN SUMMER PORGRAMS IN ENG/LANG ARTS? SCHL0155 C040704 LD001597 12TH GRADERS IN SUMMER PROGRAMS IN ARTS? SCHL0156 C040705 HE002299 12TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE? SCHL0157 C040801 HE002346 # STUDS ENROLLED IN AP SCIENCE COURSES? SCHL0158 C040802 HE002347 # STUDS ENROLLED IN AP CALCULUS COURSES? SCHL0159 C040803 HE002350 # STUDS ENROLLED IN AP COMP SCI COURSES? SCHL0160 C040804 LD001598 # STUDS ENROLLED IN AP ENGLISH COURSES? SCHL0161 C040901 HE002353 ANY 12TH GRDS TAKING COLLEGE COURSES IN MATH? SCHL0162 C040902 HE002355 ANY 12TH GRDS TAKING COLLEGE COURSES IN SCIENCE? SCHL0163 C040903 LD001599 ANY 12TH GRDS TAKING COLLEGE COURSES ENG/LANG ARTS SCHL0164 C040904 LD001600 ANY 12TH GRDS TAKING COLLEGE COURSES IN ARTS? SCHL0165 C040905 HE002358 ANY 12TH GRDS TAKING COLLEGE COURSES IN NONE ABOVE SCHL0166 C041001 HE002359 WHAT % 12TH GRDS HELD BACK AND REPEAT 12TH GRADE? SCHL0167 C041101 HE003193 LAST YR WHAT % OF 12TH GRDS GRADUATED? SCHL0168 C036001 HE001002 WHAT % OF GRADUATING CLASS NOW IN 2-YR COLLEGE? SCHL0169 C036002 HE001003 WHAT % OF GRADUATING CLASS NOW IN 4-YR COLLEGE? SCHL0170 C036003 HE002360 WHAT % OF GRADUATING CLASS NOW IN VO-TEC SCHOOL? SCHL0171 C036004 HE002361 WHAT % OF GRAD CLASS NOW IN EMPLOYER TRAINING? SCHL0172 C036005 HE002362 % OF GRADUATING CLASS NOW IN MILITARY SERVICE? SCHL0173 NTLUNSC PERCENT OF STUDENTS WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0174 NTLUNSC PERCENT OF STUDENTS WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0175 REMRDSC PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL READING SCHL0176 REMRDSC PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL READING 
Table C-3 (continued)

Summary Table of the 1996 Science Conditioning Variable Specifications

Conditioning NAEP TDDC 

ID ID ID Description 

474SCHL0177 REMMHSC PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL MATH SCHL0178 REMMHSC PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL MATH SCHL0179 NTLUNGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0180 NTLUNGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0181 REMRDGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL READING SCHL0182 REMRDGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL READING SCHL0183 REMMHGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL MATH SCHL0184 REMMHGR PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL MATH SCHL0185 NTLUNGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0186 NTLUNGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM SCHL0187 REMRDGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL READING SCHL0188 REMRDGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL READING SCHL0189 REMMHGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL MATH SCHL0190 REMMHGR PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL MATH SCHL0191 NTLUNGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRASCHL0192 NTLUNGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRASCHL0193 REMRDGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL READING SCHL0194 REMRDGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL READING SCHL0195 REMMHGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL MATH SCHL0196 REMMHGR PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL MATH 
Table C-4

Conditioning Variables Specific to the 1996 Mathematics Assessment

475

 CONDITIONING VARIABLE ID: BACK0001

 DESCRIPTION: GRAND MEAN

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: OVERALL

 NAEP ID: BKSER TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: OTHER NUMBER OF INDEPENDENT CONTRASTS: 1

 001 OVERALL (@ ) 1 GRAND MEAN

 CONDITIONING VARIABLE ID: BACK0002

 DESCRIPTION: DERIVED SEX

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: GENDER

 NAEP ID: DSEX TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 MALE (1 ) 0 MALE

 002 FEMALE (2 ) 1 FEMALE

 CONDITIONING VARIABLE ID: BACK0003

 DESCRIPTION: DERIVED RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/ETH

 NAEP ID: DRACE TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 WHI/AI/O (1,5,6 ) 000 RACE/ETHNICITY: WHITE, AMERICAN INDIAN/ALASKAN 

 NATIVE, OTHER, MISSING, UNCLASSIFIED

 002 BLACK (2 ) 100 RACE/ETHNICITY: BLACK

 003 HISPANIC (3 ) 010 RACE/ETHNICITY: HISPANIC

 004 ASIAN (4 ) 001 RACE/ETHNICITY: ASIAN

 CONDITIONING VARIABLE ID: BACK0004

 DESCRIPTION: IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: HISPANIC

 NAEP ID: B003101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 NOT HISP (1 ) 0000 HISPANIC: NOT HISPANIC

 002 MEXICAN (2 ) 1000 HISPANIC: MEXICAN, MEXICAN AMERICAN, CHICANO

 003 PUER RIC (3 ) 0100 HISPANIC: PUERTO RICAN

 004 CUBN,OTH (4,5 ) 0010 HISPANIC: CUBAN, OTHER

 005 HISP-? (M ) 0001 HISPANIC: MISSING

 CONDITIONING VARIABLE ID: BACK0005

 DESCRIPTION: MSA/NON-MSA

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: MSANAT

 NAEP ID: TOL8 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 MSA (1,2,3,4,7,9) 0 MSA

 002 NON MSA (5,6,8 ) 1 NON-MSA

 CONDITIONING VARIABLE ID: BACK0006

 DESCRIPTION: TYPE OF LOCALE (5 CATEGORIES)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL5

 NAEP ID: TOL5 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 BIG CTY5 (1 ) 0000 TOL5: LARGE CITY

 002 MID CTY5 (2,M ) 1000 TOL5: MID-SIZE CITY

 003 FR/BTWN5 (3 ) 0100 TOL5: URBAN FRINGE OF LARGE CITY, URBAN FRINGE OF

 MID-SIZE CITY

 004 SML TWN5 (4 ) 0010 TOL5: SMALL TOWN

 005 RURAL5 (5 ) 0001 TOL5: RURAL (MSA AND NON-MSA)

 CONDITIONING VARIABLE ID: BACK0007

 DESCRIPTION: DESCRIPTION OF COMMUNITY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: DOC

 NAEP ID: DOC TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 BIG CITY (1 ) 000 DOC: BIG CITY

 002 URBAN FR (2 ) 100 DOC: URBAN FRINGE

 003 MED CITY (3,9,M ) 010 DOC: MEDIUM CITY

 004 SM PLACE (4 ) 001 DOC: SMALL PLACE

 CONDITIONING VARIABLE ID: BACK0008

 DESCRIPTION: PARENTS’ HIGHEST LEVEL OF EDUCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: PARED

 NAEP ID: PARED TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

476

 001 < HS (1 ) 0000 PARED: LESS THAN HIGH SCHOOL

 002 HS GRAD (2 ) 1000 PARED: HIGH SCHOOL GRADUATE

 003 POST HS (3 ) 0100 PARED: POST HIGH SCHOOL

 004 COL GRAD (4 ) 0010 PARED: COLLEGE GRADUATE

 005 PARED-? (5,M ) 0001 PARED: MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0009

 DESCRIPTION: REGION OF THE COUNTRY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: REGION

 NAEP ID: REGION TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 N EAST (1 ) 000 REGION: NORTHEAST

 002 S EAST (2 ) 100 REGION: SOUTHEAST

 003 CENTRAL (3 ) 010 REGION: CENTRAL

 004 WEST (4,5 ) 001 REGION: WEST, TERRITORIES (NONE)

 CONDITIONING VARIABLE ID: BACK0010

 DESCRIPTION: SCHOOL TYPE (PQ)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: SCHTYPE

 NAEP ID: SCHTYPE TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 PUBLIC (1 ) 00 SCHOOL TYPE: PUBLIC

 002 PRIVATE (2,4,5,M ) 10 SCHOOL TYPE: PRIVATE, BUREAU OF INDIAN AFFAIRS, 

 DEPARTMENT OF DEFENSE, MISSING

 003 CATHOLIC (3 ) 01 SCHOOL TYPE: CATHOLIC

 CONDITIONING VARIABLE ID: BACK0011

 DESCRIPTION: INDIVIDUALIZED EDUCATION PROGRAM

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: IEP

 NAEP ID: IEP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 IEP-YES (1 ) 0 IEP: YES

 002 IEP-NO (2 ) 1 IEP: NO

 CONDITIONING VARIABLE ID: BACK0012

 DESCRIPTION: LIMITED ENGLISH PROFICIENCY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: LEP

 NAEP ID: LEP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 LEP-YES (1 ) 0 LEP: YES

 002 LEP-NO (2 ) 1 LEP: NO

 CONDITIONING VARIABLE ID: BACK0013

 DESCRIPTION: CHAPTER 1 (BOOK COVER)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: CHAPTER1

 NAEP ID: CHAP1 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 CHAP1-Y (1 ) 0 CHAPTER 1: YES

 002 CHAP1-N (2 ) 1 CHAPTER 1: NO

 CONDITIONING VARIABLE ID: BACK0014

 DESCRIPTION: DO YOU RECEIVE A FREE OR REDUCED-PRICE LUNCH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: LUNCH

 NAEP ID: SLUNCH TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 NOT ELIG (1 ) 0000 LUNCH PROGRAM: NOT ELIGIBLE

 002 RED PRIC (2 ) 1000 LUNCH PROGRAM: REDUCED PRICE

 003 FREE (3 ) 0100 LUNCH PROGRAM: FREE

 004 INFO N/A (4,M ) 0010 LUNCH PROGRAM: INFO NOT AVAILABLE

 005 SCH/REF (5 ) 0001 LUNCH PROGRAM: SCHOOL REFUAL

 CONDITIONING VARIABLE ID: BACK0015

 DESCRIPTION: HOW OFTEN DO THE PEOPLE IN YOUR HOME SPEAK A LANGUAGE OTHER THAN ENGLISH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMELANG

 NAEP ID: B003201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 HL-NEVER (1 ) 000 HOMELANG: NEVER

 002 HL-SOME (2 ) 100 HOMELANG: SOMETIMES

 003 HL-ALWAY (3 ) 010 HOMELANG: ALWAYS

 004 HL-? (M ) 001 HOMELANG: MISSING

 CONDITIONING VARIABLE ID: BACK0016

 DESCRIPTION: DO YOU HAVE YOUR OWN STUDY DESK OR TABLE AT HOME?
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

477

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B008901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 B008901Y (01 ) 000 YES

 002 B008901N (02 ) 100 NO

 003 B008901I (03 ) 010 IDK25 (FILL 1 SHELF)

 004 B008901M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: BACK0017

 DESCRIPTION: HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (LINEAR)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: TVWATCHL

 NAEP ID: B001801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TVLIN-0 (1 ) 0 TV WATCHING (LINEAR) (0 TO 6+ HOURS PER DAY)

 002 TVLIN-1 (2 ) 1 TV WATCHING (LINEAR)

 003 TVLIN-2 (3 ) 2 TV WATCHING (LINEAR)

 004 TVLIN-3 (4,M ) 3 TV WATCHING (LINEAR)

 005 TVLIN-4 (5 ) 4 TV WATCHING (LINEAR)

 006 TVLIN-5 (6 ) 5 TV WATCHING (LINEAR)

 007 TVLIN-6 (7 ) 6 TV WATCHING (LINEAR)

 CONDITIONING VARIABLE ID: BACK0018

 DESCRIPTION: HOW MUCH TELEVISION DO YOU USUALLY WATCH EACH DAY? (QUADRATIC)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: TVWATCHQ

 NAEP ID: B001801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: QUADRATIC NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TV-QUAD (1-7,M=4 ) 1.0 + -2.0*X + 1.0*X**2 TV WATCHING (QUADRATIC)

 CONDITIONING VARIABLE ID: BACK0019

 DESCRIPTION: HOMEWORK ASSIGNED?: BASED ON TIME SPENT ON HOMEWORK EACH DAY.

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: HWASSIGN

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 HW-MISS (M ) 00 HOMEWORK ASSIGNED?: MISSING

 002 HW-NO (1 ) 10 HOMEWORK ASSIGNED?: NO

 003 HW-YES (2-5 ) 01 HOMEWORK ASSIGNED?: YES

 CONDITIONING VARIABLE ID: BACK0020

 DESCRIPTION: HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY? (LINEAR)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEWRKL

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 HWLIN-0 (1,2,M ) 0 HOMEWORK (LINEAR): DON’T HAVE ANY, DON’T DO ANY, 

 MISSING

 002 HWLIN-1 (3 ) 1 HOMEWORK (LINEAR): 1/2 HOUR OR LESS

 003 HWLIN-2 (4 ) 2 HOMEWORK (LINEAR): 1 HOUR

 004 HWLIN-3 (5 ) 3 HOMEWORK (LINEAR): MORE THAN 1 HOUR

 CONDITIONING VARIABLE ID: BACK0021

 DESCRIPTION: HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY (QUADRATIC)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEWRKQ

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: SCALE NUMBER OF INDEPENDENT CONTRASTS: 1

 001 HWQUAD-0 (1,2,M ) 0 HOMEWORK (QUADRATIC): DON’T HAVE ANY, DON’T DO 

 ANY, MISSING

 002 HWQUAD-1 (3 ) 1 HOMEWORK (QUADRATIC): 1/2 HOUR OR LESS

 003 HWQUAD-2 (4 ) 4 HOMEWORK (QUADRATIC): 1 HOUR

 004 HWQUAD-3 (5 ) 9 HOMEWORK (QUADRATIC): MORE THAN 1 HOUR

 CONDITIONING VARIABLE ID: BACK0022

 DESCRIPTION: NUMBER OF ITEMS IN THE HOME (NEWSPAPER, > 25 BOOKS, ENCYCLOPEDIA, MAGAZINES) 

 (DERIVED) 

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEITMS

 NAEP ID: HOMEEN2 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 HITEM<=2 (1,M ) 00 ITEMS IN HOME: ZERO TO TWO ITEMS, MISSING

 002 HITEM=3 (2 ) 10 ITEMS IN HOME: THREE ITEMS

 003 HITEM=4 (3 ) 01 ITEMS IN HOME: FOUR ITEMS

 CONDITIONING VARIABLE ID: BACK0023

 DESCRIPTION: DOES MOTHER OR STEPMOTHER LIVE AT HOME WITH YOU?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: MOM@HOME
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

478

 NAEP ID: B005601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 MOMHOM-Y (1 ) 00 MOTHER AT HOME: YES

 002 MOMHOM-N (2 ) 10 MOTHER AT HOME: NO

 003 MOMHOM-? (M ) 01 MOTHER AT HOME: MISSING

 CONDITIONING VARIABLE ID: BACK0024

 DESCRIPTION: DOES FATHER OR STEPFATHER LIVE AT HOME WITH YOU?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: DAD@HOME

 NAEP ID: B005701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 DADHOM-Y (1 ) 00 FATHER AT HOME: YES

 002 DADHOM-N (2 ) 10 FATHER AT HOME: NO

 003 DADHOM-? (M ) 01 FATHER AT HOME: MISSING

 CONDITIONING VARIABLE ID: BACK0025

 DESCRIPTION: HOW MANY DAYS OF SCHOOL MISSED LAST MONTH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: SCH MISS

 NAEP ID: S004001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 MISS->2 (3,4,5,M ) 0 DAYS OF SCHOOL MISSED: 3-4, 5-10, 10 OR MORE 

 DAYS, MISSING

 002 MISS-2< (1,2 ) 1 DAYS OF SCHOOL MISSED: 0-1, 2 DAYS

 CONDITIONING VARIABLE ID: BACK0026

 DESCRIPTION: HOW LONG LIVED IN THE UNITED STATES?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: YRSINUSA

 NAEP ID: B008001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 ALL MY L (1 ) 0000 ALL MY LIFE

 002 USA >5 (2 ) 1000 LIVED IN US MORE THAN 5 YEARS

 003 USA 3-5 (3 ) 0100 LIVED IN US 3-5 YEARS

 004 USA <3 (4 ) 0010 LIVED IN US LESS THAN 3 YEARS

 005 USA-? (M ) 0001 LIVED IN US MISSING

 CONDITIONING VARIABLE ID: BACK0027

 DESCRIPTION: HOW MANY GRADES IN THIS STATE? (4TH GRADE)

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL: STGRADE4

 NAEP ID: B007601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 STGRD<1 (1,M ) 00 GRADES IN STATE: LESS THAN 1 GRADE, MISSING

 002 STGRD1-2 (2 ) 10 GRADES IN STATE: 1-2 GRADES

 003 STGRD3> (3 ) 01 GRADES IN STATE: 3 OR MORE GRADES

 CONDITIONING VARIABLE ID: SCHL0001

 DESCRIPTION: SCHOOL LEVEL AVERAGE MATH NORMIT (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: SCH NORM

 NAEP ID: SCHNORM TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SCHNRM-? (M ) 0 SCHOOL LEVEL AVERAGE MATH NORMIT MISSING

 002 SCHNRM-Y (@ ) 1 SCHOOL LEVEL AVERAGE MATH NORMIT NOT-MISSING

 CONDITIONING VARIABLE ID: SCHL0002

 DESCRIPTION: SCHOOL LEVEL AVERAGE MATH NORMIT

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: SNRM-LIN

 NAEP ID: SCHNORM TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: SCALE NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SNRM-LIN (# ) (F8.4) SCHOOL LEVEL AVERAGE MATH NORMIT MEAN

 002 SNRM-LIN (M ) 0 SCHOOL LEVEL AVERAGE MATH NORMIT MISSING

 CONDITIONING VARIABLE ID: BACK0028

 DESCRIPTION: HOW MANY GRADES IN THIS STATE (12TH GRADE)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: STGRAD12

 NAEP ID: B008301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 STGRD<1 (1,M ) 000 GRADES IN STATE: LESS THAN 1 GRADE, MISSING

 002 STGRD1-2 (2 ) 100 GRADES IN STATE: 1-2 GRADES

 003 STGRD3-5 (3 ) 010 GRADES IN STATE: 3-5 GRADES

 004 STGRD6> (4,5 ) 001 GRADES IN STATE: MORE THAN 5 GRADES

 CONDITIONING VARIABLE ID: BACK0029

 DESCRIPTION: HOW MANY TIMES HAVE YOU CHANGED SCHOOLS IN PAST TWO YEARS BECAUSE YOU MOVED?
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

479

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: SCH CHGS

 NAEP ID: B007301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 SCHCHG-0 (1 ) 000 SCHOOL CHANGES: NONE

 002 SCHCHG-1 (2 ) 100 SCHOOL CHANGES: ONE

 003 SCHCHG-2 (3 ) 010 SCHOOL CHANGES: TWO

 004 SCHCHG-3 (4,M ) 001 SCHOOL CHANGES: THREE OR MORE, MISSING

 CONDITIONING VARIABLE ID: BACK0030

 DESCRIPTION: HOW OFTEN DO YOU DISCUSS THINGS STUDIED IN SCHOOL WITH SOMEONE AT HOME?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: DISC@HOM

 NAEP ID: B007401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 DIS@HOM1 (1 ) 000 DISCUSS STUDIES AT HOME: ALMOST EVERY DAY

 002 DIS@HOM2 (2 ) 100 DISCUSS STUDIES AT HOME: ONCE OR TWICE A WEEK

 003 DIS@HOM3 (3 ) 010 DISCUSS STUDIES AT HOME: ONCE OR TWICE A MONTH

 004 DIS@HOM4 (4,M ) 001 DISCUSS STUDIES AT HOME: NEVER OR HARDLY EVER, 

 MISSING

 CONDITIONING VARIABLE ID: BACK0031

 DESCRIPTION: ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: PGSREAD1

 NAEP ID: B001101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 PGS<6,? (5,M ) 0 PAGES READ: 5 OR FEWER A DAY, MISSING

 002 PGS>5 (1,2,3,4 ) 1 PAGES READ: 6-10, 11-15, 16-20, 20 OR MORE

 CONDITIONING VARIABLE ID: BACK0032

 DESCRIPTION: ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: PGSREAD2

 NAEP ID: B001101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 PGS<11,? (4,5,M ) 0 PAGES READ: 6-10, 5 OR FEWER A DAY, MISSING

 002 PGS>10 (1,2,3 ) 1 PAGES READ: 11-15, 16-20, 20 OR MORE

 CONDITIONING VARIABLE ID: SUBJ0001

 DESCRIPTION: WHAT KIND OF MATH CLASS ARE YOU TAKING THIS YEAR?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: MATTAKE TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 NO MATH (01 ) 0000000 NOT TAKING THIS YEAR

 002 8TH GRD (02 ) 1000000 EIGHTH GRADE MATH

 003 PREALG (03 ) 0100000 PREALGEBRA

 004 ALGEBRA (04 ) 0010000 ALGEBRA

 005 INT/SEQ (05 ) 0001000 INTEGRATED/SEQUENTAL

 006 APPLIED (06 ) 0000100 APPLIED MATH

 007 OTHER (07 ) 0000010 OTHER MATH CLASSUR

 008 MISSING (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: BACK0033

 DESCRIPTION: WHICH BEST DESCRIBES YOUR HIGH-SCHOOL PROGRAM?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: HS PROG

 NAEP ID: B008501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 GENERAL (1 ) 0000 HIGH SCHOOL PROGRAM: GENERAL

 002 ACADEMIC (2 ) 1000 HIGH SCHOOL PROGRAM: ACADEMIC/COLLEGE PREP

 003 VOC/TECH (3 ) 0100 HIGH SCHOOL PROGRAM: VOCATIONAL OR TECHNICAL

 004 OTHERPGM (4 ) 0010 HIGH SCHOOL PROGRAM: OTHER

 005 HS PGM-? (M ) 0001 HIGH SCHOOL PROGRAM: MISSING

 CONDITIONING VARIABLE ID: BACK0034

 DESCRIPTION: SEMESTERS ENGLISH/LITERATURE/WRITING (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM ENG

 NAEP ID: B007101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMENG-? (M ) 0 ENGLISH SEMESTERS: MISSING

 002 SEMENG-^ (1-9 ) 1 ENGLISH SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0035

 DESCRIPTION: NUMBER OF SEMESTERS ENGLISH/LITERATURE/WRITING (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #ENG-LIN

 NAEP ID: B007101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

480

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #ENG-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS ENGLISH/LITERATURE/WRITING 

 (LINEAR) 

 CONDITIONING VARIABLE ID: BACK0036

 DESCRIPTION: SEMESTERS MATHEMATICS (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM MAT

 NAEP ID: B007102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 SEMMAT01 (1,2,3 ) 0000 MATH SEMEST ERS: ZERO TO 2

 002 SEMMAT02 (4,5 ) 1000 MATH SEMESTERS: 3 OR 4

 003 SEMMAT03 (6,7 ) 0100 MATH SEMESTERS: 5 OR 6

 004 SEMMAT04 (8,9 ) 0010 MATH SEMESTERS: 7 OR MORE

 005 SEMMAT-? (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0037

 DESCRIPTION: NUMBER OF SEMESTERS MATHEMATICS (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #MAT-LIN

 NAEP ID: B007102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #MAT-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS MATHEMATICS (LINEAR)

 CONDITIONING VARIABLE ID: BACK0038

 DESCRIPTION: SEMESTERS SCIENCE (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM SCI

 NAEP ID: B007103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMSCI-? (M ) 0 SCIENCE SEMESTERS: MISSING

 002 SEMSCI-^ (1-9 ) 1 SCIENCE SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0039

 DESCRIPTION: NUMBER OF SEMESTERS SCIENCE (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #SCI-LIN

 NAEP ID: B007103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #SCI-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS SCIENCE (LINEAR)

 CONDITIONING VARIABLE ID: BACK0040

 DESCRIPTION: SEMESTERS HISTORY/SOCIAL STUDIES/GEOGRAPHY (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM HIS

 NAEP ID: B007104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMHIS-? (M ) 0 HISTORY SEMESTERS: MISSING

 002 SEMHIS-^ (1-9 ) 1 HISTORY SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0041

 DESCRIPTION: NUMBER OF SEMESTERS HISTORY/SOCIAL STUDIES/GEOGRAPHY (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #HIS-LIN

 NAEP ID: B007104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #HIS-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS HISTORY/SOCIAL 

 STUDIES/GEOGRAPHY (LINEAR) 

 CONDITIONING VARIABLE ID: BACK0042

 DESCRIPTION: SEMESTERS FOREIGN LANGUAGES (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM FLG

 NAEP ID: B007105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMFLG-? (M ) 0 FOREIGN LANGUAGE SEMESTERS: MISSING

 002 SEMFLG-^ (1-9 ) 1 FOREIGN LANGUAGE SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0043

 DESCRIPTION: NUMBER OF SEMESTERS FOREIGN LANGUAGES (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #FLG-LIN

 NAEP ID: B007105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #FLG-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS FOREIGN LANGUAGES (LINEAR)

 CONDITIONING VARIABLE ID: BACK0044

 DESCRIPTION: SEMESTERS VOCATIONAL/TECHNICAL/BUSINESS EDUCATION (MISSING VS NON-MISSING)
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

481

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM VOC

 NAEP ID: B007106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMVOC-? (M ) 0 VOC/TECH SEMESTERS: MISSING

 002 SEMVOC-^ (1-9 ) 1 VOC/TECH SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0045

 DESCRIPTION: NUMBER OF SEMESTERS VOCATIONAL/TECHNICAL/BUSINESS EDUCATION (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #VOC-LIN

 NAEP ID: B007106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #VOC-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS VOCATIONAL/TECHNICAL/BUSINESS 

 EDUCATION (LINEAR)

 CONDITIONING VARIABLE ID: BACK0046

 DESCRIPTION: SEMESTERS ART/MUSIC (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM ART

 NAEP ID: B007107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMART-? (M ) 0 ART/MUSIC SEMESTERS: MISSING

 002 SEMART-^ (1-9 ) 1 ART/MUSIC SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0047

 DESCRIPTION: NUMBER OF SEMESTERS ART/MUSIC (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #ART-LIN

 NAEP ID: B007107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #ART-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS ART/MUSIC (LINEAR)

 CONDITIONING VARIABLE ID: BACK0048

 DESCRIPTION: INTERACTION: GENDER BY RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/RAC

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 3

 001 G/R 11 (11 ) 010101 GEND/RAC INTACT: 1. MALE 1. WHI/AI/O

 002 G/R 12 (12 ) -10000 GEND/RAC INTACT: 1. MALE 2. BLACK

 003 G/R 13 (13 ) 00-100 GEND/RAC INTACT: 1. MALE 3. HISPANIC

 004 G/R 14 (14 ) 0000-1 GEND/RAC INTACT: 1. MALE 4. ASIAN

 005 G/R 21 (21 ) -1-1-1 GEND/RAC INTACT: 2. FEMALE 1. WHI/AI/O

 006 G/R 22 (22 ) 010000 GEND/RAC INTACT: 2. FEMALE 2. BLACK

 007 G/R 23 (23 ) 000100 GEND/RAC INTACT: 2. FEMALE 3. HISPANIC

 008 G/R 24 (24 ) 000001 GEND/RAC INTACT: 2. FEMALE 4. ASIAN

 CONDITIONING VARIABLE ID: BACK0049

 DESCRIPTION: INTERACTION: GENDER BY TYPE OF LOCALE (5 CATEGORIES)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 G/T 11 (11 ) 01010101 GEND/TOL INTACT: 1. MALE 1. BIG CTY5

 002 G/T 12 (12 ) -1000000 GEND/TOL INTACT: 1. MALE 2. MID CTY5

 003 G/T 13 (13 ) 00-10000 GEND/TOL INTACT: 1. MALE 3. FR/BTWN5

 004 G/T 14 (14 ) 0000-100 GEND/TOL INTACT: 1. MALE 4. SML TWN5

 005 G/T 15 (15 ) 000000-1 GEND/TOL INTACT: 1. MALE 5. RURAL5

 006 G/T 21 (21 ) -1-1-1-1 GEND/TOL INTACT: 2. FEMALE 1. BIG CTY5

 007 G/T 22 (22 ) 01000000 GEND/TOL INTACT: 2. FEMALE 2. MID CTY5

 008 G/T 23 (23 ) 00010000 GEND/TOL INTACT: 2. FEMALE 3. FR/BTWN5

 009 G/T 24 (24 ) 00000100 GEND/TOL INTACT: 2. FEMALE 4. SML TWN5

 010 G/T 25 (25 ) 00000001 GEND/TOL INTACT: 2. FEMALE 5. RURAL5

 CONDITIONING VARIABLE ID: BACK0050

 DESCRIPTION: INTERACTION: GENDER BY PARENTS’ EDUCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 G/P 11 (11 ) 01010101 GEND/PAR INTACT: 1. MALE 1. < HS

 002 G/P 12 (12 ) -1000000 GEND/PAR INTACT: 1. MALE 2. HS GRAD

 003 G/P 13 (13 ) 00-10000 GEND/PAR INTACT: 1. MALE 3. POST HS

 004 G/P 14 (14 ) 0000-100 GEND/PAR INTACT: 1. MALE 4. COL GRAD

 005 G/P 15 (15 ) 000000-1 GEND/PAR INTACT: 1. MALE 5. PARED-?

 006 G/P 21 (21 ) -1-1-1-1 GEND/PAR INTACT: 2. FEMALE 1. < HS

 007 G/P 22 (22 ) 01000000 GEND/PAR INTACT: 2. FEMALE 2. HS GRAD

 008 G/P 23 (23 ) 00010000 GEND/PAR INTACT: 2. FEMALE 3. POST HS

 009 G/P 24 (24 ) 00000100 GEND/PAR INTACT: 2. FEMALE 4. COL GRAD
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

482

 010 G/P 25 (25 ) 00000001 GEND/PAR INTACT: 2. FEMALE 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0051

 DESCRIPTION: INTERACTION: GENDER BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 2

 001 G/S 11 (11 ) 0101 GEND/SCH INTACT: 1. MALE 1. PUBLIC

 002 G/S 12 (12 ) -100 GEND/SCH INTACT: 1. MALE 2. PRIVATE

 003 G/S 13 (13 ) 00-1 GEND/SCH INTACT: 1. MALE 3. CATHOLIC

 004 G/S 21 (21 ) -1-1 GEND/SCH INTACT: 2. FEMALE 1. PUBLIC

 005 G/S 22 (22 ) 0100 GEND/SCH INTACT: 2. FEMALE 2. PRIVATE

 006 G/S 23 (23 ) 0001 GEND/SCH INTACT: 2. FEMALE 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0052

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY TYPE OF LOCALE (5 CATEGORIES)

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 20

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 R/T 11 (11 ) 010101010101010101010101 RACE/TOL INTACT: 1. WHI/AI/O 1. BIG CTY5

 002 R/T 12 (12 ) -1000000-1000000-1000000 RACE/TOL INTACT: 1. WHI/AI/O 2. MID CTY5

 003 R/T 13 (13 ) 00-1000000-1000000-10000 RACE/TOL INTACT: 1. WHI/AI/O 3. FR/BTWN5

 004 R/T 14 (14 ) 0000-1000000-1000000-100 RACE/TOL INTACT: 1. WHI/AI/O 4. SML TWN5

 005 R/T 15 (15 ) 000000-1000000-1000000-1 RACE/TOL INTACT: 1. WHI/AI/O 5. RURAL5

 006 R/T 21 (21 ) -1-1-1-10000000000000000 RACE/TOL INTACT: 2. BLACK 1. BIG CTY5

 007 R/T 22 (22 ) 010000000000000000000000 RACE/TOL INTACT: 2. BLACK 2. MID CTY5

 008 R/T 23 (23 ) 000100000000000000000000 RACE/TOL INTACT: 2. BLACK 3. FR/BTWN5

 009 R/T 24 (24 ) 000001000000000000000000 RACE/TOL INTACT: 2. BLACK 4. SML TWN5

 010 R/T 25 (25 ) 000000010000000000000000 RACE/TOL INTACT: 2. BLACK 5. RURAL5

 011 R/T 31 (31 ) 00000000-1-1-1-100000000 RACE/TOL INTACT: 3. HISPANIC 1. BIG CTY5

 012 R/T 32 (32 ) 000000000100000000000000 RACE/TOL INTACT: 3. HISPANIC 2. MID CTY5

 013 R/T 33 (33 ) 000000000001000000000000 RACE/TOL INTACT: 3. HISPANIC 3. FR/BTWN5

 014 R/T 34 (34 ) 000000000000010000000000 RACE/TOL INTACT: 3. HISPANIC 4. SML TWN5

 015 R/T 35 (35 ) 000000000000000100000000 RACE/TOL INTACT: 3. HISPANIC 5. RURAL5

 016 R/T 41 (41 ) 0000000000000000-1-1-1-1 RACE/TOL INTACT: 4. ASIAN 1. BIG CTY5

 017 R/T 42 (42 ) 000000000000000001000000 RACE/TOL INTACT: 4. ASIAN 2. MID CTY5

 018 R/T 43 (43 ) 000000000000000000010000 RACE/TOL INTACT: 4. ASIAN 3. FR/BTWN5

 019 R/T 44 (44 ) 000000000000000000000100 RACE/TOL INTACT: 4. ASIAN 4. SML TWN5

 020 R/T 45 (45 ) 000000000000000000000001 RACE/TOL INTACT: 4. ASIAN 5. RURAL5

 CONDITIONING VARIABLE ID: BACK0053

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 20

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 R/P 11 (11 ) 010101010101010101010101 RACE/PAR INTACT: 1. WHI/AI/O 1. < HS

 002 R/P 12 (12 ) -1000000-1000000-1000000 RACE/PAR INTACT: 1. WHI/AI/O 2. HS GRAD

 003 R/P 13 (13 ) 00-1000000-1000000-10000 RACE/PAR INTACT: 1. WHI/AI/O 3. POST HS

 004 R/P 14 (14 ) 0000-1000000-1000000-100 RACE/PAR INTACT: 1. WHI/AI/O 4. COL GRAD

 005 R/P 15 (15 ) 000000-1000000-1000000-1 RACE/PAR INTACT: 1. WHI/AI/O 5. PARED-?

 006 R/P 21 (21 ) -1-1-1-10000000000000000 RACE/PAR INTACT: 2. BLACK 1. < HS

 007 R/P 22 (22 ) 010000000000000000000000 RACE/PAR INTACT: 2. BLACK 2. HS GRAD

 008 R/P 23 (23 ) 000100000000000000000000 RACE/PAR INTACT: 2. BLACK 3. POST HS

 009 R/P 24 (24 ) 000001000000000000000000 RACE/PAR INTACT: 2. BLACK 4. COL GRAD

 010 R/P 25 (25 ) 000000010000000000000000 RACE/PAR INTACT: 2. BLACK 5. PARED-?

 011 R/P 31 (31 ) 00000000-1-1-1-100000000 RACE/PAR INTACT: 3. HISPANIC 1. < HS

 012 R/P 32 (32 ) 000000000100000000000000 RACE/PAR INTACT: 3. HISPANIC 2. HS GRAD

 013 R/P 33 (33 ) 000000000001000000000000 RACE/PAR INTACT: 3. HISPANIC 3. POST HS

 014 R/P 34 (34 ) 000000000000010000000000 RACE/PAR INTACT: 3. HISPANIC 4. COL GRAD

 015 R/P 35 (35 ) 000000000000000100000000 RACE/PAR INTACT: 3. HISPANIC 5. PARED-?

 016 R/P 41 (41 ) 0000000000000000-1-1-1-1 RACE/PAR INTACT: 4. ASIAN 1. < HS

 017 R/P 42 (42 ) 000000000000000001000000 RACE/PAR INTACT: 4. ASIAN 2. HS GRAD

 018 R/P 43 (43 ) 000000000000000000010000 RACE/PAR INTACT: 4. ASIAN 3. POST HS

 019 R/P 44 (44 ) 000000000000000000000100 RACE/PAR INTACT: 4. ASIAN 4. COL GRAD

 020 R/P 45 (45 ) 000000000000000000000001 RACE/PAR INTACT: 4. ASIAN 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0054

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 12

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 R/S 11 (11 ) 010101010101 RACE/SCH INTACT: 1. WHI/AI/O 1. PUBLIC

 002 R/S 12 (12 ) -100-100-100 RACE/SCH INTACT: 1. WHI/AI/O 2. PRIVATE

 003 R/S 13 (13 ) 00-100-100-1 RACE/SCH INTACT: 1. WHI/AI/O 3. CATHOLIC

 004 R/S 21 (21 ) -1-100000000 RACE/SCH INTACT: 2. BLACK 1. PUBLIC

 005 R/S 22 (22 ) 010000000000 RACE/SCH INTACT: 2. BLACK 2. PRIVATE

 006 R/S 23 (23 ) 000100000000 RACE/SCH INTACT: 2. BLACK 3. CATHOLIC

 007 R/S 31 (31 ) 0000-1-10000 RACE/SCH INTACT: 3. HISPANIC 1. PUBLIC

 008 R/S 32 (32 ) 000001000000 RACE/SCH INTACT: 3. HISPANIC 2. PRIVATE
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

483

 009 R/S 33 (33 ) 000000010000 RACE/SCH INTACT: 3. HISPANIC 3. CATHOLIC

 010 R/S 41 (41 ) 00000000-1-1 RACE/SCH INTACT: 4. ASIAN 1. PUBLIC

 011 R/S 42 (42 ) 000000000100 RACE/SCH INTACT: 4. ASIAN 2. PRIVATE

 012 R/S 43 (43 ) 000000000001 RACE/SCH INTACT: 4. ASIAN 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0055

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY PARENT’S EDUCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL5/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 25

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 16

 001 T/P 11 (11 ) 01010101010101010101010101010101 TOL5/PAR INTACT: 1. BIG CTY5 1. < HS

 002 T/P 12 (12 ) -1000000-1000000-1000000-1000000 TOL5/PAR INTACT: 1. BIG CTY5 2. HS GRAD

 003 T/P 13 (13 ) 00-1000000-1000000-1000000-10000 TOL5/PAR INTACT: 1. BIG CTY5 3. POST HS

 004 T/P 14 (14 ) 0000-1000000-1000000-1000000-100 TOL5/PAR INTACT: 1. BIG CTY5 4. COL GRAD

 005 T/P 15 (15 ) 000000-1000000-1000000-1000000-1 TOL5/PAR INTACT: 1. BIG CTY5 5. PARED-?

 006 T/P 21 (21 ) -1-1-1-1000000000000000000000000 TOL5/PAR INTACT: 2. MID CTY5 1. < HS

 007 T/P 22 (22 ) 01000000000000000000000000000000 TOL5/PAR INTACT: 2. MID CTY5 2. HS GRAD

 008 T/P 23 (23 ) 00010000000000000000000000000000 TOL5/PAR INTACT: 2. MID CTY5 3. POST HS

 009 T/P 24 (24 ) 00000100000000000000000000000000 TOL5/PAR INTACT: 2. MID CTY5 4. COL GRAD

 010 T/P 25 (25 ) 00000001000000000000000000000000 TOL5/PAR INTACT: 2. MID CTY5 5. PARED-?

 011 T/P 31 (31 ) 00000000-1-1-1-10000000000000000 TOL5/PAR INTACT: 3. FR/BTWN5 1. < HS

 012 T/P 32 (32 ) 00000000010000000000000000000000 TOL5/PAR INTACT: 3. FR/BTWN5 2. HS GRAD

 013 T/P 33 (33 ) 00000000000100000000000000000000 TOL5/PAR INTACT: 3. FR/BTWN5 3. POST HS

 014 T/P 34 (34 ) 00000000000001000000000000000000 TOL5/PAR INTACT: 3. FR/BTWN5 4. COL GRAD

 015 T/P 35 (35 ) 00000000000000010000000000000000 TOL5/PAR INTACT: 3. FR/BTWN5 5. PARED-?

 016 T/P 41 (41 ) 0000000000000000-1-1-1-100000000 TOL5/PAR INTACT: 4. SML TWN5 1. < HS

 017 T/P 42 (42 ) 00000000000000000100000000000000 TOL5/PAR INTACT: 4. SML TWN5 2. HS GRAD

 018 T/P 43 (43 ) 00000000000000000001000000000000 TOL5/PAR INTACT: 4. SML TWN5 3. POST HS

 019 T/P 44 (44 ) 00000000000000000000010000000000 TOL5/PAR INTACT: 4. SML TWN5 4. COL GRAD

 020 T/P 45 (45 ) 00000000000000000000000100000000 TOL5/PAR INTACT: 4. SML TWN5 5. PARED-?

 021 T/P 51 (51 ) 000000000000000000000000-1-1-1-1 TOL5/PAR INTACT: 5. RURAL5 1. < HS

 022 T/P 52 (52 ) 00000000000000000000000001000000 TOL5/PAR INTACT: 5. RURAL5 2. HS GRAD

 023 T/P 53 (53 ) 00000000000000000000000000010000 TOL5/PAR INTACT: 5. RURAL5 3. POST HS

 024 T/P 54 (54 ) 00000000000000000000000000000100 TOL5/PAR INTACT: 5. RURAL5 4. COL GRAD

 025 T/P 55 (55 ) 00000000000000000000000000000001 TOL5/PAR INTACT: 5. RURAL5 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0056

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL5/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 T/S 11 (11 ) 0101010101010101 TOL5/SCH INTACT: 1. BIG CTY5 1. PUBLIC

 002 T/S 12 (12 ) -100-100-100-100 TOL5/SCH INTACT: 1. BIG CTY5 2. PRIVATE

 003 T/S 13 (13 ) 00-100-100-100-1 TOL5/SCH INTACT: 1. BIG CTY5 3. CATHOLIC

 004 T/S 21 (21 ) -1-1000000000000 TOL5/SCH INTACT: 2. MID CTY5 1. PUBLIC

 005 T/S 22 (22 ) 0100000000000000 TOL5/SCH INTACT: 2. MID CTY5 2. PRIVATE

 006 T/S 23 (23 ) 0001000000000000 TOL5/SCH INTACT: 2. MID CTY5 3. CATHOLIC

 007 T/S 31 (31 ) 0000-1-100000000 TOL5/SCH INTACT: 3. FR/BTWN5 1. PUBLIC

 008 T/S 32 (32 ) 0000010000000000 TOL5/SCH INTACT: 3. FR/BTWN5 2. PRIVATE

 009 T/S 33 (33 ) 0000000100000000 TOL5/SCH INTACT: 3. FR/BTWN5 3. CATHOLIC

 010 T/S 41 (41 ) 00000000-1-10000 TOL5/SCH INTACT: 4. SML TWN5 1. PUBLIC

 011 T/S 42 (42 ) 0000000001000000 TOL5/SCH INTACT: 4. SML TWN5 2. PRIVATE

 012 T/S 43 (43 ) 0000000000010000 TOL5/SCH INTACT: 4. SML TWN5 3. CATHOLIC

 013 T/S 51 (51 ) 000000000000-1-1 TOL5/SCH INTACT: 5. RURAL5 1. PUBLIC

 014 T/S 52 (52 ) 0000000000000100 TOL5/SCH INTACT: 5. RURAL5 2. PRIVATE

 015 T/S 53 (53 ) 0000000000000001 TOL5/SCH INTACT: 5. RURAL5 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0057

 DESCRIPTION: INTERACTION: PARENTS’ EDUCATION BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: PARE/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 P/S 11 (11 ) 0101010101010101 PARE/SCH INTACT: 1. < HS 1. PUBLIC

 002 P/S 12 (12 ) -100-100-100-100 PARE/SCH INTACT: 1. < HS 2. PRIVATE

 003 P/S 13 (13 ) 00-100-100-100-1 PARE/SCH INTACT: 1. < HS 3. CATHOLIC

 004 P/S 21 (21 ) -1-1000000000000 PARE/SCH INTACT: 2. HS GRAD 1. PUBLIC

 005 P/S 22 (22 ) 0100000000000000 PARE/SCH INTACT: 2. HS GRAD 2. PRIVATE

 006 P/S 23 (23 ) 0001000000000000 PARE/SCH INTACT: 2. HS GRAD 3. CATHOLIC

 007 P/S 31 (31 ) 0000-1-100000000 PARE/SCH INTACT: 3. POST HS 1. PUBLIC

 008 P/S 32 (32 ) 0000010000000000 PARE/SCH INTACT: 3. POST HS 2. PRIVATE

 009 P/S 33 (33 ) 0000000100000000 PARE/SCH INTACT: 3. POST HS 3. CATHOLIC

 010 P/S 41 (41 ) 00000000-1-10000 PARE/SCH INTACT: 4. COL GRAD 1. PUBLIC

 011 P/S 42 (42 ) 0000000001000000 PARE/SCH INTACT: 4. COL GRAD 2. PRIVATE

 012 P/S 43 (43 ) 0000000000010000 PARE/SCH INTACT: 4. COL GRAD 3. CATHOLIC

 013 P/S 51 (51 ) 000000000000-1-1 PARE/SCH INTACT: 5. PARED-? 1. PUBLIC

 014 P/S 52 (52 ) 0000000000000100 PARE/SCH INTACT: 5. PARED-? 2. PRIVATE

 015 P/S 53 (53 ) 0000000000000001 PARE/SCH INTACT: 5. PARED-? 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0058

 DESCRIPTION: INTERACTION: GENDER BY MATH COURSES TAKING THIS YEAR

 GRADES/ASSESSMENTS: N08, S08
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

484

 CONDITIONING VAR LABEL: GEND/

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 16

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 7

 001 G/ 11 (11 ) 01010101010101 GEND/ INTACT: 1. MALE 1. NO MATH

 002 G/ 12 (12 ) -1000000000000 GEND/ INTACT: 1. MALE 2. 8TH GRD

 003 G/ 13 (13 ) 00-10000000000 GEND/ INTACT: 1. MALE 3. PREALG

 004 G/ 14 (14 ) 0000-100000000 GEND/ INTACT: 1. MALE 4. ALGEBRA

 005 G/ 15 (15 ) 000000-1000000 GEND/ INTACT: 1. MALE 5. INT/SEQ

 006 G/ 16 (16 ) 00000000-10000 GEND/ INTACT: 1. MALE 6. APPLIED

 007 G/ 17 (17 ) 0000000000-100 GEND/ INTACT: 1. MALE 7. OTHER

 008 G/ 18 (18 ) 000000000000-1 GEND/ INTACT: 1. MALE 8. MISSING

 009 G/ 21 (21 ) -1-1-1-1-1-1-1 GEND/ INTACT: 2. FEMALE 1. NO MATH

 010 G/ 22 (22 ) 01000000000000 GEND/ INTACT: 2. FEMALE 2. 8TH GRD

 011 G/ 23 (23 ) 00010000000000 GEND/ INTACT: 2. FEMALE 3. PREALG

 012 G/ 24 (24 ) 00000100000000 GEND/ INTACT: 2. FEMALE 4. ALGEBRA

 013 G/ 25 (25 ) 00000001000000 GEND/ INTACT: 2. FEMALE 5. INT/SEQ

 014 G/ 26 (26 ) 00000000010000 GEND/ INTACT: 2. FEMALE 6. APPLIED

 015 G/ 27 (27 ) 00000000000100 GEND/ INTACT: 2. FEMALE 7. OTHER

 016 G/ 28 (28 ) 00000000000001 GEND/ INTACT: 2. FEMALE 8. MISSING

 CONDITIONING VARIABLE ID: BACK0060

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY MATH COURSES TAKING THIS YEAR

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL: RACE/

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 32

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 21

 001 R/ 11 (11 ) 333333333333333333333 RACE/ INTACT: 1. WHI/AI/O 1. NO MATH

 002 R/ 12 (12 ) 122222212222221222222 RACE/ INTACT: 1. WHI/AI/O 2. 8TH GRD

 003 R/ 13 (13 ) 212222221222222122222 RACE/ INTACT: 1. WHI/AI/O 3. PREALG

 004 R/ 14 (14 ) 221222222122222212222 RACE/ INTACT: 1. WHI/AI/O 4. ALGEBRA

 005 R/ 15 (15 ) 222122222212222221222 RACE/ INTACT: 1. WHI/AI/O 5. INT/SEQ

 006 R/ 16 (16 ) 222212222221222222122 RACE/ INTACT: 1. WHI/AI/O 6. APPLIED

 007 R/ 17 (17 ) 222221222222122222212 RACE/ INTACT: 1. WHI/AI/O 7. OTHER

 008 R/ 18 (18 ) 222222122222212222221 RACE/ INTACT: 1. WHI/AI/O 8. MISSING

 009 R/ 21 (21 ) 111111122222222222222 RACE/ INTACT: 2. BLACK 1. NO MATH

 010 R/ 22 (22 ) 322222222222222222222 RACE/ INTACT: 2. BLACK 2. 8TH GRD

 011 R/ 23 (23 ) 232222222222222222222 RACE/ INTACT: 2. BLACK 3. PREALG

 012 R/ 24 (24 ) 223222222222222222222 RACE/ INTACT: 2. BLACK 4. ALGEBRA

 013 R/ 25 (25 ) 222322222222222222222 RACE/ INTACT: 2. BLACK 5. INT/SEQ

 014 R/ 26 (26 ) 222232222222222222222 RACE/ INTACT: 2. BLACK 6. APPLIED

 015 R/ 27 (27 ) 222223222222222222222 RACE/ INTACT: 2. BLACK 7. OTHER

 016 R/ 28 (28 ) 222222322222222222222 RACE/ INTACT: 2. BLACK 8. MISSING

 017 R/ 31 (31 ) 222222211111112222222 RACE/ INTACT: 3. HISPANIC 1. NO MATH

 018 R/ 32 (32 ) 222222232222222222222 RACE/ INTACT: 3. HISPANIC 2. 8TH GRD

 019 R/ 33 (33 ) 222222223222222222222 RACE/ INTACT: 3. HISPANIC 3. PREALG

 020 R/ 34 (34 ) 222222222322222222222 RACE/ INTACT: 3. HISPANIC 4. ALGEBRA

 021 R/ 35 (35 ) 222222222232222222222 RACE/ INTACT: 3. HISPANIC 5. INT/SEQ

 022 R/ 36 (36 ) 222222222223222222222 RACE/ INTACT: 3. HISPANIC 6. APPLIED

 023 R/ 37 (37 ) 222222222222322222222 RACE/ INTACT: 3. HISPANIC 7. OTHER

 024 R/ 38 (38 ) 222222222222232222222 RACE/ INTACT: 3. HISPANIC 8. MISSING

 025 R/ 41 (41 ) 222222222222221111111 RACE/ INTACT: 4. ASIAN 1. NO MATH

 026 R/ 42 (42 ) 222222222222223222222 RACE/ INTACT: 4. ASIAN 2. 8TH GRD

 027 R/ 43 (43 ) 222222222222222322222 RACE/ INTACT: 4. ASIAN 3. PREALG

 028 R/ 44 (44 ) 222222222222222232222 RACE/ INTACT: 4. ASIAN 4. ALGEBRA

 029 R/ 45 (45 ) 222222222222222223222 RACE/ INTACT: 4. ASIAN 5. INT/SEQ

 030 R/ 46 (46 ) 222222222222222222322 RACE/ INTACT: 4. ASIAN 6. APPLIED

 031 R/ 47 (47 ) 222222222222222222232 RACE/ INTACT: 4. ASIAN 7. OTHER

 032 R/ 48 (48 ) 222222222222222223223 RACE/ INTACT: 4. ASIAN 8. MISSING

 CONDITIONING VARIABLE ID: BACK0062

 DESCRIPTION: INTERACTION: PARENTS’ EDUCATION BY MATH COURSES TAKING THIS YEAR

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL: PARE/

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 40

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 28

 001 P/ 11 (11 ) 3333333333333333333333333333 PARE/ INTACT: 1. < HS 1. NO MATH

 002 P/ 12 (12 ) 1222222122222212222221222222 PARE/ INTACT: 1. < HS 2. 8TH GRD

 003 P/ 13 (13 ) 2122222212222221222222122222 PARE/ INTACT: 1. < HS 3. PREALG

 004 P/ 14 (14 ) 2212222221222222122222212222 PARE/ INTACT: 1. < HS 4. ALGEBRA

 005 P/ 15 (15 ) 2221222222122222212222221222 PARE/ INTACT: 1. < HS 5. INT/SEQ

 006 P/ 16 (16 ) 2222122222212222221222222122 PARE/ INTACT: 1. < HS 6. APPLIED

 007 P/ 17 (17 ) 2222212222221222222122222212 PARE/ INTACT: 1. < HS 7. OTHER

 008 P/ 18 (18 ) 2222221222222122222212222221 PARE/ INTACT: 1. < HS 8. MISSING

 009 P/ 21 (21 ) 1111111222222222222222222222 PARE/ INTACT: 2. HS GRAD 1. NO MATH

 010 P/ 22 (22 ) 3222222222222222222222222222 PARE/ INTACT: 2. HS GRAD 2. 8TH GRD

 011 P/ 23 (23 ) 2322222222222222222222222222 PARE/ INTACT: 2. HS GRAD 3. PREALG

 012 P/ 24 (24 ) 2232222222222222222222222222 PARE/ INTACT: 2. HS GRAD 4. ALGEBRA

 013 P/ 25 (25 ) 2223222222222222222222222222 PARE/ INTACT: 2. HS GRAD 5. INT/SEQ

 014 P/ 26 (26 ) 2222322222222222222222222222 PARE/ INTACT: 2. HS GRAD 6. APPLIED

 015 P/ 27 (27 ) 2222232222222222222222222222 PARE/ INTACT: 2. HS GRAD 7. OTHER

 016 P/ 28 (28 ) 2222223222222222222222222222 PARE/ INTACT: 2. HS GRAD 8. MISSING

 017 P/ 31 (31 ) 2222222111111122222222222222 PARE/ INTACT: 3. POST HS 1. NO MATH

 018 P/ 32 (32 ) 2222222322222222222222222222 PARE/ INTACT: 3. POST HS 2. 8TH GRD

 019 P/ 33 (33 ) 2222222232222222222222222222 PARE/ INTACT: 3. POST HS 3. PREALG
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

485

 020 P/ 34 (34 ) 2222222223222222222222222222 PARE/ INTACT: 3. POST HS 4. ALGEBRA

 021 P/ 35 (35 ) 2222222222322222222222222222 PARE/ INTACT: 3. POST HS 5. INT/SEQ

 022 P/ 36 (36 ) 2222222222232222222222222222 PARE/ INTACT: 3. POST HS 6. APPLIED

 023 P/ 37 (37 ) 2222222222223222222222222222 PARE/ INTACT: 3. POST HS 7. OTHER

 024 P/ 38 (38 ) 2222222222222322222222222222 PARE/ INTACT: 3. POST HS 8. MISSING

 025 P/ 41 (41 ) 2222222222222211111112222222 PARE/ INTACT: 4. COL GRAD 1. NO MATH

 026 P/ 42 (42 ) 2222222222222232222222222222 PARE/ INTACT: 4. COL GRAD 2. 8TH GRD

 027 P/ 43 (43 ) 2222222222222223222222222222 PARE/ INTACT: 4. COL GRAD 3. PREALG

 028 P/ 44 (44 ) 2222222222222222322222222222 PARE/ INTACT: 4. COL GRAD 4. ALGEBRA

 029 P/ 45 (45 ) 2222222222222222232222222222 PARE/ INTACT: 4. COL GRAD 5. INT/SEQ

 030 P/ 46 (46 ) 2222222222222222223222222222 PARE/ INTACT: 4. COL GRAD 6. APPLIED

 031 P/ 47 (47 ) 2222222222222222222322222222 PARE/ INTACT: 4. COL GRAD 7. OTHER

 032 P/ 48 (48 ) 2222222222222222222232222222 PARE/ INTACT: 4. COL GRAD 8. MISSING

 033 P/ 51 (51 ) 2222222222222222222221111111 PARE/ INTACT: 5. PARED-? 1. NO MATH

 034 P/ 52 (52 ) 2222222222222222222223222222 PARE/ INTACT: 5. PARED-? 2. 8TH GRD

 035 P/ 53 (53 ) 2222222222222222222222322222 PARE/ INTACT: 5. PARED-? 3. PREALG

 036 P/ 54 (54 ) 2222222222222222222222232222 PARE/ INTACT: 5. PARED-? 4. ALGEBRA

 037 P/ 55 (55 ) 2222222222222222222222223222 PARE/ INTACT: 5. PARED-? 5. INT/SEQ

 038 P/ 56 (56 ) 2222222222222222222222222322 PARE/ INTACT: 5. PARED-? 6. APPLIED

 039 P/ 57 (57 ) 2222222222222222222222222232 PARE/ INTACT: 5. PARED-? 7. OTHER

 040 P/ 58 (58 ) 2222222222222222222222222223 PARE/ INTACT: 5. PARED-? 8. MISSING

 CONDITIONING VARIABLE ID: BACK0064

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY MATH COURSES TAKING THIS YEAR

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL: TOL5/

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 40

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 28

 001 T/ 11 (11 ) 3333333333333333333333333333 TOL5/ INTACT: 1. BIG CTY5 1. NO MATH

 002 T/ 12 (12 ) 1222222122222212222221222222 TOL5/ INTACT: 1. BIG CTY5 2. 8TH GRD

 003 T/ 13 (13 ) 2122222212222221222222122222 TOL5/ INTACT: 1. BIG CTY5 3. PREALG

 004 T/ 14 (14 ) 2212222221222222122222212222 TOL5/ INTACT: 1. BIG CTY5 4. ALGEBRA

 005 T/ 15 (15 ) 2221222222122222212222221222 TOL5/ INTACT: 1. BIG CTY5 5. INT/SEQ

 006 T/ 16 (16 ) 2222122222212222221222222122 TOL5/ INTACT: 1. BIG CTY5 6. APPLIED

 007 T/ 17 (17 ) 2222212222221222222122222212 TOL5/ INTACT: 1. BIG CTY5 7. OTHER

 008 T/ 18 (18 ) 2222221222222122222212222221 TOL5/ INTACT: 1. BIG CTY5 8. MISSING

 009 T/ 21 (21 ) 1111111222222222222222222222 TOL5/ INTACT: 2. MID CTY5 1. NO MATH

 010 T/ 22 (22 ) 3222222222222222222222222222 TOL5/ INTACT: 2. MID CTY5 2. 8TH GRD

 011 T/ 23 (23 ) 2322222222222222222222222222 TOL5/ INTACT: 2. MID CTY5 3. PREALG

 012 T/ 24 (24 ) 2232222222222222222222222222 TOL5/ INTACT: 2. MID CTY5 4. ALGEBRA

 013 T/ 25 (25 ) 2223222222222222222222222222 TOL5/ INTACT: 2. MID CTY5 5. INT/SEQ

 014 T/ 26 (26 ) 2222322222222222222222222222 TOL5/ INTACT: 2. MID CTY5 6. APPLIED

 015 T/ 27 (27 ) 2222232222222222222222222222 TOL5/ INTACT: 2. MID CTY5 7. OTHER

 016 T/ 28 (28 ) 2222223222222222222222222222 TOL5/ INTACT: 2. MID CTY5 8. MISSING

 017 T/ 31 (31 ) 2222222111111122222222222222 TOL5/ INTACT: 3. FR/BTWN5 1. NO MATH

 018 T/ 32 (32 ) 2222222322222222222222222222 TOL5/ INTACT: 3. FR/BTWN5 2. 8TH GRD

 019 T/ 33 (33 ) 2222222232222222222222222222 TOL5/ INTACT: 3. FR/BTWN5 3. PREALG

 020 T/ 34 (34 ) 2222222223222222222222222222 TOL5/ INTACT: 3. FR/BTWN5 4. ALGEBRA

 021 T/ 35 (35 ) 2222222222322222222222222222 TOL5/ INTACT: 3. FR/BTWN5 5. INT/SEQ

 022 T/ 36 (36 ) 2222222222232222222222222222 TOL5/ INTACT: 3. FR/BTWN5 6. APPLIED

 023 T/ 37 (37 ) 2222222222223222222222222222 TOL5/ INTACT: 3. FR/BTWN5 7. OTHER

 024 T/ 38 (38 ) 2222222222222322222222222222 TOL5/ INTACT: 3. FR/BTWN5 8. MISSING

 025 T/ 41 (41 ) 2222222222222211111112222222 TOL5/ INTACT: 4. SML TWN5 1. NO MATH

 026 T/ 42 (42 ) 2222222222222232222222222222 TOL5/ INTACT: 4. SML TWN5 2. 8TH GRD

 027 T/ 43 (43 ) 2222222222222223222222222222 TOL5/ INTACT: 4. SML TWN5 3. PREALG

 028 T/ 44 (44 ) 2222222222222222322222222222 TOL5/ INTACT: 4. SML TWN5 4. ALGEBRA

 029 T/ 45 (45 ) 2222222222222222232222222222 TOL5/ INTACT: 4. SML TWN5 5. INT/SEQ

 030 T/ 46 (46 ) 2222222222222222223222222222 TOL5/ INTACT: 4. SML TWN5 6. APPLIED

 031 T/ 47 (47 ) 2222222222222222222322222222 TOL5/ INTACT: 4. SML TWN5 7. OTHER

 032 T/ 48 (48 ) 2222222222222222222232222222 TOL5/ INTACT: 4. SML TWN5 8. MISSING

 033 T/ 51 (51 ) 2222222222222222222221111111 TOL5/ INTACT: 5. RURAL5 1. NO MATH

 034 T/ 52 (52 ) 2222222222222222222223222222 TOL5/ INTACT: 5. RURAL5 2. 8TH GRD

 035 T/ 53 (53 ) 2222222222222222222222322222 TOL5/ INTACT: 5. RURAL5 3. PREALG

 036 T/ 54 (54 ) 2222222222222222222222232222 TOL5/ INTACT: 5. RURAL5 4. ALGEBRA

 037 T/ 55 (55 ) 2222222222222222222222223222 TOL5/ INTACT: 5. RURAL5 5. INT/SEQ

 038 T/ 56 (56 ) 2222222222222222222222222322 TOL5/ INTACT: 5. RURAL5 6. APPLIED

 039 T/ 57 (57 ) 2222222222222222222222222232 TOL5/ INTACT: 5. RURAL5 7. OTHER

 040 T/ 58 (58 ) 2222222222222222222222222223 TOL5/ INTACT: 5. RURAL5 8. MISSING

 CONDITIONING VARIABLE ID: BACK0066

 DESCRIPTION: INTERACTION: SCHOOL TYPE BY MATH COURSES TAKING THIS YEAR

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL: SCHT/

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 24

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 14

 001 S/ 11 (11 ) 0101010101010101010101010101 SCHT/ INTACT: 1. PUBLIC 1. NO MATH

 002 S/ 12 (12 ) -1000000000000-1000000000000 SCHT/ INTACT: 1. PUBLIC 2. 8TH GRD

 003 S/ 13 (13 ) 00-1000000000000-10000000000 SCHT/ INTACT: 1. PUBLIC 3. PREALG

 004 S/ 14 (14 ) 0000-1000000000000-100000000 SCHT/ INTACT: 1. PUBLIC 4. ALGEBRA

 005 S/ 15 (15 ) 000000-1000000000000-1000000 SCHT/ INTACT: 1. PUBLIC 5. INT/SEQ

 006 S/ 16 (16 ) 00000000-1000000000000-10000 SCHT/ INTACT: 1. PUBLIC 6. APPLIED

 007 S/ 17 (17 ) 0000000000-1000000000000-100 SCHT/ INTACT: 1. PUBLIC 7. OTHER

 008 S/ 18 (18 ) 000000000000-1000000000000-1 SCHT/ INTACT: 1. PUBLIC 8. MISSING

 009 S/ 21 (21 ) -1-1-1-1-1-1-100000000000000 SCHT/ INTACT: 2. PRIVATE 1. NO MATH

 010 S/ 22 (22 ) 0100000000000000000000000000 SCHT/ INTACT: 2. PRIVATE 2. 8TH GRD
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

486

 011 S/ 23 (23 ) 0001000000000000000000000000 SCHT/ INTACT: 2. PRIVATE 3. PREALG

 012 S/ 24 (24 ) 0000010000000000000000000000 SCHT/ INTACT: 2. PRIVATE 4. ALGEBRA

 013 S/ 25 (25 ) 0000000100000000000000000000 SCHT/ INTACT: 2. PRIVATE 5. INT/SEQ

 014 S/ 26 (26 ) 0000000001000000000000000000 SCHT/ INTACT: 2. PRIVATE 6. APPLIED

 015 S/ 27 (27 ) 0000000000010000000000000000 SCHT/ INTACT: 2. PRIVATE 7. OTHER

 016 S/ 28 (28 ) 0000000000000100000000000000 SCHT/ INTACT: 2. PRIVATE 8. MISSING

 017 S/ 31 (31 ) 00000000000000-1-1-1-1-1-1-1 SCHT/ INTACT: 3. CATHOLIC 1. NO MATH

 018 S/ 32 (32 ) 0000000000000001000000000000 SCHT/ INTACT: 3. CATHOLIC 2. 8TH GRD

 019 S/ 33 (33 ) 0000000000000000010000000000 SCHT/ INTACT: 3. CATHOLIC 3. PREALG

 020 S/ 34 (34 ) 0000000000000000000100000000 SCHT/ INTACT: 3. CATHOLIC 4. ALGEBRA

 021 S/ 35 (35 ) 0000000000000000000001000000 SCHT/ INTACT: 3. CATHOLIC 5. INT/SEQ

 022 S/ 36 (36 ) 0000000000000000000000010000 SCHT/ INTACT: 3. CATHOLIC 6. APPLIED

 023 S/ 37 (37 ) 0000000000000000000000000100 SCHT/ INTACT: 3. CATHOLIC 7. OTHER

 024 S/ 38 (38 ) 0000000000000000000000000001 SCHT/ INTACT: 3. CATHOLIC 8. MISSING

 CONDITIONING VARIABLE ID: BACK0059

 DESCRIPTION: INTERACTION: GENDER BY NUMBER OF SEMESTERS MATH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: GEND/SEM

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 G/S 11 (11 ) 01010101 GEND/SEM INTACT: 1. MALE 1. SEMMAT01

 002 G/S 12 (12 ) -1000000 GEND/SEM INTACT: 1. MALE 2. SEMMAT02

 003 G/S 13 (13 ) 00-10000 GEND/SEM INTACT: 1. MALE 3. SEMMAT03

 004 G/S 14 (14 ) 0000-100 GEND/SEM INTACT: 1. MALE 4. SEMMAT04

 005 G/S 15 (15 ) 0000-100 GEND/SEM INTACT: 1. MALE 5. SEMMAT-?

 006 G/S 21 (21 ) -1-1-1-1 GEND/SEM INTACT: 2. FEMALE 1. SEMMAT01

 007 G/S 22 (22 ) 01000000 GEND/SEM INTACT: 2. FEMALE 2. SEMMAT02

 008 G/S 23 (23 ) 00010000 GEND/SEM INTACT: 2. FEMALE 3. SEMMAT03

 009 G/S 24 (24 ) 00000100 GEND/SEM INTACT: 2. FEMALE 4. SEMMAT04

 010 G/S 25 (25 ) 00000001 GEND/SEM INTACT: 2. FEMALE 5. SEMMAT-?

 CONDITIONING VARIABLE ID: BACK0061

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY NUMBER OF SEMESTERS MATH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: RACE/SEM

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 20

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 R/S 11 (11 ) 010101010101010101010101 RACE/SEM INTACT: 1. WHI/AI/O 1. SEMMAT01

 002 R/S 12 (12 ) -1000000-1000000-1000000 RACE/SEM INTACT: 1. WHI/AI/O 2. SEMMAT02

 003 R/S 13 (13 ) 00-1000000-1000000-10000 RACE/SEM INTACT: 1. WHI/AI/O 3. SEMMAT03

 004 R/S 14 (14 ) 0000-1000000-1000000-100 RACE/SEM INTACT: 1. WHI/AI/O 4. SEMMAT04

 005 R/S 15 (15 ) 000000-1000000-1000000-1 RACE/SEM INTACT: 1. WHI/AI/O 5. SEMMAT-?

 006 R/S 21 (21 ) -1-1-1-10000000000000000 RACE/SEM INTACT: 2. BLACK 1. SEMMAT01

 007 R/S 22 (22 ) 010000000000000000000000 RACE/SEM INTACT: 2. BLACK 2. SEMMAT02

 008 R/S 23 (23 ) 000100000000000000000000 RACE/SEM INTACT: 2. BLACK 3. SEMMAT03

 009 R/S 24 (24 ) 000001000000000000000000 RACE/SEM INTACT: 2. BLACK 4. SEMMAT04

 010 R/S 25 (25 ) 000000010000000000000000 RACE/SEM INTACT: 2. BLACK 5. SEMMAT-?

 011 R/S 31 (31 ) 00000000-1-1-1-100000000 RACE/SEM INTACT: 3. HISPANIC 1. SEMMAT01

 012 R/S 32 (32 ) 000000000100000000000000 RACE/SEM INTACT: 3. HISPANIC 2. SEMMAT02

 013 R/S 33 (33 ) 000000000001000000000000 RACE/SEM INTACT: 3. HISPANIC 3. SEMMAT03

 014 R/S 34 (34 ) 000000000000010000000000 RACE/SEM INTACT: 3. HISPANIC 4. SEMMAT04

 015 R/S 35 (35 ) 000000000000000100000000 RACE/SEM INTACT: 3. HISPANIC 5. SEMMAT-?

 016 R/S 41 (41 ) 0000000000000000-1-1-1-1 RACE/SEM INTACT: 4. ASIAN 1. SEMMAT01

 017 R/S 42 (42 ) 000000000000000001000000 RACE/SEM INTACT: 4. ASIAN 2. SEMMAT02

 018 R/S 43 (43 ) 000000000000000000010000 RACE/SEM INTACT: 4. ASIAN 3. SEMMAT03

 019 R/S 44 (44 ) 000000000000000000000100 RACE/SEM INTACT: 4. ASIAN 4. SEMMAT04

 020 R/S 45 (45 ) 000000000000000000000001 RACE/SEM INTACT: 4. ASIAN 5. SEMMAT-?

 CONDITIONING VARIABLE ID: BACK0063

 DESCRIPTION: INTERACTION: PARENTS’ EDUCATION BY NUMBER OF SEMESTERS MATH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: PARE/SEM

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 25

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 16

 001 P/S 11 (11 ) 01010101010101010101010101010101 PARE/SEM INTACT: 1. < HS 1. SEMMAT01

 002 P/S 12 (12 ) -1000000-1000000-1000000-1000000 PARE/SEM INTACT: 1. < HS 2. SEMMAT02

 003 P/S 13 (13 ) 00-1000000-1000000-1000000-10000 PARE/SEM INTACT: 1. < HS 3. SEMMAT03

 004 P/S 14 (14 ) 0000-1000000-1000000-1000000-100 PARE/SEM INTACT: 1. < HS 4. SEMMAT04

 005 P/S 15 (15 ) 000000-1000000-1000000-1000000-1 PARE/SEM INTACT: 1. < HS 5. SEMMAT-?

 006 P/S 21 (21 ) -1-1-1-1000000000000000000000000 PARE/SEM INTACT: 2. HS GRAD 1. SEMMAT01

 007 P/S 22 (22 ) 01000000000000000000000000000000 PARE/SEM INTACT: 2. HS GRAD 2. SEMMAT02

 008 P/S 23 (23 ) 00010000000000000000000000000000 PARE/SEM INTACT: 2. HS GRAD 3. SEMMAT03

 009 P/S 24 (24 ) 00000100000000000000000000000000 PARE/SEM INTACT: 2. HS GRAD 4. SEMMAT04

 010 P/S 25 (25 ) 00000001000000000000000000000000 PARE/SEM INTACT: 2. HS GRAD 5. SEMMAT-?

 011 P/S 31 (31 ) 00000000-1-1-1-10000000000000000 PARE/SEM INTACT: 3. POST HS 1. SEMMAT01

 012 P/S 32 (32 ) 00000000010000000000000000000000 PARE/SEM INTACT: 3. POST HS 2. SEMMAT02

 013 P/S 33 (33 ) 00000000000100000000000000000000 PARE/SEM INTACT: 3. POST HS 3. SEMMAT03

 014 P/S 34 (34 ) 00000000000001000000000000000000 PARE/SEM INTACT: 3. POST HS 4. SEMMAT04

 015 P/S 35 (35 ) 00000000000000010000000000000000 PARE/SEM INTACT: 3. POST HS 5. SEMMAT-?

 016 P/S 41 (41 ) 0000000000000000-1-1-1-100000000 PARE/SEM INTACT: 4. COL GRAD 1. SEMMAT01

 017 P/S 42 (42 ) 00000000000000000100000000000000 PARE/SEM INTACT: 4. COL GRAD 2. SEMMAT02

 018 P/S 43 (43 ) 00000000000000000001000000000000 PARE/SEM INTACT: 4. COL GRAD 3. SEMMAT03

 019 P/S 44 (44 ) 00000000000000000000010000000000 PARE/SEM INTACT: 4. COL GRAD 4. SEMMAT04
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

487

 020 P/S 45 (45 ) 00000000000000000000000100000000 PARE/SEM INTACT: 4. COL GRAD 5. SEMMAT-?

 021 P/S 51 (51 ) 000000000000000000000000-1-1-1-1 PARE/SEM INTACT: 5. PARED-? 1. SEMMAT01

 022 P/S 52 (52 ) 00000000000000000000000001000000 PARE/SEM INTACT: 5. PARED-? 2. SEMMAT02

 023 P/S 53 (53 ) 00000000000000000000000000010000 PARE/SEM INTACT: 5. PARED-? 3. SEMMAT03

 024 P/S 54 (54 ) 00000000000000000000000000000100 PARE/SEM INTACT: 5. PARED-? 4. SEMMAT04

 025 P/S 55 (55 ) 00000000000000000000000000000001 PARE/SEM INTACT: 5. PARED-? 5. SEMMAT-?

 CONDITIONING VARIABLE ID: BACK0065

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY NUMBER OF SEMESTERS MATH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: TOL5/SEM

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 25

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 16

 001 T/S 11 (11 ) 01010101010101010101010101010101 TOL5/SEM INTACT: 1. BIG CTY5 1. SEMMAT01

 002 T/S 12 (12 ) -1000000-1000000-1000000-1000000 TOL5/SEM INTACT: 1. BIG CTY5 2. SEMMAT02

 003 T/S 13 (13 ) 00-1000000-1000000-1000000-10000 TOL5/SEM INTACT: 1. BIG CTY5 3. SEMMAT03

 004 T/S 14 (14 ) 0000-1000000-1000000-1000000-100 TOL5/SEM INTACT: 1. BIG CTY5 4. SEMMAT04

 005 T/S 15 (15 ) 000000-1000000-1000000-1000000-1 TOL5/SEM INTACT: 1. BIG CTY5 5. SEMMAT-?

 006 T/S 21 (21 ) -1-1-1-1000000000000000000000000 TOL5/SEM INTACT: 2. MID CTY5 1. SEMMAT01

 007 T/S 22 (22 ) 01000000000000000000000000000000 TOL5/SEM INTACT: 2. MID CTY5 2. SEMMAT02

 008 T/S 23 (23 ) 00010000000000000000000000000000 TOL5/SEM INTACT: 2. MID CTY5 3. SEMMAT03

 009 T/S 24 (24 ) 00000100000000000000000000000000 TOL5/SEM INTACT: 2. MID CTY5 4. SEMMAT04

 010 T/S 25 (25 ) 00000001000000000000000000000000 TOL5/SEM INTACT: 2. MID CTY5 5. SEMMAT-?

 011 T/S 31 (31 ) 00000000-1-1-1-10000000000000000 TOL5/SEM INTACT: 3. FR/BTWN5 1. SEMMAT01

 012 T/S 32 (32 ) 00000000010000000000000000000000 TOL5/SEM INTACT: 3. FR/BTWN5 2. SEMMAT02

 013 T/S 33 (33 ) 00000000000100000000000000000000 TOL5/SEM INTACT: 3. FR/BTWN5 3. SEMMAT03

 014 T/S 34 (34 ) 00000000000001000000000000000000 TOL5/SEM INTACT: 3. FR/BTWN5 4. SEMMAT04

 015 T/S 35 (35 ) 00000000000000010000000000000000 TOL5/SEM INTACT: 3. FR/BTWN5 5. SEMMAT-?

 016 T/S 41 (41 ) 0000000000000000-1-1-1-100000000 TOL5/SEM INTACT: 4. SML TWN5 1. SEMMAT01

 017 T/S 42 (42 ) 00000000000000000100000000000000 TOL5/SEM INTACT: 4. SML TWN5 2. SEMMAT02

 018 T/S 43 (43 ) 00000000000000000001000000000000 TOL5/SEM INTACT: 4. SML TWN5 3. SEMMAT03

 019 T/S 44 (44 ) 00000000000000000000010000000000 TOL5/SEM INTACT: 4. SML TWN5 4. SEMMAT04

 020 T/S 45 (45 ) 00000000000000000000000100000000 TOL5/SEM INTACT: 4. SML TWN5 5. SEMMAT-?

 021 T/S 51 (51 ) 000000000000000000000000-1-1-1-1 TOL5/SEM INTACT: 5. RURAL5 1. SEMMAT01

 022 T/S 52 (52 ) 00000000000000000000000001000000 TOL5/SEM INTACT: 5. RURAL5 2. SEMMAT02

 023 T/S 53 (53 ) 00000000000000000000000000010000 TOL5/SEM INTACT: 5. RURAL5 3. SEMMAT03

 024 T/S 54 (54 ) 00000000000000000000000000000100 TOL5/SEM INTACT: 5. RURAL5 4. SEMMAT04

 025 T/S 55 (55 ) 00000000000000000000000000000001 TOL5/SEM INTACT: 5. RURAL5 5. SEMMAT-?

 CONDITIONING VARIABLE ID: BACK0067

 DESCRIPTION: INTERACTION: SCHOOL TYPE BY NUMBER OF SEMESTERS MATH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SCHT/SEM

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 S/S 11 (11 ) 0101010101010101 SCHT/SEM INTACT: 1. PUBLIC 1. SEMMAT01

 002 S/S 12 (12 ) -1000000-1000000 SCHT/SEM INTACT: 1. PUBLIC 2. SEMMAT02

 003 S/S 13 (13 ) 00-1000000-10000 SCHT/SEM INTACT: 1. PUBLIC 3. SEMMAT03

 004 S/S 14 (14 ) 0000-1000000-100 SCHT/SEM INTACT: 1. PUBLIC 4. SEMMAT04

 005 S/S 15 (15 ) 000000-1000000-1 SCHT/SEM INTACT: 1. PUBLIC 5. SEMMAT-?

 006 S/S 21 (21 ) -1-1-1-100000000 SCHT/SEM INTACT: 2. PRIVATE 1. SEMMAT01

 007 S/S 22 (22 ) 0100000000000000 SCHT/SEM INTACT: 2. PRIVATE 2. SEMMAT02

 008 S/S 23 (23 ) 0001000000000000 SCHT/SEM INTACT: 2. PRIVATE 3. SEMMAT03

 009 S/S 24 (24 ) 0000010000000000 SCHT/SEM INTACT: 2. PRIVATE 4. SEMMAT04

 010 S/S 25 (25 ) 0000000100000000 SCHT/SEM INTACT: 2. PRIVATE 5. SEMMAT-?

 011 S/S 31 (31 ) 00000000-1-1-1-1 SCHT/SEM INTACT: 3. CATHOLIC 1. SEMMAT01

 012 S/S 32 (32 ) 0000000001000000 SCHT/SEM INTACT: 3. CATHOLIC 2. SEMMAT02

 013 S/S 33 (33 ) 0000000000010000 SCHT/SEM INTACT: 3. CATHOLIC 3. SEMMAT03

 014 S/S 34 (34 ) 0000000000000100 SCHT/SEM INTACT: 3. CATHOLIC 4. SEMMAT04

 015 S/S 35 (35 ) 0000000000000001 SCHT/SEM INTACT: 3. CATHOLIC 5. SEMMAT-?

 CONDITIONING VARIABLE ID: BACK0071

 DESCRIPTION: SAMPLE TYPE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: SUBSAMP TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 SAMP S1 (01 ) 00 SAMPLE S1

 002 SAMP S2 (02 ) 10 SAMPLE S2

 003 SAMP S3 (03 ) 01 SAMPLE S3

 CONDITIONING VARIABLE ID: BACK0072

 DESCRIPTION: INTERACTION: SAMPLE BY RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: /RAC

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 12

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 /R 11 (11 ) 010101010101 /RAC INTACT: 1. SAMP S1 1. WHI/AI/O

 002 /R 12 (12 ) -10000-10000 /RAC INTACT: 1. SAMP S1 2. BLACK

 003 /R 13 (13 ) 00-10000-100 /RAC INTACT: 1. SAMP S1 3. HISPANIC

 004 /R 14 (14 ) 0000-10000-1 /RAC INTACT: 1. SAMP S1 4. ASIAN

 005 /R 21 (21 ) -1-1-1000000 /RAC INTACT: 2. SAMP S2 1. WHI/AI/O

 006 /R 22 (22 ) 010000000000 /RAC INTACT: 2. SAMP S2 2. BLACK
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

488

 007 /R 23 (23 ) 000100000000 /RAC INTACT: 2. SAMP S2 3. HISPANIC

 008 /R 24 (24 ) 000001000000 /RAC INTACT: 2. SAMP S2 4. ASIAN

 009 /R 31 (31 ) 000000-1-1-1 /RAC INTACT: 3. SAMP S3 1. WHI/AI/O

 010 /R 32 (32 ) 000000010000 /RAC INTACT: 3. SAMP S3 2. BLACK

 011 /R 33 (33 ) 000000000100 /RAC INTACT: 3. SAMP S3 3. HISPANIC

 012 /R 34 (34 ) 000000000001 /RAC INTACT: 3. SAMP S3 4. ASIAN

 CONDITIONING VARIABLE ID: BACK0073

 DESCRIPTION: REPORTING SAMPLE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: RPTSAMP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 RPTSAMP (01 ) 0 YES

 002 RPT NO (02 ) 1 NO

 CONDITIONING VARIABLE ID: BACK0074

 DESCRIPTION: HOW OFTEN USE A HOME COMPUTER FOR SCHOOLWORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B009301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B009301A (01 ) 00000 ALMOST EVERY DAY

 002 B009301B (02 ) 10000 ONCE OR TWICE A WEEK

 003 B009301C (03 ) 01000 ONCE OR TWICE A MTH

 004 B009301D (04 ) 00100 NEVER OR HARDLY EVER

 005 B009301E (05 ) 00010 NO COMPUTER AT HOME

 006 B009301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: BACK0075

 DESCRIPTION: HOW SAFE DO YOU FEEL AT SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B009401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B009401A (01 ) 0000 VERY SAFE

 002 B009401B (02 ) 1000 SOMEWHAT SAFE

 003 B009401C (03 ) 0100 SOMEWHAT UNSAFE

 004 B009401D (04 ) 0010 VERY UNSAFERDLY EVER

 005 B009401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0076

 DESCRIPTION: DOES YOUR STEP/MOTHER WORK AT A JOB FOR PAY?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: B009501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 B009501A (01 ) 000 YES

 002 B009501B (02 ) 100 NO

 003 B009501C (03 ) 010 DON’T LIVE W/ MOTHER

 004 B009501M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: BACK0077

 DESCRIPTION: DOES YOUR STEP/FATHER WORK AT A JOB FOR PAY?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: B009502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 B009502A (01 ) 000 YES

 002 B009502B (02 ) 100 NO

 003 B009502C (03 ) 010 DON’T LIVE W/ FATHER

 004 B009502M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0002

 DESCRIPTION: HOW OFTEN DO YOU DO MATH PROBLEMS FROM TEXTBOOK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812701A (01 ) 0000 ALMOST EVERY DAY

 002 M812701B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812701C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812701D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812701M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0003

 DESCRIPTION: HOW OFTEN DO YOU DO MATH PROBLEMS ON WORK SHEETS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

489

 001 M812702A (01 ) 0000 ALMOST EVERY DAY

 002 M812702B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812702C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812701D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812701M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0004

 DESCRIPTION: HOW OFTEN SOLVE MATH PROBLEMS WITH PARTNER/GROUP?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812703A (01 ) 0000 ALMOST EVERY DAY

 002 M812703B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812703C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812703D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812703M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0005

 DESCRIPTION: HOW OFTEN IN MATH WORK W/ RULERS, GEOM SHAPES?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: M812704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812704A (01 ) 0000 ALMOST EVERY DAY

 002 M812704B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812704C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812704D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812704M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0006

 DESCRIPTION: HOW OFTEN WRITE ABOUT SOLVING A MATH PROBLEM?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812705A (01 ) 0000 ALMOST EVERY DAY

 002 M812705B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812705C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812705D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812705M (M ) 0001 MISSING RDLY EVER

 CONDITIONING VARIABLE ID: SUBJ0007

 DESCRIPTION: HOW OFTEN DO YOU TAKE MATHEMATICS TESTS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812706 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812706A (01 ) 0000 ALMOST EVERY DAY

 002 M812706B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812706C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812706D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812706M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0008

 DESCRIPTION: HOW OFTEN DO YOU TALK TO THE CLASS RE MATH WORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812707 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812707A (01 ) 0000 ALMOST EVERY DAY

 002 M812707B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812707C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812707D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812707M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0009

 DESCRIPTION: HOW OFTEN DO 10 OR MORE MATH PROBLEMS BY YOURSELF?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812708 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812708A (01 ) 0000 ALMOST EVERY DAY

 002 M812708B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812708C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812708D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812708M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0010

 DESCRIPTION: HOW OFTEN DISCUSS MATH PROBLEMS W/ OTHER STUDENTS?
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

490

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812709 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812709A (01 ) 0000 ALMOST EVERY DAY

 002 M812709B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812709C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812709D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812709M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0011

 DESCRIPTION: HOW OFTEN DO YOU USE A COMPUTER FOR MATH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812710 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812710A (01 ) 0000 ALMOST EVERY DAY

 002 M812710B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812710C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812710D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812710M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0012

 DESCRIPTION: HOW OFTEN DO YOU USE A CALCULATOR FOR MATH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812711 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812711A (01 ) 0000 ALMOST EVERY DAY

 002 M812711B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812711C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812711D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812711M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0013

 DESCRIPTION: DO YOU HAVE CALCULATOR TO USE WITH MATH HOMEWORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 M811201Y (01 ) 00 YES

 002 M811201N (02 ) 10 NODECIDED

 003 M811201M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0014

 DESCRIPTION: HOW OFTEN USE A CALCULATOR FOR MATH CLASSWORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812001A (01 ) 0000 ALMOST EVERY DAY

 002 M812001B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812001C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812001D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812001M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0015

 DESCRIPTION: HOW OFTEN USE A CALCULATOR FOR MATH HOMEWORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812002A (01 ) 0000 ALMOST EVERY DAY

 002 M812002B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812001C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812001D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812001M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0016

 DESCRIPTION: HOW OFTEN USE A CALCULATOR FOR MATH TESTS/QUIZZES?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812003A (01 ) 0000 ALMOST EVERY DAY

 002 M812003B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812003C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812003D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812003M (M ) 0001 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

491

 CONDITIONING VARIABLE ID: SUBJ0017

 DESCRIPTION: HOW MUCH TIME SPEND DAILY ON MATH HOMEWORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 M812101A (01 ) 0000000 NOT TAKING THIS YEAR

 002 M812101B (02 ) 1000000 NONE

 003 M812101C (03 ) 0100000 15 MINUTES

 004 M812101D (04 ) 0010000 30 MINUTES

 005 M812101E (05 ) 0001000 45 MINUTES

 006 M812101F (06 ) 0000100 ONE HOUR

 007 M812101G (07 ) 0000010 MORE THAN ONE HOUR

 008 M812101M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0018

 DESCRIPTION: GET MATH HELP FROM SPECIAL TEACHER/AIDE/TUTOR?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: M811401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 M811401Y (01 ) 00 YES

 002 M811401N (02 ) 10 NODECIDED

 003 M811401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0019

 DESCRIPTION: AGREE/DISAGREE: I LIKE MATH

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: M811101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 M811101A (01 ) 000 AGREE

 002 M811101B (02 ) 100 UNDECIDED

 003 M811101C (03 ) 010 DISAGREE OR LESS

 004 M811101M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0020

 DESCRIPTION: AGREE/DISAGREE: I AM GOOD AT MATH

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: M811103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 M811103A (01 ) 000 AGREE

 002 M811103B (02 ) 100 UNDECIDED

 003 M811103C (03 ) 010 DISAGREE OR LESS

 004 M811103M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0021

 DESCRIPTION: HOW MUCH AGREE-UNDERSTAND MOST OF MATH CLASS?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: M811106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 M811106A (01 ) 000 AGREE

 002 M811106B (02 ) 100 UNDECIDED

 003 M811106C (03 ) 010 DISAGREE OR LESS

 004 M811106M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0022

 DESCRIPTION: HOW MUCH AGREE-ONLY 1 CORRECT WAY TO SOLVE PROB?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: M811109 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 M811109A (01 ) 000 AGREE

 002 M811109B (02 ) 100 UNDECIDED

 003 M811109C (03 ) 010 DISAGREE OR LESS

 004 M811109M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0023

 DESCRIPTION: HOW MUCH AGREE-LEARNING MATH IS MEMORIZING FACTS?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: M811107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 M811107A (01 ) 000 AGREE

 002 M811107B (02 ) 100 UNDECIDED

 003 M811107C (03 ) 010 DISAGREE OR LESS

 004 M811107M (M ) 001 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

492

 CONDITIONING VARIABLE ID: SUBJ0024

 DESCRIPTION: AGREE/DISAGREE: MATH USED FOR SOLVING PROBLEMS

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: M811105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 M811105A (01 ) 000 AGREE

 002 M811105B (02 ) 100 UNDECIDED

 003 M811105C (03 ) 010 DISAGREE OR LESS

 004 M811105M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0025

 DESCRIPTION: HOW MUCH AGREE-IF CHOICE I WOULDN’T STUDY MORE?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: M811108 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 M811108A (01 ) 000 AGREE

 002 M811108B (02 ) 100 UNDECIDED

 003 M811108C (03 ) 010 DISAGREE OR LESS

 004 M811108M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0026

 DESCRIPTION: HOW MUCH AGREE-EVERYONE CAN DO WELL IF TRY?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: M811110 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 M811110A (01 ) 000 AGREE

 002 M811110B (02 ) 100 UNDECIDED

 003 M811110C (03 ) 010 DISAGREE OR LESS

 004 M811110M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0027

 DESCRIPTION: ABOUT HOW MANY QUESTIONS RIGHT ON TEST?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: MM00101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 MM00101A (01 ) 0000 ALMOST ALL

 002 MM00101B (02 ) 1000 MORE THAN HALF

 003 MM00101C (03 ) 0100 ABOUT HALF

 004 MM00101D (04 ) 0010 LESS THAN HALF

 005 MM00101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0028

 DESCRIPTION: HOW HARD TEST COMPARED TO THOSE IN SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: MM00201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 MM00201A (01 ) 0000 MUCH HARDER

 002 MM00201B (02 ) 1000 HARDER THAN OTHERS

 003 MM00201C (03 ) 0100 ABOUT THE SAME

 004 MM00201D (04 ) 0010 EASIER THEN OTHES

 005 MM00201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0029

 DESCRIPTION: HOW HARD DID YOU TRY ON TEST COMPARED TO OTHERS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: MM00301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 MM00301A (01 ) 0000 MUCH HARDER

 002 MM00301B (02 ) 1000 HARDER THAN OTHERS

 003 MM00301C (03 ) 0100 ABOUT AS HARD

 004 MM00301D (04 ) 0010 NOT AS HARD AS OTHER

 005 MM00301M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0030

 DESCRIPTION: HOW IMPORTANT WAS IT YOU DO WELL ON THIS TEST?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: MM00401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 MM00401A (01 ) 0000 VERY IMPORTANT

 002 MM00401B (02 ) 1000 IMPORTANT

 003 MM00401C (03 ) 0100 SOMEWHAT IMPORTANT

 004 MM00401D (04 ) 0010 NOT VERY IMPORTANTER

 005 MM00401M (M ) 0001 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

493

 CONDITIONING VARIABLE ID: SUBJ0031

 DESCRIPTION: HOW OFTEN HAD TO WRITE LONG ANSWERS TO QSTS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: MM00501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 MM00501A (01 ) 0000 AT LEAST ONCE A WEEK

 002 MM00501B (02 ) 1000 ONCE OR TWICE A MNTH

 003 MM00501C (03 ) 0100 ONCE OR TWICE A YEAR

 004 MM00501D (04 ) 0010 NEVERERY IMPORTANTER

 005 MM00501M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0078

 DESCRIPTION: DESCRIBE YOUR OVERALL GRADES SINCE 6TH GRADE

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: B009701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 B009701A (01 ) 000000 MOSTLY A’S

 002 B009701B (02 ) 100000 MOSTLY B’S

 003 B009701C (03 ) 010000 MOSTLY C’S

 004 B009701D (04 ) 001000 MOSTLY D’S

 005 B009701E (05 ) 000100 MOSTLY BELOW D’S

 006 B009701F (06 ) 000010 CLASSES NOT GRADED

 007 B009701M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0079

 DESCRIPTION: HOW FAR IN SCHOOL DO YOU THINK YOU WILL GO?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: B009801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 NO HS (01 ) 000000 NOT FINISH HS

 002 GRAD HS (02 ) 100000 GRADUATE HS

 003 ED > HS (03 ) 010000 SOME ED PAST HS

 004 GRAD CLG (04 ) 001000 GRADUATE COLLEGE

 005 GRAD SCH (05 ) 000100 GO GRADUATE SCHOOL

 006 IDK (06 ) 000010 I DON’T KNOWGRADED

 007 MISSING (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0080

 DESCRIPTION: DOES YOUR STEP/MOTHER WORK AT A JOB FOR PAY?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B009601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B009601A (01 ) 0000 YES, FULL-TIME

 002 B009601B (02 ) 1000 YES, PART-TIME

 003 B009601C (03 ) 0100 NO

 004 B009601D (04 ) 0010 DON’T LIVE W/ MOTHER

 005 B009601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0081

 DESCRIPTION: DOES YOUR STEP/FATHER WORK AT A JOB FOR PAY?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B009602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B009601A (01 ) 0000 YES, FULL-TIME

 002 B009601B (02 ) 1000 YES, PART-TIME

 003 B009601C (03 ) 0100 NO

 004 B009601D (04 ) 0010 DON’T LIVE W/ FATHER

 005 B009601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0032

 DESCRIPTION: WORK W/ MEAS. INSTRUMENTS/GEOM SOLIDS FOR MATH?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812712 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812712A (01 ) 0000 ALMOST EVERY DAY

 002 M812712B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812712C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812712D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812712M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0033

 DESCRIPTION: HOW OFTEN WRITE REPORTS OR DO MATH PROJECTS?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

494

 NAEP ID: M812713 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M812713A (01 ) 0000 ALMOST EVERY DAY

 002 M812713B (02 ) 1000 ONCE OR TWICE A WEEK

 003 M812713C (03 ) 0100 ONCE OR TWICE MONTH

 004 M812713D (04 ) 0010 NEVER OR HARDLY EVER

 005 M812713M (M ) 0001 MISSING RDLY EVER

 CONDITIONING VARIABLE ID: SUBJ0034

 DESCRIPTION: IS THERE A PORTFOLIO W/ YOUR MATH WORK IN IT?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 M812201Y (01 ) 00 YES

 002 M812201N (02 ) 10 NONE

 003 M812201M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0035

 DESCRIPTION: DO YOU USE A SCIENTIFIC CALCULATOR FOR MATH WORK?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 M812301Y (01 ) 00 YES

 002 M812301N (02 ) 10 NONE

 003 M812301M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0036

 DESCRIPTION: DO YOU USE A GRAPHING CALCULATOR FOR MATH WORK?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 M812401Y (01 ) 00 YES

 002 M812401N (02 ) 10 NONE

 003 M812401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0037

 DESCRIPTION: WHAT KIND OF MATH CLASS WILL YOU TAKE IN 9TH GR?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: MATEXP TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 9

 001 NO MATH (01 ) 000000000 NOT EXPECT TO TAKE

 002 BASIC (02 ) 100000000 BASIC,GEN,BUSINESS

 003 APPLIED (03 ) 010000000 APPLIED MATH

 004 PREALG (04 ) 001000000 PREALGEBRA

 005 ALG 1 (05 ) 000100000 ALGEBRA 1/ELEM ALG

 006 GEOMETRY (06 ) 000010000 GEOMETRY

 007 INT/SEQ (07 ) 000001000 INTERGATED/SEQUENTAL

 008 OTHER (08 ) 000000100 OTHER MATH CLASS

 009 IDK (09 ) 000000010 I DON’T KNOW

 010 MISSING (M ) 000000001 MISSING W

 CONDITIONING VARIABLE ID: SUBJ0038

 DESCRIPTION: DO YOU AGREE: I LIKE MATH

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M810701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 M812701A (01 ) 00000 STRONGLY AGREE

 002 M812701B (02 ) 10000 AGREE

 003 M812701C (03 ) 01000 UNDECIDED

 004 M812701D (04 ) 00100 DISAGREE

 005 M812701E (05 ) 00010 STRONGLY DISAGREE

 006 M812701M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0039

 DESCRIPTION: DO YOU AGREE: I AM GOOD IN MATH

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M810703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 M810701A (01 ) 00000 STRONGLY AGREE

 002 M810701B (02 ) 10000 AGREE

 003 M810701C (03 ) 01000 UNDECIDED

 004 M810701D (04 ) 00100 DISAGREE

 005 M810701E (05 ) 00010 STRONGLY DISAGREE

 006 M810701M (M ) 00001 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

495

 CONDITIONING VARIABLE ID: SUBJ0040

 DESCRIPTION: AGREE/DISAGREE: UNDERSTAND MOST OF MATH CLASS

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M810707 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 M810707A (01 ) 00000 STRONGLY AGREE

 002 M810707B (02 ) 10000 AGREE

 003 M810707C (03 ) 01000 UNDECIDED

 004 M810707D (04 ) 00100 DISAGREE

 005 M810707E (05 ) 00010 STRONGLY DISAGREE

 006 M810707M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0041

 DESCRIPTION: AGREE/DISAGREE: ONLY ONE WAY TO SOLVE MATH PROBLEM

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M810709 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 M810709A (01 ) 00000 STRONGLY AGREE

 002 M810709B (02 ) 10000 AGREE

 003 M810709C (03 ) 01000 UNDECIDED

 004 M810709D (04 ) 00100 DISAGREE

 005 M810709E (05 ) 00010 STRONGLY DISAGREE

 006 M810709M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0042

 DESCRIPTION: AGREE/DISAGREE: MATH IS MOSTLY MEMORIZING FACTS

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M810708 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 M810708A (01 ) 00000 STRONGLY AGREE

 002 M810708B (02 ) 10000 AGREE

 003 M810708C (03 ) 01000 UNDECIDED

 004 M810708D (04 ) 00100 DISAGREE

 005 M810708E (05 ) 00010 STRONGLY DISAGREE

 006 M810708M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0043

 DESCRIPTION: AGREE/DISAGREE: CONCEPTS ARE AS IMPORTANT AS OPER

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M810710 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 M810710A (01 ) 00000 STRONGLY AGREE

 002 M810710B (02 ) 10000 AGREE

 003 M810710C (03 ) 01000 UNDECIDED

 004 M810710D (04 ) 00100 DISAGREE

 005 M810710E (05 ) 00010 STRONGLY DISAGREE

 006 M810710M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0044

 DESCRIPTION: DO YOU AGREE: MATH USEFUL/SOLVING EVERYDAY PROBLEM

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M810705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 M810705A (01 ) 00000 STRONGLY AGREE

 002 M810705B (02 ) 10000 AGREE

 003 M810705C (03 ) 01000 UNDECIDED

 004 M810705D (04 ) 00100 DISAGREE

 005 M810705E (05 ) 00010 STRONGLY DISAGREE

 006 M810705M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0045

 DESCRIPTION: AGREE/DISAGREE: WOULD NOT STUDY MORE MATH

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M810706 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 M810706A (01 ) 00000 STRONGLY AGREE

 002 M810706B (02 ) 10000 AGREE

 003 M810706C (03 ) 01000 UNDECIDED

 004 M810706D (04 ) 00100 DISAGREE

 005 M810706E (05 ) 00010 STRONGLY DISAGREE

 006 M810706M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0046

 DESCRIPTION: AGREE/DISAGREE: ALL CAN DO WELL IN MATH IF TRY
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

496

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: M810711 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 M810711A (01 ) 00000 STRONGLY AGREE

 002 M810711B (02 ) 10000 AGREE

 003 M810711C (03 ) 01000 UNDECIDED

 004 M810711D (04 ) 00100 DISAGREE

 005 M810711E (05 ) 00010 STRONGLY DISAGREE

 006 M810711M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: BACK0082

 DESCRIPTION: SINCE KDG, GRADES ATTENDED IN THIS STATE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B008301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B008301A (01 ) 00000 LESS THAN 1 GRADE

 002 B008301B (02 ) 10000 1-2 GRADES

 003 B008301C (03 ) 01000 3-5 GRADES

 004 B008301D (04 ) 00100 6-9 GRADES

 005 B008301E (05 ) 00010 MORE THAN 9 GRADES

 006 B008301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: BACK0083

 DESCRIPTION: DESCRIBE YOUR OVERALL GRADES SINCE 9TH GRADE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B009901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 B009901A (01 ) 000000 MOSTLY A’S

 002 B009901B (02 ) 100000 MOSTLY B’S

 003 B009901C (03 ) 010000 MOSTLY C’S

 004 B009901D (04 ) 001000 MOSTLY D’S

 005 B009901E (05 ) 000100 MOSTLY BELOW D’S

 006 B009901F (06 ) 000010 CLASSES NOT GRADED

 007 B009901M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0084

 DESCRIPTION: WHICH BEST DESCRIBES YOUR HIGH SCHOOL PROGRAM

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B008501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 GENERAL (01 ) 0000 GENERAL

 002 ACADEMIC (02 ) 1000 ACADEMIC/COLLEGE PRE

 003 VOC/TECH (03 ) 0100 VOCATIONAL OR TECHN

 004 OTHER (04 ) 0010 OTHERRADES

 005 MISSING (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0085

 DESCRIPTION: WHAT WILL TAKE LARGEST AMT. OF TIME AFTER HIGH-SCH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B005501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 WORKING (01 ) 000000 WORKING FULL-TIME

 002 VOC COLL (02 ) 100000 VOCATIONAL COLLEGE

 003 2 YR COL (03 ) 010000 ATTEND 2 YR COLLEGE

 004 4 YR COL (04 ) 001000 ATTEND 4 YEAR COLLEG

 005 MILITARY (05 ) 000100 MILITARY SERVICE

 006 OTHER (06 ) 000010 OTHERS

 007 MISSING (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0086

 DESCRIPTION: KIND OF WORK DONE BY YOUR MOTHER/FEMALE GUARDIAN?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: MOTHOCC TOTAL NUMBER OF SPECIFIED CONTRASTS: 17

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 16

 001 NOT PRE (01,M ) 0000000000000000 NOT PRESENT IN HOUSEHOLD

 002 OFFICE W (02 ) 1000000000000000 OFFICE WORKER

 003 TECHNIC (03 ) 0100000000000000 TECHNICAL

 004 PROT SRV (04 ) 0010000000000000 PROTECTIVE SERVICE

 005 SALES (05 ) 0001000000000000 SALES

 006 OWNER (06 ) 0000100000000000 OWNER

 007 SERV WRK (07 ) 0000010000000000 SERVICE WORKER

 008 SKILL T (08 ) 0000001000000000 SKILLED TRADESPERSON

 009 LABORER (09 ) 0000000100000000 LABORER

 010 OPERATOR (10 ) 0000000010000000 OPERATOR

 011 FARMER (11 ) 0000000001000000 FARMER/FARM MANAGER
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

497

 012 HOMEMAKE (12 ) 0000000000100000 HOMEMAKER

 013 MANAGER (13 ) 0000000000010000 MANAGER

 014 MILITARY (14 ) 0000000000001000 MILITARY

 015 TEACHER (15 ) 0000000000000100 SCHOOL TEACHER

 016 PROFESS1 (16 ) 0000000000000010 PROFESSIONAL 1

 017 PROFESS2 (17 ) 0000000000000001 PROFESSIONAL 2

 CONDITIONING VARIABLE ID: BACK0087

 DESCRIPTION: KIND OF WORK DONE BY YOUR MOTHER/FEMALE GUARDIAN?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B011A01 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 B011A01M (M ) 0 MISSING

 002 B011A01N (@ ) 1 NOT PRESENT IN HOUSEHOLD

 CONDITIONING VARIABLE ID: BACK0088

 DESCRIPTION: KIND OF WORK DONE BY YOUR FATHER/MALE GUARDIAN?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: FATHOCC TOTAL NUMBER OF SPECIFIED CONTRASTS: 17

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 16

 001 NOT PRES (01,M ) 0000000000000000 NOT PRESENT IN HOUSEHOLD

 002 OFFICE (02 ) 1000000000000000 OFFICE WORKER

 003 TECHNIC (03 ) 0100000000000000 TECHNICAL

 004 PROT SRV (04 ) 0010000000000000 PROTECTIVE SERVICES

 005 SALES (05 ) 0001000000000000 SALES

 006 OWNER (06 ) 0000100000000000 OWNER

 007 SERV WRK (07 ) 0000010000000000 SERVICE WORKER

 008 SKILL TR (08 ) 0000001000000000 SKILLED TRADEPERSON

 009 LABORER (09 ) 0000000100000000 LABORER

 010 OPERATOR (10 ) 0000000010000000 OPERATOR

 011 FARMER (11 ) 0000000001000000 FARMER/FARM MANAGER

 012 HOMEMAKE (12 ) 0000000000100000 HOMEMAKER

 013 MANAGER (13 ) 0000000000010000 MANAGER

 014 MILITARY (14 ) 0000000000001000 MILITARY

 015 TEACHER (15 ) 0000000000000100 SCHOOL TEACHER

 016 PROFESS1 (16 ) 0000000000000010 PROFESSIONAL 1

 017 PROFESS2 (17 ) 0000000000000001 PROFESSIONAL 2

 CONDITIONING VARIABLE ID: BACK0089

 DESCRIPTION: KIND OF WORK DONE BY YOUR FATHER/MALE GUARDIAN?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B012A02 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 B012A02M (M ) 0 MISSING

 002 B012A02N (@ ) 1 NOT PRESENT IN HOUSEHOLD

 CONDITIONING VARIABLE ID: SUBJ0047

 DESCRIPTION: ARE YOU TAKING MATH CLASSES THIS YEAR

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M810901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 M810901Y (01 ) 00 YES

 002 M810901N (02 ) 10 NO

 003 M810901M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0048

 DESCRIPTION: WHAT GRADE DID YOU TAKE 1ST YR ALGEBRA?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 M811801A (01 ) 00000 HAVNT TAKEN 1ST YEAR

 002 M811801B (02 ) 10000 BEFORE 9TH GRADE

 003 M811801C (03 ) 01000 9TH GRADE

 004 M811801D (04 ) 00100 10TH GRADE

 005 M811801E (05 ) 00010 11TH OR 12TH GRADE

 006 M811801M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0049

 DESCRIPTION: HOW LONG HAVE YOU TAKEN GENERAL MATH (9-12)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M811001A (01 ) 0000 MORE THAN 1 YEAR

 002 M811001B (02 ) 1000 1 SCHOOL YEAR

 003 M811001C (03 ) 0100 1/2 YEAR OR LESS
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

498

 004 M811001D (04 ) 0010 NOT STUDIED

 005 M811001M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0050

 DESCRIPTION: HOW LONG HAVE YOU TAKEN BUSINESS OR CONSUMER MATH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M811002A (01 ) 0000 MORE THAN 1 YEAR

 002 M811002B (02 ) 1000 1 SCHOOL YEAR

 003 M811002C (03 ) 0100 1/2 YEAR OR LESS

 004 M811002D (04 ) 0010 NOT STUDIED

 005 M811002M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0051

 DESCRIPTION: HOW LONG TAKEN APPLIED MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811014 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M811014A (01 ) 0000 MORE THAN 1 YEAR

 002 M811014B (02 ) 1000 1 SCHOOL YEAR

 003 M811014C (03 ) 0100 1/2 YEAR OR LESS

 004 M811014D (04 ) 0010 NOT STUDIED

 005 M811014M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0052

 DESCRIPTION: HOW LONG HAVE YOU TAKEN INTRO (PRE) - ALGEBRA

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M811014A (01 ) 0000 MORE THAN 1 YEAR

 002 M811014B (02 ) 1000 1 SCHOOL YEAR

 003 M811014C (03 ) 0100 1/2 YEAR OR LESS

 004 M811014D (04 ) 0010 NOT STUDIED

 005 M811014M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0053

 DESCRIPTION: HOW LONG HAVE YOU STUDIED FIRST-YEAR ALGEBRA

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M811004A (01 ) 0000 MORE THAN 1 YEAR

 002 M811004B (02 ) 1000 1 SCHOOL YEAR

 003 M811004C (03 ) 0100 1/2 YEAR OR LESS

 004 M811004D (04 ) 0010 NOT STUDIED

 005 M811004M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0054

 DESCRIPTION: HOW LONG HAVE YOU TAKEN GEOMETRY

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M811005A (01 ) 0000 MORE THAN 1 YEAR

 002 M811005B (02 ) 1000 1 SCHOOL YEAR

 003 M811005C (03 ) 0100 1/2 YEAR OR LESS

 004 M811005D (04 ) 0010 NOT STUDIED

 005 M811005M (M ) 0001 NOT STUDIED

 CONDITIONING VARIABLE ID: SUBJ0055

 DESCRIPTION: HOW LONG HAVE YOU TAKEN SECOND-YEAR ALGEBRA

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M811006A (01 ) 0000 MORE THAN 1 YEAR

 002 M811006B (02 ) 1000 1 SCHOOL YEAR

 003 M811006C (03 ) 0100 1/2 YEAR OR LESS

 004 M811006D (04 ) 0010 NOT STUDIED

 005 M811006M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0056

 DESCRIPTION: HOW LONG HAVE YOU TAKEN TRIGONOMETRY

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

499

 001 M811007A (01 ) 0000 MORE THAN 1 YEAR

 002 M811007B (02 ) 1000 1 SCHOOL YEAR

 003 M811007C (03 ) 0100 1/2 YEAR OR LESS

 004 M811007D (04 ) 0010 NOT STUDIED

 005 M811007M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0057

 DESCRIPTION: HOW LONG HAVE YOU TAKEN PRE-CALC (3RD YR ALGEBRA)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811008 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M811008A (01 ) 0000 MORE THAN 1 YEAR

 002 M811008B (02 ) 1000 1 SCHOOL YEAR

 003 M811008C (03 ) 0100 1/2 YEAR OR LESS

 004 M811008D (04 ) 0010 NOT STUDIED

 005 M811008M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0058

 DESCRIPTION: HOW LONG TAKEN UNIFIED/INTEGRTD/SEQ MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811012 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M811012A (01 ) 0000 MORE THAN 1 YEAR

 002 M811012B (02 ) 1000 1 SCHOOL YEAR

 003 M811012C (03 ) 0100 1/2 YEAR OR LESS

 004 M811012D (04 ) 0010 NOT STUDIED

 005 M811012M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0059

 DESCRIPTION: HOW LONG HAVE YOU TAKEN PROBABILITY OR STATISTICS

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811009 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M811009A (01 ) 0000 MORE THAN 1 YEAR

 002 M811009B (02 ) 1000 1 SCHOOL YEAR

 003 M811009C (03 ) 0100 1/2 YEAR OR LESS

 004 M811009D (04 ) 0010 NOT STUDIED

 005 M811009M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0060

 DESCRIPTION: HOW LONG HAVE YOU TAKEN CALCULUS

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M811011 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 M811011A (01 ) 0000 MORE THAN 1 YEAR

 002 M811011B (02 ) 1000 1 SCHOOL YEAR

 003 M811011C (03 ) 0100 1/2 YEAR OR LESS

 004 M811011D (04 ) 0010 NOT STUDIED

 005 M811011M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0061

 DESCRIPTION: IN/HAVE TAKEN MATH ADVANCED PLACEMENT CLASS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: M812801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 M812801Y (01 ) 00 YES

 002 M812801N (02 ) 10 NOCE OR TWICE A WEEK

 003 M812801M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0003

 DESCRIPTION: BEST DESCRIBES HOW 4TH GR ARE ORGANIZED?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C030901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C030901A (01 ) 000 SELF CONTAINED

 002 C030901B (02 ) 100 DEPARTMENTALIZED

 003 C030901C (03 ) 010 REGROUPED

 004 C030901M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0004

 DESCRIPTION: 4TH GRADERS ASSIGNED BY ABILITY/ACHIEVEMENT LEVEL?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

500

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037101A (01 ) 0000 YES, BY MATH ABILITY

 002 C037101B (02 ) 1000 YES, READING ABILITY

 003 C037101C (03 ) 0100 YES, GENERAL ABILITY

 004 C037101N (04 ) 0010 NO

 005 C037101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0005

 DESCRIPTION: HOW OFTEN IS 4TH-GRADER INSTRUCTED IN MATH?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C031212 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C031212A (01 ) 00000 EVERY DAY

 002 C031212B (02 ) 10000 THREE OR FOUR/WEEK

 003 C031212C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C031212D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C031212E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C031212M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0006

 DESCRIPTION: HOW OFTEN IS 4TH-GRADER INSTRUCTED IN SCIENCE?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C031205 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C031205A (01 ) 00000 EVERY DAY

 002 C031205B (02 ) 10000 THREE OR FOUR/WEEK

 003 C031205C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C031205D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C031205E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C031205M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0007

 DESCRIPTION: HOW OFTEN IS 4TH-GRADER INSTRUCTED IN READING?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C031213 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C031213A (01 ) 00000 EVERY DAY

 002 C031213B (02 ) 10000 THREE OR FOUR/WEEK

 003 C031213C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C031213D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C031213E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C031213M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0008

 DESCRIPTION: HOW OFTEN IS 4TH-GRADER INSTRUCTED IN ARTS?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C031214 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C031214A (01 ) 00000 EVERY DAY

 002 C031214B (02 ) 10000 THREE OR FOUR/WEEK

 003 C031214C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C031214D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C031214E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C031214M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0009

 DESCRIPTION: HAS MATH BEEN IDENTIFIED AS A PRIORITY?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C031603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031603Y (01 ) 00 YES

 002 C031603N (02 ) 10 NO

 003 C031603M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0010

 DESCRIPTION: HAS SCIENCE BEEN IDENTIFIED AS A PRIORITY?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C031607 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031607Y (01 ) 00 YES

 002 C031607N (02 ) 10 NO

 003 C031607M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0011
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

501

 DESCRIPTION: HAS READING BEEN IDENTIFIED AS A PRIORITY?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C031601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031601Y (01 ) 00 YES

 002 C031601N (02 ) 10 NO

 003 C031601M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0012

 DESCRIPTION: HAS ARTS BEEN IDENTIFIED AS A PRIORITY?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C031610 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031610Y (01 ) 00 YES

 002 C031610N (02 ) 10 NO

 003 C031610M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0013

 DESCRIPTION: HAS SUBJECT INTEGRATION BEEN A PRIORITY?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C031606 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031606Y (01 ) 00 YES

 002 C031606N (02 ) 10 NO

 003 C031606M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0014

 DESCRIPTION: COMPUTERS AVAILABLE ALL THE TIME IN CLASSROOM?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035701Y (01 ) 00 YES

 002 C035701N (02 ) 10 NO

 003 C035701M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0015

 DESCRIPTION: COMPUTERS GROUPED IN SEPARATE LAB AND AVAILABLE?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035702Y (01 ) 00 YES

 002 C035702N (02 ) 10 NO

 003 C035702M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0016

 DESCRIPTION: COMPUTERS AVAILABLE TO BRING TO ROOM WHEN NEEDED?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035703Y (01 ) 00 YES

 002 C035703N (02 ) 10 NO

 003 C035703M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0017

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON MATH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037201Y (01 ) 0 YES

 002 C037201M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0018

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON SCIENCE?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037202Y (01 ) 0 YES

 002 C037202M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0019

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON MATH?
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

502

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037207 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037207Y (01 ) 0 YES

 002 C037207M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0020

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON ARTS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037204 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037204Y (01 ) 0 YES

 002 C037204M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0021

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON OTHER?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037205 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037205Y (01 ) 0 YES

 002 C037205M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0022

 DESCRIPTION: SCHOOL NOT A SPECIAL FOCUS SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037206 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037206Y (01 ) 0 YES

 002 C037206M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0023

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE MATH CURRICULUM?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037301Y (01 ) 0 YES

 002 C037301M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0024

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE SCIENCE CURRICULUM?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037302Y (01 ) 0 YES

 002 C037302M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0025

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE READING CURRICULUM?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037303Y (01 ) 0 YES

 002 C037303M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0026

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE ARTS CURRICULUM?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037304Y (01 ) 0 YES

 002 C037304M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0027

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE FOR NONE OF ABOVE?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037305Y (01 ) 0 YES

 002 C037305M (M ) 1 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

503

 CONDITIONING VARIABLE ID: SCHL0028

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR MATH?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037401Y (01 ) 0 YES

 002 C037401M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0029

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR SCIENCE?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037402Y (01 ) 0 YES

 002 C037402M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0030

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR READING?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037403Y (01 ) 0 YES

 002 C037403M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0031

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR ARTS?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037404Y (01 ) 0 YES

 002 C037404M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0032

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR OTHER?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037405Y (01 ) 0 YES

 002 C037405M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0033

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR NONE ABOVE?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037406Y (01 ) 0 YES

 002 C037406M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0034

 DESCRIPTION: 4TH GRADERS IN EXTRACURR ACTS FOR MATH?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037501Y (01 ) 0 YES

 002 C037501M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0035

 DESCRIPTION: 4TH GRADERS IN EXTRACURR ACTS FOR SCIENCE?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037502Y (01 ) 0 YES

 002 C037502M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0036

 DESCRIPTION: 4TH GRADERS IN EXTRACURR ACTS FOR READING?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

504

 001 C037503Y (01 ) 0 YES

 002 C037503M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0037

 DESCRIPTION: 4TH GRADERS IN EXTRACURR ACTS FOR ARTS?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037504Y (01 ) 0 YES

 002 C037504M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0038

 DESCRIPTION: 4TH GRADERS IN EXTRACURR ACTS FOR NONE OF ABOVE?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037505Y (01 ) 0 YES

 002 C037505M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0039

 DESCRIPTION: 4TH GRADERS IN SUMMER PROGRAMS IN MATH?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037601Y (01 ) 0 YES

 002 C037601M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0040

 DESCRIPTION: 4TH GRADERS IN SUMMER PROGRAMS IN SCIENCE?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037602Y (01 ) 0 YES

 002 C037602M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0041

 DESCRIPTION: 4TH GRADERS IN SUMMER PROGRAMS IN READING?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037603Y (01 ) 0 YES

 002 C037603M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0042

 DESCRIPTION: 4TH GRADERS IN SUMMER PROGRAMS IN ARTS?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037604Y (01 ) 0 YES

 002 C037604M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0043

 DESCRIPTION: 4TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037605Y (01 ) 0 YES

 002 C037605M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0044

 DESCRIPTION: WHICH BEST DESCRIBES PRIMARY WAY LIBRARY STAFFED?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C036601A (01 ) 0000 NO LIBRARY IN SCHOOL

 002 C036601B (02 ) 1000 LIBRARY-NO/VOL STAFF

 003 C036601C (03 ) 0100 PART-TIME STAFF

 004 C036601D (04 ) 0010 FULL-TIME STAFF

 005 C036601M (M ) 0001 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

505

 CONDITIONING VARIABLE ID: SCHL0045

 DESCRIPTION: INVOLVE PARENTS AS AIDES IN CLASSROOM?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032207 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C032207A (01 ) 000 YES, ROUTINELY

 002 C032207B (02 ) 100 YES, OCCASIONALLY

 003 C032207N (03 ) 010 NO

 004 C032207M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0046

 DESCRIPTION: HAVE PARENTS REVIEW/SIGN HOMEWORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032209 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C032209A (01 ) 000 YES, ROUTINELY

 002 C032209B (02 ) 100 YES, OCCASIONALLY

 003 C032209N (03 ) 010 NO

 004 C032209M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0047

 DESCRIPTION: ASSIGN HOMEWORK STUDENTS DO WITH PARENTS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032210 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C032210A (01 ) 000 YES, ROUTINELY

 002 C032210B (02 ) 100 YES, OCCASIONALLY

 003 C032210N (03 ) 010 NO

 004 C032210M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0048

 DESCRIPTION: HAVE A PARENT VOLUNTEER PROGRAM?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032211 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C032211A (01 ) 000 YES, ROUTINELY

 002 C032211B (02 ) 100 YES, OCCASIONALLY

 003 C032211N (03 ) 010 NO

 004 C032211M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0049

 DESCRIPTION: WHAT % OF PARENTS IN PARENT-TEACHER ORGS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037701A (01 ) 0000 0-25%

 002 C037701B (02 ) 1000 26-50%

 003 C037701C (03 ) 0100 51-75%

 004 C037701D (04 ) 0010 76-100%

 005 C037701M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0050

 DESCRIPTION: WHAT % OF PARENTS IN OPEN HOUSE/BACK SCHOOL NIGHT?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037702A (01 ) 0000 0-25%

 002 C037702B (02 ) 1000 26-50%

 003 C037702C (03 ) 0100 51-75%

 004 C037702D (04 ) 0010 76-100%

 005 C037702M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0051

 DESCRIPTION: WHAT % OF PARENTS IN PARENT-TEACHER COMFERENCES?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037703A (01 ) 0000 0-25%

 002 C037703B (02 ) 1000 26-50%

 003 C037703C (03 ) 0100 51-75%

 004 C037703D (04 ) 0010 76-100%

 005 C037703M (M ) 0001 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

506

 CONDITIONING VARIABLE ID: SCHL0052

 DESCRIPTION: WHAT % PARENTS INVOLVED MAKING CURRICULUM DECISION

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037704A (01 ) 0000 0-25%

 002 C037704B (02 ) 1000 26-50%

 003 C037704C (03 ) 0100 51-75%

 004 C037704D (04 ) 0010 76-100%

 005 C037704M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0053

 DESCRIPTION: WHAT % OF PARENTS IN VOLUNTEER PROGRAMS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037705A (01 ) 0000 0-25%

 002 C037705B (02 ) 1000 26-50%

 003 C037705C (03 ) 0100 51-75%

 004 C037705D (04 ) 0010 76-100%

 005 C037705M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0054

 DESCRIPTION: IS STUDENT ABSENTEEISM A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032402A (01 ) 0000 SERIOUS

 002 C032402B (02 ) 1000 MODERATE

 003 C032402C (03 ) 0100 MINOR

 004 C032402D (04 ) 0010 NOT A PROBLEM

 005 C032402M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0055

 DESCRIPTION: IS STUDENT TARDINESS A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032401A (01 ) 0000 SERIOUS

 002 C032401B (02 ) 1000 MODERATE

 003 C032401C (03 ) 0100 MINOR

 004 C032401D (04 ) 0010 NOT A PROBLEM

 005 C032401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0056

 DESCRIPTION: ARE PHYSICAL CONFLICTS A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032404A (01 ) 0000 SERIOUS

 002 C032404B (02 ) 1000 MODERATE

 003 C032404C (03 ) 0100 MINOR

 004 C032404D (04 ) 0010 NOT A PROBLEM

 005 C032404M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0057

 DESCRIPTION: IS TEACHER ABSENTEEISM A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032406A (01 ) 0000 SERIOUS

 002 C032406B (02 ) 1000 MODERATE

 003 C032406C (03 ) 0100 MINOR

 004 C032406D (04 ) 0010 NOT A PROBLEM

 005 C032406M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0058

 DESCRIPTION: ARE RACE/CULT. CONFLICTS A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032407 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032407A (01 ) 0000 SERIOUS

 002 C032407B (02 ) 1000 MODERATE
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

507

 003 C032407C (03 ) 0100 MINOR

 004 C032407D (04 ) 0010 NOT A PROBLEM

 005 C032407M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0059

 DESCRIPTION: IS STUDENT HEALTH A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032408 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032408A (01 ) 0000 SERIOUS

 002 C032408B (02 ) 1000 MODERATE

 003 C032408C (03 ) 0100 MINOR

 004 C032408D (04 ) 0010 NOT A PROBLEM

 005 C032408M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0060

 DESCRIPTION: IS LACK OF PARENT INVLMT A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032409 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032409A (01 ) 0000 SERIOUS

 002 C032409B (02 ) 1000 MODERATE

 003 C032409C (03 ) 0100 MINOR

 004 C032409D (04 ) 0010 NOT A PROBLEM

 005 C032409M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0061

 DESCRIPTION: IS STUD USE OF ALCOHOL A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032410 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032410A (01 ) 0000 SERIOUS

 002 C032410B (02 ) 1000 MODERATE

 003 C032410C (03 ) 0100 MINOR

 004 C032410D (04 ) 0010 NOT A PROBLEM

 005 C032410M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0062

 DESCRIPTION: IS STUDENT TOBACCO USE A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032411 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032411A (01 ) 0000 SERIOUS

 002 C032411B (02 ) 1000 MODERATE

 003 C032411C (03 ) 0100 MINOR

 004 C032411D (04 ) 0010 NOT A PROBLEM

 005 C032411M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0063

 DESCRIPTION: IS STUDENT DRUG USE A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032412 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032412A (01 ) 0000 SERIOUS

 002 C032412B (02 ) 1000 MODERATE

 003 C032412C (03 ) 0100 MINOR

 004 C032412D (04 ) 0010 NOT A PROBLEM

 005 C032412M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0064

 DESCRIPTION: ARE GANG ACTIVITIES A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032413 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032413A (01 ) 0000 SERIOUS

 002 C032413B (02 ) 1000 MODERATE

 003 C032413C (03 ) 0100 MINOR

 004 C032413D (04 ) 0010 NOT A PROBLEM

 005 C032413M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0065

 DESCRIPTION: IS STUDENT MISBEHAVIOR A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032414 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

508

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032414A (01 ) 0000 SERIOUS

 002 C032414B (02 ) 1000 MODERATE

 003 C032414C (03 ) 0100 MINOR

 004 C032414D (04 ) 0010 NOT A PROBLEM

 005 C032414M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0066

 DESCRIPTION: IS STUDENT CHEATING A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032415 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032415A (01 ) 0000 SERIOUS

 002 C032415B (02 ) 1000 MODERATE

 003 C032415C (03 ) 0100 MINOR

 004 C032415D (04 ) 0010 NOT A PROBLEM

 005 C032415M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0067

 DESCRIPTION: IS TEACHER MORALE POS. OR NEG.?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032502A (01 ) 0000 VERY POSITIVE

 002 C032502B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032502C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032502D (04 ) 0010 VERY NEGATIVE

 005 C032502M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0068

 DESCRIPTION: ARE STUDENT ATTITUDES TO ACADEMICS POS. OR NEG.?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032503A (01 ) 0000 VERY POSITIVE

 002 C032503B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032503C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032503D (04 ) 0010 VERY NEGATIVE

 005 C032503M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0069

 DESCRIPTION: IS PARENT SUPPORT FOR ACHIEVEMENT POS. OR NEG.?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032505A (01 ) 0000 VERY POSITIVE

 002 C032505B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032505C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032505D (04 ) 0010 VERY NEGATIVE

 005 C032505M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0070

 DESCRIPTION: IS REGARD FOR SCHOOL PROPERTY POS. OR NEG.?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032506A (01 ) 0000 VERY POSITIVE

 002 C032506B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032506C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032506D (04 ) 0010 VERY NEGATIVE

 005 C032506M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0071

 DESCRIPTION: % ABSENT ON AVERAGE DAY?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C033601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C033601A (01 ) 0000 0-2%

 002 C033601B (02 ) 1000 3-5%

 003 C033601C (03 ) 0100 6-10%

 004 C033601D (04 ) 0010 MORE THAN 10%

 005 C033601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0072
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

509

 DESCRIPTION: WHAT % OF TEACHERS ABSENT ON GIVEN DAY?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C036501A (01 ) 0000 0-2%

 002 C036501B (02 ) 1000 3-5%

 003 C036501C (03 ) 0100 6-10%

 004 C036501D (04 ) 0010 MORE THAN 10%

 005 C036501M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0073

 DESCRIPTION: % OF STUDS EROLLED AT START OF YR EROLLED AT END?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C037801A (01 ) 00000000 98-100%

 002 C037801B (02 ) 10000000 95-97%

 003 C037801C (03 ) 01000000 90-94%

 004 C037801D (04 ) 00100000 80-89%

 005 C037801E (05 ) 00010000 70-79%

 006 C037801F (06 ) 00001000 60-69%

 007 C037801G (07 ) 00000100 50-59%

 008 C037801H (08 ) 00000010 LESS THAN 50%

 009 C037801M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0074

 DESCRIPTION: % OF 4TH GRADERS HELD BACK & REPEATING 4TH GRADE?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: C037901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C037901A (01 ) 00000 0%

 002 C037901B (02 ) 10000 1-2%

 003 C037901C (03 ) 01000 3-5%

 004 C037901D (04 ) 00100 6-10%

 005 C037901E (05 ) 00010 MORE THAN 10%

 006 C037901M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0075

 DESCRIPTION: % OF FULL TIME TEACHERS LEFT BEFORE END OF YR?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C038001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C038001A (01 ) 0000000 0%

 002 C038001B (02 ) 1000000 1-2%

 003 C038001C (03 ) 0100000 3-5%

 004 C038001D (04 ) 0010000 6-10%

 005 C038001E (05 ) 0001000 11-15%

 006 C038001F (06 ) 0000100 16-20%

 007 C038001G (07 ) 0000010 MORE THAN 20%

 008 C038001M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0076

 DESCRIPTION: IS SCHOOL IN NATIONAL SCHOOL LUNCH PROGRAM?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C038301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C038301Y (01 ) 00 YES

 002 C038301N (02 ) 10 NO

 003 C038301M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0077

 DESCRIPTION: SCHOOL RECEIVE CHAP 1/TITLE 1 FUNDING?

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C038801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C038801Y (01 ) 00 YES

 002 C038801N (02 ) 10 NO

 003 C038801M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0078

 DESCRIPTION: DID PRINCIPAL FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

510

 001 C034101Y (01 ) 0 YES

 002 C034101M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0079

 DESCRIPTION: DID HEADMASTER/HEADMISTRESS FILL OUT QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034102Y (01 ) 0 YES

 002 C034102M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0080

 DESCRIPTION: DID HEAD TEACHER FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034103Y (01 ) 0 YES

 002 C034103M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0081

 DESCRIPTION: DID VICE PRINCIPAL FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034104Y (01 ) 0 YES

 002 C034104M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0082

 DESCRIPTION: DID COUNSELOR FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034105Y (01 ) 0 YES

 002 C034105M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0083

 DESCRIPTION: DID CURRICULUM COORD FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034106Y (01 ) 0 YES

 002 C034106M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0084

 DESCRIPTION: DID TEACHER FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034107Y (01 ) 0 YES

 002 C034107M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0085

 DESCRIPTION: DID SECRETARY FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034108 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034108Y (01 ) 0 YES

 002 C034108M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0086

 DESCRIPTION: DID OTHER PERSON FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034109 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034109Y (01 ) 0 YES

 002 C034109M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0001

 DESCRIPTION: WHAT IS YOUR GENDER?

 GRADES/ASSESSMENTS: N04, S04, N08, S08
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

511

 CONDITIONING VAR LABEL:

 NAEP ID: T055901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T055901A (01 ) 00 MALE

 002 T055901B (02 ) 10 FEMALE

 003 T055901M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0002

 DESCRIPTION: WHICH BEST DESRIBES YOU?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T056001A (01 ) 000000 WHITE

 002 T056001B (02 ) 100000 BLACK

 003 T056001C (03 ) 010000 HISPANIC

 004 T056001D (04 ) 001000 ASIAN/PACIFIC AMERIC

 005 T056001E (05 ) 000100 AMER IND/ALASKA NATV

 006 T056001F (06 ) 000010 OTHER

 007 T056001M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0003

 DESCRIPTION: YEARS TAUGHT

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T040301A (01 ) 00000 2 YEARS OR LESS

 002 T040301B (02 ) 10000 3-5 YEARS

 003 T040301C (03 ) 01000 6-10 YEARS

 004 T040301D (04 ) 00100 11-24 YEARS

 005 T040301E (05 ) 00010 25 YEARS OR MORE

 006 T040301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0004

 DESCRIPTION: HOW MANY YRS TOTAL YOU TAUGHT MATH?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T056101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T056101A (01 ) 00000 2 YEARS OR LESS

 002 T056101B (02 ) 10000 3-5 YEARS

 003 T056101C (03 ) 01000 6-10 YEARS

 004 T056101D (04 ) 00100 11-24 YEARS

 005 T056101E (05 ) 00010 25 YEARS OR MORE

 006 T056101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0005

 DESCRIPTION: HOW MANY YRS TOTAL YOU TAUGHT SCIENCE?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T056102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T056102A (01 ) 00000 2 YEARS OR LESS

 002 T056102B (02 ) 10000 3-5 YEARS

 003 T056102C (03 ) 01000 6-10 YEARS

 004 T056102D (04 ) 00100 11-24 YEARS

 005 T056102E (05 ) 00010 25 YEARS OR MORE

 006 T056102M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0006

 DESCRIPTION: TYPE TCHNG CERT IN THIS ST IN MAIN FIELD?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T056201A (01 ) 000000 ADVANCED PROFESSIONL

 002 T056201B (02 ) 100000 REGULAR/STANDARD ST

 003 T056201C (03 ) 010000 PROBATIONARY STATE

 004 T056201D (04 ) 001000 TEMPORARY/PROVISIONL

 005 T056201E (05 ) 000100 OTHER THAN STATE CRT

 006 T056201F (06 ) 000010 NOT HAVE CERT MAIN

 007 T056201M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0007

 DESCRIPTION: CERTIFICATION, ELEMENTARY OR MIDDLE/JUNIOR HS ED?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

512

 001 T040501Y (01 ) 000 YES

 002 T040501N (02 ) 100 NO

 003 T040501C (03 ) 010 NOT OFFERED IN STATE

 004 T040501M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0008

 DESCRIPTION: DO YOU HAVE CERTIFICATION IN ELEMENTARY MATH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T040506Y (01 ) 000 YES

 002 T040506N (02 ) 100 NO

 003 T040506C (03 ) 010 NOT OFFERED IN STATE

 004 T040506M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0009

 DESCRIPTION: DO YOU HAVE CERTIFICATION IN JR HIGH/SEC MATH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T040504Y (01 ) 000 YES

 002 T040504N (02 ) 100 NO

 003 T040504C (03 ) 010 NOT OFFERED IN STATE

 004 T040504M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0010

 DESCRIPTION: CERTIFICATION, ELEMENTARY SCIENCE?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040507 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T040507Y (01 ) 000 YES

 002 T040507N (02 ) 100 NO

 003 T040507C (03 ) 010 NOT OFFERED IN STATE

 004 T040507M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0011

 DESCRIPTION: CERTIFICATION, MIDDLE/JUNIOR SCIENCE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040508 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T040508Y (01 ) 000 YES

 002 T040508N (02 ) 100 NO

 003 T040508C (03 ) 010 NOT OFFERED IN STATE

 004 T040508M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0012

 DESCRIPTION: CERTIFICATION, OTHER

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T040505Y (01 ) 000 YES

 002 T040505N (02 ) 100 NO

 003 T040505C (03 ) 010 NOT OFFERED IN STATE

 004 T040505M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0013

 DESCRIPTION: HIGHEST ACADEMIC DEGREE YOU HOLD?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 T056301A (01 ) 0000000 HIGH SCHOOL DIPLOMA

 002 T056301B (02 ) 1000000 ASSOCIATES/VOCATIONL

 003 T056301C (03 ) 0100000 BACHELORS DEGREE

 004 T056301D (04 ) 0010000 MASTER’S DEGREE

 005 T056301E (05 ) 0001000 EDUCATION SPECIALIST

 006 T056301F (06 ) 0000100 DOCTORATE

 007 T056301G (07 ) 0000010 PROFESSIONAL DEGREE

 008 T056301M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0014

 DESCRIPTION: EDUCATION UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

513

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040701Y (01 ) 0 YES

 002 T040701M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0015

 DESCRIPTION: ELMENT ED UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040706 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040706Y (01 ) 0 YES

 002 T040706M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0016

 DESCRIPTION: SEC ED UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040707 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040707Y (01 ) 0 YES

 002 T040707M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0017

 DESCRIPTION: WAS YOUR UNDERGRADUATE MAJOR MATH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040703Y (01 ) 0 YES

 002 T040703M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0018

 DESCRIPTION: WAS YOUR UNDERGRADUATE MAJOR MATH ED?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040704Y (01 ) 0 YES

 002 T040704M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0019

 DESCRIPTION: SCIENCE ED UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040710 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040710Y (01 ) 0 YES

 002 T040710M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0020

 DESCRIPTION: LIFE SCIENCE UNDERGRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040711 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040711Y (01 ) 0 YES

 002 T040711M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0021

 DESCRIPTION: PHYSICAL SCIENCE UNDERGRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040712 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040712Y (01 ) 0 YES

 002 T040712M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0022

 DESCRIPTION: EARTH SCIENCE UNDERGRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040713 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040713Y (01 ) 0 YES

 002 T040713M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0023

 DESCRIPTION: SPECIAL EDUCATION UNDERGRAD MAJOR
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

514

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040708 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040708Y (01 ) 0 YES

 002 T040708M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0024

 DESCRIPTION: BILINGUAL ED/ESL UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040709 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040709Y (01 ) 0 YES

 002 T040709M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0025

 DESCRIPTION: OTHER UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040705Y (01 ) 0 YES

 002 T040705M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0026

 DESCRIPTION: EDUCATION GRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040801Y (01 ) 0 YES

 002 T040801M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0027

 DESCRIPTION: ELEMENTARY ED GRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040807 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040807Y (01 ) 0 YES

 002 T040807M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0028

 DESCRIPTION: SECONDARY ED GRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040808 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040808Y (01 ) 0 YES

 002 T040808M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0029

 DESCRIPTION: WAS YOUR GRADUATE MAJOR MATHEMATICS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040803Y (01 ) 0 YES

 002 T040803M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0030

 DESCRIPTION: WAS YOUR GRADUATE MAJOR MATH ED?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040804Y (01 ) 0 YES

 002 T040804M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0031

 DESCRIPTION: SCIENCE ED GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040814 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040814Y (01 ) 0 YES

 002 T040814M (M ) 1 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

515

 CONDITIONING VARIABLE ID: TCHR0032

 DESCRIPTION: LIFE SCIENCE GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040815 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040815Y (01 ) 0 YES

 002 T040815M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0033

 DESCRIPTION: PHYSICAL SCIENCE GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040816 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040816Y (01 ) 0 YES

 002 T040816M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0034

 DESCRIPTION: EARTH SCIENCE GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040817 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040817Y (01 ) 0 YES

 002 T040817M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0035

 DESCRIPTION: SPECIAL ED GRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040809 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040809Y (01 ) 0 YES

 002 T040809M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0036

 DESCRIPTION: BILINGUAL GRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040810 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040810Y (01 ) 0 YES

 002 T040810M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0037

 DESCRIPTION: ADMIN/SUPERVISION GRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040811 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040811Y (01 ) 0 YES

 002 T040811M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0038

 DESCRIPTION: CURRICULUM/INSTRUCTION GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040812 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040812Y (01 ) 0 YES

 002 T040812M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0039

 DESCRIPTION: COUNSELING GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040813 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040813Y (01 ) 0 YES

 002 T040813M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0040

 DESCRIPTION: OTHER GRAD MAJOR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040805 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

516

 001 T040805Y (01 ) 0 YES

 002 T040805M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0041

 DESCRIPTION: NO GRADUATE STUDY

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040806 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040806Y (01 ) 0 YES

 002 T040806M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0042

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-EDUCATION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056401Y (01 ) 0 YES

 002 T056401M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0043

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-ELEMENTARY ED

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056402Y (01 ) 0 YES

 002 T056402M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0044

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-SECONDARY ED

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056403Y (01 ) 0 YES

 002 T056403M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0045

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-MATHEMATICS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056404Y (01 ) 0 YES

 002 T056404M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0046

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-MATHEMATICS ED

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056405Y (01 ) 0 YES

 002 T056405M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0047

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-SCIENCE ED

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056413 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056413Y (01 ) 0 YES

 002 T056413M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0048

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-LIFE SCIENCE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056414 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056414Y (01 ) 0 YES

 002 T056414M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0049

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-PHYSICAL SCIENCE

 GRADES/ASSESSMENTS: N04, N08, S08
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

517

 CONDITIONING VAR LABEL:

 NAEP ID: T056415 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056415Y (01 ) 0 YES

 002 T056415M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0050

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-EARTH SCIENCE

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T056416 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056416Y (01 ) 0 YES

 002 T056416M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0051

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-SPECIAL ED

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056406Y (01 ) 0 YES

 002 T056406M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0052

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-BILINGUAL ED

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056407 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056407Y (01 ) 0 YES

 002 T056407M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0053

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-ADMIN & SUPERVISION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056408 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056408Y (01 ) 0 YES

 002 T056408M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0054

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-CURRICULUM & INSTRUC

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056409 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056409Y (01 ) 0 YES

 002 T056409M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0055

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-COUNSELING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056410 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056410Y (01 ) 0 YES

 002 T056410M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0056

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-OTHER

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056411 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056411Y (01 ) 0 YES

 002 T056411M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0057

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-NONE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056412 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056412Y (01 ) 0 YES

 002 T056412M (M ) 1 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

518

 CONDITIONING VARIABLE ID: TCHR0058

 DESCRIPTION: LAST YR, HOW MUCH TIME IN MATH/MATH ED SEM/WRKSHP?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T056501A (01 ) 00000 NONE

 002 T056501B (02 ) 10000 LESS THAN 6 HOURS

 003 T056501C (03 ) 01000 6-15 HOURS

 004 T056501D (04 ) 00100 16-35 HOURS

 005 T056501E (05 ) 00010 MORE THAN 35 HOURS

 006 T056501M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0059

 DESCRIPTION: LAST YR, HOW MUCH TIME IN SCI/SCI ED SEM/WRKSHPS?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T058101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T058101A (01 ) 00000 NONE

 002 T058101B (02 ) 10000 LESS THAN 6 HOURS

 003 T058101C (03 ) 01000 6-15 HOURS

 004 T058101D (04 ) 00100 16-35 HOURS

 005 T058101E (05 ) 00010 MORE THAN 35 HOURS

 006 T058101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0060

 DESCRIPTION: LAST 2 YRS, HOW MANY MATH/MATH ED UNIV COURSES?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T056601A (01 ) 00000 NONE

 002 T056601B (02 ) 10000 ONE

 003 T056601C (03 ) 01000 TWO

 004 T056601D (04 ) 00100 THREE

 005 T056601E (05 ) 00010 FOUR OR MORE

 006 T056601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0061

 DESCRIPTION: LAST 2 YRS, HOW MANY SCI/SCI ED UNIV COURSES?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T058201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T058201A (01 ) 00000 NONE

 002 T058201B (02 ) 10000 ONE

 003 T058201C (03 ) 01000 TWO

 004 T058201D (04 ) 00100 THREE

 005 T058201E (05 ) 00010 FOUR OR MORE

 006 T058201M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0062

 DESCRIPTION: PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-TELECOMM USE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056701Y (01 ) 0 YES

 002 T056701M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0063

 DESCRIPTION: PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-TECH USE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056702Y (01 ) 0 YES

 002 T056702M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0064

 DESCRIPTION: PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-COOP INSTRCT

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056703Y (01 ) 0 YES

 002 T056703M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0065
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

519

 DESCRIPTION: PAST 5 YRS, COURSES/IN PRO DEVLP-INTERDISP INSTRCT

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056704Y (01 ) 0 YES

 002 T056704M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0066

 DESCRIPTION: PAST 5 YRS, COURSES/IN PRO DEVLP-PORTFOLIO ASSMNT

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056705Y (01 ) 0 YES

 002 T056705M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0067

 DESCRIPTION: PAST 5 YRS, COURSES/IN PRO DEVLP-PERF BASED ASSMNT

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056706 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056706Y (01 ) 0 YES

 002 T056706M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0068

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-TEACH HIGHORDER THKG

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056707 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056707Y (01 ) 0 YES

 002 T056707M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0069

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-TEACH DIFF CULT BKGD

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056708 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056708Y (01 ) 0 YES

 002 T056708M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0070

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-TEACH LEP STUDENTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056709 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056709Y (01 ) 0 YES

 002 T056709M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0071

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-TEACH SPEC NEED STDS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056710 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056710Y (01 ) 0 YES

 002 T056710M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0072

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-CLASSRM MNGMT/ORG

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056711 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056711Y (01 ) 0 YES

 002 T056711M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0073

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-OTHER PROF ISSUES

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056712 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056712Y (01 ) 0 YES
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

520

 002 T056712M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0074

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-NONE OF ABOVE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056713 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056713Y (01 ) 0 YES

 002 T056713M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0075

 DESCRIPTION: AVAILABILITY OF RESOURCES

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T041201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T041201A (01 ) 0000 GET ALL RESOURCES

 002 T041201B (02 ) 1000 GET MOST RESOURCES

 003 T041201C (03 ) 0100 GET SOME RESOURCES

 004 T041201D (04 ) 0010 DONT GET RESOURCES

 005 T041201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0076

 DESCRIPTION: ARE CURRICULUM SPECIALISTS AVAILABLE FOR MATH?

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T041302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T041302Y (01 ) 00 YES

 002 T041302N (02 ) 10 NO

 003 T041302M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0077

 DESCRIPTION: SCIENCE CURRICULUM SPECIALIST

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: T041303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T041303Y (01 ) 00 YES

 002 T041303N (02 ) 10 NO

 003 T041303M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0078

 DESCRIPTION: HOW MANY SCHOOL HOURS ARE PREP TIME PER WEEK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T056801A (01 ) 000000 NONE

 002 T056801B (02 ) 100000 LESS THAN ONE

 003 T056801C (03 ) 010000 1-2

 004 T056801D (04 ) 001000 3-4

 005 T056801E (05 ) 000100 5

 006 T056801F (06 ) 000010 MORE THAN 5

 007 T056801M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0079

 DESCRIPTION: METHODS OF TEACHING ELEM MATH- 1+COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056901Y (01 ) 0 YES

 002 T056901M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0080

 DESCRIPTION: METHODS OF TEACHING ELEM MATH-PART COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569A1 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569A1Y (01 ) 0 YES

 002 T0569A1M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0081

 DESCRIPTION: METHODS OF TEACHING ELEM MATH-SEMINAR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569B1 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

521

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569B1Y (01 ) 0 YES

 002 T0569B1M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0082

 DESCRIPTION: METHODS OF TEACHING ELEM MATH-LITTLE NO EXPOSURE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569C1 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569C1Y (01 ) 0 YES

 002 T0569C1M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0083

 DESCRIPTION: NUMBER SYSTEMS & NUMERATION-1+ COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056902 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056902Y (01 ) 0 YES

 002 T056902M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0084

 DESCRIPTION: NUMBER SYSTEMS & NUMERATION-PART COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569A2 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569A2Y (01 ) 0 YES

 002 T0569A2M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0085

 DESCRIPTION: NUMBER SYSTEMS & NUMERATION-SEMINAR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569B2 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569B2Y (01 ) 0 YES

 002 T0569B2M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0086

 DESCRIPTION: NUMBER SYSTEMS & NUMERATION-LITTLE/NO EXPOSURE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569C2 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569C2Y (01 ) 0 YES

 002 T0569C2M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0087

 DESCRIPTION: MEASUREMENT IN MATH- 1+COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056903 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056903Y (01 ) 0 YES

 002 T056903M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0088

 DESCRIPTION: MEASUREMENT IN MATH- PART COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569A3 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569A3Y (01 ) 0 YES

 002 T0569A3M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0089

 DESCRIPTION: MEASUREMENT IN MATH- -SEMINAR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569B3 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569B3Y (01 ) 0 YES

 002 T0569B3M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0090

 DESCRIPTION: MEASUREMENT IN MATH- -LITTLE NO EXPOSURE
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

522

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569C3 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569C3Y (01 ) 0 YES

 002 T0569C3M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0091

 DESCRIPTION: GEOMETRY- 1+COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056904 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056904Y (01 ) 0 YES

 002 T056904M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0092

 DESCRIPTION: GEOMETRY-PART COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569A4 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569A4Y (01 ) 0 YES

 002 T0569A4M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0093

 DESCRIPTION: GEOMETRY-SEMINAR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569B4 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569B4Y (01 ) 0 YES

 002 T0569B4M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0094

 DESCRIPTION: GEOMETRY-LITTLE NO EXPOSURE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569C4 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569C4Y (01 ) 0 YES

 002 T0569C4M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0095

 DESCRIPTION: PROBABILITY/STATISTICS- 1+COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056905 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056905Y (01 ) 0 YES

 002 T056905M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0096

 DESCRIPTION: PROBABILITY/STATISTICS-PART COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569A5 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569A5Y (01 ) 0 YES

 002 T0569A5M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0097

 DESCRIPTION: PROBABILITY/STATISTICS-SEMINAR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569B5 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569B5Y (01 ) 0 YES

 002 T0569B5M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0098

 DESCRIPTION: PROBABILITY/STATISTICS-LITTLE NO EXPOSURE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569C5 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569C5Y (01 ) 0 YES

 002 T0569C5M (M ) 1 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

523

 CONDITIONING VARIABLE ID: TCHR0099

 DESCRIPTION: COLLEGE ALGEBRA- 1+COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056906 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056906Y (01 ) 0 YES

 002 T056906M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0100

 DESCRIPTION: COLLEGE ALGEBRA-PART COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569A6 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569A6Y (01 ) 0 YES

 002 T0569A6M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0101

 DESCRIPTION: COLLEGE ALGEBRA-SEMINAR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569B6 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569B6Y (01 ) 0 YES

 002 T0569B6M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0102

 DESCRIPTION: COLLEGE ALGEBRA-LITTLE NO EXPOSURE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569C6 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569C6Y (01 ) 0 YES

 002 T0569C6M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0103

 DESCRIPTION: CALCULUS- 1+COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056907 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056907Y (01 ) 0 YES

 002 T056907M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0104

 DESCRIPTION: CALCULUS-PART COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569A7 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569A7Y (01 ) 0 YES

 002 T0569A7M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0105

 DESCRIPTION: CALCULUS-SEMINAR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569B7 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569B7Y (01 ) 0 YES

 002 T0569B7M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0106

 DESCRIPTION: CALCULUS-LITTLE NO EXPOSURE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569C7 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569C7Y (01 ) 0 YES

 002 T0569C7M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0107

 DESCRIPTION: ABSTRACT/LINEAR ALGEBRA- 1+COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056908 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

524

 001 T056908Y (01 ) 0 YES

 002 T056908M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0108

 DESCRIPTION: ABSTRACT/LINEAR ALGEBRA-PART COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569A8 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569A8Y (01 ) 0 YES

 002 T0569A8M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0109

 DESCRIPTION: ABSTRACT/LINEAR ALGEBRA-SEMINAR

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569B8 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569B8Y (01 ) 0 YES

 002 T0569B8M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0110

 DESCRIPTION: ABSTRACT/LINEAR ALGEBRA-LITTLE NO EXPOSURE

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T0569C8 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T0569C8Y (01 ) 0 YES

 002 T0569C8M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0111

 DESCRIPTION: EVER STUDIED ESTIMATION?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T057001Y (01 ) 00 YES

 002 T057001N (02 ) 10 NO

 003 T057001M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0112

 DESCRIPTION: EVER STUDIED PROBLEM SOLVING IN MATH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T057002Y (01 ) 00 YES

 002 T057002N (02 ) 10 NO

 003 T057002M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0113

 DESCRIPTION: EVER STUDIED USE OF MANIPULATIVES?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T057003Y (01 ) 00 YES

 002 T057003N (02 ) 10 NO

 003 T057003M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0114

 DESCRIPTION: EVER STUDIED USE OF CALCULATORS IN MATH INSTRC?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T057004Y (01 ) 00 YES

 002 T057004N (02 ) 10 NO

 003 T057004M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0115

 DESCRIPTION: EVER STUDIED UNDERSTANDING STUDS MATH THINKING?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T057005Y (01 ) 00 YES

 002 T057005N (02 ) 10 NO
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

525

 003 T057005M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0116

 DESCRIPTION: EVER STUDIED GENDER ISSUES IN TEACHING MATH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T057006Y (01 ) 00 YES

 002 T057006N (02 ) 10 NO

 003 T057006M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0117

 DESCRIPTION: EVER STUDIED TEACHING STUDS OF DIFF CULTURES?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T057007Y (01 ) 00 YES

 002 T057007N (02 ) 10 NO

 003 T057007M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0118

 DESCRIPTION: KNOWLEDGE OF NCTM CURR & EVAL STANDARDS FOR MATH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057101A (01 ) 0000 VERY KNOWLEDGEABLE

 002 T057101B (02 ) 1000 KNOWLEDGEABLE

 003 T057101C (03 ) 0100 SOMEWHAT KNOWLEDGEAB

 004 T057101D (04 ) 0010 LITTLE/NO KNOWLEDGE

 005 T057101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0119

 DESCRIPTION: PRO ACTVTS-STRATEGIES LOCAL WORKSHOPS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T057201Y (01 ) 0 YES

 002 T057201M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0120

 DESCRIPTION: PRO ACTVTS-STRATEGIES REGIONAL NCTM MEETING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057211 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T057211Y (01 ) 0 YES

 002 T057211M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0121

 DESCRIPTION: PRO ACTVTS-STRATEGIES NATIONAL NCTM MEETING

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057221 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T057221Y (01 ) 0 YES

 002 T057221M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0122

 DESCRIPTION: PRO ACTVTS-STRATEGIES OTHER

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057231 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T057231Y (01 ) 0 YES

 002 T057231M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0123

 DESCRIPTION: PRO ACTVTS-STRATEGIES NO

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057241 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T057241Y (01 ) 0 YES

 002 T057241M (M ) 1 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

526

 CONDITIONING VARIABLE ID: TSUB0001

 DESCRIPTION: IMPORTANCE W/ STUDS-APPLYING MATH IDEAS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057301A (01 ) 0000 VERY IMPORTANT

 002 T057301B (02 ) 1000 SOMEWHAT IMPORTANT

 003 T057301C (03 ) 0100 NOT VERY IMPORTANT

 004 T057301D (04 ) 0010 NOT IMPORTANT

 005 T057301M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0002

 DESCRIPTION: IMPORTANCE W/ STUDS-PROB SOLVING=GOAL & CONCEPT?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057302A (01 ) 0000 VERY IMPORTANT

 002 T057302B (02 ) 1000 SOMEWHAT IMPORTANT

 003 T057302C (03 ) 0100 NOT VERY IMPORTANT

 004 T057302D (04 ) 0010 NOT IMPORTANT

 005 T057302M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0003

 DESCRIPTION: IMPORTANCE W/ STUDS-? TECHS PROMOTE STUD TALK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057303A (01 ) 0000 VERY IMPORTANT

 002 T057303B (02 ) 1000 SOMEWHAT IMPORTANT

 003 T057303C (03 ) 0100 NOT VERY IMPORTANT

 004 T057303D (04 ) 0010 NOT IMPORTANT

 005 T057303M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0004

 DESCRIPTION: IMPORTANCE W/ STUDS-USE RESULTS TO INFORM DECISION

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057304A (01 ) 0000 VERY IMPORTANT

 002 T057304B (02 ) 1000 SOMEWHAT IMPORTANT

 003 T057304C (03 ) 0100 NOT VERY IMPORTANT

 004 T057304D (04 ) 0010 NOT IMPORTANT

 005 T057304M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0005

 DESCRIPTION: TO ACCESS PROGRESS HOW OFTEN USE MULT CHOICE TESTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057401A (01 ) 0000 ONCE OR TWICE A WEEK

 002 T057401B (02 ) 1000 ONCE OR TWICE MONTH

 003 T057401C (03 ) 0100 ONCE OR TWICE A YEAR

 004 T057401D (04 ) 0010 NEVER/HARDLY EVER

 005 T057401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0006

 DESCRIPTION: TO ACCESS PROGRESS HOW OFTEN USE PROBLEM SETS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057402A (01 ) 0000 ONCE OR TWICE A WEEK

 002 T057402B (02 ) 1000 ONCE OR TWICE MONTH

 003 T057402C (03 ) 0100 ONCE OR TWICE A YEAR

 004 T057402D (04 ) 0010 NEVER/HARDLY EVER

 005 T057402M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0007

 DESCRIPTION: TO ACCESS PROGRESS HOW OFTEN USE SHORT WRITTEN RSP

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057403A (01 ) 0000 ONCE OR TWICE A WEEK

 002 T057403B (02 ) 1000 ONCE OR TWICE MONTH
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

527

 003 T057403C (03 ) 0100 ONCE OR TWICE A YEAR

 004 T057403D (04 ) 0010 NEVER/HARDLY EVER

 005 T057403M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0008

 DESCRIPTION: TO ACCESS PROGRESS HOW OFTEN USE INDV/GROUP PRJCTS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057404A (01 ) 0000 ONCE OR TWICE A WEEK

 002 T057404B (02 ) 1000 ONCE OR TWICE MONTH

 003 T057404C (03 ) 0100 ONCE OR TWICE A YEAR

 004 T057404D (04 ) 0010 NEVER/HARDLY EVER

 005 T057404M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0009

 DESCRIPTION: TO ACCESS PROGRESS HOW OFTEN USE PORTFOLIOS

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057405A (01 ) 0000 ONCE OR TWICE A WEEK

 002 T057405B (02 ) 1000 ONCE OR TWICE MONTH

 003 T057405C (03 ) 0100 ONCE OR TWICE A YEAR

 004 T057405D (04 ) 0010 NEVER/HARDLY EVER

 005 T057405M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0010

 DESCRIPTION: BEST DESCRIPTION OF COMPUTER AVAILABLITY IN MATH

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T057501A (01 ) 000000 NONE AVAILABLE

 002 T057501B (02 ) 100000 ONE IN CLASS

 003 T057501C (03 ) 010000 2 OR 3 IN CLASS

 004 T057501D (04 ) 001000 4 OR MORE IN CLASS

 005 T057501E (05 ) 000100 DIFFICULT TO ACCESS

 006 T057501F (06 ) 000010 EASY TO ACCESS

 007 T057501M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TSUB0011

 DESCRIPTION: PRIMARY USE OF COMPUTERS FOR MATH INSTRUCTION?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T057601A (01 ) 00000 DRILL AND PRACTICE

 002 T057601B (02 ) 10000 DEMO OF NEW TOPICS

 003 T057601C (03 ) 01000 PLAYING MATH GAMES

 004 T057601D (04 ) 00100 STIMULATIONS/APPS

 005 T057601E (05 ) 00010 DO NOT USE COMPUTERS

 006 T057601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0012

 DESCRIPTION: ARE STUDENTS ASSIGNED TO THIS CLASS BY ABILITY?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T044002Y (01 ) 00 YES

 002 T044002N (02 ) 10 NO

 003 T044002M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TSUB0013

 DESCRIPTION: IF ASSIGNED BY ABILITY, WHAT BASIS ASSIGNED?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057701A (01 ) 0000 NOT BY ABILITY

 002 T057701B (02 ) 1000 BY READING ABILITY

 003 T057701C (03 ) 0100 BY MATH ABILITY

 004 T057701D (04 ) 0010 BY GENERAL ABILITY

 005 T057701M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0014

 DESCRIPTION: IF ASSIGNED BY ABILITY, WHAT IS MATH ABILITY?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

528

 NAEP ID: T057801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T057801A (01 ) 00000 NOT BY ABILITY

 002 T057801B (02 ) 10000 HIGH ABILITY

 003 T057801C (03 ) 01000 AVERAGE ABILITY

 004 T057801D (04 ) 00100 LOW ABILITY

 005 T057801E (05 ) 00010 MIXED ABILITY

 006 T057801M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0015

 DESCRIPTION: CREATE GROUPS IN CLASS FOR MATH ON ABILITY BASIS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T044201Y (01 ) 00 YES

 002 T044201N (02 ) 10 NO

 003 T044201M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TSUB0016

 DESCRIPTION: TIME/WEEK ON MATH INSTRUTION?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T044301A (01 ) 000 2 1/2 HOURS OR LESS

 002 T044301B (02 ) 100 > 2 1/2 HRS-< 4 HRS

 003 T044301C (03 ) 010 4 HRS OR MORE

 004 T044301M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0017

 DESCRIPTION: HOW MUCH TIME PER WEEK STUDS DO MATH W/ PARTNER?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T057901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T057901A (01 ) 0000 NONE

 002 T057901B (02 ) 1000 LESS THAN 1/2 HOUR

 003 T057901C (03 ) 0100 1/2-1 HOUR

 004 T057901D (04 ) 0010 MORE THAN 1 HOUR

 005 T057901M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0018

 DESCRIPTION: AMOUNT MATH HOMEWORK ASSIGN/DAY?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T044401A (01 ) 000000 NONE

 002 T044401B (02 ) 100000 15 MINUTES

 003 T044401C (03 ) 010000 30 MINUTES

 004 T044401D (04 ) 001000 45 MINUTES

 005 T044401E (05 ) 000100 ONE HOUR

 006 T044401F (06 ) 000010 MORE THAN ONE HOUR

 007 T044401M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TSUB0019

 DESCRIPTION: HOW OFTEN DO STUDENTS DO MATH FROM TEXTBOOKS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044501A (01 ) 0000 ALMOST EVERY DAY

 002 T044501B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044501C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044501D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044501M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0020

 DESCRIPTION: HOW OFTEN DO STUDENTS DO MATH ON WORKSHEETS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044502A (01 ) 0000 ALMOST EVERY DAY

 002 T044502B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044502C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044502D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044502M (M ) 0001 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

529

 CONDITIONING VARIABLE ID: TSUB0021

 DESCRIPTION: HOW OFTEN DO STUDENTS SOLVE PROBS W/ OTHER STUDS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044512 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044512A (01 ) 0000 ALMOST EVERY DAY

 002 T044512B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044512C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044512D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044512M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0022

 DESCRIPTION: HOW OFTEN DO STUDENTS WORK W/ OBJECTS LIKE RULERS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044513 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044513A (01 ) 0000 ALMOST EVERY DAY

 002 T044513B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044513C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044513D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044513M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0023

 DESCRIPTION: HOW OFTEN WORK W/ COUNTING BLOCKS.GEOMETRIC SHAPES

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044514 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044514A (01 ) 0000 ALMOST EVERY DAY

 002 T044514B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044514C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044514D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044514M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0024

 DESCRIPTION: HOW OFTEN DO STUDENTS USE A CALCULATOR?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044505A (01 ) 0000 ALMOST EVERY DAY

 002 T044505B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044505C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044505D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044505M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0025

 DESCRIPTION: HOW OFTEN DO STUDENTS TAKE MATH TESTS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044515 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044515A (01 ) 0000 ALMOST EVERY DAY

 002 T044515B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044515C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044515D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044515M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0026

 DESCRIPTION: HOW OFTEN DO STUDENTS WRITE ABOUT PROBLEM-SOLVING?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044507 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044507A (01 ) 0000 ALMOST EVERY DAY

 002 T044507B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044507C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044507D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044507M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0027

 DESCRIPTION: HOW OFTEN DO STUDENTS TALK ABOUT MATH WORK?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044516 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044516A (01 ) 0000 ALMOST EVERY DAY

 002 T044516B (02 ) 1000 ONCE OR TWICE A WEEK
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

530

 003 T044516C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044516D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044516M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0028

 DESCRIPTION: HOW OFTEN DO STUDENTS WRITE REPORTS/DO PROJECTS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044508 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044508A (01 ) 0000 ALMOST EVERY DAY

 002 T044508B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044508C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044508D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044508M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0029

 DESCRIPTION: HOW OFTEN DO STUDENTS DISCUSS MATH W/OTHER STDNTS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044509 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044509A (01 ) 0000 ALMOST EVERY DAY

 002 T044509B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044509C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044509D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044509M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0030

 DESCRIPTION: HOW OFTEN DO STUDENTS WORK REAL-LIFE MATH PRBLMS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044510 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044510A (01 ) 0000 ALMOST EVERY DAY

 002 T044510B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044510C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044510D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044510M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0031

 DESCRIPTION: HOW OFTEN DO STUDENTS USE A COMPUTER?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T044506A (01 ) 0000 ALMOST EVERY DAY

 002 T044506B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T044506C (03 ) 0100 ONCE OR TWICE MONTH

 004 T044506D (04 ) 0010 NEVER OR HARDLY EVER

 005 T044506M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0032

 DESCRIPTION: IN MATH CLASS HOW OFTEN ADDRESS-NUMBERS & OPS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T058001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T058001A (01 ) 0000 A LOT

 002 T058001B (02 ) 1000 SOME

 003 T058001C (03 ) 0100 A LITTLE

 004 T058001D (04 ) 0010 NONE

 005 T058001M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0033

 DESCRIPTION: IN MATH CLASS HOW OFTEN ADDRESS-MEASUREMENT?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T058002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T058002A (01 ) 0000 A LOT

 002 T058002B (02 ) 1000 SOME

 003 T058002C (03 ) 0100 A LITTLE

 004 T058002D (04 ) 0010 NONE

 005 T058002M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0034

 DESCRIPTION: IN MATH CLASS HOW OFTEN ADDRESS-GEOMETRY?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T058003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

531

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T058003A (01 ) 0000 A LOT

 002 T058003B (02 ) 1000 SOME

 003 T058003C (03 ) 0100 A LITTLE

 004 T058003D (04 ) 0010 NONE

 005 T058003M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0035

 DESCRIPTION: IN MATH CLASS HOW OFTEN ADDRESS-DATA ANALYSIS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T058004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T058004A (01 ) 0000 A LOT

 002 T058004B (02 ) 1000 SOME

 003 T058004C (03 ) 0100 A LITTLE

 004 T058004D (04 ) 0010 NONE

 005 T058004M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0036

 DESCRIPTION: IN MATH CLASS HOW OFTEN ADDRESS-ALGEBRA & FUNCT?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T058005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T058005A (01 ) 0000 A LOT

 002 T058005B (02 ) 1000 SOME

 003 T058005C (03 ) 0100 A LITTLE

 004 T058005D (04 ) 0010 NONE

 005 T058005M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0037

 DESCRIPTION: IN MATH HOW OFTEN ADDRESS-LRN MATH FACTS/CONCEPTS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T058006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T058006A (01 ) 0000 A LOT

 002 T058006B (02 ) 1000 SOME

 003 T058006C (03 ) 0100 A LITTLE

 004 T058006D (04 ) 0010 NONE

 005 T058006M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0038

 DESCRIPTION: IN MATH HOW OFTEN ADDRESS-LRN SKILLS/PROCEDURES?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T058007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T058007A (01 ) 0000 A LOT

 002 T058007B (02 ) 1000 SOME

 003 T058007C (03 ) 0100 A LITTLE

 004 T058007D (04 ) 0010 NONE

 005 T058007M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0039

 DESCRIPTION: IN MATH HOW OFTEN ADDRESS-DEVELOP REASONING ABLTY?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T058008 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T058008A (01 ) 0000 A LOT

 002 T058008B (02 ) 1000 SOME

 003 T058008C (03 ) 0100 A LITTLE

 004 T058008D (04 ) 0010 NONE

 005 T058008M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0040

 DESCRIPTION: IN MATH HOW OFTEN ADDRESS-LRN TO COMMUNICATE MATH?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T058009 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T058009A (01 ) 0000 A LOT

 002 T058009B (02 ) 1000 SOME

 003 T058009C (03 ) 0100 A LITTLE

 004 T058009D (04 ) 0010 NONE

 005 T058009M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0041
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

532

 DESCRIPTION: DO YOU PERMIT UNRESTRICTED USE OF CALCULATORS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T045401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T045401Y (01 ) 00 YES

 002 T045401N (02 ) 10 NO

 003 T045401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TSUB0042

 DESCRIPTION: DO YOU PERMIT USE OF CALCULATORS ON TESTS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T044801Y (01 ) 00 YES

 002 T044801N (02 ) 10 NO

 003 T044801M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TSUB0043

 DESCRIPTION: STUDENTS HAVE ACCESS TO SCHL-OWNED CALCULATORS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T045001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T045001Y (01 ) 00 YES

 002 T045001N (02 ) 10 NO

 003 T045001M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TSUB0044

 DESCRIPTION: DO YOU PROVIDE INSTRUCTION IN USE OF CALCULATORS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T044901Y (01 ) 00 YES

 002 T044901N (02 ) 10 NO

 003 T044901M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TSUB0045

 DESCRIPTION: HOW PREPARED TO TEACH MATH CONCEPTS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T045304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T045304A (01 ) 0000 VERY WELL PREPARED

 002 T045304B (02 ) 1000 MODERATELY PREPARED

 003 T045304C (03 ) 0100 NOT VERY PREPARED

 004 T045304D (04 ) 0010 NOT AT ALL PREPARED

 005 T045304M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0046

 DESCRIPTION: HOW PREPARED TO TEACH MATH PROCEDURES?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T045305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T045305A (01 ) 0000 VERY WELL PREPARED

 002 T045305B (02 ) 1000 MODERATELY PREPARED

 003 T045305C (03 ) 0100 NOT VERY PREPARED

 004 T045305D (04 ) 0010 NOT AT ALL PREPARED

 005 T045305M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0047

 DESCRIPTION: HOW PREPARED TO TEACH USE OF COMPUTERS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T045302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T045302A (01 ) 0000 VERY WELL PREPARED

 002 T045302B (02 ) 1000 MODERATELY PREPARED

 003 T045302C (03 ) 0100 NOT VERY PREPARED

 004 T045302D (04 ) 0010 NOT AT ALL PREPARED

 005 T045302M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0048

 DESCRIPTION: HOW PREPARED TO TEACH USE OF CALCULATORS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T045303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

533

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T045303A (01 ) 0000 VERY WELL PREPARED

 002 T045303B (02 ) 1000 MODERATELY PREPARED

 003 T045303C (03 ) 0100 NOT VERY PREPARED

 004 T045303D (04 ) 0010 NOT AT ALL PREPARED

 005 T045303M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0049

 DESCRIPTION: NUMBER OF STUDENTS IN CLASS?

 GRADES/ASSESSMENTS: N04, S04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T044000 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T044000A (01 ) 00000 1-20 STUDENTS

 002 T044000B (02 ) 10000 21-25 STUDENTS

 003 T044000C (03 ) 01000 26-30 STUDENTS

 004 T044000D (04 ) 00100 31-35 STUDENTS

 005 T044000E (05 ) 00010 36 OR MORE STUDENTS

 006 T044000M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0087

 DESCRIPTION: BEST DESCRIBES HOW 8TH GRADES ARE ORGANIZED?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C034201A (01 ) 000 SELF-CONTAINED

 002 C034201B (02 ) 100 SEMI-DEPARTMENTALIZE

 003 C034201C (03 ) 010 DEPARTMENTALIZED

 004 C034201M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0088

 DESCRIPTION: ARE 8TH-GRADERS ASSIGNED TO MATH BY ABILITY?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C034402Y (01 ) 00 YES

 002 C034402N (02 ) 10 NO

 003 C034402M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0089

 DESCRIPTION: ARE 8TH-GRADERS ASSIGNED TO SCIENCE BY ABILITY?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C034403Y (01 ) 00 YES

 002 C034403N (02 ) 10 NO

 003 C034403M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0090

 DESCRIPTION: ARE 8TH-GRADERS ASSIGNED TO ENGLISH BY ABILITY?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C034401Y (01 ) 00 YES

 002 C034401N (02 ) 10 NO

 003 C034401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0091

 DESCRIPTION: ARE 8TH-GRADERS ASSIGNED TO ARTS BY ABILITY?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C034406Y (01 ) 00 YES

 002 C034406N (02 ) 10 NO

 003 C034406M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0092

 DESCRIPTION: HOW OFTEN 8TH GRDS RECEIVE COMP SCI INSTRUCTION?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034510 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C034510A (01 ) 00000 EVERY DAY

 002 C034510B (02 ) 10000 3 OR 4 TIMES A WEEK
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

534

 003 C034510C (03 ) 01000 ONCE OR TWICE/WEEK

 004 C034510D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C034510E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C034510M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0093

 DESCRIPTION: HOW OFTEN 8TH GRDS RECEIVE MATH INSTRUCTION?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034511 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C034511A (01 ) 00000 EVERY DAY

 002 C034511B (02 ) 10000 3 OR 4 TIMES A WEEK

 003 C034511C (03 ) 01000 ONCE OR TWICE/WEEK

 004 C034511D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C034511E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C034511M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0094

 DESCRIPTION: HOW OFTEN 8TH GRDS RECEIVE SCIENCE INSTRUCTION?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034512 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C034512A (01 ) 00000 EVERY DAY

 002 C034512B (02 ) 10000 3 OR 4 TIMES A WEEK

 003 C034512C (03 ) 01000 ONCE OR TWICE/WEEK

 004 C034512D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C034512E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C034512M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0095

 DESCRIPTION: HOW OFTEN 8TH GRDS RECEIVE ENGLISH INSTRUCTION?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034513 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C034513A (01 ) 00000 EVERY DAY

 002 C034513B (02 ) 10000 3 OR 4 TIMES A WEEK

 003 C034513C (03 ) 01000 ONCE OR TWICE/WEEK

 004 C034513D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C034513E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C034513M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0096

 DESCRIPTION: HOW OFTEN 8TH GRDS RECEIVE ARTS INSTRUCTION?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034514 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C034514A (01 ) 00000 EVERY DAY

 002 C034514B (02 ) 10000 3 OR 4 TIMES A WEEK

 003 C034514C (03 ) 01000 ONCE OR TWICE/WEEK

 004 C034514D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C034514E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C034514M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0097

 DESCRIPTION: HAS ENGLISH BEEN IDENTIFIED AS A PRIORITY?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C031611 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031611Y (01 ) 00 YES

 002 C031611N (02 ) 10 NO

 003 C031611M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0098

 DESCRIPTION: SCHOOL OFFER 8TH GR STUDS ALGEBRA FOR HS CREDIT?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C034601Y (01 ) 00 YES

 002 C034601N (02 ) 10 NO

 003 C034601M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0099

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON ENGLISH?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

535

 NAEP ID: C037203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037203Y (01 ) 0 YES

 002 C037203M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0100

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE ENGLISH CURRICULUM?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037306 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037306Y (01 ) 0 YES

 002 C037306M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0101

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR MATH?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039401Y (01 ) 0 YES

 002 C039401M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0102

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR SCIENCE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039402Y (01 ) 0 YES

 002 C039402M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0103

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR READING?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039403Y (01 ) 0 YES

 002 C039403M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0104

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR ARTS?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039404Y (01 ) 0 YES

 002 C039404M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0105

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR OTHER?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039405Y (01 ) 0 YES

 002 C039405M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0106

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR NONE ABOVE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039406Y (01 ) 0 YES

 002 C039406M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0107

 DESCRIPTION: 8TH GRADERS IN EXTRACURR ACTS FOR MATH?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039501Y (01 ) 0 YES

 002 C039501M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0108
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

536

 DESCRIPTION: 8TH GRADERS IN EXTRACURR ACTS FOR SCIENCE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039502Y (01 ) 0 YES

 002 C039502M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0109

 DESCRIPTION: 8TH GRADERS IN EXTRACURR ACTS FOR ENG/LANG ARTS?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039503Y (01 ) 0 YES

 002 C039503M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0110

 DESCRIPTION: 8TH GRADERS IN EXTRACURR ACTS FOR ARTS?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039504Y (01 ) 0 YES

 002 C039504M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0111

 DESCRIPTION: 8TH GRADERS IN EXTRACURR ACTS FOR NONE OF ABOVE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039505Y (01 ) 0 YES

 002 C039505M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0112

 DESCRIPTION: 8TH GRADERS IN SUMMER PROGRAMS IN MATH?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039601Y (01 ) 0 YES

 002 C039601M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0113

 DESCRIPTION: 8TH GRADERS IN SUMMER PROGRAMS IN SCIENCE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039602Y (01 ) 0 YES

 002 C039602M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0114

 DESCRIPTION: 8TH GRADERS IN SUMMER PROGRAMS IN ENG/LANG ARTS?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039603Y (01 ) 0 YES

 002 C039603M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0115

 DESCRIPTION: 8TH GRADERS IN SUMMER PROGRAMS IN ARTS?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039604Y (01 ) 0 YES

 002 C039604M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0116

 DESCRIPTION: 8TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039605Y (01 ) 0 YES
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

537

 002 C039605M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0117

 DESCRIPTION: WHAT % OF 8TH GRDS HELD BACK/REPEAT 8TH GRADE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C041901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C041901A (01 ) 00000 0%

 002 C041901B (02 ) 10000 1-2%

 003 C041901C (03 ) 01000 3-5%

 004 C041901D (04 ) 00100 6-10%

 005 C041901E (05 ) 00010 MORE THAN 10%

 006 C041901M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0124

 DESCRIPTION: YRS TOTAL TAUGHT MATH

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T063001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T063001A (01 ) 00000 2 YEARS OR LESS

 002 T063001B (02 ) 10000 3-5 YEARS

 003 T063001C (03 ) 01000 6-10 YEARS

 004 T063001D (04 ) 00100 11-24 YEARS

 005 T063001E (05 ) 00010 25 YEARS OR MORE

 006 T063001M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0125

 DESCRIPTION: CURRICULUM SPECIALIST TO HELP/ADVISE IN MATH?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T058301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T058301Y (01 ) 00 YES

 002 T058301N (02 ) 10 NO

 003 T058301M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0118

 DESCRIPTION: ARE 12TH-GRADERS ASSIGNED TO MATH BY ABILITY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035002Y (01 ) 00 YES

 002 C035002N (02 ) 10 NO

 003 C035002M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0119

 DESCRIPTION: ARE 12TH-GRADERS ASSIGNED TO SCIENCE BY ABILITY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035003Y (01 ) 00 YES

 002 C035003N (02 ) 10 NO

 003 C035003M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0120

 DESCRIPTION: ARE 12TH-GRADERS ASSIGNED TO ENGLISH BY ABILITY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035006Y (01 ) 00 YES

 002 C035006N (02 ) 10 NO

 003 C035006M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0121

 DESCRIPTION: ARE 12TH-GRADERS ASSIGNED TO ARTS BY ABILITY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035007Y (01 ) 00 YES

 002 C035007N (02 ) 10 NO

 003 C035007M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0122

 DESCRIPTION: FROM 9TH ON HOW MANY YRS REQUIRED FOR MATH?
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

538

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040201A (01 ) 0000000 NONE

 002 C040201B (02 ) 1000000 ONE HALF YEAR

 003 C040201C (03 ) 0100000 ONE YEAR

 004 C040201D (04 ) 0010000 TWO YEARS

 005 C040201E (05 ) 0001000 THREE YEARS

 006 C040201F (06 ) 0000100 FOUR YEARS

 007 C040201G (07 ) 0000010 MORE THAN FOUR YEARS

 008 C040201M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0123

 DESCRIPTION: FROM 9TH ON HOW MANY YRS REQUIRED FOR SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040202A (01 ) 0000000 NONE

 002 C040202B (02 ) 1000000 ONE HALF YEAR

 003 C040202C (03 ) 0100000 ONE YEAR

 004 C040202D (04 ) 0010000 TWO YEARS

 005 C040202E (05 ) 0001000 THREE YEARS

 006 C040202F (06 ) 0000100 FOUR YEARS

 007 C040202G (07 ) 0000010 MORE THAN FOUR YEARS

 008 C040202M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0124

 DESCRIPTION: FROM 9TH ON HOW MANY YRS REQUIRED FOR ENG/LIT?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040203A (01 ) 0000000 NONE

 002 C040203B (02 ) 1000000 ONE HALF YEAR

 003 C040203C (03 ) 0100000 ONE YEAR

 004 C040203D (04 ) 0010000 TWO YEARS

 005 C040203E (05 ) 0001000 THREE YEARS

 006 C040203F (06 ) 0000100 FOUR YEARS

 007 C040203G (07 ) 0000010 MORE THAN FOUR YEARS

 008 C040203M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0125

 DESCRIPTION: FROM 9TH ON HOW MANY YRS REQUIRED FINE/PERF ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040204 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040204A (01 ) 0000000 NONE

 002 C040204B (02 ) 1000000 ONE HALF YEAR

 003 C040204C (03 ) 0100000 ONE YEAR

 004 C040204D (04 ) 0010000 TWO YEARS

 005 C040204E (05 ) 0001000 THREE YEARS

 006 C040204F (06 ) 0000100 FOUR YEARS

 007 C040204G (07 ) 0000010 MORE THAN FOUR YEARS

 008 C040204M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0126

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN ADVANCED BIO?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040301Y (01 ) 0 YES

 002 C040301M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0127

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN ADVANCED CHEM?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040302Y (01 ) 0 YES

 002 C040302M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0128

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN ADV PHYSICS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

539

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040303Y (01 ) 0 YES

 002 C040303M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0129

 DESCRIPTION: NO ADVANCED SCIENCE COURSES TAUGHT

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040304Y (01 ) 0 YES

 002 C040304M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0130

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN COMP SCI?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040305Y (01 ) 0 YES

 002 C040305M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0131

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN CALCULUS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040306 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040306Y (01 ) 0 YES

 002 C040306M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0132

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN TRIGONOMETRY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040307 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040307Y (01 ) 0 YES

 002 C040307M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0133

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN PRECALCULUS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040308 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040308Y (01 ) 0 YES

 002 C040308M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0134

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN PROB/STAT?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040309 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040309Y (01 ) 0 YES

 002 C040309M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0135

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN UNI/INTEG MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040310 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040310Y (01 ) 0 YES

 002 C040310M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0136

 DESCRIPTION: NO ADVANCED MATH COURSES TAUGHT?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040311 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040311Y (01 ) 0 YES

 002 C040311M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0137

 DESCRIPTION: STUDS REQUIRED TO PASS STATE TEST IN MATH?
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

540

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C040401Y (01 ) 00 YES

 002 C040401N (02 ) 10 NO

 003 C040401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0138

 DESCRIPTION: STUDS REQUIRED TO PASS STATE TEST IN SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C040402Y (01 ) 00 YES

 002 C040402N (02 ) 10 NO

 003 C040402M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0139

 DESCRIPTION: STUDS REQUIRED TO PASS STATE TEST IN ENG/LANG ARTS

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C040403Y (01 ) 00 YES

 002 C040403N (02 ) 10 NO

 003 C040403M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0140

 DESCRIPTION: STUDS REQUIRED TO PASS STATE TEST IN FINE/PERF ART

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C040404Y (01 ) 00 YES

 002 C040404N (02 ) 10 NO

 003 C040404M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0141

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS FOR MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040501Y (01 ) 0 YES

 002 C040501M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0142

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS FOR SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040502Y (01 ) 0 YES

 002 C040502M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0143

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS IN ENG/LANG ART

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040503Y (01 ) 0 YES

 002 C040503M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0144

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS IN ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040504Y (01 ) 0 YES

 002 C040504M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0145

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS IN OTHER?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

541

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040505Y (01 ) 0 YES

 002 C040505M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0146

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS IN NONE ABOVE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040506Y (01 ) 0 YES

 002 C040506M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0147

 DESCRIPTION: 12TH GRADERS IN EXTRACURR ACTS IN MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040601Y (01 ) 0 YES

 002 C040601M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0148

 DESCRIPTION: 12TH GRADERS IN EXTRACURR ACTS IN SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040602Y (01 ) 0 YES

 002 C040602M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0149

 DESCRIPTION: 12TH GRADERS IN EXTRACURR ACTS IN ENG/LANG ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040603Y (01 ) 0 YES

 002 C040603M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0150

 DESCRIPTION: 12TH GRADERS IN EXTRACURR ACTS IN ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040604Y (01 ) 0 YES

 002 C040604M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0151

 DESCRIPTION: 12TH GRADERS IN EXTRACURR ACTS IN NONE OF ABOVE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040605Y (01 ) 0 YES

 002 C040605M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0152

 DESCRIPTION: 12TH GRADERS IN SUMMER PROGRAMS IN MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040701Y (01 ) 0 YES

 002 C040701M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0153

 DESCRIPTION: 12TH GRADERS IN SUMMER PROGRAMS IN SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040702Y (01 ) 0 YES

 002 C040702M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0154

 DESCRIPTION: 12TH GRADERS IN SUMMER PORGRAMS IN ENG/LANG ARTS?
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

542

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040703Y (01 ) 0 YES

 002 C040703M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0155

 DESCRIPTION: 12TH GRADERS IN SUMMER PROGRAMS IN ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040704Y (01 ) 0 YES

 002 C040704M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0156

 DESCRIPTION: 12TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040705Y (01 ) 0 YES

 002 C040705M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0157

 DESCRIPTION: # STUDS ENROLLED IN AP SCIENCE COURSES?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040801A (01 ) 0000000 NONE

 002 C040801B (02 ) 1000000 1-10

 003 C040801C (03 ) 0100000 11-25

 004 C040801D (04 ) 0010000 26-50

 005 C040801E (05 ) 0001000 51-75

 006 C040801F (06 ) 0000100 76-99

 007 C040801G (07 ) 0000010 100 OR MORE

 008 C040801M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0158

 DESCRIPTION: # STUDS ENROLLED IN AP CALCULUS COURSES?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040802A (01 ) 0000000 NONE

 002 C040802B (02 ) 1000000 1-10

 003 C040802C (03 ) 0100000 11-25

 004 C040802D (04 ) 0010000 26-50

 005 C040802E (05 ) 0001000 51-75

 006 C040802F (06 ) 0000100 76-99

 007 C040802G (07 ) 0000010 100 OR MORE

 008 C040802M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0159

 DESCRIPTION: # STUDS ENROLLED IN AP COMP SCI COURSES?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040803A (01 ) 0000000 NONE

 002 C040803B (02 ) 1000000 1-10

 003 C040803C (03 ) 0100000 11-25

 004 C040803D (04 ) 0010000 26-50

 005 C040803E (05 ) 0001000 51-75

 006 C040803F (06 ) 0000100 76-99

 007 C040803G (07 ) 0000010 100 OR MORE

 008 C040803M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0160

 DESCRIPTION: # STUDS ENROLLED IN AP ENGLISH COURSES?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040804A (01 ) 0000000 NONE

 002 C040804B (02 ) 1000000 1-10

 003 C040804C (03 ) 0100000 11-25

 004 C040804D (04 ) 0010000 26-50
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

543

 005 C040804E (05 ) 0001000 51-75

 006 C040804F (06 ) 0000100 76-99

 007 C040804G (07 ) 0000010 100 OR MORE

 008 C040804M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0161

 DESCRIPTION: ANY 12TH GRDS TAKING COLLEGE COURSES IN MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040901Y (01 ) 0 YES

 002 C040901M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0162

 DESCRIPTION: ANY 12TH GRDS TAKING COLLEGE COURSES IN SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040902 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040902Y (01 ) 0 YES

 002 C040902M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0163

 DESCRIPTION: ANY 12TH GRDS TAKING COLLEGE COURSES ENG/LANG ARTS

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040903 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040903Y (01 ) 0 YES

 002 C040903M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0164

 DESCRIPTION: ANY 12TH GRDS TAKING COLLEGE COURSES IN ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040904 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040904Y (01 ) 0 YES

 002 C040904M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0165

 DESCRIPTION: ANY 12TH GRDS TAKING COLLEGE COURSES IN NONE ABOVE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040905 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040905Y (01 ) 0 YES

 002 C040905M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0166

 DESCRIPTION: WHAT % 12TH GRDS HELD BACK AND REPEAT 12TH GRADE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C041001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C041001A (01 ) 00000 0%

 002 C041001B (02 ) 10000 1-2%

 003 C041001C (03 ) 01000 3-5%

 004 C041001D (04 ) 00100 6-10%

 005 C041001E (05 ) 00010 MORE THAN 10%

 006 C041001M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0167

 DESCRIPTION: LAST YR WHAT % OF 12TH GRDS GRADUATED?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C041101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C041101A (01 ) 00000 99-100%

 002 C041101B (02 ) 10000 95-98%

 003 C041101C (03 ) 01000 90-94%

 004 C041101D (04 ) 00100 75-89%

 005 C041101E (05 ) 00010 LESS THAN 75%

 006 C041101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0168

 DESCRIPTION: WHAT % OF GRADUATING CLASS NOW IN 2-YR COLLEGE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

544

 NAEP ID: C036001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C036001A (01 ) 000000 0-10%

 002 C036001B (02 ) 100000 11-25%

 003 C036001C (03 ) 010000 26-50%

 004 C036001D (04 ) 001000 51-75%

 005 C036001E (05 ) 000100 76-90%

 006 C036001F (06 ) 000010 91-100%

 007 C036001M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0169

 DESCRIPTION: WHAT % OF GRADUATING CLASS NOW IN 4-YR COLLEGE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C036002A (01 ) 000000 0-10%

 002 C036002B (02 ) 100000 11-25%

 003 C036002C (03 ) 010000 26-50%

 004 C036002D (04 ) 001000 51-75%

 005 C036002E (05 ) 000100 76-90%

 006 C036002F (06 ) 000010 91-100%

 007 C036002M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0170

 DESCRIPTION: WHAT % OF GRADUATING CLASS NOW IN VO-TEC SCHOOL?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C036003A (01 ) 000000 0-10%

 002 C036003B (02 ) 100000 11-25%

 003 C036003C (03 ) 010000 26-50%

 004 C036003D (04 ) 001000 51-75%

 005 C036003E (05 ) 000100 76-90%

 006 C036003F (06 ) 000010 91-100%

 007 C036003M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0171

 DESCRIPTION: WHAT % OF GRAD CLASS NOW IN EMPLOYER TRAINING?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C036004A (01 ) 000000 0-10%

 002 C036004B (02 ) 100000 11-25%

 003 C036004C (03 ) 010000 26-50%

 004 C036004D (04 ) 001000 51-75%

 005 C036004E (05 ) 000100 76-90%

 006 C036004F (06 ) 000010 91-100%

 007 C036004M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0172

 DESCRIPTION: % OF GRADUATING CLASS NOW IN MILITARY SERVICE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C036005A (01 ) 000000 0-10%

 002 C036005B (02 ) 100000 11-25%

 003 C036005C (03 ) 010000 26-50%

 004 C036005D (04 ) 001000 51-75%

 005 C036005E (05 ) 000100 76-90%

 006 C036005F (06 ) 000010 91-100%

 007 C036005M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0173

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLUNCM (M ) 0 MISSING

 002 NATLUNCH (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0174

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

545

 001 NATLUNCL (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0175

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMREADM (M ) 0 MISSING

 002 REM READ (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0176

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMREADL (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0177

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMATHM (M ) 0 MISSING

 002 REM MATH (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0178

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMATHL (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0179

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLUN4M (M ) 0 MISSING

 002 NATLUN4 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0180

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLUN4L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0181

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMREA4M (M ) 0 MISSING

 002 REMREAD4 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0182

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMREA4L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0183

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMAT4M (M ) 0 MISSING
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

546

 002 REMMATH4 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0184

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMAT4L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0185

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLUN8M (M ) 0 MISSING

 002 NATLUN8 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0186

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLUN8L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0187

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMREA8M (M ) 0 MISSING

 002 REMREAD8 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0188

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMREA8L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0189

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMAT8M (M ) 0 MISSING

 002 REMMATH8 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0190

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMAT8L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0190

 DESCRIPTION: RCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLU12M (M ) 0 MISSING

 002 NATLUN12 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0191

 DESCRIPTION: PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLU12L (0-100,M=0 ) 0.0 + 1.0*X LINEAR
Table C-4 (continued)

Conditioning Variables Specific to the 1996 Mathematics Assessment

547

 CONDITIONING VARIABLE ID: SCHL0192

 DESCRIPTION: RCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMRD12M (M ) 0 MISSING

 002 REMRD12 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0193

 DESCRIPTION: PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMRD12L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0194

 DESCRIPTION: RCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMH12M (M ) 0 MISSING

 002 REMMH12 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0195

 DESCRIPTION: PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMH12L (0-100,M=0 ) 0.0 + 1.0*X LINEAR
Table C-5

Conditioning Variables Specific to the 1996 Science Assessment

548

 CONDITIONING VARIABLE ID: BACK0001

 DESCRIPTION: GRAND MEAN

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: OVERALL

 NAEP ID: BKSER TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: OTHER NUMBER OF INDEPENDENT CONTRASTS: 1

 001 OVERALL (@ ) 1 GRAND MEAN

 CONDITIONING VARIABLE ID: BACK0002

 DESCRIPTION: DERIVED SEX

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: GENDER

 NAEP ID: DSEX TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 MALE (1 ) 0 MALE

 002 FEMALE (2 ) 1 FEMALE

 CONDITIONING VARIABLE ID: BACK0003

 DESCRIPTION: DERIVED RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/ETH

 NAEP ID: DRACE TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 WHI/AI/O (1,5,6 ) 000 RACE/ETHNICITY: WHITE, AMERICAN INDIAN/ALASKAN 

 NATIVE, OTHER, MISSING, UNCLASSIFIED

 002 BLACK (2 ) 100 RACE/ETHNICITY: BLACK

 003 HISPANIC (3 ) 010 RACE/ETHNICITY: HISPANIC

 004 ASIAN (4 ) 001 RACE/ETHNICITY: ASIAN

 CONDITIONING VARIABLE ID: BACK0004

 DESCRIPTION: IF HISPANIC, WHAT IS YOUR HISPANIC BACKGROUND?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: HISPANIC

 NAEP ID: B003101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 NOT HISP (1 ) 0000 HISPANIC: NOT HISPANIC

 002 MEXICAN (2 ) 1000 HISPANIC: MEXICAN, MEXICAN AMERICAN, CHICANO

 003 PUER RIC (3 ) 0100 HISPANIC: PUERTO RICAN

 004 CUBN,OTH (4,5 ) 0010 HISPANIC: CUBAN, OTHER

 005 HISP-? (M ) 0001 HISPANIC: MISSING

 CONDITIONING VARIABLE ID: BACK0005

 DESCRIPTION: MSA/NON-MSA

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: MSANAT

 NAEP ID: TOL8 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 MSA (1,2,3,4,7,9) 0 MSA

 002 NON MSA (5,6,8 ) 1 NON-MSA

 CONDITIONING VARIABLE ID: BACK0006

 DESCRIPTION: TYPE OF LOCALE (5 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL5

 NAEP ID: TOL5 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 BIG CTY5 (1 ) 0000 TOL5: LARGE CITY

 002 MID CTY5 (2,M ) 1000 TOL5: MID-SIZE CITY

 003 FR/BTWN5 (3 ) 0100 TOL5: URBAN FRINGE OF LARGE CITY, URBAN FRINGE OF

 MID-SIZE CITY

 004 SML TWN5 (4 ) 0010 TOL5: SMALL TOWN

 005 RURAL5 (5 ) 0001 TOL5: RURAL (MSA AND NON-MSA)

 CONDITIONING VARIABLE ID: BACK0007

 DESCRIPTION: DESCRIPTION OF COMMUNITY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: DOC

 NAEP ID: DOC TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 BIG CITY (1 ) 000 DOC: BIG CITY

 002 URBAN FR (2 ) 100 DOC: URBAN FRINGE

 003 MED CITY (3,9,M ) 010 DOC: MEDIUM CITY

 004 SM PLACE (4 ) 001 DOC: SMALL PLACE

 CONDITIONING VARIABLE ID: BACK0008

 DESCRIPTION: PARENTS’ HIGHEST LEVEL OF EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: PARED

 NAEP ID: PARED TOTAL NUMBER OF SPECIFIED CONTRASTS: 5
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

549

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 < HS (1 ) 0000 PARED: LESS THAN HIGH SCHOOL

 002 HS GRAD (2 ) 1000 PARED: HIGH SCHOOL GRADUATE

 003 POST HS (3 ) 0100 PARED: POST HIGH SCHOOL

 004 COL GRAD (4 ) 0010 PARED: COLLEGE GRADUATE

 005 PARED-? (5,M ) 0001 PARED: MISSING, I DON’T KNOW

 CONDITIONING VARIABLE ID: BACK0009

 DESCRIPTION: REGION OF THE COUNTRY

 GRADES/ASSESSMENTS: N04, N08, N12

 CONDITIONING VAR LABEL: REGION

 NAEP ID: REGION TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 N EAST (1 ) 000 REGION: NORTHEAST

 002 S EAST (2 ) 100 REGION: SOUTHEAST

 003 CENTRAL (3 ) 010 REGION: CENTRAL

 004 WEST (4,5 ) 001 REGION: WEST, TERRITORIES (NONE)

 CONDITIONING VARIABLE ID: BACK0010

 DESCRIPTION: SCHOOL TYPE (PQ)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: SCHTYPE

 NAEP ID: SCHTYPE TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 PUBLIC (1 ) 00 SCHOOL TYPE: PUBLIC

 002 PRIVATE (2,4,5,M ) 10 SCHOOL TYPE: PRIVATE, BUREAU OF INDIAN AFFAIRS, 

 DEPARTMENT OF DEFENSE, MISSING

 003 CATHOLIC (3 ) 01 SCHOOL TYPE: CATHOLIC

 CONDITIONING VARIABLE ID: BACK0011

 DESCRIPTION: INDIVIDUALIZED EDUCATION PROGRAM

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: IEP

 NAEP ID: IEP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 IEP-YES (1 ) 0 IEP: YES

 002 IEP-NO (2 ) 1 IEP: NO

 CONDITIONING VARIABLE ID: BACK0012

 DESCRIPTION: LIMITED ENGLISH PROFICIENCY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: LEP

 NAEP ID: LEP TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 LEP-YES (1 ) 0 LEP: YES

 002 LEP-NO (2 ) 1 LEP: NO

 CONDITIONING VARIABLE ID: BACK0013

 DESCRIPTION: CHAPTER 1 (BOOK COVER)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: CHAPTER1

 NAEP ID: CHAP1 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 CHAP1-Y (1 ) 0 CHAPTER 1: YES

 002 CHAP1-N (2 ) 1 CHAPTER 1: NO

 CONDITIONING VARIABLE ID: BACK0014

 DESCRIPTION: DO YOU RECEIVE A FREE OR REDUCED-PRICE LUNCH?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: LUNCH

 NAEP ID: SLUNCH TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 NOT ELIG (1 ) 0000 LUNCH PROGRAM: NOT ELIGIBLE

 002 REDUCED (2 ) 1000 LUNCH PROGRAM: REDUCED PRICE

 003 FREE (3 ) 0100 LUNCH PROGRAM: FREE

 004 INFO NA (4,M ) 0010 LUNCH PROGRAM: INFO NOT AVAILABLE

 005 SCH REF (5 ) 0001 LUNCH PROGRAM: SCHOOL REFUAL

 CONDITIONING VARIABLE ID: BACK0015

 DESCRIPTION: HOW OFTEN DO THE PEOPLE IN YOUR HOME SPEAK A LANGUAGE OTHER THAN ENGLISH?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMELANG

 NAEP ID: B003201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 HL-NEVER (1 ) 000 HOMELANG: NEVER

 002 HL-SOME (2 ) 100 HOMELANG: SOMETIMES

 003 HL-ALWAY (3 ) 010 HOMELANG: ALWAYS

 004 HL-? (M ) 001 HOMELANG: MISSING
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

550

 CONDITIONING VARIABLE ID: BACK0016

 DESCRIPTION: HOW MUCH TV/VIDEO DO YOU USUALLY WATCH EACH DAY? (LINEAR)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: TVWATCHL

 NAEP ID: B009001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TVLIN-0 (1 ) 0 TV WATCHING (LINEAR) (0 TO 6+ HOURS PER DAY)

 002 TVLIN-1 (2 ) 1 TV WATCHING (LINEAR)

 003 TVLIN-2 (3 ) 2 TV WATCHING (LINEAR)

 004 TVLIN-3 (4,M ) 3 TV WATCHING (LINEAR)

 005 TVLIN-4 (5 ) 4 TV WATCHING (LINEAR)

 006 TVLIN-5 (6 ) 5 TV WATCHING (LINEAR)

 007 TVLIN-6 (7 ) 6 TV WATCHING (LINEAR)

 CONDITIONING VARIABLE ID: BACK0017

 DESCRIPTION: HOW MUCH TV/VIDEO DO YOU USUALLY WATCH EACH DAY? (QUADRATIC)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: TVWATCHQ

 NAEP ID: B009001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: QUADRATIC NUMBER OF INDEPENDENT CONTRASTS: 1

 001 TV-QUAD (1-7,M=4 ) 1.0 + -2.0*X + 1.0*X**2 TV WATCHING (QUADRATIC)

 CONDITIONING VARIABLE ID: BACK0018

 DESCRIPTION: HOMEWORK ASSIGNED?: BASED ON TIME SPENT ON HOMEWORK EACH DAY.

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: HWASSIGN

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 HW-MISS (M ) 00 HOMEWORK ASSIGNED?: MISSING

 002 HW-NO (1 ) 10 HOMEWORK ASSIGNED?: NO

 003 HW-YES (2-5 ) 01 HOMEWORK ASSIGNED?: YES

 CONDITIONING VARIABLE ID: BACK0019

 DESCRIPTION: HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY? (LINEAR)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEWRKL

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 HWLIN-0 (1,2,M ) 0 HOMEWORK (LINEAR): DON’T HAVE ANY, DON’T DO ANY, 

 MISSING 

 002 HWLIN-1 (3 ) 1 HOMEWORK (LINEAR): 1/2 HOUR OR LESS

 003 HWLIN-2 (4 ) 2 HOMEWORK (LINEAR): 1 HOUR

 004 HWLIN-3 (5 ) 3 HOMEWORK (LINEAR): MORE THAN 1 HOUR

 CONDITIONING VARIABLE ID: BACK0020

 DESCRIPTION: HOW MUCH TIME DO YOU USUALLY SPEND ON HOMEWORK EACH DAY (QUADRATIC)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEWRKQ

 NAEP ID: B006601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: SCALE NUMBER OF INDEPENDENT CONTRASTS: 1

 001 HWQUAD-0 (1,2,M ) 0 HOMEWORK (QUADRATIC): DON’T HAVE ANY, DON’T DO 

 ANY, MISSING 

 002 HWQUAD-1 (3 ) 1 HOMEWORK (QUADRATIC): 1/2 HOUR OR LESS

 003 HWQUAD-2 (4 ) 4 HOMEWORK (QUADRATIC): 1 HOUR

 004 HWQUAD-3 (5 ) 9 HOMEWORK (QUADRATIC): MORE THAN 1 HOUR

 CONDITIONING VARIABLE ID: BACK0021

 DESCRIPTION: HOURS EXTRA READING/WK, NOT CONNECTED W/ SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: B009101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 NONE (01 ) 0000000 NONE

 002 1-2 HRS (02 ) 1000000 1-2 HOURS

 003 3-4 HRS (03 ) 0100000 3-4 HOURS

 004 5-6 HRS (04 ) 0010000 5-6 HOURS

 005 7-8 HRS (05 ) 0001000 7-8 HOURS

 006 9-10 HRS (06 ) 0000100 9-10 HOURS

 007 > 10 HRS (07 ) 0000010 MORE THAN 10 HOURS

 008 B009101M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: BACK0022

 DESCRIPTION: NUMBER OF ITEMS IN THE HOME (NEWSPAPER, > 25 BOOKS, ENCYCLOPEDIA, MAGAZINES) 

 (DERIVED)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: HOMEITMS

 NAEP ID: HOMEEN3 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 HITEM<=2 (1,M ) 00 ITEMS IN HOME: ZERO TO TWO ITEMS, MISSING

 002 HITEM=3 (2 ) 10 ITEMS IN HOME: THREE ITEMS
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

551

 003 HITEM=4 (3 ) 01 ITEMS IN HOME: FOUR ITEMS

 CONDITIONING VARIABLE ID: BACK0023

 DESCRIPTION: HOW MANY DAYS OF SCHOOL MISSED LAST MONTH?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: SCH MISS

 NAEP ID: S004001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 MISS->2 (3,4,5,M ) 0 DAYS OF SCHOOL MISSED: 3-4, 5-10, 10 OR MORE 

 DAYS, MISSING

 002 MISS-2< (1,2 ) 1 DAYS OF SCHOOL MISSED: 0-1, 2 DAYS

 CONDITIONING VARIABLE ID: BACK0024

 DESCRIPTION: HOW MANY GRADES IN THIS STATE (4)

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: B007602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 < ONE GR (01 ) 0000 LESS THAN ON GRADE

 002 1-2 GRDS (02 ) 1000 1-2 GRADES

 003 3-5 GRDS (03 ) 0100 3 -5 GRADES

 004 > 5 GRDS (04 ) 0010 5 OR MORE GRADESEVER

 005 MISSING (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0025

 DESCRIPTION: SINCE 1ST GR, NOT PROMOTION, HOW OFTEN DIFF SCHLS?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: B010101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 B010101A (01 ) 00000000 NONE

 002 B010101B (02 ) 10000000 1

 003 B010101C (03 ) 01000000 2

 004 B010101D (04 ) 00100000 3

 005 B010101E (05 ) 00010000 4

 006 B010101F (06 ) 00001000 5

 007 B010101G (07 ) 00000100 6

 008 B010101H (08 ) 00000010 7 OR MOREMORE

 009 B010101M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: BACK0026

 DESCRIPTION: HOW LONG LIVED IN THE UNITED STATES?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: YRSINUSA

 NAEP ID: B008001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 AALL LIF (1 ) 0000 ALL MY LIFE

 002 USA >5 (2 ) 1000 LIVED IN US MORE THAN 5 YEARS

 003 USA 3-5 (3 ) 0100 LIVED IN US 3-5 YEARS

 004 USA <3 (4 ) 0010 LIVED IN US LESS THAN 3 YEARS

 005 USA-? (M ) 0001 LIVED IN US MISSING

 CONDITIONING VARIABLE ID: BACK0027

 DESCRIPTION: HOW MANY GRADES IN THIS STATE? (4TH GRADE)

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL: STGRADE4

 NAEP ID: B007601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 STGRD<1 (1,M ) 00 GRADES IN STATE: LESS THAN 1 GRADE, MISSING

 002 STGRD1-2 (2 ) 10 GRADES IN STATE: 1-2 GRADES

 003 STGRD3> (3 ) 01 GRADES IN STATE: 3 OR MORE GRADES

 CONDITIONING VARIABLE ID: BACK0029

 DESCRIPTION: TIMES CHANGED SCHOOL SINCE FIRST GRADE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B007302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 B007302A (01 ) 00000000 NONE

 002 B007302B (02 ) 10000000 1

 003 B007302C (03 ) 01000000 2

 004 B007302D (04 ) 00100000 3

 005 B007302E (05 ) 00010000 4

 006 B007302F (06 ) 00001000 5

 007 B007302G (07 ) 00000100 6

 008 B007302H (08 ) 00000010 7 OR MORE

 009 B007302M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: BACK0030

 DESCRIPTION: HOW MANY GRADES IN THIS STATE (12TH GRADE)

 GRADES/ASSESSMENTS: N12
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

552

 CONDITIONING VAR LABEL: STGRAD12

 NAEP ID: B008301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 STGRD<1 (1,M ) 000 GRADES IN STATE: LESS THAN 1 GRADE, MISSING

 002 STGRD1-2 (2 ) 100 GRADES IN STATE: 1-2 GRADES

 003 STGRD3-5 (3 ) 010 GRADES IN STATE: 3-5 GRADES

 004 STGRD6> (4,5 ) 001 GRADES IN STATE: MORE THAN 5 GRADES

 CONDITIONING VARIABLE ID: SCHL0001

 DESCRIPTION: SCHOOL LEVEL AVERAGE SCIENCE NORMIT (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: SCH NORM

 NAEP ID: SCHNORM TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SCHNRM-? (M ) 0 SCHOOL LEVEL AVERAGE SCIENCE NORMIT MISSING

 002 SCHNRM-Y (@ ) 1 SCHOOL LEVEL AVERAGE SCIENCE NORMIT NOT-MISSING

 CONDITIONING VARIABLE ID: SCHL0002

 DESCRIPTION: SCHOOL LEVEL AVERAGE SCIENCE NORMIT

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: SNRM-LIN

 NAEP ID: SCHNORM TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: SCALE NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SNRM-LIN (# ) (F8.4) SCHOOL LEVEL AVERAGE SCIENCE NORMIT MEAN

 002 SNRM-LIN (M ) 0 SCHOOL LEVEL AVERAGE SCIENCE NORMIT MISSING

 CONDITIONING VARIABLE ID: BACK0028

 DESCRIPTION: TIMES CHANGED SCHLS SINCE 1ST GR; NOT PROMOTIONS?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: B009201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B009201A (01 ) 0000 NONE

 002 B009201B (02 ) 1000 1

 003 B009201C (03 ) 0100 2

 004 B009201D (04 ) 0010 3 OR GREATER

 005 B009201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0031

 DESCRIPTION: HOW OFTEN DO YOU DISCUSS THINGS STUDIED IN SCHOOL WITH SOMEONE AT HOME?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: DISC@HOM

 NAEP ID: B007401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 DIS@HOM1 (1 ) 000 DISCUSS STUDIES AT HOME: ALMOST EVERY DAY

 002 DIS@HOM2 (2 ) 100 DISCUSS STUDIES AT HOME: ONCE OR TWICE A WEEK

 003 DIS@HOM3 (3 ) 010 DISCUSS STUDIES AT HOME: ONCE OR TWICE A MONTH

 004 DIS@HOM4 (4,M ) 001 DISCUSS STUDIES AT HOME: NEVER OR HARDLY EVER, 

 MISSING

 CONDITIONING VARIABLE ID: BACK0032

 DESCRIPTION: HOW OFTEN USE A HOME COMPUTER FOR SCHOOLWORK?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B009301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 DAILY (01 ) 00000 ALMOST EVERY DAY

 002 1-2 WEEK (02 ) 10000 ONCE OR TWICE A WEEK

 003 1-2 MTH (03 ) 01000 ONCE OR TWICE A MTH

 004 NEVER (04 ) 00100 NEVER OR HARDLY EVER

 005 NO COMP (05 ) 00010 NO COMPUTER AT HOME

 006 MISSING (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: BACK0033

 DESCRIPTION: ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: PGSREAD1

 NAEP ID: B001101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 PGS<6,? (5,M ) 0 PAGES READ: 5 OR FEWER A DAY, MISSING

 002 PGS>5 (1,2,3,4 ) 1 PAGES READ: 6-10, 11-15, 16-20, 20 OR MORE

 CONDITIONING VARIABLE ID: BACK0034

 DESCRIPTION: ABOUT HOW MANY PAGES A DAY DO YOU HAVE TO READ FOR SCHOOL AND HOMEWORK?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: PGSREAD2

 NAEP ID: B001101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 PGS<11,? (4,5,M ) 0 PAGES READ: 6-10, 5 OR FEWER A DAY, MISSING
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

553

 002 PGS>10 (1,2,3 ) 1 PAGES READ: 11-15, 16-20, 20 OR MORE

 CONDITIONING VARIABLE ID: BACK0035

 DESCRIPTION: WHICH BEST DESCRIBES YOUR HIGH-SCHOOL PROGRAM?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: HS PROG

 NAEP ID: B008501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 GENERAL (1 ) 0000 HIGH SCHOOL PROGRAM: GENERAL

 002 ACADEMIC (2 ) 1000 HIGH SCHOOL PROGRAM: ACADEMIC/COLLEGE PREP

 003 VOC/TECH (3 ) 0100 HIGH SCHOOL PROGRAM: VOCATIONAL OR TECHNICAL

 004 OTHERPGM (4 ) 0010 HIGH SCHOOL PROGRAM: OTHER

 005 HS PGM-? (M ) 0001 HIGH SCHOOL PROGRAM: MISSING

 CONDITIONING VARIABLE ID: BACK0036

 DESCRIPTION: SEMESTERS ENGLISH/LITERATURE/WRITING (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM ENG

 NAEP ID: B007101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMENG-? (M ) 0 ENGLISH SEMESTERS: MISSING

 002 SEMENG-^ (1-9 ) 1 ENGLISH SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0037

 DESCRIPTION: NUMBER OF SEMESTERS ENGLISH/LITERATURE/WRITING (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #ENG-LIN

 NAEP ID: B007101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #ENG-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS ENGLISH/LITERATURE/WRITING

(LINEAR)

 CONDITIONING VARIABLE ID: BACK0038

 DESCRIPTION: SEMESTERS MATHEMATICS (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM MAT

 NAEP ID: B007102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMMAT-? (M ) 0 MATH SEMESTERS: MISSING

 002 SEMMAT-^ (1-9 ) 1 MATH SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0039

 DESCRIPTION: NUMBER OF SEMESTERS MATHEMATICS (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #MAT-LIN

 NAEP ID: B007102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #MAT-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS MATHEMATICS (LINEAR)

 CONDITIONING VARIABLE ID: BACK0040

 DESCRIPTION: SEMESTERS SCIENCE (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM SCI

 NAEP ID: B007103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 SEMSCI-? (M ) 0000 SCIENCE SEMESTERS: MISSING

 002 SEMSCI-^ (1,2,3 ) 1000 SCIENCE SEMESTERS: ZERO TO 2

 003 SEMSCI-^ (4,5 ) 0100 SCIENCE SEMESTERS: 3 OR 4

 004 SEMSCI-^ (6,7 ) 0010 SCIENCE SEMESTERS: 5 OR 6

 005 SEMSCI-^ (8,9 ) 0001 SCIENCE SEMESTERS: 7 OR MORE

 CONDITIONING VARIABLE ID: BACK0041

 DESCRIPTION: NUMBER OF SEMESTERS SCIENCE (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #SCI-LIN

 NAEP ID: B007103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #SCI-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS SCIENCE (LINEAR)

 CONDITIONING VARIABLE ID: BACK0042

 DESCRIPTION: SEMESTERS HISTORY/SOCIAL STUDIES/GEOGRAPHY (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM HIS

 NAEP ID: B007104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMHIS-? (M ) 0 HISTORY SEMESTERS: MISSING

 002 SEMHIS-^ (1-9 ) 1 HISTORY SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0043
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

554

 DESCRIPTION: NUMBER OF SEMESTERS HISTORY/SOCIAL STUDIES/GEOGRAPHY (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #HIS-LIN

 NAEP ID: B007104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #HIS-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS HISTORY/SOCIAL 

 STUDIES/GEOGRAPHY (LINEAR)

 CONDITIONING VARIABLE ID: BACK0044

 DESCRIPTION: SEMESTERS FOREIGN LANGUAGES (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM FLG

 NAEP ID: B007105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMFLG-? (M ) 0 FOREIGN LANGUAGE SEMESTERS: MISSING

 002 SEMFLG-^ (1-9 ) 1 FOREIGN LANGUAGE SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0045

 DESCRIPTION: NUMBER OF SEMESTERS FOREIGN LANGUAGES (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #FLG-LIN

 NAEP ID: B007105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #FLG-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS FOREIGN LANGUAGES (LINEAR)

 CONDITIONING VARIABLE ID: BACK0046

 DESCRIPTION: SEMESTERS VOCATIONAL/TECHNICAL/BUSINESS EDUCATION (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM VOC

 NAEP ID: B007106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMVOC-? (M ) 0 VOC/TECH SEMESTERS: MISSING

 002 SEMVOC-^ (1-9 ) 1 VOC/TECH SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0047

 DESCRIPTION: NUMBER OF SEMESTERS VOCATIONAL/TECHNICAL/BUSINESS EDUCATION (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #VOC-LIN

 NAEP ID: B007106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #VOC-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS VOCATIONAL/TECHNICAL/BUSINESS 

 EDUCATION (LINEAR)

 CONDITIONING VARIABLE ID: BACK0048

 DESCRIPTION: SEMESTERS ART/MUSIC (MISSING VS NON-MISSING)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SEM ART

 NAEP ID: B007107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 SEMART-? (M ) 0 ART/MUSIC SEMESTERS: MISSING

 002 SEMART-^ (1-9 ) 1 ART/MUSIC SEMESTERS: NOT MISSING

 CONDITIONING VARIABLE ID: BACK0049

 DESCRIPTION: NUMBER OF SEMESTERS ART/MUSIC (LINEAR)

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: #ART-LIN

 NAEP ID: B007107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 #ART-LIN (1-9,M=0 ) 0.0 + 1.0*X NUMBER OF SEMESTERS ART/MUSIC (LINEAR)

 CONDITIONING VARIABLE ID: SUBJ0001

 DESCRIPTION: WHAT KIND OF SCI CLASS ARE YOU TAKING THIS YEAR?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: SCITAKE TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 NO SCIEN (01 ) 000000 NOT TAKING THIS YEAR

 002 LIFESCI (02 ) 100000 LIFE SCIENCE

 003 PHYSSCI (03 ) 010000 PHYSICAL SCIENCE

 004 EATHSCI (04 ) 001000 EARTH SCIENCE

 005 GEN SCI (05 ) 000100 GENERAL SCIENCE

 006 INTESCI (06 ) 000010 INTEGRATED SCIENCE

 007 MISSING (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0050

 DESCRIPTION: INTERACTION: GENDER BY RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/RAC

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 8
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

555

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 3

 001 G/R 11 (11 ) 010101 GEND/RAC INTACT: 1. MALE 1. WHI/AI/O

 002 G/R 12 (12 ) -10000 GEND/RAC INTACT: 1. MALE 2. BLACK

 003 G/R 13 (13 ) 00-100 GEND/RAC INTACT: 1. MALE 3. HISPANIC

 004 G/R 14 (14 ) 0000-1 GEND/RAC INTACT: 1. MALE 4. ASIAN

 005 G/R 21 (21 ) -1-1-1 GEND/RAC INTACT: 2. FEMALE 1. WHI/AI/O

 006 G/R 22 (22 ) 010000 GEND/RAC INTACT: 2. FEMALE 2. BLACK

 007 G/R 23 (23 ) 000100 GEND/RAC INTACT: 2. FEMALE 3. HISPANIC

 008 G/R 24 (24 ) 000001 GEND/RAC INTACT: 2. FEMALE 4. ASIAN

 CONDITIONING VARIABLE ID: BACK0051

 DESCRIPTION: INTERACTION: GENDER BY TYPE OF LOCALE (5 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 G/T 11 (11 ) 01010101 GEND/TOL INTACT: 1. MALE 1. BIG CTY5

 002 G/T 12 (12 ) -1000000 GEND/TOL INTACT: 1. MALE 2. MID CTY5

 003 G/T 13 (13 ) 00-10000 GEND/TOL INTACT: 1. MALE 3. FR/BTWN5

 004 G/T 14 (14 ) 0000-100 GEND/TOL INTACT: 1. MALE 4. SML TWN5

 005 G/T 15 (15 ) 000000-1 GEND/TOL INTACT: 1. MALE 5. RURAL5

 006 G/T 21 (21 ) -1-1-1-1 GEND/TOL INTACT: 2. FEMALE 1. BIG CTY5

 007 G/T 22 (22 ) 01000000 GEND/TOL INTACT: 2. FEMALE 2. MID CTY5

 008 G/T 23 (23 ) 00010000 GEND/TOL INTACT: 2. FEMALE 3. FR/BTWN5

 009 G/T 24 (24 ) 00000100 GEND/TOL INTACT: 2. FEMALE 4. SML TWN5

 010 G/T 25 (25 ) 00000001 GEND/TOL INTACT: 2. FEMALE 5. RURAL5

 CONDITIONING VARIABLE ID: BACK0052

 DESCRIPTION: INTERACTION: GENDER BY PARENTS’ EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 G/P 11 (11 ) 01010101 GEND/PAR INTACT: 1. MALE 1. < HS

 002 G/P 12 (12 ) -1000000 GEND/PAR INTACT: 1. MALE 2. HS GRAD

 003 G/P 13 (13 ) 00-10000 GEND/PAR INTACT: 1. MALE 3. POST HS

 004 G/P 14 (14 ) 0000-100 GEND/PAR INTACT: 1. MALE 4. COL GRAD

 005 G/P 15 (15 ) 000000-1 GEND/PAR INTACT: 1. MALE 5. PARED-?

 006 G/P 21 (21 ) -1-1-1-1 GEND/PAR INTACT: 2. FEMALE 1. < HS

 007 G/P 22 (22 ) 01000000 GEND/PAR INTACT: 2. FEMALE 2. HS GRAD

 008 G/P 23 (23 ) 00010000 GEND/PAR INTACT: 2. FEMALE 3. POST HS

 009 G/P 24 (24 ) 00000100 GEND/PAR INTACT: 2. FEMALE 4. COL GRAD

 010 G/P 25 (25 ) 00000001 GEND/PAR INTACT: 2. FEMALE 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0053

 DESCRIPTION: INTERACTION: GENDER BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: GEND/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 2

 001 G/S 11 (11 ) 0101 GEND/SCH INTACT: 1. MALE 1. PUBLIC

 002 G/S 12 (12 ) -100 GEND/SCH INTACT: 1. MALE 2. PRIVATE

 003 G/S 13 (13 ) 00-1 GEND/SCH INTACT: 1. MALE 3. CATHOLIC

 004 G/S 21 (21 ) -1-1 GEND/SCH INTACT: 2. FEMALE 1. PUBLIC

 005 G/S 22 (22 ) 0100 GEND/SCH INTACT: 2. FEMALE 2. PRIVATE

 006 G/S 23 (23 ) 0001 GEND/SCH INTACT: 2. FEMALE 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0054

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY TYPE OF LOCALE (5 CATEGORIES)

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/TOL

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 20

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 R/T 11 (11 ) 010101010101010101010101 RACE/TOL INTACT: 1. WHI/AI/O 1. BIG CTY5

 002 R/T 12 (12 ) -1000000-1000000-1000000 RACE/TOL INTACT: 1. WHI/AI/O 2. MID CTY5

 003 R/T 13 (13 ) 00-1000000-1000000-10000 RACE/TOL INTACT: 1. WHI/AI/O 3. FR/BTWN5

 004 R/T 14 (14 ) 0000-1000000-1000000-100 RACE/TOL INTACT: 1. WHI/AI/O 4. SML TWN5

 005 R/T 15 (15 ) 000000-1000000-1000000-1 RACE/TOL INTACT: 1. WHI/AI/O 5. RURAL5

 006 R/T 21 (21 ) -1-1-1-10000000000000000 RACE/TOL INTACT: 2. BLACK 1. BIG CTY5

 007 R/T 22 (22 ) 010000000000000000000000 RACE/TOL INTACT: 2. BLACK 2. MID CTY5

 008 R/T 23 (23 ) 000100000000000000000000 RACE/TOL INTACT: 2. BLACK 3. FR/BTWN5

 009 R/T 24 (24 ) 000001000000000000000000 RACE/TOL INTACT: 2. BLACK 4. SML TWN5

 010 R/T 25 (25 ) 000000010000000000000000 RACE/TOL INTACT: 2. BLACK 5. RURAL5

 011 R/T 31 (31 ) 00000000-1-1-1-100000000 RACE/TOL INTACT: 3. HISPANIC 1. BIG CTY5

 012 R/T 32 (32 ) 000000000100000000000000 RACE/TOL INTACT: 3. HISPANIC 2. MID CTY5

 013 R/T 33 (33 ) 000000000001000000000000 RACE/TOL INTACT: 3. HISPANIC 3. FR/BTWN5

 014 R/T 34 (34 ) 000000000000010000000000 RACE/TOL INTACT: 3. HISPANIC 4. SML TWN5

 015 R/T 35 (35 ) 000000000000000100000000 RACE/TOL INTACT: 3. HISPANIC 5. RURAL5

 016 R/T 41 (41 ) 0000000000000000-1-1-1-1 RACE/TOL INTACT: 4. ASIAN 1. BIG CTY5

 017 R/T 42 (42 ) 000000000000000001000000 RACE/TOL INTACT: 4. ASIAN 2. MID CTY5

 018 R/T 43 (43 ) 000000000000000000010000 RACE/TOL INTACT: 4. ASIAN 3. FR/BTWN5

 019 R/T 44 (44 ) 000000000000000000000100 RACE/TOL INTACT: 4. ASIAN 4. SML TWN5
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

556

 020 R/T 45 (45 ) 000000000000000000000001 RACE/TOL INTACT: 4. ASIAN 5. RURAL5

 CONDITIONING VARIABLE ID: BACK0055

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY PARENTS’ EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 20

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 R/P 11 (11 ) 010101010101010101010101 RACE/PAR INTACT: 1. WHI/AI/O 1. < HS

 002 R/P 12 (12 ) -1000000-1000000-1000000 RACE/PAR INTACT: 1. WHI/AI/O 2. HS GRAD

 003 R/P 13 (13 ) 00-1000000-1000000-10000 RACE/PAR INTACT: 1. WHI/AI/O 3. POST HS

 004 R/P 14 (14 ) 0000-1000000-1000000-100 RACE/PAR INTACT: 1. WHI/AI/O 4. COL GRAD

 005 R/P 15 (15 ) 000000-1000000-1000000-1 RACE/PAR INTACT: 1. WHI/AI/O 5. PARED-?

 006 R/P 21 (21 ) -1-1-1-10000000000000000 RACE/PAR INTACT: 2. BLACK 1. < HS

 007 R/P 22 (22 ) 010000000000000000000000 RACE/PAR INTACT: 2. BLACK 2. HS GRAD

 008 R/P 23 (23 ) 000100000000000000000000 RACE/PAR INTACT: 2. BLACK 3. POST HS

 009 R/P 24 (24 ) 000001000000000000000000 RACE/PAR INTACT: 2. BLACK 4. COL GRAD

 010 R/P 25 (25 ) 000000010000000000000000 RACE/PAR INTACT: 2. BLACK 5. PARED-?

 011 R/P 31 (31 ) 00000000-1-1-1-100000000 RACE/PAR INTACT: 3. HISPANIC 1. < HS

 012 R/P 32 (32 ) 000000000100000000000000 RACE/PAR INTACT: 3. HISPANIC 2. HS GRAD

 013 R/P 33 (33 ) 000000000001000000000000 RACE/PAR INTACT: 3. HISPANIC 3. POST HS

 014 R/P 34 (34 ) 000000000000010000000000 RACE/PAR INTACT: 3. HISPANIC 4. COL GRAD

 015 R/P 35 (35 ) 000000000000000100000000 RACE/PAR INTACT: 3. HISPANIC 5. PARED-?

 016 R/P 41 (41 ) 0000000000000000-1-1-1-1 RACE/PAR INTACT: 4. ASIAN 1. < HS

 017 R/P 42 (42 ) 000000000000000001000000 RACE/PAR INTACT: 4. ASIAN 2. HS GRAD

 018 R/P 43 (43 ) 000000000000000000010000 RACE/PAR INTACT: 4. ASIAN 3. POST HS

 019 R/P 44 (44 ) 000000000000000000000100 RACE/PAR INTACT: 4. ASIAN 4. COL GRAD

 020 R/P 45 (45 ) 000000000000000000000001 RACE/PAR INTACT: 4. ASIAN 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0056

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: RACE/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 12

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 R/S 11 (11 ) 010101010101 RACE/SCH INTACT: 1. WHI/AI/O 1. PUBLIC

 002 R/S 12 (12 ) -100-100-100 RACE/SCH INTACT: 1. WHI/AI/O 2. PRIVATE

 003 R/S 13 (13 ) 00-100-100-1 RACE/SCH INTACT: 1. WHI/AI/O 3. CATHOLIC

 004 R/S 21 (21 ) -1-100000000 RACE/SCH INTACT: 2. BLACK 1. PUBLIC

 005 R/S 22 (22 ) 010000000000 RACE/SCH INTACT: 2. BLACK 2. PRIVATE

 006 R/S 23 (23 ) 000100000000 RACE/SCH INTACT: 2. BLACK 3. CATHOLIC

 007 R/S 31 (31 ) 0000-1-10000 RACE/SCH INTACT: 3. HISPANIC 1. PUBLIC

 008 R/S 32 (32 ) 000001000000 RACE/SCH INTACT: 3. HISPANIC 2. PRIVATE

 009 R/S 33 (33 ) 000000010000 RACE/SCH INTACT: 3. HISPANIC 3. CATHOLIC

 010 R/S 41 (41 ) 00000000-1-1 RACE/SCH INTACT: 4. ASIAN 1. PUBLIC

 011 R/S 42 (42 ) 000000000100 RACE/SCH INTACT: 4. ASIAN 2. PRIVATE

 012 R/S 43 (43 ) 000000000001 RACE/SCH INTACT: 4. ASIAN 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0057

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY PARENT’S EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL5/PAR

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 25

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 16

 001 T/P 11 (11 ) 01010101010101010101010101010101 TOL5/PAR INTACT: 1. BIG CTY5 1. < HS

 002 T/P 12 (12 ) -1000000-1000000-1000000-1000000 TOL5/PAR INTACT: 1. BIG CTY5 2. HS GRAD

 003 T/P 13 (13 ) 00-1000000-1000000-1000000-10000 TOL5/PAR INTACT: 1. BIG CTY5 3. POST HS

 004 T/P 14 (14 ) 0000-1000000-1000000-1000000-100 TOL5/PAR INTACT: 1. BIG CTY5 4. COL GRAD

 005 T/P 15 (15 ) 000000-1000000-1000000-1000000-1 TOL5/PAR INTACT: 1. BIG CTY5 5. PARED-?

 006 T/P 21 (21 ) -1-1-1-1000000000000000000000000 TOL5/PAR INTACT: 2. MID CTY5 1. < HS

 007 T/P 22 (22 ) 01000000000000000000000000000000 TOL5/PAR INTACT: 2. MID CTY5 2. HS GRAD

 008 T/P 23 (23 ) 00010000000000000000000000000000 TOL5/PAR INTACT: 2. MID CTY5 3. POST HS

 009 T/P 24 (24 ) 00000100000000000000000000000000 TOL5/PAR INTACT: 2. MID CTY5 4. COL GRAD

 010 T/P 25 (25 ) 00000001000000000000000000000000 TOL5/PAR INTACT: 2. MID CTY5 5. PARED-?

 011 T/P 31 (31 ) 00000000-1-1-1-10000000000000000 TOL5/PAR INTACT: 3. FR/BTWN5 1. < HS

 012 T/P 32 (32 ) 00000000010000000000000000000000 TOL5/PAR INTACT: 3. FR/BTWN5 2. HS GRAD

 013 T/P 33 (33 ) 00000000000100000000000000000000 TOL5/PAR INTACT: 3. FR/BTWN5 3. POST HS

 014 T/P 34 (34 ) 00000000000001000000000000000000 TOL5/PAR INTACT: 3. FR/BTWN5 4. COL GRAD

 015 T/P 35 (35 ) 00000000000000010000000000000000 TOL5/PAR INTACT: 3. FR/BTWN5 5. PARED-?

 016 T/P 41 (41 ) 0000000000000000-1-1-1-100000000 TOL5/PAR INTACT: 4. SML TWN5 1. < HS

 017 T/P 42 (42 ) 00000000000000000100000000000000 TOL5/PAR INTACT: 4. SML TWN5 2. HS GRAD

 018 T/P 43 (43 ) 00000000000000000001000000000000 TOL5/PAR INTACT: 4. SML TWN5 3. POST HS

 019 T/P 44 (44 ) 00000000000000000000010000000000 TOL5/PAR INTACT: 4. SML TWN5 4. COL GRAD

 020 T/P 45 (45 ) 00000000000000000000000100000000 TOL5/PAR INTACT: 4. SML TWN5 5. PARED-?

 021 T/P 51 (51 ) 000000000000000000000000-1-1-1-1 TOL5/PAR INTACT: 5. RURAL5 1. < HS

 022 T/P 52 (52 ) 00000000000000000000000001000000 TOL5/PAR INTACT: 5. RURAL5 2. HS GRAD

 023 T/P 53 (53 ) 00000000000000000000000000010000 TOL5/PAR INTACT: 5. RURAL5 3. POST HS

 024 T/P 54 (54 ) 00000000000000000000000000000100 TOL5/PAR INTACT: 5. RURAL5 4. COL GRAD

 025 T/P 55 (55 ) 00000000000000000000000000000001 TOL5/PAR INTACT: 5. RURAL5 5. PARED-?

 CONDITIONING VARIABLE ID: BACK0058

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: TOL5/SCH
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

557

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 T/S 11 (11 ) 0101010101010101 TOL5/SCH INTACT: 1. BIG CTY5 1. PUBLIC

 002 T/S 12 (12 ) -100-100-100-100 TOL5/SCH INTACT: 1. BIG CTY5 2. PRIVATE

 003 T/S 13 (13 ) 00-100-100-100-1 TOL5/SCH INTACT: 1. BIG CTY5 3. CATHOLIC

 004 T/S 21 (21 ) -1-1000000000000 TOL5/SCH INTACT: 2. MID CTY5 1. PUBLIC

 005 T/S 22 (22 ) 0100000000000000 TOL5/SCH INTACT: 2. MID CTY5 2. PRIVATE

 006 T/S 23 (23 ) 0001000000000000 TOL5/SCH INTACT: 2. MID CTY5 3. CATHOLIC

 007 T/S 31 (31 ) 0000-1-100000000 TOL5/SCH INTACT: 3. FR/BTWN5 1. PUBLIC

 008 T/S 32 (32 ) 0000010000000000 TOL5/SCH INTACT: 3. FR/BTWN5 2. PRIVATE

 009 T/S 33 (33 ) 0000000100000000 TOL5/SCH INTACT: 3. FR/BTWN5 3. CATHOLIC

 010 T/S 41 (41 ) 00000000-1-10000 TOL5/SCH INTACT: 4. SML TWN5 1. PUBLIC

 011 T/S 42 (42 ) 0000000001000000 TOL5/SCH INTACT: 4. SML TWN5 2. PRIVATE

 012 T/S 43 (43 ) 0000000000010000 TOL5/SCH INTACT: 4. SML TWN5 3. CATHOLIC

 013 T/S 51 (51 ) 000000000000-1-1 TOL5/SCH INTACT: 5. RURAL5 1. PUBLIC

 014 T/S 52 (52 ) 0000000000000100 TOL5/SCH INTACT: 5. RURAL5 2. PRIVATE

 015 T/S 53 (53 ) 0000000000000001 TOL5/SCH INTACT: 5. RURAL5 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0059

 DESCRIPTION: INTERACTION: PARENTS’ EDUCATION BY SCHOOL TYPE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: PARE/SCH

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 P/S 11 (11 ) 0101010101010101 PARE/SCH INTACT: 1. < HS 1. PUBLIC

 002 P/S 12 (12 ) -100-100-100-100 PARE/SCH INTACT: 1. < HS 2. PRIVATE

 003 P/S 13 (13 ) 00-100-100-100-1 PARE/SCH INTACT: 1. < HS 3. CATHOLIC

 004 P/S 21 (21 ) -1-1000000000000 PARE/SCH INTACT: 2. HS GRAD 1. PUBLIC

 005 P/S 22 (22 ) 0100000000000000 PARE/SCH INTACT: 2. HS GRAD 2. PRIVATE

 006 P/S 23 (23 ) 0001000000000000 PARE/SCH INTACT: 2. HS GRAD 3. CATHOLIC

 007 P/S 31 (31 ) 0000-1-100000000 PARE/SCH INTACT: 3. POST HS 1. PUBLIC

 008 P/S 32 (32 ) 0000010000000000 PARE/SCH INTACT: 3. POST HS 2. PRIVATE

 009 P/S 33 (33 ) 0000000100000000 PARE/SCH INTACT: 3. POST HS 3. CATHOLIC

 010 P/S 41 (41 ) 00000000-1-10000 PARE/SCH INTACT: 4. COL GRAD 1. PUBLIC

 011 P/S 42 (42 ) 0000000001000000 PARE/SCH INTACT: 4. COL GRAD 2. PRIVATE

 012 P/S 43 (43 ) 0000000000010000 PARE/SCH INTACT: 4. COL GRAD 3. CATHOLIC

 013 P/S 51 (51 ) 000000000000-1-1 PARE/SCH INTACT: 5. PARED-? 1. PUBLIC

 014 P/S 52 (52 ) 0000000000000100 PARE/SCH INTACT: 5. PARED-? 2. PRIVATE

 015 P/S 53 (53 ) 0000000000000001 PARE/SCH INTACT: 5. PARED-? 3. CATHOLIC

 CONDITIONING VARIABLE ID: BACK0060

 DESCRIPTION: INTERACTION: GENDER BY SCIENCE COURSES TAKING THIS YEAR

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL: GEND/

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 14

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 G/ 11 (11 ) 010101010101 GEND/ INTACT: 1. MALE 1. NO SCIEN

 002 G/ 12 (12 ) -10000000000 GEND/ INTACT: 1. MALE 2. LIFESCI

 003 G/ 13 (13 ) 00-100000000 GEND/ INTACT: 1. MALE 3. PHYSSCI

 004 G/ 14 (14 ) 0000-1000000 GEND/ INTACT: 1. MALE 4. EATHSCI

 005 G/ 15 (15 ) 000000-10000 GEND/ INTACT: 1. MALE 5. GEN SCI

 006 G/ 16 (16 ) 00000000-100 GEND/ INTACT: 1. MALE 6. INTESCI

 007 G/ 17 (17 ) 0000000000-1 GEND/ INTACT: 1. MALE 7. MISSING

 008 G/ 21 (21 ) -1-1-1-1-1-1 GEND/ INTACT: 2. FEMALE 1. NO SCIEN

 009 G/ 22 (22 ) 010000000000 GEND/ INTACT: 2. FEMALE 2. LIFESCI

 010 G/ 23 (23 ) 000100000000 GEND/ INTACT: 2. FEMALE 3. PHYSSCI

 011 G/ 24 (24 ) 000001000000 GEND/ INTACT: 2. FEMALE 4. EATHSCI

 012 G/ 25 (25 ) 000000010000 GEND/ INTACT: 2. FEMALE 5. GEN SCI

 013 G/ 26 (26 ) 000000000100 GEND/ INTACT: 2. FEMALE 6. INTESCI

 014 G/ 27 (27 ) 000000000001 GEND/ INTACT: 2. FEMALE 7. MISSING

 CONDITIONING VARIABLE ID: BACK0062

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY SCIENCE COURSES TAKING THIS YEAR

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL: RACE/

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 28

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 18

 001 R/ 11 (11 ) 333333333333333333 RACE/ INTACT: 1. WHI/AI/O 1. NO SCIEN

 002 R/ 12 (12 ) 122222122222122222 RACE/ INTACT: 1. WHI/AI/O 2. LIFESCI

 003 R/ 13 (13 ) 212222212222212222 RACE/ INTACT: 1. WHI/AI/O 3. PHYSSCI

 004 R/ 14 (14 ) 221222221222221222 RACE/ INTACT: 1. WHI/AI/O 4. EATHSCI

 005 R/ 15 (15 ) 222122222122222122 RACE/ INTACT: 1. WHI/AI/O 5. GEN SCI

 006 R/ 16 (16 ) 222212222212222212 RACE/ INTACT: 1. WHI/AI/O 6. INTESCI

 007 R/ 17 (17 ) 222221222221222221 RACE/ INTACT: 1. WHI/AI/O 7. MISSING

 008 R/ 21 (21 ) 111111222222222222 RACE/ INTACT: 2. BLACK 1. NO SCIEN

 009 R/ 22 (22 ) 322222222222222222 RACE/ INTACT: 2. BLACK 2. LIFESCI

 010 R/ 23 (23 ) 232222222222222222 RACE/ INTACT: 2. BLACK 3. PHYSSCI

 011 R/ 24 (24 ) 223222222222222222 RACE/ INTACT: 2. BLACK 4. EATHSCI

 012 R/ 25 (25 ) 222322222222222222 RACE/ INTACT: 2. BLACK 5. GEN SCI

 013 R/ 26 (26 ) 222232222222222222 RACE/ INTACT: 2. BLACK 6. INTESCI

 014 R/ 27 (27 ) 222223222222222222 RACE/ INTACT: 2. BLACK 7. MISSING

 015 R/ 31 (31 ) 222222111111222222 RACE/ INTACT: 3. HISPANIC 1. NO SCIEN

 016 R/ 32 (32 ) 222222322222222222 RACE/ INTACT: 3. HISPANIC 2. LIFESCI
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

558

 017 R/ 33 (33 ) 222222232222222222 RACE/ INTACT: 3. HISPANIC 3. PHYSSCI

 018 R/ 34 (34 ) 222222223222222222 RACE/ INTACT: 3. HISPANIC 4. EATHSCI

 019 R/ 35 (35 ) 222222222322222222 RACE/ INTACT: 3. HISPANIC 5. GEN SCI

 020 R/ 36 (36 ) 222222222232222222 RACE/ INTACT: 3. HISPANIC 6. INTESCI

 021 R/ 37 (37 ) 222222222223222222 RACE/ INTACT: 3. HISPANIC 7. MISSING

 022 R/ 41 (41 ) 222222222222111111 RACE/ INTACT: 4. ASIAN 1. NO SCIEN

 023 R/ 42 (42 ) 222222222222322222 RACE/ INTACT: 4. ASIAN 2. LIFESCI

 024 R/ 43 (43 ) 222222222222232222 RACE/ INTACT: 4. ASIAN 3. PHYSSCI

 025 R/ 44 (44 ) 222222222222223222 RACE/ INTACT: 4. ASIAN 4. EATHSCI

 026 R/ 45 (45 ) 222222222222222322 RACE/ INTACT: 4. ASIAN 5. GEN SCI

 027 R/ 46 (46 ) 222222222222222232 RACE/ INTACT: 4. ASIAN 6. INTESCI

 028 R/ 47 (47 ) 222222222222222223 RACE/ INTACT: 4. ASIAN 7. MISSING

 CONDITIONING VARIABLE ID: BACK0064

 DESCRIPTION: INTERACTION: PARENTS’ EDUCATION BY SCIENCE COURSES TAKING THIS YEAR

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL: PARE/

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 35

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 24

 001 P/ 11 (11 ) 333333333333333333333333 PARE/ INTACT: 1. < HS 1. NO SCIEN

 002 P/ 12 (12 ) 122222122222122222122222 PARE/ INTACT: 1. < HS 2. LIFESCI

 003 P/ 13 (13 ) 212222212222212222212222 PARE/ INTACT: 1. < HS 3. PHYSSCI

 004 P/ 14 (14 ) 221222221222221222221222 PARE/ INTACT: 1. < HS 4. EATHSCI

 005 P/ 15 (15 ) 222122222122222122222122 PARE/ INTACT: 1. < HS 5. GEN SCI

 006 P/ 16 (16 ) 222212222212222212222212 PARE/ INTACT: 1. < HS 6. INTESCI

 007 P/ 17 (17 ) 222221222221222221222221 PARE/ INTACT: 1. < HS 7. MISSING

 008 P/ 21 (21 ) 111111222222222222222222 PARE/ INTACT: 2. HS GRAD 1. NO SCIEN

 009 P/ 22 (22 ) 322222222222222222222222 PARE/ INTACT: 2. HS GRAD 2. LIFESCI

 010 P/ 23 (23 ) 232222222222222222222222 PARE/ INTACT: 2. HS GRAD 3. PHYSSCI

 011 P/ 24 (24 ) 223222222222222222222222 PARE/ INTACT: 2. HS GRAD 4. EATHSCI

 012 P/ 25 (25 ) 222322222222222222222222 PARE/ INTACT: 2. HS GRAD 5. GEN SCI

 013 P/ 26 (26 ) 222232222222222222222222 PARE/ INTACT: 2. HS GRAD 6. INTESCI

 014 P/ 27 (27 ) 222223222222222222222222 PARE/ INTACT: 2. HS GRAD 7. MISSING

 015 P/ 31 (31 ) 222222111111222222222222 PARE/ INTACT: 3. POST HS 1. NO SCIEN

 016 P/ 32 (32 ) 222222322222222222222222 PARE/ INTACT: 3. POST HS 2. LIFESCI

 017 P/ 33 (33 ) 222222232222222222222222 PARE/ INTACT: 3. POST HS 3. PHYSSCI

 018 P/ 34 (34 ) 222222223222222222222222 PARE/ INTACT: 3. POST HS 4. EATHSCI

 019 P/ 35 (35 ) 222222222322222222222222 PARE/ INTACT: 3. POST HS 5. GEN SCI

 020 P/ 36 (36 ) 222222222232222222222222 PARE/ INTACT: 3. POST HS 6. INTESCI

 021 P/ 37 (37 ) 222222222223222222222222 PARE/ INTACT: 3. POST HS 7. MISSING

 022 P/ 41 (41 ) 222222222222111111222222 PARE/ INTACT: 4. COL GRAD 1. NO SCIEN

 023 P/ 42 (42 ) 222222222222322222222222 PARE/ INTACT: 4. COL GRAD 2. LIFESCI

 024 P/ 43 (43 ) 222222222222232222222222 PARE/ INTACT: 4. COL GRAD 3. PHYSSCI

 025 P/ 44 (44 ) 222222222222223222222222 PARE/ INTACT: 4. COL GRAD 4. EATHSCI

 026 P/ 45 (45 ) 222222222222222322222222 PARE/ INTACT: 4. COL GRAD 5. GEN SCI

 027 P/ 46 (46 ) 222222222222222232222222 PARE/ INTACT: 4. COL GRAD 6. INTESCI

 028 P/ 47 (47 ) 222222222222222223222222 PARE/ INTACT: 4. COL GRAD 7. MISSING

 029 P/ 51 (51 ) 222222222222222222111111 PARE/ INTACT: 5. PARED-? 1. NO SCIEN

 030 P/ 52 (52 ) 222222222222222222322222 PARE/ INTACT: 5. PARED-? 2. LIFESCI

 031 P/ 53 (53 ) 222222222222222222232222 PARE/ INTACT: 5. PARED-? 3. PHYSSCI

 032 P/ 54 (54 ) 222222222222222222223222 PARE/ INTACT: 5. PARED-? 4. EATHSCI

 033 P/ 55 (55 ) 222222222222222222222322 PARE/ INTACT: 5. PARED-? 5. GEN SCI

 034 P/ 56 (56 ) 222222222222222222222232 PARE/ INTACT: 5. PARED-? 6. INTESCI

 035 P/ 57 (57 ) 222222222222222222222223 PARE/ INTACT: 5. PARED-? 7. MISSING

 CONDITIONING VARIABLE ID: BACK0066

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY SCIENCE COURSES TAKING THIS YEAR

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL: TOL5/

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 35

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 24

 001 T/ 11 (11 ) 333333333333333333333333 TOL5/ INTACT: 1. BIG CTY5 1. NO SCIEN

 002 T/ 12 (12 ) 122222122222122222122222 TOL5/ INTACT: 1. BIG CTY5 2. LIFESCI

 003 T/ 13 (13 ) 212222212222212222212222 TOL5/ INTACT: 1. BIG CTY5 3. PHYSSCI

 004 T/ 14 (14 ) 221222221222221222221222 TOL5/ INTACT: 1. BIG CTY5 4. EATHSCI

 005 T/ 15 (15 ) 222122222122222122222122 TOL5/ INTACT: 1. BIG CTY5 5. GEN SCI

 006 T/ 16 (16 ) 222212222212222212222212 TOL5/ INTACT: 1. BIG CTY5 6. INTESCI

 007 T/ 17 (17 ) 222221222221222221222221 TOL5/ INTACT: 1. BIG CTY5 7. MISSING

 008 T/ 21 (21 ) 111111222222222222222222 TOL5/ INTACT: 2. MID CTY5 1. NO SCIEN

 009 T/ 22 (22 ) 322222222222222222222222 TOL5/ INTACT: 2. MID CTY5 2. LIFESCI

 010 T/ 23 (23 ) 232222222222222222222222 TOL5/ INTACT: 2. MID CTY5 3. PHYSSCI

 011 T/ 24 (24 ) 223222222222222222222222 TOL5/ INTACT: 2. MID CTY5 4. EATHSCI

 012 T/ 25 (25 ) 222322222222222222222222 TOL5/ INTACT: 2. MID CTY5 5. GEN SCI

 013 T/ 26 (26 ) 222232222222222222222222 TOL5/ INTACT: 2. MID CTY5 6. INTESCI

 014 T/ 27 (27 ) 222223222222222222222222 TOL5/ INTACT: 2. MID CTY5 7. MISSING

 015 T/ 31 (31 ) 222222111111222222222222 TOL5/ INTACT: 3. FR/BTWN5 1. NO SCIEN

 016 T/ 32 (32 ) 222222322222222222222222 TOL5/ INTACT: 3. FR/BTWN5 2. LIFESCI

 017 T/ 33 (33 ) 222222232222222222222222 TOL5/ INTACT: 3. FR/BTWN5 3. PHYSSCI

 018 T/ 34 (34 ) 222222223222222222222222 TOL5/ INTACT: 3. FR/BTWN5 4. EATHSCI

 019 T/ 35 (35 ) 222222222322222222222222 TOL5/ INTACT: 3. FR/BTWN5 5. GEN SCI

 020 T/ 36 (36 ) 222222222232222222222222 TOL5/ INTACT: 3. FR/BTWN5 6. INTESCI

 021 T/ 37 (37 ) 222222222223222222222222 TOL5/ INTACT: 3. FR/BTWN5 7. MISSING

 022 T/ 41 (41 ) 222222222222111111222222 TOL5/ INTACT: 4. SML TWN5 1. NO SCIEN

 023 T/ 42 (42 ) 222222222222322222222222 TOL5/ INTACT: 4. SML TWN5 2. LIFESCI

 024 T/ 43 (43 ) 222222222222232222222222 TOL5/ INTACT: 4. SML TWN5 3. PHYSSCI
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

559

 025 T/ 44 (44 ) 222222222222223222222222 TOL5/ INTACT: 4. SML TWN5 4. EATHSCI

 026 T/ 45 (45 ) 222222222222222322222222 TOL5/ INTACT: 4. SML TWN5 5. GEN SCI

 027 T/ 46 (46 ) 222222222222222232222222 TOL5/ INTACT: 4. SML TWN5 6. INTESCI

 028 T/ 47 (47 ) 222222222222222223222222 TOL5/ INTACT: 4. SML TWN5 7. MISSING

 029 T/ 51 (51 ) 222222222222222222111111 TOL5/ INTACT: 5. RURAL5 1. NO SCIEN

 030 T/ 52 (52 ) 222222222222222222322222 TOL5/ INTACT: 5. RURAL5 2. LIFESCI

 031 T/ 53 (53 ) 222222222222222222232222 TOL5/ INTACT: 5. RURAL5 3. PHYSSCI

 032 T/ 54 (54 ) 222222222222222222223222 TOL5/ INTACT: 5. RURAL5 4. EATHSCI

 033 T/ 55 (55 ) 222222222222222222222322 TOL5/ INTACT: 5. RURAL5 5. GEN SCI

 034 T/ 56 (56 ) 222222222222222222222232 TOL5/ INTACT: 5. RURAL5 6. INTESCI

 035 T/ 57 (57 ) 222222222222222222222223 TOL5/ INTACT: 5. RURAL5 7. MISSING

 CONDITIONING VARIABLE ID: BACK0068

 DESCRIPTION: INTERACTION: SCHOOL TYPE BY SCIENCE COURSES TAKING THIS YEAR

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL: SCHT/

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 21

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 S/ 11 (11 ) -1-1-1-1-1-1-1-1-1-1-1-1 SCHT/ INTACT: 1. PUBLIC 1. NO SCIEN

 002 S/ 12 (12 ) -10000000000-10000000000 SCHT/ INTACT: 1. PUBLIC 2. LIFESCI

 003 S/ 13 (13 ) 00-10000000000-100000000 SCHT/ INTACT: 1. PUBLIC 3. PHYSSCI

 004 S/ 14 (14 ) 0000-10000000000-1000000 SCHT/ INTACT: 1. PUBLIC 4. EATHSCI

 005 S/ 15 (15 ) 000000-10000000000-10000 SCHT/ INTACT: 1. PUBLIC 5. GEN SCI

 006 S/ 16 (16 ) 00000000-10000000000-100 SCHT/ INTACT: 1. PUBLIC 6. INTESCI

 007 S/ 17 (17 ) 0000000000-10000000000-1 SCHT/ INTACT: 1. PUBLIC 7. MISSING

 008 S/ 21 (21 ) -1-1-1-1-1-1000000000000 SCHT/ INTACT: 2. PRIVATE 1. NO SCIEN

 009 S/ 22 (22 ) 010000000000000000000000 SCHT/ INTACT: 2. PRIVATE 2. LIFESCI

 010 S/ 23 (23 ) 000100000000000000000000 SCHT/ INTACT: 2. PRIVATE 3. PHYSSCI

 011 S/ 24 (24 ) 000001000000000000000000 SCHT/ INTACT: 2. PRIVATE 4. EATHSCI

 012 S/ 25 (25 ) 000000010000000000000000 SCHT/ INTACT: 2. PRIVATE 5. GEN SCI

 013 S/ 26 (26 ) 000000000100000000000000 SCHT/ INTACT: 2. PRIVATE 6. INTESCI

 014 S/ 27 (27 ) 000000000001000000000000 SCHT/ INTACT: 2. PRIVATE 7. MISSING

 015 S/ 31 (31 ) 000000000000010101010101 SCHT/ INTACT: 3. CATHOLIC 1. NO SCIEN

 016 S/ 32 (32 ) 000000000000010000000000 SCHT/ INTACT: 3. CATHOLIC 2. LIFESCI

 017 S/ 33 (33 ) 000000000000000100000000 SCHT/ INTACT: 3. CATHOLIC 3. PHYSSCI

 018 S/ 34 (34 ) 000000000000000001000000 SCHT/ INTACT: 3. CATHOLIC 4. EATHSCI

 019 S/ 35 (35 ) 000000000000000000010000 SCHT/ INTACT: 3. CATHOLIC 5. GEN SCI

 020 S/ 36 (36 ) 000000000000000000000100 SCHT/ INTACT: 3. CATHOLIC 6. INTESCI

 021 S/ 37 (37 ) 000000000000000000000001 SCHT/ INTACT: 3. CATHOLIC 7. MISSING

 CONDITIONING VARIABLE ID: BACK0061

 DESCRIPTION: INTERACTION: GENDER BY NUMBER OF SEMESTERS SCIENCE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: GEND/SEM

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 10

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 4

 001 G/S 11 (11 ) 01010101 GEND/SEM INTACT: 1. MALE 1. SEMSCI-?

 002 G/S 12 (12 ) -1000000 GEND/SEM INTACT: 1. MALE 2. SEMSCI-^

 003 G/S 13 (13 ) 00-10000 GEND/SEM INTACT: 1. MALE 3. SEMSCI-^

 004 G/S 14 (14 ) 0000-100 GEND/SEM INTACT: 1. MALE 4. SEMSCI-^

 005 G/S 15 (15 ) 0000-100 GEND/SEM INTACT: 1. MALE 5. SEMSCI-^

 006 G/S 21 (21 ) -1-1-1-1 GEND/SEM INTACT: 2. FEMALE 1. SEMSCI-?

 007 G/S 22 (22 ) 01000000 GEND/SEM INTACT: 2. FEMALE 2. SEMSCI-^

 008 G/S 23 (23 ) 00010000 GEND/SEM INTACT: 2. FEMALE 3. SEMSCI-^

 009 G/S 24 (24 ) 00000100 GEND/SEM INTACT: 2. FEMALE 4. SEMSCI-^

 010 G/S 25 (25 ) 00000100 GEND/SEM INTACT: 2. FEMALE 5. SEMSCI-^

 CONDITIONING VARIABLE ID: BACK0063

 DESCRIPTION: INTERACTION: RACE/ETHNICITY BY NUMBER OF SEMESTERS SCIENCE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: RACE/SEM

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 20

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 12

 001 R/S 11 (11 ) 010101010101010101010101 RACE/SEM INTACT: 1. WHI/AI/O 1. SEMSCI-?

 002 R/S 12 (12 ) -1000000-1000000-1000000 RACE/SEM INTACT: 1. WHI/AI/O 2. SEMSCI-^

 003 R/S 13 (13 ) 00-1000000-1000000-10000 RACE/SEM INTACT: 1. WHI/AI/O 3. SEMSCI-^

 004 R/S 14 (14 ) 0000-1000000-1000000-100 RACE/SEM INTACT: 1. WHI/AI/O 4. SEMSCI-^

 005 R/S 15 (15 ) 000000-1000000-1000000-1 RACE/SEM INTACT: 1. WHI/AI/O 5. SEMSCI-^

 006 R/S 21 (21 ) -1-1-1000000000000000000 RACE/SEM INTACT: 2. BLACK 1. SEMSCI-?

 007 R/S 22 (22 ) 010000000000000000000000 RACE/SEM INTACT: 2. BLACK 2. SEMSCI-^

 008 R/S 23 (23 ) 000100000000000000000000 RACE/SEM INTACT: 2. BLACK 3. SEMSCI-^

 009 R/S 24 (24 ) 000001000000000000000000 RACE/SEM INTACT: 2. BLACK 4. SEMSCI-^

 010 R/S 25 (25 ) 000000010000000000000000 RACE/SEM INTACT: 2. BLACK 5. SEMSCI-^

 011 R/S 31 (31 ) 000000-1-1-1000000000000 RACE/SEM INTACT: 3. HISPANIC 1. SEMSCI-?

 012 R/S 32 (32 ) 000000000100000000000000 RACE/SEM INTACT: 3. HISPANIC 2. SEMSCI-^

 013 R/S 33 (33 ) 000000000001000000000000 RACE/SEM INTACT: 3. HISPANIC 3. SEMSCI-^

 014 R/S 34 (34 ) 000000000000010000000000 RACE/SEM INTACT: 3. HISPANIC 4. SEMSCI-^

 015 R/S 35 (35 ) 000000000000000100000000 RACE/SEM INTACT: 3. HISPANIC 5. SEMSCI-^

 016 R/S 41 (41 ) 000000000000-1-1-1000000 RACE/SEM INTACT: 4. ASIAN 1. SEMSCI-?

 017 R/S 42 (42 ) 000000000000000001000000 RACE/SEM INTACT: 4. ASIAN 2. SEMSCI-^

 018 R/S 43 (43 ) 000000000000000000010000 RACE/SEM INTACT: 4. ASIAN 3. SEMSCI-^

 019 R/S 44 (44 ) 000000000000000000000100 RACE/SEM INTACT: 4. ASIAN 4. SEMSCI-^

 020 R/S 45 (45 ) 000000000000000000000001 RACE/SEM INTACT: 4. ASIAN 5. SEMSCI-^
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

560

 CONDITIONING VARIABLE ID: BACK0065

 DESCRIPTION: INTERACTION: PARENTS’ EDUCATION BY NUMBER OF SEMESTERS SCIENCE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: PARE/SEM

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 25

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 16

 001 P/S 11 (11 ) 01010101010101010101010101010101 PARE/SEM INTACT: 1. < HS 1. SEMSCI-?

 002 P/S 12 (12 ) -1000000-1000000-1000000-1000000 PARE/SEM INTACT: 1. < HS 2. SEMSCI-^

 003 P/S 13 (13 ) 00-1000000-1000000-1000000-10000 PARE/SEM INTACT: 1. < HS 3. SEMSCI-^

 004 P/S 14 (14 ) 0000-1000000-1000000-1000000-100 PARE/SEM INTACT: 1. < HS 4. SEMSCI-^

 005 P/S 15 (15 ) 000000-1000000-1000000-1000000-1 PARE/SEM INTACT: 1. < HS 5. SEMSCI-^

 006 P/S 21 (21 ) -1-1-100000000000000000000000000 PARE/SEM INTACT: 2. HS GRAD 1. SEMSCI-?

 007 P/S 22 (22 ) 01000000000000000000000000000000 PARE/SEM INTACT: 2. HS GRAD 2. SEMSCI-^

 008 P/S 23 (23 ) 00010000000000000000000000000000 PARE/SEM INTACT: 2. HS GRAD 3. SEMSCI-^

 009 P/S 24 (24 ) 00000100000000000000000000000000 PARE/SEM INTACT: 2. HS GRAD 4. SEMSCI-^

 010 P/S 25 (25 ) 00000001000000000000000000000000 PARE/SEM INTACT: 2. HS GRAD 5. SEMSCI-^

 011 P/S 31 (31 ) 000000-1-1-100000000000000000000 PARE/SEM INTACT: 3. POST HS 1. SEMSCI-?

 012 P/S 32 (32 ) 00000000010000000000000000000000 PARE/SEM INTACT: 3. POST HS 2. SEMSCI-^

 013 P/S 33 (33 ) 00000000000100000000000000000000 PARE/SEM INTACT: 3. POST HS 3. SEMSCI-^

 014 P/S 34 (34 ) 00000000000001000000000000000000 PARE/SEM INTACT: 3. POST HS 4. SEMSCI-^

 015 P/S 35 (35 ) 00000000000000010000000000000000 PARE/SEM INTACT: 3. POST HS 5. SEMSCI-^

 016 P/S 41 (41 ) 000000000000-1-1-100000000000000 PARE/SEM INTACT: 4. COL GRAD 1. SEMSCI-?

 017 P/S 42 (42 ) 00000000000000000100000000000000 PARE/SEM INTACT: 4. COL GRAD 2. SEMSCI-^

 018 P/S 43 (43 ) 00000000000000000001000000000000 PARE/SEM INTACT: 4. COL GRAD 3. SEMSCI-^

 019 P/S 44 (44 ) 00000000000000000000010000000000 PARE/SEM INTACT: 4. COL GRAD 4. SEMSCI-^

 020 P/S 45 (45 ) 00000000000000000000000100000000 PARE/SEM INTACT: 4. COL GRAD 5. SEMSCI-^

 021 P/S 51 (51 ) 000000000000000000-1-1-100000000 PARE/SEM INTACT: 5. PARED-? 1. SEMSCI-?

 022 P/S 52 (52 ) 00000000000000000000000001000000 PARE/SEM INTACT: 5. PARED-? 2. SEMSCI-^

 023 P/S 53 (53 ) 00000000000000000000000000010000 PARE/SEM INTACT: 5. PARED-? 3. SEMSCI-^

 024 P/S 54 (54 ) 00000000000000000000000000000100 PARE/SEM INTACT: 5. PARED-? 4. SEMSCI-^

 025 P/S 55 (55 ) 00000000000000000000000000000001 PARE/SEM INTACT: 5. PARED-? 5. SEMSCI-^

 CONDITIONING VARIABLE ID: BACK0067

 DESCRIPTION: INTERACTION: TYPE OF LOCALE (5 CATEGORIES) BY NUMBER OF SEMESTERS SCIENCE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: TOL5/SEM

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 25

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 16

 001 T/S 11 (11 ) 01010101010101010101010101010101 TOL5/SEM INTACT: 1. BIG CTY5 1. SEMSCI-?

 002 T/S 12 (12 ) -1000000-1000000-1000000-1000000 TOL5/SEM INTACT: 1. BIG CTY5 2. SEMSCI-^

 003 T/S 13 (13 ) 00-1000000-1000000-1000000-10000 TOL5/SEM INTACT: 1. BIG CTY5 3. SEMSCI-^

 004 T/S 14 (14 ) 0000-1000000-1000000-1000000-100 TOL5/SEM INTACT: 1. BIG CTY5 4. SEMSCI-^

 005 T/S 15 (15 ) 000000-1000000-1000000-1000000-1 TOL5/SEM INTACT: 1. BIG CTY5 5. SEMSCI-^

 006 T/S 21 (21 ) -1-1-100000000000000000000000000 TOL5/SEM INTACT: 2. MID CTY5 1. SEMSCI-?

 007 T/S 22 (22 ) 01000000000000000000000000000000 TOL5/SEM INTACT: 2. MID CTY5 2. SEMSCI-^

 008 T/S 23 (23 ) 00010000000000000000000000000000 TOL5/SEM INTACT: 2. MID CTY5 3. SEMSCI-^

 009 T/S 24 (24 ) 00000100000000000000000000000000 TOL5/SEM INTACT: 2. MID CTY5 4. SEMSCI-^

 010 T/S 25 (25 ) 00000001000000000000000000000000 TOL5/SEM INTACT: 2. MID CTY5 5. SEMSCI-^

 011 T/S 31 (31 ) 000000-1-1-100000000000000000000 TOL5/SEM INTACT: 3. FR/BTWN5 1. SEMSCI-?

 012 T/S 32 (32 ) 00000000010000000000000000000000 TOL5/SEM INTACT: 3. FR/BTWN5 2. SEMSCI-^

 013 T/S 33 (33 ) 00000000000100000000000000000000 TOL5/SEM INTACT: 3. FR/BTWN5 3. SEMSCI-^

 014 T/S 34 (34 ) 00000000000001000000000000000000 TOL5/SEM INTACT: 3. FR/BTWN5 4. SEMSCI-^

 015 T/S 35 (35 ) 00000000000000010000000000000000 TOL5/SEM INTACT: 3. FR/BTWN5 5. SEMSCI-^

 016 T/S 41 (41 ) 000000000000-1-1-100000000000000 TOL5/SEM INTACT: 4. SML TWN5 1. SEMSCI-?

 017 T/S 42 (42 ) 00000000000000000100000000000000 TOL5/SEM INTACT: 4. SML TWN5 2. SEMSCI-^

 018 T/S 43 (43 ) 00000000000000000001000000000000 TOL5/SEM INTACT: 4. SML TWN5 3. SEMSCI-^

 019 T/S 44 (44 ) 00000000000000000000010000000000 TOL5/SEM INTACT: 4. SML TWN5 4. SEMSCI-^

 020 T/S 45 (45 ) 00000000000000000000000100000000 TOL5/SEM INTACT: 4. SML TWN5 5. SEMSCI-^

 021 T/S 51 (51 ) 000000000000000000-1-1-100000000 TOL5/SEM INTACT: 5. RURAL5 1. SEMSCI-?

 022 T/S 52 (52 ) 00000000000000000000000001000000 TOL5/SEM INTACT: 5. RURAL5 2. SEMSCI-^

 023 T/S 53 (53 ) 00000000000000000000000000010000 TOL5/SEM INTACT: 5. RURAL5 3. SEMSCI-^

 024 T/S 54 (54 ) 00000000000000000000000000000100 TOL5/SEM INTACT: 5. RURAL5 4. SEMSCI-^

 025 T/S 55 (55 ) 00000000000000000000000000000001 TOL5/SEM INTACT: 5. RURAL5 5. SEMSCI-^

 CONDITIONING VARIABLE ID: BACK0069

 DESCRIPTION: INTERACTION: SCHOOL TYPE BY NUMBER OF SEMESTERS SCIENCE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL: SCHT/SEM

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 15

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 8

 001 S/S 11 (11 ) 0101010101010101 SCHT/SEM INTACT: 1. PUBLIC 1. SEMSCI-?

 002 S/S 12 (12 ) -1000000-1000000 SCHT/SEM INTACT: 1. PUBLIC 2. SEMSCI-^

 003 S/S 13 (13 ) 00-1000000-10000 SCHT/SEM INTACT: 1. PUBLIC 3. SEMSCI-^

 004 S/S 14 (14 ) 0000-1000000-100 SCHT/SEM INTACT: 1. PUBLIC 4. SEMSCI-^

 005 S/S 15 (15 ) 000000-1000000-1 SCHT/SEM INTACT: 1. PUBLIC 5. SEMSCI-^

 006 S/S 21 (21 ) -1-1-10000000000 SCHT/SEM INTACT: 2. PRIVATE 1. SEMSCI-?

 007 S/S 22 (22 ) 0100000000000000 SCHT/SEM INTACT: 2. PRIVATE 2. SEMSCI-^

 008 S/S 23 (23 ) 0001000000000000 SCHT/SEM INTACT: 2. PRIVATE 3. SEMSCI-^

 009 S/S 24 (24 ) 0000010000000000 SCHT/SEM INTACT: 2. PRIVATE 4. SEMSCI-^

 010 S/S 25 (25 ) 0000000100000000 SCHT/SEM INTACT: 2. PRIVATE 5. SEMSCI-^

 011 S/S 31 (31 ) 000000-1-1-10000 SCHT/SEM INTACT: 3. CATHOLIC 1. SEMSCI-?

 012 S/S 32 (32 ) 0000000001000000 SCHT/SEM INTACT: 3. CATHOLIC 2. SEMSCI-^

 013 S/S 33 (33 ) 0000000000010000 SCHT/SEM INTACT: 3. CATHOLIC 3. SEMSCI-^

 014 S/S 34 (34 ) 0000000000000100 SCHT/SEM INTACT: 3. CATHOLIC 4. SEMSCI-^
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

561

 015 S/S 35 (35 ) 0000000000000001 SCHT/SEM INTACT: 3. CATHOLIC 5. SEMSCI-^

 CONDITIONING VARIABLE ID: BACK0073

 DESCRIPTION: SAMPLE TYPE

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: SUBSAMP TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 SAMP S1 (01 ) 00 SAMPLE S1

 002 SAMP S2 (02 ) 10 SAMPLE S2 AND S3

 003 SAMP S3 (03 ) 01 SAMPLE S3 AND S3

 CONDITIONING VARIABLE ID: BACK0074

 DESCRIPTION: INTERACTION: SAMPLE BY RACE/ETHNICITY

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL: /RAC

 NAEP ID: N/A TOTAL NUMBER OF SPECIFIED CONTRASTS: 12

 TYPE OF CONTRAST: INTERACTION NUMBER OF INDEPENDENT CONTRASTS: 6

 001 /R 11 (11 ) 010101010101 /RAC INTACT: 1. SAMP S1 1. WHI/AI/O

 002 /R 12 (12 ) -10000-10000 /RAC INTACT: 1. SAMP S1 2. BLACK

 003 /R 13 (13 ) 00-10000-100 /RAC INTACT: 1. SAMP S1 3. HISPANIC

 004 /R 14 (14 ) 0000-10000-1 /RAC INTACT: 1. SAMP S1 4. ASIAN

 005 /R 21 (21 ) -1-1-1000000 /RAC INTACT: 2. SAMP S2 1. WHI/AI/O

 006 /R 22 (22 ) 010000000000 /RAC INTACT: 2. SAMP S2 2. BLACK

 007 /R 23 (23 ) 000100000000 /RAC INTACT: 2. SAMP S2 3. HISPANIC

 008 /R 24 (24 ) 000001000000 /RAC INTACT: 2. SAMP S2 4. ASIAN

 009 /R 31 (31 ) 000000-1-1-1 /RAC INTACT: 3. SAMP S3 1. WHI/AI/O

 010 /R 32 (32 ) 000000010000 /RAC INTACT: 3. SAMP S3 2. BLACK

 011 /R 33 (33 ) 000000000100 /RAC INTACT: 3. SAMP S3 3. HISPANIC

 012 /R 34 (34 ) 000000000001 /RAC INTACT: 3. SAMP S3 4. ASIAN

 CONDITIONING VARIABLE ID: BACK0075

 DESCRIPTION: DO YOU HAVE YOUR OWN STUDY DESK OR TABLE AT HOME?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B008901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 B008901Y (01 ) 000 YES

 002 B008901N (02 ) 100 NO

 003 B008901C (03 ) 010 IDK

 004 B008901M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: BACK0076

 DESCRIPTION: HOW SAFE DO YOU FEEL AT SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B009401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 VRY SAFE (01 ) 0000 VERY SAFE

 002 SAFE (02 ) 1000 SOMEWHAT SAFE

 003 UNSAFE (03 ) 0100 SOMEWHAT UNSAFE

 004 VRUNSAFE (04 ) 0010 VERY UNSAFE

 005 MISSING (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0077

 DESCRIPTION: DOES MOTHER OR STEPMOTHER LIVE AT HOME WITH YOU?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: MOM@HOME

 NAEP ID: B005601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 MOMHOM-Y (1 ) 00 MOTHER AT HOME: YES

 002 MOMHOM-N (2 ) 10 MOTHER AT HOME: NO

 003 MOMHOM-? (M ) 01 MOTHER AT HOME: MISSING

 CONDITIONING VARIABLE ID: BACK0078

 DESCRIPTION: DOES FATHER OR STEPFATHER LIVE AT HOME WITH YOU?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL: DAD@HOME

 NAEP ID: B005701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 DADHOM-Y (1 ) 00 FATHER AT HOME: YES

 002 DADHOM-N (2 ) 10 FATHER AT HOME: NO

 003 DADHOM-? (M ) 01 FATHER AT HOME: MISSING

 CONDITIONING VARIABLE ID: SUBJ0002

 DESCRIPTION: AGREE/DISAGREE: I LIKE SCIENCE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

562

 001 K811001A (01 ) 000 AGREE

 002 K811001B (02 ) 100 NOT SURE

 003 K811001C (03 ) 010 DISAGREE

 004 K811001M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0003

 DESCRIPTION: AGREE/DISAGREE: I AM GOOD AT SCIENCE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 K811002A (01 ) 000 AGREE

 002 K811002B (02 ) 100 NOT SURE

 003 K811002C (03 ) 010 DISAGREE

 004 K811002M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0004

 DESCRIPTION: AGREE/DISAGREE: LEARNING SCI MOSTLY MEMORIZATION

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 K811003A (01 ) 000 AGREE

 002 K811003B (02 ) 100 NOT SURE

 003 K811003C (03 ) 010 DISAGREE

 004 K811003M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0005

 DESCRIPTION: AGREE/DISAGREE: SCI USEFUL FOR EVERYDAY PROBLEMS

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 K811004A (01 ) 000 AGREE

 002 K811004B (02 ) 100 NOT SURE

 003 K811004C (03 ) 010 DISAGREE

 004 K811004M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0006

 DESCRIPTION: AGREE/DISAGREE: IF CHOICE, WOULD NOT STUDY SCIENCE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 K811005A (01 ) 000 AGREE

 002 K811005B (02 ) 100 NOT SURE

 003 K811005C (03 ) 010 DISAGREE

 004 K811005M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0007

 DESCRIPTION: AGREE/DISAGREE: ALL CAN DO WELL IN SCI IF THEY TRY

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 K811006A (01 ) 000 AGREE

 002 K811006B (02 ) 100 NOT SURE

 003 K811006C (03 ) 010 DISAGREE

 004 K811006M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0008

 DESCRIPTION: AGREE/DISAGREE: SCIENCE IS BORING

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 K811007A (01 ) 000 AGREE

 002 K811007B (02 ) 100 NOT SURE

 003 K811007C (03 ) 010 DISAGREE

 004 K811007M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0009

 DESCRIPTION: AGREE/DISAGREE: SCIENCE IS A HARD SUBJECT

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811008 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 K811008A (01 ) 000 AGREE

 002 K811008B (02 ) 100 NOT SURE

 003 K811008C (03 ) 010 DISAGREE
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

563

 004 K811008M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0010

 DESCRIPTION: EVER DONE HANDS-ON PROJECT WITH LIVING THINGS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: K811101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K811101Y (01 ) 0 YES

 002 K811101M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0011

 DESCRIPTION: EVER DONE HANDS-ON PROJECT WITH ELECTRICITY?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: K811102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K811102Y (01 ) 0 YES

 002 K811102M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0012

 DESCRIPTION: EVER DONE HANDS-ON PROJECT WITH CHEMICALS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: K811103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K811103Y (01 ) 0 YES

 002 K811103M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0013

 DESCRIPTION: EVER DONE HANDS-ON PROJECT WITH ROCKS OR MINERALS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: K811104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K811104Y (01 ) 0 YES

 002 K811104M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0014

 DESCRIPTION: DONE HANDS-ON PROJ W/ MAGNIFYING GLASS/MICROSCOPE?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: K811105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K811105Y (01 ) 0 YES

 002 K811105M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0015

 DESCRIPTION: DONE HANDS-ON PROJ W/ THERMOMETER OR BAROMETER?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: K811106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K811106Y (01 ) 0 YES

 002 K811106M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0016

 DESCRIPTION: EVER DONE HANDS-ON PROJECT WITH SIMPLE MACHINES?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: K811107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K811107Y (01 ) 0 YES

 002 K811107M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0017

 DESCRIPTION: HAVE DONE HANDS-ON PROJECT WITH NONE OF THE ABOVE?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: K811108 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K811108Y (01 ) 0 YES

 002 K811108M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0018

 DESCRIPTION: HOW OFTEN DO YOU STUDY SCIENCE IN SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: K811201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

564

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 K811201A (01 ) 00000 EVERY DAY

 002 K811201B (02 ) 10000 3-4 TIMES A WEEK

 003 K811201C (03 ) 01000 1-2 TIMES A WEEK

 004 K811201D (04 ) 00100 LESS THEN ONCE WEEK

 005 K811201E (05 ) 00010 NEVER

 006 K811201M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0019

 DESCRIPTION: HOW MUCH TIME PER WEEK DOING SCIENCE HOMEWORK?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: K811301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 K811301A (01 ) 000000 DON’T HAVE SCIENCE

 002 K811301B (02 ) 100000 NONE

 003 K811301C (03 ) 010000 1/2 HOUR

 004 K811301D (04 ) 001000 1 HOUR

 005 K811301E (05 ) 000100 2 HOURS

 006 K811301F (06 ) 000010 MORE THAN 2 HOURS

 007 K811301M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0020

 DESCRIPTION: DO SCI PROJECTS IN SCHOOL THAT TAKE 1 OR MORE WKS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: K811401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 K811401Y (01 ) 00 YES

 002 K811401N (02 ) 10 NO

 003 K811401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0021

 DESCRIPTION: LAST 2 YRS, BEEN IN SCI FAIR, FESTIVAL, SCI DAY?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 K811501Y (01 ) 00 YES

 002 K811501N (02 ) 10 NO

 003 K811501M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0022

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN DO YOU READ TEXTBOOK?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811601A (01 ) 0000 ALMOST EVERY DAY

 002 K811601B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811601C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811601D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0023

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN DO YOU READ MAGS/BKS?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811602A (01 ) 0000 ALMOST EVERY DAY

 002 K811602B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811602C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811602D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811602M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0024

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN DISCUSS SCIENCE NEWS?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811603A (01 ) 0000 ALMOST EVERY DAY

 002 K811603B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811603C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811603D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811603M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0025

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN WORK WITH OTHERS?
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

565

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811604A (01 ) 0000 ALMOST EVERY DAY

 002 K811604B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811604C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811604D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811604M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0026

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN GIVE ORAL REPORT?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811605A (01 ) 0000 ALMOST EVERY DAY

 002 K811605B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811605C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811605D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811605M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0027

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN GIVE WRITTEN REPORT?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811606 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811606A (01 ) 0000 ALMOST EVERY DAY

 002 K811606B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811606C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811606D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811606M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0028

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN DO HANDS-ON PROJECT?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: K811607 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811607A (01 ) 0000 ALMOST EVERY DAY

 002 K811607B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811607C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811607D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811607M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0029

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN DISCUSS RESULTS?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: K811608 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811608A (01 ) 0000 ALMOST EVERY DAY

 002 K811608B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811608C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811608D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811608M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0030

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN DO YOU USE COMPUTER?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811609 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811609A (01 ) 0000 ALMOST EVERY DAY

 002 K811609B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811609C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811609D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811609M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0031

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN TAKE TEST OR QUIZ?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811610 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811610A (01 ) 0000 ALMOST EVERY DAY

 002 K811610B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811610C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811610D (04 ) 0010 NEVER OR HARDLY EVER
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

566

 005 K811610M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0032

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN DO YOU USE LIBRARY?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811611 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811611A (01 ) 0000 ALMOST EVERY DAY

 002 K811611B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811611C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811611D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811611M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0033

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN OBSERVE/MEAS OUTSIDE?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811612 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811612A (01 ) 0000 ALMOST EVERY DAY

 002 K811612B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811612C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811612D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811612M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0034

 DESCRIPTION: HOW OFTEN DOES SCIENCE TEACHER TALK TO CLASS?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811701A (01 ) 0000 ALMOST EVERY DAY

 002 K811701B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811701C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811701D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811701M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0035

 DESCRIPTION: HOW OFTEN DOES SCIENCE TEACHER DO DEMONSTRATION?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811702A (01 ) 0000 ALMOST EVERY DAY

 002 K811702B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811702C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811702D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811702M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0036

 DESCRIPTION: HOW OFTEN DOES SCIENCE TEACHER SHOW VIDEO OR TV?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811703A (01 ) 0000 ALMOST EVERY DAY

 002 K811703B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811703C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811703D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811703M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0037

 DESCRIPTION: HOW OFTEN DOES SCIENCE TEACHER USE COMPUTER?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811704A (01 ) 0000 ALMOST EVERY DAY

 002 K811704B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811704C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811704D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811704M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0038

 DESCRIPTION: HOW OFTEN DOES SCI TEACHER USE CD’S/LASER DISCS?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

567

 001 K811705A (01 ) 0000 ALMOST EVERY DAY

 002 K811705B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811705C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811705D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811705M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0039

 DESCRIPTION: HOW OFTEN DOES SCI CLASS GO ON A FIELD TRIP?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 K811801A (01 ) 000 3 OR MORE A DAY

 002 K811801B (02 ) 100 1 OR 2 TIMES A YEAR

 003 K811801C (03 ) 010 NEVER OR HARDLY EVER

 004 K811801M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0040

 DESCRIPTION: HOW OFTEN DOES GUEST SPEAKER COME TO SCI CLASS?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 K811901A (01 ) 000 3 OR MORE A DAY

 002 K811901B (02 ) 100 1 OR 2 TIMES A YEAR

 003 K811901C (03 ) 010 NEVER OR HARDLY EVER

 004 K811901M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0041

 DESCRIPTION: ABOUT HOW MANY QUESTIONS RIGHT ON TEST?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: SM00101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 SM00101A (01 ) 0000 ALMOST ALL

 002 SM00101B (02 ) 1000 MORE THAN HALF

 003 SM00101C (03 ) 0100 ABOUT HALF

 004 SM00101D (04 ) 0010 LESS THAN HALF

 005 SM00101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0042

 DESCRIPTION: HOW HARD TEST COMPARED TO THOSE IN SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: SM00201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 SM00201A (01 ) 0000 MUCH HARDER

 002 SM00201B (02 ) 1000 HARDER THAN OTHERS

 003 SM00201C (03 ) 0100 ABOUT THE SAME

 004 SM00201D (04 ) 0010 EASIER THEN OTHES

 005 SM00201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0043

 DESCRIPTION: HOW HARD DID YOU TRY ON TEST COMPARED TO OTHERS?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: SM00301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 SM00301A (01 ) 0000 MUCH HARDER

 002 SM00301B (02 ) 1000 HARDER THAN OTHERS

 003 SM00301C (03 ) 0100 ABOUT AS HARD

 004 SM00301D (04 ) 0010 NOT AS HARD AS OTHER

 005 SM00301M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0044

 DESCRIPTION: HOW IMPORTANT WAS IT YOU DO WELL ON THIS TEST?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: SM00401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 SM00401A (01 ) 0000 VERY IMPORTANT

 002 SM00401B (02 ) 1000 IMPORTANT

 003 SM00401C (03 ) 0100 SOMEWHAT IMPORTANT

 004 SM00401D (04 ) 0010 NOT VERY IMPORTANTER

 005 SM00401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0045

 DESCRIPTION: HOW OFTEN HAD TO WRITE LONG ANSWERS TO QSTS?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: SM00501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

568

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 SM00501A (01 ) 0000 AT LEAST ONCE A WEEK

 002 SM00501B (02 ) 1000 ONCE OR TWICE A MNTH

 003 SM00501C (03 ) 0100 ONCE OR TWICE A YEAR

 004 SM00501D (04 ) 0010 NEVERERY IMPORTANTER

 005 SM00501M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0079

 DESCRIPTION: DESCRIBE YOUR OVERALL GRADES SINCE 6TH GRADE

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: B009701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 B009701A (01 ) 000000 MOSTLY A’S

 002 B009701B (02 ) 100000 MOSTLY B’S

 003 B009701C (03 ) 010000 MOSTLY C’S

 004 B009701D (04 ) 001000 MOSTLY D’S

 005 B009701E (05 ) 000100 MOSTLY BELOW D’S

 006 B009701F (06 ) 000010 CLASSES NOT GRADED

 007 B009701M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0080

 DESCRIPTION: HOW FAR IN SCHOOL DO YOU THINK YOU WILL GO?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: B009801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 B009801A (01 ) 000000 NOT FINISH HS

 002 B009801B (02 ) 100000 GRADUATE HS

 003 B009801C (03 ) 010000 SOME ED PAST HS

 004 B009801D (04 ) 001000 GRADUATE COLLEGE

 005 B009801E (05 ) 000100 GO GRADUATE SCHOOL

 006 B009801F (06 ) 000010 I DON’T KNOW

 007 B009801M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0083

 DESCRIPTION: DOES YOUR STEP/MOTHER WORK AT A JOB FOR PAY?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B009601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B009601A (01 ) 0000 YES, FULL-TIME

 002 B009601B (02 ) 1000 YES, PART-TIME

 003 B009601N (03 ) 0100 NO

 004 B009601D (04 ) 0010 DON’T LIVE W/ MOTHER

 005 B009601E (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0084

 DESCRIPTION: DOES YOUR STEP/FATHER WORK AT A JOB FOR PAY?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: B009602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B009602A (01 ) 0000 YES, FULL-TIME

 002 B009602B (02 ) 1000 YES, PART-TIME

 003 B009602N (03 ) 0100 NO

 004 B009602D (04 ) 0010 DON’T LIVE W/ FATHER

 005 B009602M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0046

 DESCRIPTION: DO YOU/TEACHER SAVE YOUR SCI WORK IN A PORTFOLIO?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: K812101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 K812101Y (01 ) 00 YES

 002 K812101N (02 ) 10 NO

 003 K812101M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0047

 DESCRIPTION: HOW MUCH TIME WEEKLY SPENT ON SCIENCE HOMEWORK?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 K812201A (01 ) 0000000 NOT TAKING SCIENCE

 002 K812201B (02 ) 1000000 NONE

 003 K812201C (03 ) 0100000 1/2 HOUR

 004 K812201D (04 ) 0010000 1 HOUR

 005 K812201E (05 ) 0001000 2 HOURS
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

569

 006 K812201F (06 ) 0000100 3 HOURS

 007 K812201G (07 ) 0000010 MORE THAN 3 HOURS

 008 K812201M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0048

 DESCRIPTION: FOR SCI IN SCHOOL, HOW OFTEN HANDS-ON ACTIVITIES?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811613 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811613A (01 ) 0000 ALMOST EVERY DAY

 002 K811613B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811613C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811613D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811613M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0049

 DESCRIPTION: FOR SCI, HOW OFTEN DISCUSS HANDS-ON RESULTS?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811614 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811614A (01 ) 0000 ALMOST EVERY DAY

 002 K811614B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811614C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811614D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811614M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0050

 DESCRIPTION: FOR SCI, DESIGN & CARRY OUT OWN INVESTIGATION?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811615 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811615A (01 ) 0000 ALMOST EVERY DAY

 002 K811615B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811615C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811615D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811615M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0081

 DESCRIPTION: DOES YOUR STEP/MOTHER WORK AT A JOB FOR PAY?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: B009501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 B009501Y (01 ) 000 YES

 002 B009501N (02 ) 100 NO

 003 B009501C (03 ) 010 DON’T LIVE W/ MOTHER

 004 B009501M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: BACK0082

 DESCRIPTION: DOES YOUR STEP/FATHER WORK AT A JOB FOR PAY?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: B009502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 B009502Y (01 ) 000 YES

 002 B009502N (02 ) 100 NO

 003 B009502C (03 ) 010 DON’T LIVE W/ FATHER

 004 B009502M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: BACK0085

 DESCRIPTION: SINCE KDG, GRADES ATTENDED IN THIS STATE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B008301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 B008301A (01 ) 00000 LESS THAN 1 GRADE

 002 B008301B (02 ) 10000 1-2 GRADES

 003 B008301C (03 ) 01000 3-5 GRADES

 004 B008301D (04 ) 00100 6-9 GRADES

 005 B008301E (05 ) 00010 MORE THAN 9 GRADES

 006 B008301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: BACK0086

 DESCRIPTION: DESCRIBE YOUR OVERALL GRADES SINCE 9TH GRADE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B009901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

570

 001 B009901A (01 ) 000000 MOSTLY A’S

 002 B009901B (02 ) 100000 MOSTLY B’S

 003 B009901C (03 ) 010000 MOSTLY C’S

 004 B009901D (04 ) 001000 MOSTLY D’S

 005 B009901E (05 ) 000100 MOSTLY BELOW D’S

 006 B009901F (06 ) 000010 CLASSES NOT GRADED

 007 B009901M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0087

 DESCRIPTION: WHICH BEST DESCRIBES YOUR HIGH SCHOOL PROGRAM

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B008501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 B008501A (01 ) 0000 GENERAL

 002 B008501B (02 ) 1000 ACADEMIC/COLLEGE PRE

 003 B008501C (03 ) 0100 VOCATIONAL OR TECHN

 004 B008501D (04 ) 0010 OTHER

 005 B008501M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: BACK0088

 DESCRIPTION: WHAT WILL TAKE LARGEST AMT. OF TIME AFTER HIGH-SCH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B005501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 B005501A (01 ) 000000 WORKING FULL-TIME

 002 B005501B (02 ) 100000 VOCA/TECH/BUSINESS

 003 B005501C (03 ) 010000 ATTEND 2 YR COLLEGE

 004 B005501D (04 ) 001000 ATTEND 4YR COLLEGE

 005 B005501E (05 ) 000100 SERVING IN MILITARY

 006 B005501F (06 ) 000010 OTHER

 007 B005501M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: BACK0089

 DESCRIPTION: KIND OF WORK DONE BY YOUR MOTHER/FEMALE GUARDIAN?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: MOTHOCC TOTAL NUMBER OF SPECIFIED CONTRASTS: 17

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 16

 001 NOT PRE (01,M ) 0000000000000000 NOT PRESENT IN HOUSEHOLD

 002 OFFICE W (02 ) 1000000000000000 OFFICE WORKER

 003 TECHNIC (03 ) 0100000000000000 TECHNICAL

 004 PROT SRV (04 ) 0010000000000000 PROTECTIVE SERVICE

 005 SALES (05 ) 0001000000000000 SALES

 006 OWNER (06 ) 0000100000000000 OWNER

 007 SERV WRK (07 ) 0000010000000000 SERVICE WORKER

 008 SKILL T (08 ) 0000001000000000 SKILLED TRADESPERSON

 009 LABORER (09 ) 0000000100000000 LABORER

 010 OPERATOR (10 ) 0000000010000000 OPERATOR

 011 FARMER (11 ) 0000000001000000 FARMER/FARM MANAGER

 012 HOMEMAKE (12 ) 0000000000100000 HOMEMAKER

 013 MANAGER (13 ) 0000000000010000 MANAGER

 014 MILITARY (14 ) 0000000000001000 MILITARY

 015 TEACHER (15 ) 0000000000000100 SCHOOL TEACHER

 016 PROFESS1 (16 ) 0000000000000010 PROFESSIONAL 1

 017 PROFESS2 (17 ) 0000000000000001 PROFESSIONAL 2

 CONDITIONING VARIABLE ID: BACK0090

 DESCRIPTION: KIND OF WORK DONE BY YOUR MOTHER/FEMALE GUARDIAN?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B011A01 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 B011A01M (M ) 0 MISSING

 002 B011A01N (@ ) 1 NOT PRESENT IN HOUSEHOLD

 CONDITIONING VARIABLE ID: BACK0091

 DESCRIPTION: KIND OF WORK DONE BY YOUR FATHER/MALE GUARDIAN?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: FATHOCC TOTAL NUMBER OF SPECIFIED CONTRASTS: 17

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 16

 001 NOT PRES (01,M ) 0000000000000000 NOT PRESENT IN HOUSEHOLD

 002 OFFICE (02 ) 1000000000000000 OFFICE WORKER

 003 TECHNIC (03 ) 0100000000000000 TECHNICAL

 004 PROT SRV (04 ) 0010000000000000 PROTECTIVE SERVICES

 005 SALES (05 ) 0001000000000000 SALES

 006 OWNER (06 ) 0000100000000000 OWNER

 007 SERV WRK (07 ) 0000010000000000 SERVICE WORKER

 008 SKILL TR (08 ) 0000001000000000 SKILLED TRADEPERSON
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

571

 009 LABORER (09 ) 0000000100000000 LABORER

 010 OPERATOR (10 ) 0000000010000000 OPERATOR

 011 FARMER (11 ) 0000000001000000 FARMER/FARM MANAGER

 012 HOMEMAKE (12 ) 0000000000100000 HOMEMAKER

 013 MANAGER (13 ) 0000000000010000 MANAGER

 014 MILITARY (14 ) 0000000000001000 MILITARY

 015 TEACHER (15 ) 0000000000000100 SCHOOL TEACHER

 016 PROFESS1 (16 ) 0000000000000010 PROFESSIONAL 1

 017 PROFESS2 (17 ) 0000000000000001 PROFESSIONAL 2

 CONDITIONING VARIABLE ID: BACK0092

 DESCRIPTION: KIND OF WORK DONE BY YOUR FATHER/MALE GUARDIAN?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: B012A02 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 B012A02M (M ) 0 MISSING

 002 B012A02N (@ ) 1 NOT PRESENT IN HOUSEHOLD

 CONDITIONING VARIABLE ID: SUBJ0051

 DESCRIPTION: DONE SCHOOL SCI INVEST/PROJECTS W/ LIVING THINGS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K812301Y (01 ) 0 YES

 002 K812301M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0052

 DESCRIPTION: DONE SCHOOL SCI INVEST/PROJECTS W/ ELECTRICITY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K812302Y (01 ) 0 YES

 002 K812302M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0053

 DESCRIPTION: DONE SCHOOL SCI INVEST/PROJECTS W/ CHEMICALS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K812303Y (01 ) 0 YES

 002 K812303M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0054

 DESCRIPTION: DONE SCHOOL SCI INVEST/PROJECTS W/ ROCKS/MINERALS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K812304Y (01 ) 0 YES

 002 K812304M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0055

 DESCRIPTION: DONE SCI INVEST/PROJECTS W/ MAG. GLASS/MICROSCOPE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K812305Y (01 ) 0 YES

 002 K812305M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0056

 DESCRIPTION: DONE SCI INVEST/PROJECTS W/ THERMOMETER/BAROMETER?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812306 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K812306Y (01 ) 0 YES

 002 K812306M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0057

 DESCRIPTION: DONE SCI INVEST/PROJECTS W/ SIMPLE MACHINES?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812307 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

572

 001 K812307Y (01 ) 0 YES

 002 K812307M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0058

 DESCRIPTION: DONE SCI PROJECTS W/ INSTRUMENTS MEAS. SPEED?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812308 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K812308Y (01 ) 0 YES

 002 K812308M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0059

 DESCRIPTION: DONE SCHOOL SCI INVEST/PROJECTS W/ NONE OF ABOVE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812309 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 K812309Y (01 ) 0 YES

 002 K812309M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SUBJ0060

 DESCRIPTION: ARE YOU TAKING A SCIENCE COURSE THIS YEAR?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 K812401Y (01 ) 00 YES

 002 K812401N (02 ) 10 NO

 003 K812401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0061

 DESCRIPTION: SINCE 9TH GRADE, HOW MUCH GENERAL SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K812501A (01 ) 0000 MORE THAN 1 YEAR

 002 K812501B (02 ) 1000 1 YEAR

 003 K812501C (03 ) 0100 LESS THAN 1 YEAR

 004 K812501D (04 ) 0010 NONE

 005 K812501M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0062

 DESCRIPTION: SINCE 9TH GRADE, HOW MUCH EARTH & SPACE SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K812502A (01 ) 0000 MORE THAN 1 YEAR

 002 K812502B (02 ) 1000 1 YEAR

 003 K812502C (03 ) 0100 LESS THAN 1 YEAR

 004 K812502D (04 ) 0010 NONE

 005 K812502M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0063

 DESCRIPTION: SINCE 9TH GRADE, HOW MUCH BIOLOGY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K812503A (01 ) 0000 MORE THAN 1 YEAR

 002 K812503B (02 ) 1000 1 YEAR

 003 K812503C (03 ) 0100 LESS THAN 1 YEAR

 004 K812503D (04 ) 0010 NONE

 005 K812503M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0064

 DESCRIPTION: SINCE 9TH GRADE, HOW MUCH LIFE SCIENCE (NOT BIO)?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K812504A (01 ) 0000 MORE THAN 1 YEAR

 002 K812504B (02 ) 1000 1 YEAR

 003 K812504C (03 ) 0100 LESS THAN 1 YEAR

 004 K812504D (04 ) 0010 NONE

 005 K812504M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0065
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

573

 DESCRIPTION: SINCE 9TH GRADE, HOW MUCH CHEMISTRY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K812505A (01 ) 0000 MORE THAN 1 YEAR

 002 K812505B (02 ) 1000 1 YEAR

 003 K812505C (03 ) 0100 LESS THAN 1 YEAR

 004 K812505D (04 ) 0010 NONE

 005 K812505M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0066

 DESCRIPTION: SINCE 9TH GRADE, HOW MUCH PHYSICS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K812506A (01 ) 0000 MORE THAN 1 YEAR

 002 K812506B (02 ) 1000 1 YEAR

 003 K812506C (03 ) 0100 LESS THAN 1 YEAR

 004 K812506D (04 ) 0010 NONE

 005 K812506M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0067

 DESCRIPTION: SINCE 9TH GRADE, HOW MUCH OTHER PHYSICAL SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812507 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K812507A (01 ) 0000 MORE THAN 1 YEAR

 002 K812507B (02 ) 1000 1 YEAR

 003 K812507C (03 ) 0100 LESS THAN 1 YEAR

 004 K812507D (04 ) 0010 NONE

 005 K812507M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0068

 DESCRIPTION: SINCE 9TH GRADE, HOW MUCH INTEGRATED SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812508 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K812508A (01 ) 0000 MORE THAN 1 YEAR

 002 K812508B (02 ) 1000 1 YEAR

 003 K812508C (03 ) 0100 LESS THAN 1 YEAR

 004 K812508D (04 ) 0010 NONE

 005 K812508M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0069

 DESCRIPTION: SINCE 9TH GRADE, HOW MUCH SCIENCE AND TECHNOLOGY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812509 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K812509A (01 ) 0000 MORE THAN 1 YEAR

 002 K812509B (02 ) 1000 1 YEAR

 003 K812509C (03 ) 0100 LESS THAN 1 YEAR

 004 K812509D (04 ) 0010 NONE

 005 K812509M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0070

 DESCRIPTION: SINCE 9TH GRADE, HOW MANY OTHER SCIENCE COURSES?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812510 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K812510A (01 ) 0000 MORE THAN 1 YEAR

 002 K812510B (02 ) 1000 1 YEAR

 003 K812510C (03 ) 0100 LESS THAN 1 YEAR

 004 K812510D (04 ) 0010 NONE

 005 K812510M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SUBJ0071

 DESCRIPTION: ENROLLED IN SCIENCE ADVANCED PLACEMENT COURSE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 K812601Y (01 ) 00 YES

 002 K812601N (02 ) 10 NO

 003 K812601M (M ) 01 MISSING
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

574

 CONDITIONING VARIABLE ID: SUBJ0072

 DESCRIPTION: DONE SCI INVEST/PROJECTS IN SCHOOL 1 WK OR MORE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K812701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 K812701Y (01 ) 00 YES

 002 K812701N (02 ) 10 NO

 003 K812701M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SUBJ0073

 DESCRIPTION: HOW OFTEN ANALYZE OWN SCI DATA, FORM CONCLUSIONS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: K811616 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 K811616A (01 ) 0000 ALMOST EVERY DAY

 002 K811616B (02 ) 1000 ONCE OR TWICE A WEEK

 003 K811616C (03 ) 0100 ONCE OR TWICE A MTH

 004 K811616D (04 ) 0010 NEVER OR HARDLY EVER

 005 K811616M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0003

 DESCRIPTION: BEST DESCRIBES HOW 4TH GR ARE ORGANIZED?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C030901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C030901A (01 ) 000 SELF CONTAINED

 002 C030901B (02 ) 100 DEPARTMENTALIZED

 003 C030901C (03 ) 010 REGROUPED

 004 C030901M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0004

 DESCRIPTION: 4TH GRADERS ASSIGNED BY ABILITY/ACHIEVEMENT LEVEL?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037101A (01 ) 0000 YES, BY MATH ABILITY

 002 C037101B (02 ) 1000 YES, READING ABILITY

 003 C037101C (03 ) 0100 YES, GENERAL ABILITY

 004 C037101N (04 ) 0010 NO

 005 C037101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0005

 DESCRIPTION: HOW OFTEN IS 4TH-GRADER INSTRUCTED IN MATH?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C031212 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C031212A (01 ) 00000 EVERY DAY

 002 C031212B (02 ) 10000 THREE OR FOUR/WEEK

 003 C031212C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C031212D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C031212E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C031212M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0006

 DESCRIPTION: HOW OFTEN IS 4TH-GRADER INSTRUCTED IN SCIENCE?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C031205 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C031205A (01 ) 00000 EVERY DAY

 002 C031205B (02 ) 10000 THREE OR FOUR/WEEK

 003 C031205C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C031205D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C031205E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C031205M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0007

 DESCRIPTION: HOW OFTEN IS 4TH-GRADER INSTRUCTED IN READING?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C031213 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C031213A (01 ) 00000 EVERY DAY

 002 C031213B (02 ) 10000 THREE OR FOUR/WEEK
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

575

 003 C031213C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C031213D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C031213E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C031213M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0008

 DESCRIPTION: HOW OFTEN IS 4TH-GRADER INSTRUCTED IN ARTS?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C031214 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C031214A (01 ) 00000 EVERY DAY

 002 C031214B (02 ) 10000 THREE OR FOUR/WEEK

 003 C031214C (03 ) 01000 ONCE OR TWICE A WEEK

 004 C031214D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C031214E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C031214M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0009

 DESCRIPTION: HAS MATH BEEN IDENTIFIED AS A PRIORITY?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C031603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031603Y (01 ) 00 YES

 002 C031603N (02 ) 10 NO

 003 C031603M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0010

 DESCRIPTION: HAS SCIENCE BEEN IDENTIFIED AS A PRIORITY?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C031607 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031607Y (01 ) 00 YES

 002 C031607N (02 ) 10 NO

 003 C031607M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0011

 DESCRIPTION: HAS READING BEEN IDENTIFIED AS A PRIORITY?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C031601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031601Y (01 ) 00 YES

 002 C031601N (02 ) 10 NO

 003 C031601M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0012

 DESCRIPTION: HAS ARTS BEEN IDENTIFIED AS A PRIORITY?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C031610 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031610Y (01 ) 00 YES

 002 C031610N (02 ) 10 NO

 003 C031610M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0013

 DESCRIPTION: HAS SUBJECT INTEGRATION BEEN A PRIORITY?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C031606 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031606Y (01 ) 00 YES

 002 C031606N (02 ) 10 NO

 003 C031606M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0014

 DESCRIPTION: COMPUTERS AVAILABLE ALL THE TIME IN CLASSROOM?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035701Y (01 ) 00 YES

 002 C035701N (02 ) 10 NO

 003 C035701M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0015

 DESCRIPTION: COMPUTERS GROUPED IN SEPARATE LAB AND AVAILABLE?
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

576

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035702Y (01 ) 00 YES

 002 C035702N (02 ) 10 NO

 003 C035702M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0016

 DESCRIPTION: COMPUTERS AVAILABLE TO BRING TO ROOM WHEN NEEDED?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035703Y (01 ) 00 YES

 002 C035703N (02 ) 10 NO

 003 C035703M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0017

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON MATH?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037201Y (01 ) 0 YES

 002 C037201M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0018

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON SCIENCE?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037202Y (01 ) 0 YES

 002 C037202M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0019

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON MATH?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037207 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037207Y (01 ) 0 YES

 002 C037207M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0020

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON ARTS?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037204 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037204Y (01 ) 0 YES

 002 C037204M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0021

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON OTHER?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037205 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037205Y (01 ) 0 YES

 002 C037205M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0022

 DESCRIPTION: SCHOOL NOT A SPECIAL FOCUS SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037206 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037206Y (01 ) 0 YES

 002 C037206M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0023

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE MATH CURRICULUM?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

577

 001 C037301Y (01 ) 0 YES

 002 C037301M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0024

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE SCIENCE CURRICULUM?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037302Y (01 ) 0 YES

 002 C037302M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0025

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE READING CURRICULUM?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037303Y (01 ) 0 YES

 002 C037303M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0026

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE ARTS CURRICULUM?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037304Y (01 ) 0 YES

 002 C037304M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0027

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE FOR NONE OF ABOVE?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037305Y (01 ) 0 YES

 002 C037305M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0028

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR MATH?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037401Y (01 ) 0 YES

 002 C037401M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0029

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR SCIENCE?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037402Y (01 ) 0 YES

 002 C037402M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0030

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR READING?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037403Y (01 ) 0 YES

 002 C037403M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0031

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR ARTS?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037404Y (01 ) 0 YES

 002 C037404M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0032

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR OTHER?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

578

 NAEP ID: C037405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037405Y (01 ) 0 YES

 002 C037405M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0033

 DESCRIPTION: SCHOOL SPONSER 4TH GRDS FIELD TRIP FOR NONE ABOVE?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037406Y (01 ) 0 YES

 002 C037406M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0034

 DESCRIPTION: 4TH GRADERS IN EXTRACURR ACTS FOR MATH?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037501Y (01 ) 0 YES

 002 C037501M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0035

 DESCRIPTION: 4TH GRADERS IN EXTRACURR ACTS FOR SCIENCE?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037502Y (01 ) 0 YES

 002 C037502M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0036

 DESCRIPTION: 4TH GRADERS IN EXTRACURR ACTS FOR READING?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037503Y (01 ) 0 YES

 002 C037503M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0037

 DESCRIPTION: 4TH GRADERS IN EXTRACURR ACTS FOR ARTS?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037504Y (01 ) 0 YES

 002 C037504M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0038

 DESCRIPTION: 4TH GRADERS IN EXTRACURR ACTS FOR NONE OF ABOVE?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037505Y (01 ) 0 YES

 002 C037505M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0039

 DESCRIPTION: 4TH GRADERS IN SUMMER PROGRAMS IN MATH?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037601Y (01 ) 0 YES

 002 C037601M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0040

 DESCRIPTION: 4TH GRADERS IN SUMMER PROGRAMS IN SCIENCE?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037602Y (01 ) 0 YES

 002 C037602M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0041
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

579

 DESCRIPTION: 4TH GRADERS IN SUMMER PROGRAMS IN READING?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037603Y (01 ) 0 YES

 002 C037603M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0042

 DESCRIPTION: 4TH GRADERS IN SUMMER PROGRAMS IN ARTS?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037604Y (01 ) 0 YES

 002 C037604M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0043

 DESCRIPTION: 4TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037605Y (01 ) 0 YES

 002 C037605M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0044

 DESCRIPTION: WHICH BEST DESCRIBES PRIMARY WAY LIBRARY STAFFED?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C036601A (01 ) 0000 NO LIBRARY IN SCHOOL

 002 C036601B (02 ) 1000 LIBRARY-NO/VOL STAFF

 003 C036601C (03 ) 0100 PART-TIME STAFF

 004 C036601D (04 ) 0010 FULL-TIME STAFF

 005 C036601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0045

 DESCRIPTION: INVOLVE PARENTS AS AIDES IN CLASSROOM?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032207 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C032207A (01 ) 000 YES, ROUTINELY

 002 C032207B (02 ) 100 YES, OCCASIONALLY

 003 C032207N (03 ) 010 NO

 004 C032207M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0046

 DESCRIPTION: HAVE PARENTS REVIEW/SIGN HOMEWORK?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032209 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C032209A (01 ) 000 YES, ROUTINELY

 002 C032209B (02 ) 100 YES, OCCASIONALLY

 003 C032209N (03 ) 010 NO

 004 C032209M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0047

 DESCRIPTION: ASSIGN HOMEWORK STUDENTS DO WITH PARENTS?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032210 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C032210A (01 ) 000 YES, ROUTINELY

 002 C032210B (02 ) 100 YES, OCCASIONALLY

 003 C032210N (03 ) 010 NO

 004 C032210M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0048

 DESCRIPTION: HAVE A PARENT VOLUNTEER PROGRAM?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032211 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C032211A (01 ) 000 YES, ROUTINELY

 002 C032211B (02 ) 100 YES, OCCASIONALLY
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

580

 003 C032211N (03 ) 010 NO

 004 C032211M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0049

 DESCRIPTION: WHAT % OF PARENTS IN PARENT-TEACHER ORGS?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037701A (01 ) 0000 0-25%

 002 C037701B (02 ) 1000 26-50%

 003 C037701C (03 ) 0100 51-75%

 004 C037701D (04 ) 0010 76-100%

 005 C037701M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0050

 DESCRIPTION: WHAT % OF PARENTS IN OPEN HOUSE/BACK SCHOOL NIGHT?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037702A (01 ) 0000 0-25%

 002 C037702B (02 ) 1000 26-50%

 003 C037702C (03 ) 0100 51-75%

 004 C037702D (04 ) 0010 76-100%

 005 C037702M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0051

 DESCRIPTION: WHAT % OF PARENTS IN PARENT-TEACHER COMFERENCES?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037703A (01 ) 0000 0-25%

 002 C037703B (02 ) 1000 26-50%

 003 C037703C (03 ) 0100 51-75%

 004 C037703D (04 ) 0010 76-100%

 005 C037703M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0052

 DESCRIPTION: WHAT % PARENTS INVOLVED MAKING CURRICULUM DECISION

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037704A (01 ) 0000 0-25%

 002 C037704B (02 ) 1000 26-50%

 003 C037704C (03 ) 0100 51-75%

 004 C037704D (04 ) 0010 76-100%

 005 C037704M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0053

 DESCRIPTION: WHAT % OF PARENTS IN VOLUNTEER PROGRAMS?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C037705A (01 ) 0000 0-25%

 002 C037705B (02 ) 1000 26-50%

 003 C037705C (03 ) 0100 51-75%

 004 C037705D (04 ) 0010 76-100%

 005 C037705M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0054

 DESCRIPTION: IS STUDENT ABSENTEEISM A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032402A (01 ) 0000 SERIOUS

 002 C032402B (02 ) 1000 MODERATE

 003 C032402C (03 ) 0100 MINOR

 004 C032402D (04 ) 0010 NOT A PROBLEM

 005 C032402M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0055

 DESCRIPTION: IS STUDENT TARDINESS A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

581

 001 C032401A (01 ) 0000 SERIOUS

 002 C032401B (02 ) 1000 MODERATE

 003 C032401C (03 ) 0100 MINOR

 004 C032401D (04 ) 0010 NOT A PROBLEM

 005 C032401M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0056

 DESCRIPTION: ARE PHYSICAL CONFLICTS A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032404A (01 ) 0000 SERIOUS

 002 C032404B (02 ) 1000 MODERATE

 003 C032404C (03 ) 0100 MINOR

 004 C032404D (04 ) 0010 NOT A PROBLEM

 005 C032404M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0057

 DESCRIPTION: IS TEACHER ABSENTEEISM A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032406A (01 ) 0000 SERIOUS

 002 C032406B (02 ) 1000 MODERATE

 003 C032406C (03 ) 0100 MINOR

 004 C032406D (04 ) 0010 NOT A PROBLEM

 005 C032406M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0058

 DESCRIPTION: ARE RACE/CULT. CONFLICTS A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032407 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032407A (01 ) 0000 SERIOUS

 002 C032407B (02 ) 1000 MODERATE

 003 C032407C (03 ) 0100 MINOR

 004 C032407D (04 ) 0010 NOT A PROBLEM

 005 C032407M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0059

 DESCRIPTION: IS STUDENT HEALTH A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032408 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032408A (01 ) 0000 SERIOUS

 002 C032408B (02 ) 1000 MODERATE

 003 C032408C (03 ) 0100 MINOR

 004 C032408D (04 ) 0010 NOT A PROBLEM

 005 C032408M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0060

 DESCRIPTION: IS LACK OF PARENT INVLMT A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032409 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032409A (01 ) 0000 SERIOUS

 002 C032409B (02 ) 1000 MODERATE

 003 C032409C (03 ) 0100 MINOR

 004 C032409D (04 ) 0010 NOT A PROBLEM

 005 C032409M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0061

 DESCRIPTION: IS STUD USE OF ALCOHOL A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032410 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032410A (01 ) 0000 SERIOUS

 002 C032410B (02 ) 1000 MODERATE

 003 C032410C (03 ) 0100 MINOR

 004 C032410D (04 ) 0010 NOT A PROBLEM

 005 C032410M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0062

 DESCRIPTION: IS STUDENT TOBACCO USE A PROBLEM IN YOUR SCHOOL?
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

582

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032411 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032411A (01 ) 0000 SERIOUS

 002 C032411B (02 ) 1000 MODERATE

 003 C032411C (03 ) 0100 MINOR

 004 C032411D (04 ) 0010 NOT A PROBLEM

 005 C032411M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0063

 DESCRIPTION: IS STUDENT DRUG USE A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032412 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032412A (01 ) 0000 SERIOUS

 002 C032412B (02 ) 1000 MODERATE

 003 C032412C (03 ) 0100 MINOR

 004 C032412D (04 ) 0010 NOT A PROBLEM

 005 C032412M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0064

 DESCRIPTION: ARE GANG ACTIVITIES A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032413 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032413A (01 ) 0000 SERIOUS

 002 C032413B (02 ) 1000 MODERATE

 003 C032413C (03 ) 0100 MINOR

 004 C032413D (04 ) 0010 NOT A PROBLEM

 005 C032413M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0065

 DESCRIPTION: IS STUDENT MISBEHAVIOR A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032414 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032414A (01 ) 0000 SERIOUS

 002 C032414B (02 ) 1000 MODERATE

 003 C032414C (03 ) 0100 MINOR

 004 C032414D (04 ) 0010 NOT A PROBLEM

 005 C032414M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0066

 DESCRIPTION: IS STUDENT CHEATING A PROBLEM IN YOUR SCHOOL?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032415 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032415A (01 ) 0000 SERIOUS

 002 C032415B (02 ) 1000 MODERATE

 003 C032415C (03 ) 0100 MINOR

 004 C032415D (04 ) 0010 NOT A PROBLEM

 005 C032415M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0067

 DESCRIPTION: IS TEACHER MORALE POS. OR NEG.?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032502A (01 ) 0000 VERY POSITIVE

 002 C032502B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032502C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032502D (04 ) 0010 VERY NEGATIVE

 005 C032502M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0068

 DESCRIPTION: ARE STUDENT ATTITUDES TO ACADEMICS POS. OR NEG.?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032503A (01 ) 0000 VERY POSITIVE

 002 C032503B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032503C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032503D (04 ) 0010 VERY NEGATIVE
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

583

 005 C032503M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0069

 DESCRIPTION: IS PARENT SUPPORT FOR ACHIEVEMENT POS. OR NEG.?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032505A (01 ) 0000 VERY POSITIVE

 002 C032505B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032505C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032505D (04 ) 0010 VERY NEGATIVE

 005 C032505M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0070

 DESCRIPTION: IS REGARD FOR SCHOOL PROPERTY POS. OR NEG.?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C032506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C032506A (01 ) 0000 VERY POSITIVE

 002 C032506B (02 ) 1000 SOMEWHAT POSITIVE

 003 C032506C (03 ) 0100 SOMEWHAT NEGATIVE

 004 C032506D (04 ) 0010 VERY NEGATIVE

 005 C032506M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0071

 DESCRIPTION: % ABSENT ON AVERAGE DAY?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C033601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C033601A (01 ) 0000 0-2%

 002 C033601B (02 ) 1000 3-5%

 003 C033601C (03 ) 0100 6-10%

 004 C033601D (04 ) 0010 MORE THAN 10%

 005 C033601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0072

 DESCRIPTION: WHAT % OF TEACHERS ABSENT ON GIVEN DAY?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 C036501A (01 ) 0000 0-2%

 002 C036501B (02 ) 1000 3-5%

 003 C036501C (03 ) 0100 6-10%

 004 C036501D (04 ) 0010 MORE THAN 10%

 005 C036501M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: SCHL0073

 DESCRIPTION: % OF STUDS EROLLED AT START OF YR EROLLED AT END?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 9

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 8

 001 C037801A (01 ) 00000000 98-100%

 002 C037801B (02 ) 10000000 95-97%

 003 C037801C (03 ) 01000000 90-94%

 004 C037801D (04 ) 00100000 80-89%

 005 C037801E (05 ) 00010000 70-79%

 006 C037801F (06 ) 00001000 60-69%

 007 C037801G (07 ) 00000100 50-59%

 008 C037801H (08 ) 00000010 LESS THAN 50%

 009 C037801M (M ) 00000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0074

 DESCRIPTION: % OF 4TH GRADERS HELD BACK & REPEATING 4TH GRADE?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: C037901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C037901A (01 ) 00000 0%

 002 C037901B (02 ) 10000 1-2%

 003 C037901C (03 ) 01000 3-5%

 004 C037901D (04 ) 00100 6-10%

 005 C037901E (05 ) 00010 MORE THAN 10%

 006 C037901M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0075

 DESCRIPTION: % OF FULL TIME TEACHERS LEFT BEFORE END OF YR?
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

584

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C038001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C038001A (01 ) 0000000 0%

 002 C038001B (02 ) 1000000 1-2%

 003 C038001C (03 ) 0100000 3-5%

 004 C038001D (04 ) 0010000 6-10%

 005 C038001E (05 ) 0001000 11-15%

 006 C038001F (06 ) 0000100 16-20%

 007 C038001G (07 ) 0000010 MORE THAN 20%

 008 C038001M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0076

 DESCRIPTION: IS SCHOOL IN NATIONAL SCHOOL LUNCH PROGRAM?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C038301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C038301Y (01 ) 00 YES

 002 C038301N (02 ) 10 NO

 003 C038301M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0077

 DESCRIPTION: SCHOOL RECEIVE CHAP 1/TITLE 1 FUNDING?

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C038801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C038801Y (01 ) 00 YES

 002 C038801N (02 ) 10 NO

 003 C038801M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0078

 DESCRIPTION: DID PRINCIPAL FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034101Y (01 ) 0 YES

 002 C034101M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0079

 DESCRIPTION: DID HEADMASTER/HEADMISTRESS FILL OUT QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034102Y (01 ) 0 YES

 002 C034102M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0080

 DESCRIPTION: DID HEAD TEACHER FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034103Y (01 ) 0 YES

 002 C034103M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0081

 DESCRIPTION: DID VICE PRINCIPAL FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034104Y (01 ) 0 YES

 002 C034104M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0082

 DESCRIPTION: DID COUNSELOR FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034105Y (01 ) 0 YES

 002 C034105M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0083
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

585

 DESCRIPTION: DID CURRICULUM COORD FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034106Y (01 ) 0 YES

 002 C034106M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0084

 DESCRIPTION: DID TEACHER FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034107Y (01 ) 0 YES

 002 C034107M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0085

 DESCRIPTION: DID SECRETARY FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034108 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034108Y (01 ) 0 YES

 002 C034108M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0086

 DESCRIPTION: DID OTHER PERSON FILL OUT THIS QUESTIONNAIRE

 GRADES/ASSESSMENTS: N04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C034109 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C034109Y (01 ) 0 YES

 002 C034109M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0001

 DESCRIPTION: WHAT IS YOUR GENDER?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T055901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T055901A (01 ) 00 MALE

 002 T055901B (02 ) 10 FEMALE

 003 T055901M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0002

 DESCRIPTION: WHICH BEST DESRIBES YOU?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T056001A (01 ) 000000 WHITE

 002 T056001B (02 ) 100000 BLACK

 003 T056001C (03 ) 010000 HISPANIC

 004 T056001D (04 ) 001000 ASIAN/PACIFIC AMERIC

 005 T056001E (05 ) 000100 AMER IND/ALASKA NATV

 006 T056001F (06 ) 000010 OTHER

 007 T056001M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0003

 DESCRIPTION: YEARS TAUGHT

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T040301A (01 ) 00000 2 YEARS OR LESS

 002 T040301B (02 ) 10000 3-5 YEARS

 003 T040301C (03 ) 01000 6-10 YEARS

 004 T040301D (04 ) 00100 11-24 YEARS

 005 T040301E (05 ) 00010 25 YEARS OR MORE

 006 T040301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0004

 DESCRIPTION: HOW MANY YRS TOTAL YOU TAUGHT MATH?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T056101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T056101A (01 ) 00000 2 YEARS OR LESS
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

586

 002 T056101B (02 ) 10000 3-5 YEARS

 003 T056101C (03 ) 01000 6-10 YEARS

 004 T056101D (04 ) 00100 11-24 YEARS

 005 T056101E (05 ) 00010 25 YEARS OR MORE

 006 T056101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0005

 DESCRIPTION: HOW MANY YRS TOTAL YOU TAUGHT SCIENCE?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T056102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T056102A (01 ) 00000 2 YEARS OR LESS

 002 T056102B (02 ) 10000 3-5 YEARS

 003 T056102C (03 ) 01000 6-10 YEARS

 004 T056102D (04 ) 00100 11-24 YEARS

 005 T056102E (05 ) 00010 25 YEARS OR MORE

 006 T056102M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0006

 DESCRIPTION: TYPE TCHNG CERT IN THIS ST IN MAIN FIELD?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T056201A (01 ) 000000 ADVANCED PROFESSIONL

 002 T056201B (02 ) 100000 REGULAR/STANDARD ST

 003 T056201C (03 ) 010000 PROBATIONARY STATE

 004 T056201D (04 ) 001000 TEMPORARY/PROVISIONL

 005 T056201E (05 ) 000100 OTHER THAN STATE CRT

 006 T056201F (06 ) 000010 NOT HAVE CERT MAIN

 007 T056201M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0007

 DESCRIPTION: CERTIFICATION, ELEMENTARY OR MIDDLE/JUNIOR HS ED?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T040501Y (01 ) 000 YES

 002 T040501N (02 ) 100 NO

 003 T040501C (03 ) 010 NOT OFFERED IN STATE

 004 T040501M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0008

 DESCRIPTION: DO YOU HAVE CERTIFICATION IN ELEMENTARY MATH?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T040506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T040506Y (01 ) 000 YES

 002 T040506N (02 ) 100 NO

 003 T040506C (03 ) 010 NOT OFFERED IN STATE

 004 T040506M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0009

 DESCRIPTION: DO YOU HAVE CERTIFICATION IN JR HIGH/SEC MATH?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T040504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T040504Y (01 ) 000 YES

 002 T040504N (02 ) 100 NO

 003 T040504C (03 ) 010 NOT OFFERED IN STATE

 004 T040504M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0010

 DESCRIPTION: CERTIFICATION, ELEMENTARY SCIENCE?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040507 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T040507Y (01 ) 000 YES

 002 T040507N (02 ) 100 NO

 003 T040507C (03 ) 010 NOT OFFERED IN STATE

 004 T040507M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0011

 DESCRIPTION: CERTIFICATION, MIDDLE/JUNIOR SCIENCE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

587

 NAEP ID: T040508 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T040508Y (01 ) 000 YES

 002 T040508N (02 ) 100 NO

 003 T040508C (03 ) 010 NOT OFFERED IN STATE

 004 T040508M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0012

 DESCRIPTION: CERTIFICATION, OTHER

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T040505Y (01 ) 000 YES

 002 T040505N (02 ) 100 NO

 003 T040505C (03 ) 010 NOT OFFERED IN STATE

 004 T040505M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TCHR0013

 DESCRIPTION: HIGHEST ACADEMIC DEGREE YOU HOLD?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 T056301A (01 ) 0000000 HIGH SCHOOL DIPLOMA

 002 T056301B (02 ) 1000000 ASSOCIATES/VOCATIONL

 003 T056301C (03 ) 0100000 BACHELORS DEGREE

 004 T056301D (04 ) 0010000 MASTER’S DEGREE

 005 T056301E (05 ) 0001000 EDUCATION SPECIALIST

 006 T056301F (06 ) 0000100 DOCTORATE

 007 T056301G (07 ) 0000010 PROFESSIONAL DEGREE

 008 T056301M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0014

 DESCRIPTION: EDUCATION UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040701Y (01 ) 0 YES

 002 T040701M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0015

 DESCRIPTION: ELMENT ED UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040706 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040706Y (01 ) 0 YES

 002 T040706M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0016

 DESCRIPTION: SEC ED UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040707 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040707Y (01 ) 0 YES

 002 T040707M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0017

 DESCRIPTION: WAS YOUR UNDERGRADUATE MAJOR MATH?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T040703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040703Y (01 ) 0 YES

 002 T040703M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0018

 DESCRIPTION: WAS YOUR UNDERGRADUATE MAJOR MATH ED?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T040704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040704Y (01 ) 0 YES

 002 T040704M (M ) 1 MISSING
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

588

 CONDITIONING VARIABLE ID: TCHR0019

 DESCRIPTION: SCIENCE ED UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040710 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040710Y (01 ) 0 YES

 002 T040710M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0020

 DESCRIPTION: LIFE SCIENCE UNDERGRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040711 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040711Y (01 ) 0 YES

 002 T040711M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0021

 DESCRIPTION: PHYSICAL SCIENCE UNDERGRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040712 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040712Y (01 ) 0 YES

 002 T040712M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0022

 DESCRIPTION: EARTH SCIENCE UNDERGRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040713 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040713Y (01 ) 0 YES

 002 T040713M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0023

 DESCRIPTION: SPECIAL EDUCATION UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040708 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040708Y (01 ) 0 YES

 002 T040708M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0024

 DESCRIPTION: BILINGUAL ED/ESL UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040709 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040709Y (01 ) 0 YES

 002 T040709M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0025

 DESCRIPTION: OTHER UNDERGRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040705Y (01 ) 0 YES

 002 T040705M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0026

 DESCRIPTION: EDUCATION GRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040801Y (01 ) 0 YES

 002 T040801M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0027

 DESCRIPTION: ELEMENTARY ED GRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040807 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

589

 001 T040807Y (01 ) 0 YES

 002 T040807M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0028

 DESCRIPTION: SECONDARY ED GRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040808 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040808Y (01 ) 0 YES

 002 T040808M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0029

 DESCRIPTION: WAS YOUR GRADUATE MAJOR MATHEMATICS?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T040803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040803Y (01 ) 0 YES

 002 T040803M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0030

 DESCRIPTION: WAS YOUR GRADUATE MAJOR MATH ED?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T040804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040804Y (01 ) 0 YES

 002 T040804M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0031

 DESCRIPTION: SCIENCE ED GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040814 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040814Y (01 ) 0 YES

 002 T040814M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0032

 DESCRIPTION: LIFE SCIENCE GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040815 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040815Y (01 ) 0 YES

 002 T040815M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0033

 DESCRIPTION: PHYSICAL SCIENCE GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040816 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040816Y (01 ) 0 YES

 002 T040816M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0034

 DESCRIPTION: EARTH SCIENCE GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040817 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040817Y (01 ) 0 YES

 002 T040817M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0035

 DESCRIPTION: SPECIAL ED GRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040809 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040809Y (01 ) 0 YES

 002 T040809M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0036

 DESCRIPTION: BILINGUAL GRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

590

 NAEP ID: T040810 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040810Y (01 ) 0 YES

 002 T040810M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0037

 DESCRIPTION: ADMIN/SUPERVISION GRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040811 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040811Y (01 ) 0 YES

 002 T040811M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0038

 DESCRIPTION: CURRICULUM/INSTRUCTION GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040812 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040812Y (01 ) 0 YES

 002 T040812M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0039

 DESCRIPTION: COUNSELING GRAD MAJOR?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040813 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040813Y (01 ) 0 YES

 002 T040813M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0040

 DESCRIPTION: OTHER GRAD MAJOR

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040805 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040805Y (01 ) 0 YES

 002 T040805M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0041

 DESCRIPTION: NO GRADUATE STUDY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T040806 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T040806Y (01 ) 0 YES

 002 T040806M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0042

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-EDUCATION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056401Y (01 ) 0 YES

 002 T056401M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0043

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-ELEMENTARY ED

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056402Y (01 ) 0 YES

 002 T056402M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0044

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-SECONDARY ED

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056403Y (01 ) 0 YES

 002 T056403M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0045
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

591

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-MATHEMATICS

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T056404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056404Y (01 ) 0 YES

 002 T056404M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0046

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-MATHEMATICS ED

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056405Y (01 ) 0 YES

 002 T056405M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0047

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-SCIENCE ED

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056413 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056413Y (01 ) 0 YES

 002 T056413M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0048

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-LIFE SCIENCE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056414 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056414Y (01 ) 0 YES

 002 T056414M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0049

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-PHYSICAL SCIENCE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056415 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056415Y (01 ) 0 YES

 002 T056415M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0050

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-EARTH SCIENCE

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T056416 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056416Y (01 ) 0 YES

 002 T056416M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0051

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-SPECIAL ED

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056406Y (01 ) 0 YES

 002 T056406M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0052

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-BILINGUAL ED

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056407 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056407Y (01 ) 0 YES

 002 T056407M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0053

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-ADMIN & SUPERVISION

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056408 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056408Y (01 ) 0 YES
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

592

 002 T056408M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0054

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-CURRICULUM & INSTRUC

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056409 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056409Y (01 ) 0 YES

 002 T056409M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0055

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-COUNSELING

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056410 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056410Y (01 ) 0 YES

 002 T056410M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0056

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-OTHER

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056411 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056411Y (01 ) 0 YES

 002 T056411M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0057

 DESCRIPTION: UNDERGRAD/GRAD MINOR STUDY-NONE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056412 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056412Y (01 ) 0 YES

 002 T056412M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0058

 DESCRIPTION: LAST YR, HOW MUCH TIME IN MATH/MATH ED SEM/WRKSHP?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T056501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T056501A (01 ) 00000 NONE

 002 T056501B (02 ) 10000 LESS THAN 6 HOURS

 003 T056501C (03 ) 01000 6-15 HOURS

 004 T056501D (04 ) 00100 16-35 HOURS

 005 T056501E (05 ) 00010 MORE THAN 35 HOURS

 006 T056501M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0059

 DESCRIPTION: LAST YR, HOW MUCH TIME IN SCI/SCI ED SEM/WRKSHPS?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T058101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T058101A (01 ) 00000 NONE

 002 T058101B (02 ) 10000 LESS THAN 6 HOURS

 003 T058101C (03 ) 01000 6-15 HOURS

 004 T058101D (04 ) 00100 16-35 HOURS

 005 T058101E (05 ) 00010 MORE THAN 35 HOURS

 006 T058101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0060

 DESCRIPTION: LAST 2 YRS, HOW MANY MATH/MATH ED UNIV COURSES?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T056601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T056601A (01 ) 00000 NONE

 002 T056601B (02 ) 10000 ONE

 003 T056601C (03 ) 01000 TWO

 004 T056601D (04 ) 00100 THREE

 005 T056601E (05 ) 00010 FOUR OR MORE

 006 T056601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0061

 DESCRIPTION: LAST 2 YRS, HOW MANY SCI/SCI ED UNIV COURSES?

 GRADES/ASSESSMENTS: N04
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

593

 CONDITIONING VAR LABEL:

 NAEP ID: T058201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T058201A (01 ) 00000 NONE

 002 T058201B (02 ) 10000 ONE

 003 T058201C (03 ) 01000 TWO

 004 T058201D (04 ) 00100 THREE

 005 T058201E (05 ) 00010 FOUR OR MORE

 006 T058201M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0062

 DESCRIPTION: PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-TELECOMM USE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056701Y (01 ) 0 YES

 002 T056701M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0063

 DESCRIPTION: PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-TECH USE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056702Y (01 ) 0 YES

 002 T056702M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0064

 DESCRIPTION: PAST 5 YRS, TAKEN COURSES/IN PRO DEVP-COOP INSTRCT

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056703Y (01 ) 0 YES

 002 T056703M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0065

 DESCRIPTION: PAST 5 YRS, COURSES/IN PRO DEVLP-INTERDISP INSTRCT

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056704Y (01 ) 0 YES

 002 T056704M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0066

 DESCRIPTION: PAST 5 YRS, COURSES/IN PRO DEVLP-PORTFOLIO ASSMNT

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056705Y (01 ) 0 YES

 002 T056705M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0067

 DESCRIPTION: PAST 5 YRS, COURSES/IN PRO DEVLP-PERF BASED ASSMNT

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056706 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056706Y (01 ) 0 YES

 002 T056706M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0068

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-TEACH HIGHORDER THKG

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056707 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056707Y (01 ) 0 YES

 002 T056707M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0069

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-TEACH DIFF CULT BKGD

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056708 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

594

 001 T056708Y (01 ) 0 YES

 002 T056708M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0070

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-TEACH LEP STUDENTS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056709 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056709Y (01 ) 0 YES

 002 T056709M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0071

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-TEACH SPEC NEED STDS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056710 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056710Y (01 ) 0 YES

 002 T056710M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0072

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-CLASSRM MNGMT/ORG

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056711 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056711Y (01 ) 0 YES

 002 T056711M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0073

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-OTHER PROF ISSUES

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056712 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056712Y (01 ) 0 YES

 002 T056712M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0074

 DESCRIPTION: PAST 5 YRS, COURSES/PRO DEVLP-NONE OF ABOVE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056713 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T056713Y (01 ) 0 YES

 002 T056713M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0075

 DESCRIPTION: AVAILABILITY OF RESOURCES

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T041201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T041201A (01 ) 0000 GET ALL RESOURCES

 002 T041201B (02 ) 1000 GET MOST RESOURCES

 003 T041201C (03 ) 0100 GET SOME RESOURCES

 004 T041201D (04 ) 0010 DONT GET RESOURCES

 005 T041201M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TCHR0076

 DESCRIPTION: ARE CURRICULUM SPECIALISTS AVAILABLE FOR MATH?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T041302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T041302Y (01 ) 00 YES

 002 T041302N (02 ) 10 NO

 003 T041302M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0077

 DESCRIPTION: SCIENCE CURRICULUM SPECIALIST

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T041303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T041303Y (01 ) 00 YES

 002 T041303N (02 ) 10 NO
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

595

 003 T041303M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TCHR0078

 DESCRIPTION: HOW MANY SCHOOL HOURS ARE PREP TIME PER WEEK?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T056801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T056801A (01 ) 000000 NONE

 002 T056801B (02 ) 100000 LESS THAN ONE

 003 T056801C (03 ) 010000 1-2

 004 T056801D (04 ) 001000 3-4

 005 T056801E (05 ) 000100 5

 006 T056801F (06 ) 000010 MORE THAN 5

 007 T056801M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0079

 DESCRIPTION: HOW MANY YRS TAUGHT SCI IN PUB/PRIV SCHOOLS?

 GRADES/ASSESSMENTS: N04

 CONDITIONING VAR LABEL:

 NAEP ID: T060201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T060201A (01 ) 000000 NOT TAUGHT SCIENCE

 002 T060201B (02 ) 100000 LESS THAN 3 YEARS

 003 T060201C (03 ) 010000 3-5 YEARS

 004 T060201D (04 ) 001000 6-9 YEARS

 005 T060201E (05 ) 000100 10-20 YEARS

 006 T060201F (06 ) 000010 MORE THAN 20 YEARS

 007 T060201M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TCHR0080

 DESCRIPTION: METHODS OF TCHING SCI? COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060301Y (01 ) 0 YES

 002 T060301M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0081

 DESCRIPTION: METHODS OF TCHING SCI?WRKSHP >1 WK

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060311 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060311Y (01 ) 0 YES

 002 T060311M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0082

 DESCRIPTION: METHODS OF TCHING SCI?WRKSHP <1 WK >1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060321 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060321Y (01 ) 0 YES

 002 T060321M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0083

 DESCRIPTION: METHODS OF TCHING SCI?WRKSHP <= 1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060331 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060331Y (01 ) 0 YES

 002 T060331M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0084

 DESCRIPTION: METHODS OF TCHING SCI?OTHER PROF. DEV

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060341 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060341Y (01 ) 0 YES

 002 T060341M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0085

 DESCRIPTION: UNIV COURSES IN-BIO/LIFE SCI? COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

596

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060302Y (01 ) 0 YES

 002 T060302M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0086

 DESCRIPTION: UNIV COURSES IN-BIO/LIFE SCI?WRKSHP >1 WK

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060312 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060312Y (01 ) 0 YES

 002 T060312M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0087

 DESCRIPTION: UNIV COURSES IN-BIO/LIFE SCI?WRKSHP <1 WK >1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060322 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060322Y (01 ) 0 YES

 002 T060322M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0088

 DESCRIPTION: UNIV COURSES IN-BIO/LIFE SCI?WRKSHP <= 1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060332 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060332Y (01 ) 0 YES

 002 T060332M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0089

 DESCRIPTION: UNIV COURSES IN-BIO/LIFE SCI?OTHER PROF. DEV

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060342 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060342Y (01 ) 0 YES

 002 T060342M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0090

 DESCRIPTION: UNIV COURSES IN-CHEMISTRY? COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060303Y (01 ) 0 YES

 002 T060303M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0091

 DESCRIPTION: UNIV COURSES IN-CHEMISTRY?WRKSHP >1 WK

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060313 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060313Y (01 ) 0 YES

 002 T060313M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0092

 DESCRIPTION: UNIV COURSES IN-CHEMISTRY?WRKSHP <1 WK >1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060323 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060323Y (01 ) 0 YES

 002 T060323M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0093

 DESCRIPTION: UNIV COURSES IN-CHEMISTRY?WRKSHP <= 1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060333 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060333Y (01 ) 0 YES

 002 T060333M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0094

 DESCRIPTION: UNIV COURSES IN-CHEMISTRY?OTHER PROF. DEV
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

597

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060343 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060343Y (01 ) 0 YES

 002 T060343M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0095

 DESCRIPTION: UNIV COURSES IN-PHYSICS? COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060304Y (01 ) 0 YES

 002 T060304M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0096

 DESCRIPTION: UNIV COURSES IN-PHYSICS?WRKSHP >1 WK

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060314 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060314Y (01 ) 0 YES

 002 T060314M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0097

 DESCRIPTION: UNIV COURSES IN-PHYSICS?WRKSHP <1 WK >1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060324 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060324Y (01 ) 0 YES

 002 T060324M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0098

 DESCRIPTION: UNIV COURSES IN-PHYSICS?WRKSHP <= 1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060334 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060334Y (01 ) 0 YES

 002 T060334M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0099

 DESCRIPTION: UNIV COURSES IN-PHYSICS?OTHER PROF. DEV

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060344 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060344Y (01 ) 0 YES

 002 T060344M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0100

 DESCRIPTION: UNIV COURSES IN-EARTH SCI? COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060305Y (01 ) 0 YES

 002 T060305M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0101

 DESCRIPTION: UNIV COURSES IN-EARTH SCI?WRKSHP >1 WK

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060315 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060315Y (01 ) 0 YES

 002 T060315M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0102

 DESCRIPTION: UNIV COURSES IN-EARTH SCI?WRKSHP <1 WK >1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060325 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060325Y (01 ) 0 YES

 002 T060325M (M ) 1 MISSING
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

598

 CONDITIONING VARIABLE ID: TCHR0103

 DESCRIPTION: UNIV COURSES IN-EARTH SCI?WRKSHP <= 1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060335 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060335Y (01 ) 0 YES

 002 T060335M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0104

 DESCRIPTION: UNIV COURSES IN-EARTH SCI?OTHER PROF. DEV

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060345 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060345Y (01 ) 0 YES

 002 T060345M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0105

 DESCRIPTION: UNIV COURSES-OTHER TYPES OF SCI? COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060306 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060306Y (01 ) 0 YES

 002 T060306M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0106

 DESCRIPTION: UNIV COURSES-OTHER TYPES OF SCI?WRKSHP >1 WK

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060316 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060316Y (01 ) 0 YES

 002 T060316M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0107

 DESCRIPTION: UNIV COURSES-OTHR TYPES OF SCI?WRKSHP <1 WK >1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060326 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060326Y (01 ) 0 YES

 002 T060326M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0108

 DESCRIPTION: UNIV COURSES-OTHER TYPES OF SCI?WRKSHP <= 1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060336 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060336Y (01 ) 0 YES

 002 T060336M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0109

 DESCRIPTION: UNIV COURSES-OTHER TYPES OF SCI?OTHER PROF. DEV

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060346 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060346Y (01 ) 0 YES

 002 T060346M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0110

 DESCRIPTION: UNIV COURSES IN-NONE OF ABOVE? COLLEGE COURSE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060307 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060307Y (01 ) 0 YES

 002 T060307M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0111

 DESCRIPTION: UNIV COURSES IN-NONE OF ABOVE?WRKSHP >1 WK

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060317 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

599

 001 T060317Y (01 ) 0 YES

 002 T060317M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0112

 DESCRIPTION: UNIV COURSES IN-NONE OF ABOVE?WRKSHP <1 WK >1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060327 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060327Y (01 ) 0 YES

 002 T060327M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0113

 DESCRIPTION: UNIV COURSES IN-NONE OF ABOVE?WRKSHP <= 1 DAY

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060337 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060337Y (01 ) 0 YES

 002 T060337M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0114

 DESCRIPTION: UNIV COURSES IN-NONE OF ABOVE?OTHER PROF. DEV

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060347 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060347Y (01 ) 0 YES

 002 T060347M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0115

 DESCRIPTION: PAST 5 YRS, COURSES/ACTVTS IN-COMP USE TO GET DATA

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060401Y (01 ) 0 YES

 002 T060401M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0116

 DESCRIPTION: PAST 5 YRS, COURSES/ACTVTS IN-COMP DATA ANALYSIS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060402Y (01 ) 0 YES

 002 T060402M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0117

 DESCRIPTION: PAST 5 YRS, COURSES/ACTVTS IN-MULTIMEDIA SCI ED?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060403Y (01 ) 0 YES

 002 T060403M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0118

 DESCRIPTION: PAST 5 YRS, COURSES/ACTVTS IN-LAB MNGMT/SAFETY?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060404Y (01 ) 0 YES

 002 T060404M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0119

 DESCRIPTION: PAST 5 YRS, COURSES/ACTVTS IN-INTEGRATED SCI INST?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T060405Y (01 ) 0 YES

 002 T060405M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TCHR0120

 DESCRIPTION: YOU BELONG TO 1 OR > SCI RELATED SCI ORGS?

 GRADES/ASSESSMENTS: N04, N08, S08
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

600

 CONDITIONING VAR LABEL:

 NAEP ID: T060501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T060501Y (01 ) 00 YES

 002 T060501N (02 ) 10 NO

 003 T060501M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TSUB0001

 DESCRIPTION: HOW OFTEN STUDS READ SCI TEXTBOOK?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060601A (01 ) 0000 ALMOST EVERY DAY

 002 T060601B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060601C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060601D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060601M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0002

 DESCRIPTION: HOW OFTEN STUDS READ BOOK/MAN ABOUT SCI?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060602A (01 ) 0000 ALMOST EVERY DAY

 002 T060602B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060602C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060602D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060602M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0003

 DESCRIPTION: HOW OFTEN STUDS DISCUSS SCI IN THE NEWS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060603A (01 ) 0000 ALMOST EVERY DAY

 002 T060603B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060603C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060603D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060603M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0004

 DESCRIPTION: HOW OFTEN STUDS WORK W/ OTHER STUDS ON ACT/PROJCT?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060604A (01 ) 0000 ALMOST EVERY DAY

 002 T060604B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060604C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060604D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060604M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0005

 DESCRIPTION: HOW OFTEN STUDS GIVE ORAL SCI REPORT?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060605A (01 ) 0000 ALMOST EVERY DAY

 002 T060605B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060605C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060605D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060605M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0006

 DESCRIPTION: HOW OFTEN STUDS PREPARE A WRITTEN SCI REPORT?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060606 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060606A (01 ) 0000 ALMOST EVERY DAY

 002 T060606B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060606C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060606D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060606M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0007
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

601

 DESCRIPTION: HOW OFTEN STUDS DO HANDS ON SCI ACTIVITIES IN SCI?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060607 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060607A (01 ) 0000 ALMOST EVERY DAY

 002 T060607B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060607C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060607D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060607M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0008

 DESCRIPTION: HOW OFTEN STUDS TALK ABOUT MEASURES/RESULTS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060608 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060608A (01 ) 0000 ALMOST EVERY DAY

 002 T060608B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060608C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060608D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060608M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0009

 DESCRIPTION: HOW OFTEN STUDS TAKE SCI TEST OR QUIZ?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060609 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060609A (01 ) 0000 ALMOST EVERY DAY

 002 T060609B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060609C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060609D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060609M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0010

 DESCRIPTION: HOW OFTEN STUDS USE LIBRARY RESOURCES FOR SCI?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060610 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060610A (01 ) 0000 ALMOST EVERY DAY

 002 T060610B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060610C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060610D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060610M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0011

 DESCRIPTION: HOW OFTEN STUDS USE COMPUTERS FOR SCI?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060611 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060611A (01 ) 0000 ALMOST EVERY DAY

 002 T060611B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060611C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060611D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060611M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0012

 DESCRIPTION: HOW OFTEN DO YOU TALK TO CLASS ABOUT SCI?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060701A (01 ) 0000 ALMOST EVERY DAY

 002 T060701B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060701C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060701D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060701M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0013

 DESCRIPTION: HOW OFTEN DO YOU DO A SCI DEMONSTRATION?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060702A (01 ) 0000 ALMOST EVERY DAY

 002 T060702B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060702C (03 ) 0100 ONCE OR TWICE/MONTH
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

602

 004 T060702D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060702M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0014

 DESCRIPTION: HOW OFTEN DO YOU SHOW A SCI VIDEOTAPE/TV PROGRAM?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060703A (01 ) 0000 ALMOST EVERY DAY

 002 T060703B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060703C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060703D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060703M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0015

 DESCRIPTION: HOW OFTEN DO YOU USE COMPUTERS FOR SCI?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060704A (01 ) 0000 ALMOST EVERY DAY

 002 T060704B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060704C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060704D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060704M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0016

 DESCRIPTION: HOW OFTEN DO YOU USE CDS OR LASER DISKS ON SCI?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T060705A (01 ) 0000 ALMOST EVERY DAY

 002 T060705B (02 ) 1000 ONCE OR TWICE A WEEK

 003 T060705C (03 ) 0100 ONCE OR TWICE/MONTH

 004 T060705D (04 ) 0010 NEVER OR HARDLY EVER

 005 T060705M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0017

 DESCRIPTION: HOW OFTEN YOUR SCI STUDS GO ON SCI FIELD TRIPS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T060801A (01 ) 000 3 OR MORE TIMES/YEAR

 002 T060801B (02 ) 100 1 OR 2 TIMES A YEAR

 003 T060801C (03 ) 010 NEVER OR HARDLY EVER

 004 T060801M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0018

 DESCRIPTION: HOW OFTEN DO YOU BRING GUEST SPEAKER FOR SCI STUDS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T060901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T060901A (01 ) 000 3 OR MORE TIMES/YEAR

 002 T060901B (02 ) 100 1 OR 2 TIMES A YEAR

 003 T060901C (03 ) 010 NEVER OR HARDLY EVER

 004 T060901M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0019

 DESCRIPTION: SAVE STUDS SCI WORK IN PORTFOLIOS FOR ASSESSMENT?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T061001Y (01 ) 00 YES

 002 T061001N (02 ) 10 NO

 003 T061001M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TSUB0020

 DESCRIPTION: HOW MUCH EMPHASIS-KNOWING SCI FACTS/TERMS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T061101A (01 ) 000 HEAVY EMPHASIS

 002 T061101B (02 ) 100 MODERATE EMPHASIS

 003 T061101C (03 ) 010 LITTLE/NO EMPHASIS
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

603

 004 T061101M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0021

 DESCRIPTION: HOW MUCH EMPHASIS-UNDERSTANDING KEY SCI CONCEPTS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061102 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T061102A (01 ) 000 HEAVY EMPHASIS

 002 T061102B (02 ) 100 MODERATE EMPHASIS

 003 T061102C (03 ) 010 LITTLE/NO EMPHASIS

 004 T061102M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0022

 DESCRIPTION: HOW MUCH EMPHASIS-DEVELOP SCI PROB SOLVING SKILL?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061103 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T061103A (01 ) 000 HEAVY EMPHASIS

 002 T061103B (02 ) 100 MODERATE EMPHASIS

 003 T061103C (03 ) 010 LITTLE/NO EMPHASIS

 004 T061103M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0023

 DESCRIPTION: HOW MUCH EMPHASIS-SCI RELEVANCE TO SOCIETY/TECH?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061104 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T061104A (01 ) 000 HEAVY EMPHASIS

 002 T061104B (02 ) 100 MODERATE EMPHASIS

 003 T061104C (03 ) 010 LITTLE/NO EMPHASIS

 004 T061104M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0024

 DESCRIPTION: HOW MUCH EMPHASIS-COMMUNICATE IDEAS IN SCI?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061105 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T061105A (01 ) 000 HEAVY EMPHASIS

 002 T061105B (02 ) 100 MODERATE EMPHASIS

 003 T061105C (03 ) 010 LITTLE/NO EMPHASIS

 004 T061105M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0025

 DESCRIPTION: HOW MUCH EMPHASIS-DEVELOPING LAB SKILLS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061106 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T061106A (01 ) 000 HEAVY EMPHASIS

 002 T061106B (02 ) 100 MODERATE EMPHASIS

 003 T061106C (03 ) 010 LITTLE/NO EMPHASIS

 004 T061106M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0026

 DESCRIPTION: HOW MUCH EMPHASIS-DEVELOPING STUDS SCI INTEREST?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061107 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T061107A (01 ) 000 HEAVY EMPHASIS

 002 T061107B (02 ) 100 MODERATE EMPHASIS

 003 T061107C (03 ) 010 LITTLE/NO EMPHASIS

 004 T061107M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0027

 DESCRIPTION: HOW MUCH EMPHASIS-DEVELOPING DATA ANALYSIS SKILLS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061108 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T061108A (01 ) 000 HEAVY EMPHASIS

 002 T061108B (02 ) 100 MODERATE EMPHASIS

 003 T061108C (03 ) 010 LITTLE/NO EMPHASIS

 004 T061108M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0028
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

604

 DESCRIPTION: HOW MUCH EMPHASIS-USING TECH AS SCI TOOL?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061109 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T061109A (01 ) 000 HEAVY EMPHASIS

 002 T061109B (02 ) 100 MODERATE EMPHASIS

 003 T061109C (03 ) 010 LITTLE/NO EMPHASIS

 004 T061109M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0029

 DESCRIPTION: EVER ASSIGN SOLO/GROUP SCI PROJECTS THAT TAKE >WK?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T061201Y (01 ) 00 YES

 002 T061201N (02 ) 10 NO

 003 T061201M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: TSUB0030

 DESCRIPTION: HOW OFTEN USE MULT CHOICE TESTS TO ACCESS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061301A (01 ) 00000 ONCE OR TWICE WEEK

 002 T061301B (02 ) 10000 ONCE OR TWICE MONTH

 003 T061301C (03 ) 01000 ONCE/GRADING PERIOD

 004 T061301D (04 ) 00100 ONCE OR TWICE/YEAR

 005 T061301E (05 ) 00010 NEVER OR HARDLY EVER

 006 T061301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0031

 DESCRIPTION: HOW OFTEN USE SHOR/LONG WRITTEN RESPONSE TO ACCESS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061302A (01 ) 00000 ONCE OR TWICE WEEK

 002 T061302B (02 ) 10000 ONCE OR TWICE MONTH

 003 T061302C (03 ) 01000 ONCE/GRADING PERIOD

 004 T061302D (04 ) 00100 ONCE OR TWICE/YEAR

 005 T061302E (05 ) 00010 NEVER OR HARDLY EVER

 006 T061302M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0032

 DESCRIPTION: HOW OFTEN USE SOLO PROJECTS TO ACCESS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061303A (01 ) 00000 ONCE OR TWICE WEEK

 002 T061303B (02 ) 10000 ONCE OR TWICE MONTH

 003 T061303C (03 ) 01000 ONCE/GRADING PERIOD

 004 T061303D (04 ) 00100 ONCE OR TWICE/YEAR

 005 T061303E (05 ) 00010 NEVER OR HARDLY EVER

 006 T061303M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0033

 DESCRIPTION: HOW OFTEN USE GROUP PROJECTS TO ACCESS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061304A (01 ) 00000 ONCE OR TWICE WEEK

 002 T061304B (02 ) 10000 ONCE OR TWICE MONTH

 003 T061304C (03 ) 01000 ONCE/GRADING PERIOD

 004 T061304D (04 ) 00100 ONCE OR TWICE/YEAR

 005 T061304E (05 ) 00010 NEVER OR HARDLY EVER

 006 T061304M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0034

 DESCRIPTION: HOW OFTEN USE WORK PORTFOLIOS TO ACCESS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061305A (01 ) 00000 ONCE OR TWICE WEEK

 002 T061305B (02 ) 10000 ONCE OR TWICE MONTH
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

605

 003 T061305C (03 ) 01000 ONCE/GRADING PERIOD

 004 T061305D (04 ) 00100 ONCE OR TWICE/YEAR

 005 T061305E (05 ) 00010 NEVER OR HARDLY EVER

 006 T061305M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0035

 DESCRIPTION: HOW OFTEN USE IN CLASS ESSAYS TO ACCESS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061306 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061306A (01 ) 00000 ONCE OR TWICE WEEK

 002 T061306B (02 ) 10000 ONCE OR TWICE MONTH

 003 T061306C (03 ) 01000 ONCE/GRADING PERIOD

 004 T061306D (04 ) 00100 ONCE OR TWICE/YEAR

 005 T061306E (05 ) 00010 NEVER OR HARDLY EVER

 006 T061306M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0036

 DESCRIPTION: HOW OFTEN USE SELF/PEER EVAL TO ACCESS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061307 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061307A (01 ) 00000 ONCE OR TWICE WEEK

 002 T061307B (02 ) 10000 ONCE OR TWICE MONTH

 003 T061307C (03 ) 01000 ONCE/GRADING PERIOD

 004 T061307D (04 ) 00100 ONCE OR TWICE/YEAR

 005 T061307E (05 ) 00010 NEVER OR HARDLY EVER

 006 T061307M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0037

 DESCRIPTION: HOW OFTEN USE LAB NOTEBOOKS/JOURNALS TO ACCESS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061308 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061308A (01 ) 00000 ONCE OR TWICE WEEK

 002 T061308B (02 ) 10000 ONCE OR TWICE MONTH

 003 T061308C (03 ) 01000 ONCE/GRADING PERIOD

 004 T061308D (04 ) 00100 ONCE OR TWICE/YEAR

 005 T061308E (05 ) 00010 NEVER OR HARDLY EVER

 006 T061308M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0038

 DESCRIPTION: HOW OFTEN USE HOMEWORK TO ACCESS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061309 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061309A (01 ) 00000 ONCE OR TWICE WEEK

 002 T061309B (02 ) 10000 ONCE OR TWICE MONTH

 003 T061309C (03 ) 01000 ONCE/GRADING PERIOD

 004 T061309D (04 ) 00100 ONCE OR TWICE/YEAR

 005 T061309E (05 ) 00010 NEVER OR HARDLY EVER

 006 T061309M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0039

 DESCRIPTION: HOW OFTEN USE HANDS ON ACTIVITIES TO ACCESS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061310 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061310A (01 ) 00000 ONCE OR TWICE WEEK

 002 T061310B (02 ) 10000 ONCE OR TWICE MONTH

 003 T061310C (03 ) 01000 ONCE/GRADING PERIOD

 004 T061310D (04 ) 00100 ONCE OR TWICE/YEAR

 005 T061310E (05 ) 00010 NEVER OR HARDLY EVER

 006 T061310M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0040

 DESCRIPTION: PROPORTION OF EVAL IN SCI BASED ON HANDS ON ACTVS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T061401A (01 ) 0000 MOST OR ALL OF GRADE

 002 T061401B (02 ) 1000 ABOUT HALF OF GRADE

 003 T061401C (03 ) 0100 VERY LITTLE OF GRADE

 004 T061401D (04 ) 0010 NONE OF GRADE

 005 T061401M (M ) 0001 MISSING
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

606

 CONDITIONING VARIABLE ID: TSUB0041

 DESCRIPTION: BEST DESCRIPTION OF COMPUTER AVAILABILITY FOR SCI

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 T061501A (01 ) 000000 NONE AVAILABLE

 002 T061501B (02 ) 100000 1 IN CLASSROOM

 003 T061501C (03 ) 010000 2-3 IN CLASSROOM

 004 T061501D (04 ) 001000 4 IN CLASSROOM

 005 T061501E (05 ) 000100 IN LAB BUT DIFFICULT

 006 T061501F (06 ) 000010 IN LAB AND EASY

 007 T061501M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: TSUB0042

 DESCRIPTION: USE COMPUTERS FOR SCI INSTRUCTION: PLAYING SCI

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T061601Y (01 ) 0 YES

 002 T061601M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0043

 DESCRIPTION: USE COMPUTERS FOR SCI INSTRUCTION: PLAYING SCI

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061611 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T061611Y (01 ) 0 YES

 002 T061611M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0044

 DESCRIPTION: USE COMPUTERS FOR SCI INSTRUCTION: SIMULATIONS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061621 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T061621Y (01 ) 0 YES

 002 T061621M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0045

 DESCRIPTION: USE COMPUTERS FOR SCI INSTRUCTION: DATA ANALYSIS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061631 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T061631Y (01 ) 0 YES

 002 T061631M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0046

 DESCRIPTION: USE COMPUTERS FOR SCI INSTRUCTION: WORD PROCESS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061641 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T061641Y (01 ) 0 YES

 002 T061641M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0047

 DESCRIPTION: USE COMPUTERS FOR SCI INSTRUCTION: DO NOT USE

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061651 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T061651Y (01 ) 0 YES

 002 T061651M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0048

 DESCRIPTION: STUDS ASSIGNED TO CLASS BY ABILITY/ACHVMNT LEVEL?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061701A (01 ) 00000 NOT ASSIGNED BY ABIL

 002 T061701B (02 ) 10000 ASSIGNED BY MATN ABL

 003 T061701C (03 ) 01000 ASSIGNED BY READING

 004 T061701D (04 ) 00100 ASSIGNED BY SCIENCE
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

607

 005 T061701E (05 ) 00010 ASSIGNED BY GENERAL

 006 T061701M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0049

 DESCRIPTION: IF ASSIGNED BY ABILITY WHICH BEST DESCRIBES LEVEL?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T061801A (01 ) 00000 NOT ASSIGNED BY ABIL

 002 T061801B (02 ) 10000 MOSTLY HIGH ABILITY

 003 T061801C (03 ) 01000 MOSTLY AVERAGE ABIL

 004 T061801D (04 ) 00100 MOSTLY LOW ABILITY

 005 T061801E (05 ) 00010 MOSTLY MIXED ABILITY

 006 T061801M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0050

 DESCRIPTION: COMPOSITION OF CLASS ACCORDING TO GENDER?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T061901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 T061901A (01 ) 000 MIXED

 002 T061901B (02 ) 100 ALL MALE

 003 T061901C (03 ) 010 ALL FEMALE

 004 T061901M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: TSUB0051

 DESCRIPTION: HOW MUCH TIME CLASS SPEND ON LIFE SCIENCE?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T062001A (01 ) 0000 A LOT

 002 T062001B (02 ) 1000 SOME

 003 T062001C (03 ) 0100 LITTLE

 004 T062001D (04 ) 0010 NONE

 005 T062001M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0052

 DESCRIPTION: HOW MUCH TIME CLASS SPEND ON EARTH SCIENCE?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T062002A (01 ) 0000 A LOT

 002 T062002B (02 ) 1000 SOME

 003 T062002C (03 ) 0100 LITTLE

 004 T062002D (04 ) 0010 NONE

 005 T062002M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0053

 DESCRIPTION: HOW MUCH TIME CLASS SPEND ON PHYSICAL SCIENCE?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T062003A (01 ) 0000 A LOT

 002 T062003B (02 ) 1000 SOME

 003 T062003C (03 ) 0100 LITTLE

 004 T062003D (04 ) 0010 NONE

 005 T062003M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0054

 DESCRIPTION: WHICH BEST DESCRIBES SPACE WHERE CLASS TAUGHT?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 5

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 4

 001 T062101A (01 ) 0000 CLASS ONLY

 002 T062101B (02 ) 1000 CLASS W/ H2O ACCESS

 003 T062101C (03 ) 0100 CLASS W/ LAB ACCESS

 004 T062101D (04 ) 0010 LAB W/ H2O ACCESS

 005 T062101M (M ) 0001 MISSING

 CONDITIONING VARIABLE ID: TSUB0055

 DESCRIPTION: DO STUDS PRODUCE NOTEBOOKS/REPORTS OF LAB WORK?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

608

 001 T062201Y (01 ) 0 YES

 002 T062201M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0056

 DESCRIPTION: DO STUDS PRODUCE REPORTS OF EXTENDED SCI PROJECTS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T062202Y (01 ) 0 YES

 002 T062202M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0057

 DESCRIPTION: DO STUDS PRODUCE REPORTS ON SPECIFIC TOPIC/ISSUE?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T062203Y (01 ) 0 YES

 002 T062203M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0058

 DESCRIPTION: DO STUDS PRODUCE REPORTS/RECORDS OF FIELD TRIPS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062204 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T062204Y (01 ) 0 YES

 002 T062204M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0059

 DESCRIPTION: DO STUDS PRODUCE JOURNALS/DIARIES/LOGS OF IDEAS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062205 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T062205Y (01 ) 0 YES

 002 T062205M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0060

 DESCRIPTION: DO STUDS PRODUCE PHOTO RECORDS OF PROJECTS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062206 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T062206Y (01 ) 0 YES

 002 T062206M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0061

 DESCRIPTION: DO STUDS PRODUCE AUDIO/VIDEOTAPE RECORDS OF ACTVS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062207 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T062207Y (01 ) 0 YES

 002 T062207M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0062

 DESCRIPTION: DO STUDS PRODUCE REPORTS OF PERSONAL INTERVIEWS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062208 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T062208Y (01 ) 0 YES

 002 T062208M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0063

 DESCRIPTION: DO STUDS PRODUCE 3D SCI MODELS?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062209 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T062209Y (01 ) 0 YES

 002 T062209M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0064

 DESCRIPTION: DO STUDS PRODUCE COMP GENERATED MULTMEDIA PROJECTS

 GRADES/ASSESSMENTS: N04, N08, S08
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

609

 CONDITIONING VAR LABEL:

 NAEP ID: T062210 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 T062210Y (01 ) 0 YES

 002 T062210M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: TSUB0065

 DESCRIPTION: TIME PER WEEK EXPECT STUD TO SPEND ON HOMEWORK?

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T062301A (01 ) 00000 NONE

 002 T062301B (02 ) 10000 1/2 HOUR

 003 T062301C (03 ) 01000 1 HOUR

 004 T062301D (04 ) 00100 2 HOURS

 005 T062301E (05 ) 00010 MORE THAN 2 HOURS

 006 T062301M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TSUB0066

 DESCRIPTION: CLASS PERIOD AND # OF STUDS IN CLASS

 GRADES/ASSESSMENTS: N04, N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T062401A (01 ) 00000 1-20 STUDENTS

 002 T062401B (02 ) 10000 21025 STUDENTS

 003 T062401C (03 ) 01000 26-30 STUDENTS

 004 T062401D (04 ) 00100 31-35 STUDENTS

 005 T062401E (05 ) 00010 36 OR MORE STUDENTS

 006 T062401M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0087

 DESCRIPTION: BEST DESCRIBES HOW 8TH GRADES ARE ORGANIZED?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 4

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 3

 001 C034201A (01 ) 000 SELF-CONTAINED

 002 C034201B (02 ) 100 SEMI-DEPARTMENTALIZE

 003 C034201C (03 ) 010 DEPARTMENTALIZED

 004 C034201M (M ) 001 MISSING

 CONDITIONING VARIABLE ID: SCHL0088

 DESCRIPTION: ARE 8TH-GRADERS ASSIGNED TO MATH BY ABILITY?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C034402Y (01 ) 00 YES

 002 C034402N (02 ) 10 NO

 003 C034402M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0089

 DESCRIPTION: ARE 8TH-GRADERS ASSIGNED TO SCIENCE BY ABILITY?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C034403Y (01 ) 00 YES

 002 C034403N (02 ) 10 NO

 003 C034403M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0090

 DESCRIPTION: ARE 8TH-GRADERS ASSIGNED TO ENGLISH BY ABILITY?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C034401Y (01 ) 00 YES

 002 C034401N (02 ) 10 NO

 003 C034401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0091

 DESCRIPTION: ARE 8TH-GRADERS ASSIGNED TO ARTS BY ABILITY?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

610

 001 C034406Y (01 ) 00 YES

 002 C034406N (02 ) 10 NO

 003 C034406M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0092

 DESCRIPTION: HOW OFTEN 8TH GRDS RECEIVE COMP SCI INSTRUCTION?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034510 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C034510A (01 ) 00000 EVERY DAY

 002 C034510B (02 ) 10000 3 OR 4 TIMES A WEEK

 003 C034510C (03 ) 01000 ONCE OR TWICE/WEEK

 004 C034510D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C034510E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C034510M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0093

 DESCRIPTION: HOW OFTEN 8TH GRDS RECEIVE MATH INSTRUCTION?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034511 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C034511A (01 ) 00000 EVERY DAY

 002 C034511B (02 ) 10000 3 OR 4 TIMES A WEEK

 003 C034511C (03 ) 01000 ONCE OR TWICE/WEEK

 004 C034511D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C034511E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C034511M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0094

 DESCRIPTION: HOW OFTEN 8TH GRDS RECEIVE SCIENCE INSTRUCTION?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034512 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C034512A (01 ) 00000 EVERY DAY

 002 C034512B (02 ) 10000 3 OR 4 TIMES A WEEK

 003 C034512C (03 ) 01000 ONCE OR TWICE/WEEK

 004 C034512D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C034512E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C034512M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0095

 DESCRIPTION: HOW OFTEN 8TH GRDS RECEIVE ENGLISH INSTRUCTION?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034513 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C034513A (01 ) 00000 EVERY DAY

 002 C034513B (02 ) 10000 3 OR 4 TIMES A WEEK

 003 C034513C (03 ) 01000 ONCE OR TWICE/WEEK

 004 C034513D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C034513E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C034513M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0096

 DESCRIPTION: HOW OFTEN 8TH GRDS RECEIVE ARTS INSTRUCTION?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034514 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C034514A (01 ) 00000 EVERY DAY

 002 C034514B (02 ) 10000 3 OR 4 TIMES A WEEK

 003 C034514C (03 ) 01000 ONCE OR TWICE/WEEK

 004 C034514D (04 ) 00100 LESS THAN ONCE/WEEK

 005 C034514E (05 ) 00010 SUBJECT NOT TAUGHT

 006 C034514M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0097

 DESCRIPTION: HAS ENGLISH BEEN IDENTIFIED AS A PRIORITY?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C031611 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C031611Y (01 ) 00 YES

 002 C031611N (02 ) 10 NO

 003 C031611M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0098

 DESCRIPTION: SCHOOL OFFER 8TH GR STUDS ALGEBRA FOR HS CREDIT?
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

611

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C034601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C034601Y (01 ) 00 YES

 002 C034601N (02 ) 10 NO

 003 C034601M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0099

 DESCRIPTION: SCHOOL W/ SPECIAL FOCUS ON ENGLISH?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037203Y (01 ) 0 YES

 002 C037203M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0100

 DESCRIPTION: SCHOOL FOLLOW DISTRICT/STATE ENGLISH CURRICULUM?

 GRADES/ASSESSMENTS: N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: C037306 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C037306Y (01 ) 0 YES

 002 C037306M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0101

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR MATH?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039401Y (01 ) 0 YES

 002 C039401M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0102

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR SCIENCE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039402Y (01 ) 0 YES

 002 C039402M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0103

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR READING?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039403Y (01 ) 0 YES

 002 C039403M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0104

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR ARTS?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039404Y (01 ) 0 YES

 002 C039404M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0105

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR OTHER?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039405 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039405Y (01 ) 0 YES

 002 C039405M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0106

 DESCRIPTION: SCHOOL SPONSER 8TH GRDS FIELD TRIP FOR NONE ABOVE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039406 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039406Y (01 ) 0 YES
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

612

 002 C039406M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0107

 DESCRIPTION: 8TH GRADERS IN EXTRACURR ACTS FOR MATH?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039501Y (01 ) 0 YES

 002 C039501M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0108

 DESCRIPTION: 8TH GRADERS IN EXTRACURR ACTS FOR SCIENCE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039502Y (01 ) 0 YES

 002 C039502M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0109

 DESCRIPTION: 8TH GRADERS IN EXTRACURR ACTS FOR ENG/LANG ARTS?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039503Y (01 ) 0 YES

 002 C039503M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0110

 DESCRIPTION: 8TH GRADERS IN EXTRACURR ACTS FOR ARTS?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039504Y (01 ) 0 YES

 002 C039504M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0111

 DESCRIPTION: 8TH GRADERS IN EXTRACURR ACTS FOR NONE OF ABOVE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039505Y (01 ) 0 YES

 002 C039505M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0112

 DESCRIPTION: 8TH GRADERS IN SUMMER PROGRAMS IN MATH?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039601Y (01 ) 0 YES

 002 C039601M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0113

 DESCRIPTION: 8TH GRADERS IN SUMMER PROGRAMS IN SCIENCE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039602Y (01 ) 0 YES

 002 C039602M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0114

 DESCRIPTION: 8TH GRADERS IN SUMMER PROGRAMS IN ENG/LANG ARTS?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039603Y (01 ) 0 YES

 002 C039603M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0115

 DESCRIPTION: 8TH GRADERS IN SUMMER PROGRAMS IN ARTS?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

613

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039604Y (01 ) 0 YES

 002 C039604M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0116

 DESCRIPTION: 8TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C039605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C039605Y (01 ) 0 YES

 002 C039605M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0117

 DESCRIPTION: WHAT % OF 8TH GRDS HELD BACK/REPEAT 8TH GRADE?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: C041901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C041901A (01 ) 00000 0%

 002 C041901B (02 ) 10000 1-2%

 003 C041901C (03 ) 01000 3-5%

 004 C041901D (04 ) 00100 6-10%

 005 C041901E (05 ) 00010 MORE THAN 10%

 006 C041901M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0121

 DESCRIPTION: COUNTING THIS YR, HOW MANY YRS TOTAL TAUGHT SCI?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T062501A (01 ) 00000 2 YEARS OR LESS

 002 T062501B (02 ) 10000 3-5 YEARS

 003 T062501C (03 ) 01000 6-10 YEARS

 004 T062501D (04 ) 00100 11-24 YEARS

 005 T062501E (05 ) 00010 25 YEARS OR MORE

 006 T062501M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0122

 DESCRIPTION: LAST YR, TIME IN PRO WORKSHOPS/SEMS IN SCI?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T062601A (01 ) 00000 NONE

 002 T062601B (02 ) 10000 LESS THAN 6 HOURS

 003 T062601C (03 ) 01000 6-15 HOURS

 004 T062601D (04 ) 00100 16-35 HOURS

 005 T062601E (05 ) 00010 MORE THAN 35 HOURS

 006 T062601M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0123

 DESCRIPTION: LAST 2 YRS, # OF UNIV COURSES IN SCI/SCI ED?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 T062701A (01 ) 00000 NONE

 002 T062701B (02 ) 10000 ONE

 003 T062701C (03 ) 01000 TWO

 004 T062701D (04 ) 00100 THREE

 005 T062701E (05 ) 00010 FOUR OR MORE

 006 T062701M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: TCHR0124

 DESCRIPTION: CURRICULUM SPECIALIST TO HELP/ADVISE IN SCI?

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: T062801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 T062801Y (01 ) 00 YES

 002 T062801N (02 ) 10 NO

 003 T062801M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0118

 DESCRIPTION: ARE 12TH-GRADERS ASSIGNED TO MATH BY ABILITY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

614

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035002Y (01 ) 00 YES

 002 C035002N (02 ) 10 NO

 003 C035002M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0119

 DESCRIPTION: ARE 12TH-GRADERS ASSIGNED TO SCIENCE BY ABILITY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035003Y (01 ) 00 YES

 002 C035003N (02 ) 10 NO

 003 C035003M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0120

 DESCRIPTION: ARE 12TH-GRADERS ASSIGNED TO ENGLISH BY ABILITY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035006 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035006Y (01 ) 00 YES

 002 C035006N (02 ) 10 NO

 003 C035006M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0121

 DESCRIPTION: ARE 12TH-GRADERS ASSIGNED TO ARTS BY ABILITY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C035007 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C035007Y (01 ) 00 YES

 002 C035007N (02 ) 10 NO

 003 C035007M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0122

 DESCRIPTION: FROM 9TH ON HOW MANY YRS REQUIRED FOR MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040201 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040201A (01 ) 0000000 NONE

 002 C040201B (02 ) 1000000 ONE HALF YEAR

 003 C040201C (03 ) 0100000 ONE YEAR

 004 C040201D (04 ) 0010000 TWO YEARS

 005 C040201E (05 ) 0001000 THREE YEARS

 006 C040201F (06 ) 0000100 FOUR YEARS

 007 C040201G (07 ) 0000010 MORE THAN FOUR YEARS

 008 C040201M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0123

 DESCRIPTION: FROM 9TH ON HOW MANY YRS REQUIRED FOR SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040202 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040202A (01 ) 0000000 NONE

 002 C040202B (02 ) 1000000 ONE HALF YEAR

 003 C040202C (03 ) 0100000 ONE YEAR

 004 C040202D (04 ) 0010000 TWO YEARS

 005 C040202E (05 ) 0001000 THREE YEARS

 006 C040202F (06 ) 0000100 FOUR YEARS

 007 C040202G (07 ) 0000010 MORE THAN FOUR YEARS

 008 C040202M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0124

 DESCRIPTION: FROM 9TH ON HOW MANY YRS REQUIRED FOR ENG/LIT?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040203 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040203A (01 ) 0000000 NONE

 002 C040203B (02 ) 1000000 ONE HALF YEAR

 003 C040203C (03 ) 0100000 ONE YEAR

 004 C040203D (04 ) 0010000 TWO YEARS

 005 C040203E (05 ) 0001000 THREE YEARS

 006 C040203F (06 ) 0000100 FOUR YEARS

 007 C040203G (07 ) 0000010 MORE THAN FOUR YEARS

 008 C040203M (M ) 0000001 MISSING
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

615

 CONDITIONING VARIABLE ID: SCHL0125

 DESCRIPTION: FROM 9TH ON HOW MANY YRS REQUIRED FINE/PERF ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040204 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040204A (01 ) 0000000 NONE

 002 C040204B (02 ) 1000000 ONE HALF YEAR

 003 C040204C (03 ) 0100000 ONE YEAR

 004 C040204D (04 ) 0010000 TWO YEARS

 005 C040204E (05 ) 0001000 THREE YEARS

 006 C040204F (06 ) 0000100 FOUR YEARS

 007 C040204G (07 ) 0000010 MORE THAN FOUR YEARS

 008 C040204M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0126

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN ADVANCED BIO?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040301 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040301Y (01 ) 0 YES

 002 C040301M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0127

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN ADVANCED CHEM?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040302 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040302Y (01 ) 0 YES

 002 C040302M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0128

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN ADV PHYSICS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040303 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040303Y (01 ) 0 YES

 002 C040303M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0129

 DESCRIPTION: NO ADVANCED SCIENCE COURSES TAUGHT

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040304 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040304Y (01 ) 0 YES

 002 C040304M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0130

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN COMP SCI?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040305 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040305Y (01 ) 0 YES

 002 C040305M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0131

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN CALCULUS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040306 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040306Y (01 ) 0 YES

 002 C040306M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0132

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN TRIGONOMETRY?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040307 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040307Y (01 ) 0 YES

 002 C040307M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0133
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

616

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN PRECALCULUS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040308 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040308Y (01 ) 0 YES

 002 C040308M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0134

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN PROB/STAT?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040309 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040309Y (01 ) 0 YES

 002 C040309M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0135

 DESCRIPTION: COURSES 1 OR > SEMESTERS TAUGHT IN UNI/INTEG MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040310 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040310Y (01 ) 0 YES

 002 C040310M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0136

 DESCRIPTION: NO ADVANCED MATH COURSES TAUGHT?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040311 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040311Y (01 ) 0 YES

 002 C040311M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0137

 DESCRIPTION: STUDS REQUIRED TO PASS STATE TEST IN MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040401 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C040401Y (01 ) 00 YES

 002 C040401N (02 ) 10 NO

 003 C040401M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0138

 DESCRIPTION: STUDS REQUIRED TO PASS STATE TEST IN SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040402 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C040402Y (01 ) 00 YES

 002 C040402N (02 ) 10 NO

 003 C040402M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0139

 DESCRIPTION: STUDS REQUIRED TO PASS STATE TEST IN ENG/LANG ARTS

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040403 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C040403Y (01 ) 00 YES

 002 C040403N (02 ) 10 NO

 003 C040403M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0140

 DESCRIPTION: STUDS REQUIRED TO PASS STATE TEST IN FINE/PERF ART

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040404 TOTAL NUMBER OF SPECIFIED CONTRASTS: 3

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 2

 001 C040404Y (01 ) 00 YES

 002 C040404N (02 ) 10 NO

 003 C040404M (M ) 01 MISSING

 CONDITIONING VARIABLE ID: SCHL0141

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS FOR MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

617

 NAEP ID: C040501 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040501Y (01 ) 0 YES

 002 C040501M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0142

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS FOR SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040502 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040502Y (01 ) 0 YES

 002 C040502M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0143

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS IN ENG/LANG ART

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040503 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040503Y (01 ) 0 YES

 002 C040503M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0144

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS IN ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040504 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040504Y (01 ) 0 YES

 002 C040504M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0145

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS IN OTHER?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040505 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040505Y (01 ) 0 YES

 002 C040505M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0146

 DESCRIPTION: SCHOOL SPONSER 12TH GR FIELD TRIPS IN NONE ABOVE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040506 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040506Y (01 ) 0 YES

 002 C040506M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0147

 DESCRIPTION: 12TH GRADERS IN EXTRACURR ACTS IN MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040601 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040601Y (01 ) 0 YES

 002 C040601M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0148

 DESCRIPTION: 12TH GRADERS IN EXTRACURR ACTS IN SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040602 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040602Y (01 ) 0 YES

 002 C040602M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0149

 DESCRIPTION: 12TH GRADERS IN EXTRACURR ACTS IN ENG/LANG ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040603 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040603Y (01 ) 0 YES

 002 C040603M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0150
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

618

 DESCRIPTION: 12TH GRADERS IN EXTRACURR ACTS IN ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040604 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040604Y (01 ) 0 YES

 002 C040604M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0151

 DESCRIPTION: 12TH GRADERS IN EXTRACURR ACTS IN NONE OF ABOVE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040605 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040605Y (01 ) 0 YES

 002 C040605M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0152

 DESCRIPTION: 12TH GRADERS IN SUMMER PROGRAMS IN MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040701 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040701Y (01 ) 0 YES

 002 C040701M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0153

 DESCRIPTION: 12TH GRADERS IN SUMMER PROGRAMS IN SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040702 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040702Y (01 ) 0 YES

 002 C040702M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0154

 DESCRIPTION: 12TH GRADERS IN SUMMER PORGRAMS IN ENG/LANG ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040703 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040703Y (01 ) 0 YES

 002 C040703M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0155

 DESCRIPTION: 12TH GRADERS IN SUMMER PROGRAMS IN ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040704 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040704Y (01 ) 0 YES

 002 C040704M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0156

 DESCRIPTION: 12TH GRADERS IN SUMMER PROGRAMS IN NONE OF ABOVE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040705 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040705Y (01 ) 0 YES

 002 C040705M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0157

 DESCRIPTION: # STUDS ENROLLED IN AP SCIENCE COURSES?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040801 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040801A (01 ) 0000000 NONE

 002 C040801B (02 ) 1000000 1-10

 003 C040801C (03 ) 0100000 11-25

 004 C040801D (04 ) 0010000 26-50

 005 C040801E (05 ) 0001000 51-75

 006 C040801F (06 ) 0000100 76-99

 007 C040801G (07 ) 0000010 100 OR MORE

 008 C040801M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0158

 DESCRIPTION: # STUDS ENROLLED IN AP CALCULUS COURSES?
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

619

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040802 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040802A (01 ) 0000000 NONE

 002 C040802B (02 ) 1000000 1-10

 003 C040802C (03 ) 0100000 11-25

 004 C040802D (04 ) 0010000 26-50

 005 C040802E (05 ) 0001000 51-75

 006 C040802F (06 ) 0000100 76-99

 007 C040802G (07 ) 0000010 100 OR MORE

 008 C040802M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0159

 DESCRIPTION: # STUDS ENROLLED IN AP COMP SCI COURSES?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040803 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040803A (01 ) 0000000 NONE

 002 C040803B (02 ) 1000000 1-10

 003 C040803C (03 ) 0100000 11-25

 004 C040803D (04 ) 0010000 26-50

 005 C040803E (05 ) 0001000 51-75

 006 C040803F (06 ) 0000100 76-99

 007 C040803G (07 ) 0000010 100 OR MORE

 008 C040803M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0160

 DESCRIPTION: # STUDS ENROLLED IN AP ENGLISH COURSES?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040804 TOTAL NUMBER OF SPECIFIED CONTRASTS: 8

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 7

 001 C040804A (01 ) 0000000 NONE

 002 C040804B (02 ) 1000000 1-10

 003 C040804C (03 ) 0100000 11-25

 004 C040804D (04 ) 0010000 26-50

 005 C040804E (05 ) 0001000 51-75

 006 C040804F (06 ) 0000100 76-99

 007 C040804G (07 ) 0000010 100 OR MORE

 008 C040804M (M ) 0000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0161

 DESCRIPTION: ANY 12TH GRDS TAKING COLLEGE COURSES IN MATH?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040901 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040901Y (01 ) 0 YES

 002 C040901M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0162

 DESCRIPTION: ANY 12TH GRDS TAKING COLLEGE COURSES IN SCIENCE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040902 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040902Y (01 ) 0 YES

 002 C040902M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0163

 DESCRIPTION: ANY 12TH GRDS TAKING COLLEGE COURSES ENG/LANG ARTS

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040903 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040903Y (01 ) 0 YES

 002 C040903M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0164

 DESCRIPTION: ANY 12TH GRDS TAKING COLLEGE COURSES IN ARTS?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040904 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040904Y (01 ) 0 YES

 002 C040904M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0165
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

620

 DESCRIPTION: ANY 12TH GRDS TAKING COLLEGE COURSES IN NONE ABOVE

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C040905 TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 C040905Y (01 ) 0 YES

 002 C040905M (M ) 1 MISSING

 CONDITIONING VARIABLE ID: SCHL0166

 DESCRIPTION: WHAT % 12TH GRDS HELD BACK AND REPEAT 12TH GRADE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C041001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C041001A (01 ) 00000 0%

 002 C041001B (02 ) 10000 1-2%

 003 C041001C (03 ) 01000 3-5%

 004 C041001D (04 ) 00100 6-10%

 005 C041001E (05 ) 00010 MORE THAN 10%

 006 C041001M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0167

 DESCRIPTION: LAST YR WHAT % OF 12TH GRDS GRADUATED?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C041101 TOTAL NUMBER OF SPECIFIED CONTRASTS: 6

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 5

 001 C041101A (01 ) 00000 99-100%

 002 C041101B (02 ) 10000 95-98%

 003 C041101C (03 ) 01000 90-94%

 004 C041101D (04 ) 00100 75-89%

 005 C041101E (05 ) 00010 LESS THAN 75%

 006 C041101M (M ) 00001 MISSING

 CONDITIONING VARIABLE ID: SCHL0168

 DESCRIPTION: WHAT % OF GRADUATING CLASS NOW IN 2-YR COLLEGE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036001 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C036001A (01 ) 000000 0-10%

 002 C036001B (02 ) 100000 11-25%

 003 C036001C (03 ) 010000 26-50%

 004 C036001D (04 ) 001000 51-75%

 005 C036001E (05 ) 000100 76-90%

 006 C036001F (06 ) 000010 91-100%

 007 C036001M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0169

 DESCRIPTION: WHAT % OF GRADUATING CLASS NOW IN 4-YR COLLEGE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036002 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C036002A (01 ) 000000 0-10%

 002 C036002B (02 ) 100000 11-25%

 003 C036002C (03 ) 010000 26-50%

 004 C036002D (04 ) 001000 51-75%

 005 C036002E (05 ) 000100 76-90%

 006 C036002F (06 ) 000010 91-100%

 007 C036002M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0170

 DESCRIPTION: WHAT % OF GRADUATING CLASS NOW IN VO-TEC SCHOOL?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036003 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C036003A (01 ) 000000 0-10%

 002 C036003B (02 ) 100000 11-25%

 003 C036003C (03 ) 010000 26-50%

 004 C036003D (04 ) 001000 51-75%

 005 C036003E (05 ) 000100 76-90%

 006 C036003F (06 ) 000010 91-100%

 007 C036003M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0171

 DESCRIPTION: WHAT % OF GRAD CLASS NOW IN EMPLOYER TRAINING?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036004 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

621

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C036004A (01 ) 000000 0-10%

 002 C036004B (02 ) 100000 11-25%

 003 C036004C (03 ) 010000 26-50%

 004 C036004D (04 ) 001000 51-75%

 005 C036004E (05 ) 000100 76-90%

 006 C036004F (06 ) 000010 91-100%

 007 C036004M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0172

 DESCRIPTION: % OF GRADUATING CLASS NOW IN MILITARY SERVICE?

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: C036005 TOTAL NUMBER OF SPECIFIED CONTRASTS: 7

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 6

 001 C036005A (01 ) 000000 0-10%

 002 C036005B (02 ) 100000 11-25%

 003 C036005C (03 ) 010000 26-50%

 004 C036005D (04 ) 001000 51-75%

 005 C036005E (05 ) 000100 76-90%

 006 C036005F (06 ) 000010 91-100%

 007 C036005M (M ) 000001 MISSING

 CONDITIONING VARIABLE ID: SCHL0173

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLUNCM (M ) 0 MISSING

 002 NATLUNCH (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0174

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLUNCL (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0175

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMREADM (M ) 0 MISSING

 002 REM READ (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0176

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMREADL (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0177

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMATHM (M ) 0 MISSING

 002 REM MATH (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0178

 DESCRIPTION: PERCENT OF STUDENTS WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N04, S04, N08, S08, N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHSC TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMATHL (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0179

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

622

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLUN4M (M ) 0 MISSING

 002 NATLUN4 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0180

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLUN4L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0181

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMREA4M (M ) 0 MISSING

 002 REMREAD4 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0182

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMREA4L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0183

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMAT4M (M ) 0 MISSING

 002 REMMATH4 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0184

 DESCRIPTION: PERCENT OF STUDENTS IN 4TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N04, S04

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMAT4L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0185

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLUN8M (M ) 0 MISSING

 002 NATLUN8 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0186

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLUN8L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0187

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMREA8M (M ) 0 MISSING

 002 REMREAD8 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0188

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1
Table C-5 (continued)

Conditioning Variables Specific to the 1996 Science Assessment

623

 001 REMREA8L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0189

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMAT8M (M ) 0 MISSING

 002 REMMATH8 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0190

 DESCRIPTION: PERCENT OF STUDENTS IN 8TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N08, S08

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMAT8L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0191

 DESCRIPTION: PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLU12M (M ) 0 MISSING

 002 NATLUN12 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0192

 DESCRIPTION: PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN THE NATIONAL SCHOOL LUNCH PROGRAM

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: NTLUNGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 NATLU12L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0193

 DESCRIPTION: PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMRD12M (M ) 0 MISSING

 002 REMRD12 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0194

 DESCRIPTION: PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL READING

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMRDGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMRD12L (0-100,M=0 ) 0.0 + 1.0*X LINEAR

 CONDITIONING VARIABLE ID: SCHL0195

 DESCRIPTION: PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 2

 TYPE OF CONTRAST: CLASS NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMH12M (M ) 0 MISSING

 002 REMMH12 (@ ) 1 PERCENT

 CONDITIONING VARIABLE ID: SCHL0196

 DESCRIPTION: PERCENT OF STUDENTS IN 12TH WHO PARTICIPATED IN REMEDIAL MATH

 GRADES/ASSESSMENTS: N12

 CONDITIONING VAR LABEL:

 NAEP ID: REMMHGR TOTAL NUMBER OF SPECIFIED CONTRASTS: 1

 TYPE OF CONTRAST: LINEAR NUMBER OF INDEPENDENT CONTRASTS: 1

 001 REMMH12L (0-100,M=0 ) 0.0 + 1.0*X LINEAR
Table C-6

Conditioning Variables for the Long-Term Trend Reading Assessment in 1996

624

Conditioning

Variable

Age

Classes

Variable

Name(s) Variable Coding

Contrast

Coding

Overall All  1

Gender All DSEX Male

Female

0

1

Region All REGION Northeast

Southeast

Central

West

000

100

010

001

Parental Education All PARED Less than high school

High school graduate

Post-high school

College graduate

Missing and I dont know

0000

1000

0100

0010

0001

Items in the Home All B000901

B000902

B000903

B000904

B000905

B000906

None of the six items

One of the six items

Two of the six items

Three of the six items

Four of the six items

Five of the six items

Six of the six items

Missing

00

10

20

30

40

50

60

01

Television

Watching

All B001801 None

One hour or less

Two hours

Three hours

Four hours

Five hours

Six or more hours

Missing

00

10

20

30

40

50

60

01

Homework All B001701 Don’t have any

Don’t do any

Less than 1 hour

1-2 hours

More than 2 hours

Missing

00

00

10

20

30

01

Language Spoken

at Home

All B000401 English

Spanish

Other

Missing

00

10

10

01
Table C-6 (continued)

Conditioning Variables for the Long-Term Trend Reading Assessment in 1996

625

Conditioning

Variable

Age

Classes

Variable

Name(s) Variable Coding

Contrast

Coding

Pages Read All B001101 More than 20

16-20

11-15

6-10

5 or fewer

Missing

10

10

10

10

00

01

Percent in School

Lunch Program

All C032001 None

1-5%

6-10%

11-25%

26-50%

51-75%

76-90%

over 90%

Missing

00000000

10000000

01000000

00100000

00010000

00001000

00000100

00000010

00000001

Percent White All PCTWHTQ 0-49%

50-79%

80-100%

Missing

100

010

001

000

Derived

Race/Ethnicity

All DRACE White

Black

Hispanic

Asian American

American Indian

Unclassified

Missing

000

100

010

001

000

000

000

Age by Grade All MODGRAG < age, = grade

= age, < grade

= age, = grade

= age, > grade

> age, = grade

0000

1000

0100

0010

0001

School Type All SCHTYPE Public

Private, Catholic, Bureau of

Indian Affairs, Department of

Defense

1

0

Description of

Community

All DOC Big City

Fringe

Medium City

Small Place

Missing

0000

1000

0100

0010

0001
Table C-6 (continued)

Conditioning Variables for the Long-Term Trend Reading Assessment in 1996

626

Conditioning

Variable

Age

Classes

Variable

Name(s) Variable Coding

Contrast

Coding

Type of Location All TOL8 Big City

Medium City

Fringe of Big City

Fringe of Medium City

Large Town

Small Place

Rural - MSA

Rural - Non MSA

Missing

00000000

10000000

01000000

00100000

00010000

00001000

00000100

00000010

00000001

Courses Taken 9, 13 B001001

B001002

B001003

B001004

B001005

B001006

B001007

None

1

2

3

4

5

6

7

Missing

00

10

20

30

40

50

60

70

01
Table C-7

Conditioning Variables for the Long-Term Trend Mathematics Assessment in 1996

627

Conditioning

Variable

Age

Classes

Variable

Name(s) Variable Coding

Contrast

Coding

Overall All  1

Gender All GENDER Male

Female

0

1

Observed

Race/Ethnicity

All ETHNIC White

Black

Hispanic

Asian American

American Indian

Other

Missing

000

100

010

001

000

000

000

Size and Type of

Community

All STOC 1, 4-7 all except 2 and 3

2 Low Metro

3 High Metro

01

00

10

Region All REGION Northeast

Southeast

Central

West

000

100

010

001

Parents’

Education

All PARED Less than high school

High school graduate

Post-high school

College graduate

Missing and I Don’t Know

0000

1000

0100

0010

0001

Modal Grade All MODGRD < modal grade

= modal grade, missing

> modal grade

10

00

01

Items in the Home

(of newspaper, >

25 books,

encyclopedia,

magazines)

All HOMEEN2 0 to 2 items

3 items

4 items

00

10

01
Table C-7 (continued)

Conditioning Variables for the Long-Term Trend Mathematics Assessment in 1996

628

Conditioning

Variable

Age

Classes

Variable

Name(s) Variable Coding

Contrast

Coding

Observed

Race/Ethnicity by

Gender (White

includes

American Indian

and Other)

All RACE x

GENDER

White, male

Black, male

Hispanic, male

Asian American, male

White, female

Black, female

Hispanic, female

Asian American, female

000

000

000

000

000

100

010

001

Observed

Race/Ethnicity by

Parents’

Education

(White includes

American Indian

and

Other)coded

differently for

each age class

9 RACE x

PARED

White, < HS

White, HS graduate

White, post-HS

White, college grad.

White, missing

Black, < HS

Black, HS grad & post-HS

Black, college grad.

Black, missing

Hispanic, < HS

Hispanic, HS grad & post-HS

Hispanic, coll. grad.

Hispanic, missing

Asian Amer., < HS

Asian Amer., HS grad & post-HS

Asian Amer., coll. grad.

Asian Amer., missing

0000 0000 0000

0000 0000 0000

0000 0000 0000

0000 0000 0000

0000 0000 0000

0000 0000 0000

1000 0000 0000

0010 0000 0000

0001 0000 0000

0000 0000 0000

0000 1000 0000

0000 0010 0000

0000 0001 0000

0000 0000 0000

0000 0000 1000

0000 0000 0010

0000 0000 0001
Table C-7 (continued)

Conditioning Variables for the Long-Term Trend Mathematics Assessment in 1996

629

Conditioning

Variable

Age

Classes

Variable

Name(s) Variable Coding

Contrast

Coding

Observed

Race/Ethnicity by

Parents’

Education

(White includes

American Indian

and

Other)coded

differently for

each age class

13 RACE x

PARED

White, < HS

White, HS graduate

White, post-HS

White, college grad.

White, missing

Black, < HS

Black, HS graduate

Black, post-HS

Black, college grad.

Black, missing

Hispanic, < HS

Hispanic, HS grad.

Hispanic, post-HS

Hispanic, coll. grad.

Hispanic, missing

Asian Amer., < HS

Asian Amer., HS grad.

Asian Amer., post-HS

Asian Amer., coll. grad.

Asian Amer., missing

0000 0000 0000

0000 0000 0000

0000 0000 0000

0000 0000 0000

0000 0000 0000

0000 0000 0000

1000 0000 0000

0100 0000 0000

0010 0000 0000

0001 0000 0000

0000 0000 0000

0000 1000 0000

0000 0100 0000

0000 0010 0000

0000 0001 0000

0000 0000 0000

0000 0000 1000

0000 0000 0100

0000 0000 0010

0000 0000 0001

Observed

Race/Ethnicity by

Parents’

Education

(White includes

American Indian

and

Other)coded

differently for

each age class

17 RACE x

PARED

White, < HS

White, HS graduate

White, post-HS

White, college grad.

White, missing

Black, < HS

Black, HS graduate

Black, post-HS

Black, college grad.

Black, missing

Hispanic, < HS

Hispanic, HS grad.

Hispanic, post-HS

Hispanic, coll. grad.

Hispanic, missing

Asian Amer., < HS

Asian Amer., HS grad.

Asian Amer., post-HS, coll. grad.

Asian Amer., missing

0000 0000 0000

0000 0000 0000

0000 0000 0000

0000 0000 0000

0000 0000 0000

0000 0000 0000

1000 0000 0000

0100 0000 0000

0010 0000 0000

0001 0000 0000

0000 0000 0000

0000 1000 0000

0000 0100 0000

0000 0010 0000

0000 0001 0000

0000 0000 0000

0000 0000 1000

0000 0000 0100

0000 0000 0001

Language in the

Home

All LANGHOM Never

Sometimes

Always

00

10

01

Observed Race by All RACE x White, often 00 00 00
Table C-7 (continued)

Conditioning Variables for the Long-Term Trend Mathematics Assessment in 1996

630

Conditioning

Variable

Age

Classes

Variable

Name(s) Variable Coding

Contrast

Coding

Language at

Home

LANGHOM White, sometimes

White, never

Black, often and sometimes

Black, often

Black, sometimes

Black, never

Hispanic, often and sometimes

Hispanic, often

Hispanic, sometimes

Hispanic, never

Asian American, often and sometimes

Asian American, often

Asian American, sometimes

Asian American, never

00 00 00

00 00 00

10 00 00

10 00 00

01 00 00

00 00 00

00 10 00

00 10 00

00 01 00

00 00 00

00 00 10

00 00 10

00 00 01

00 00 00

Derived

Race/Ethnicity

All DRACE White

Black

Hispanic

Asian American

Other

Missing

000

100

010

001

000

000

Homework 13, 17 HW None assigned

Didn’t do

1/2 hour or less

1 hour

2 hours

More than 2 hours

Missing

100

010

012

013

014

000

000

Highest Level of

Mathematics

Class

17 NMATH Pre-Algebra

Algebra

Geometry

Algebra 2

Calculus

Something else

10000

01000

00100

00010

00001

00000

High School

Program

17 HS_PGM General

College Preparatory

Vocational/Technical

Missing

00

10

01

00
Table C-7 (continued)

Conditioning Variables for the Long-Term Trend Mathematics Assessment in 1996

631

Conditioning

Variable

Age

Classes

Variable

Name(s) Variable Coding

Contrast

Coding

Type of Location All TOL8 Big City

Medium City

Fringe of Big City

Fringe of Medium City

Large Town

Small Place

Rural - MSA

Rural - non MSA

Missing

00000000

10000000

01000000

00100000

00010000

00001000

00000100

00000010

00000001
Table C-8

Conditioning Variables for the Long-Term Trend Science Assessment in 1996

632

Conditioning

Variable

Age

Classes

Variable

Name(s)

Variable

Coding

Contrast

Coding

Overall All  1

Gender All DSEX Male

Female

0

1

Observed Race All RACE White

Black

Hispanic

Asian American

American Indian, Pacific Islander

Other, blank, missing

000

100

010

001

000

000

Size and Type

of Community

(92 only)

All STOC Low Metro

High Metro

All others, missing

10

01

00

Type of

Location (94

and 96 only)

All TOL8 Big City

Medium City

Fringe of Big City

Fringe of Medium City

Large Town

Small Place

Rural - MSA

Rural - Non MSA

Missing

00000000

10000000

01000000

00100000

00010000

00001000

00000100

00000010

00000001

Region All REGION Northeast

Southeast

Central

West

Missing

000

100

010

001

000

Parents’

Education

All PARED Less than high school

High school graduate

Post-high school

College graduate

Missing and I dont know

0000

1000

0100

0010

0001

Modal Grade All MODGRD < modal grade

= modal grade

> modal grade

Missing

10

00

01

00
Table C-8 (continued)

Conditioning Variables for the Long-Term Trend Science Assessment in 1996

633

Conditioning

Variable

Age

Classes

Variable

Name(s)

Variable

Coding

Contrast

Coding

Observed Race

by Gender

All RACE x

DSEX

White, Male

Black, Male

Hispanic, Male

Asian American, Male

White, Female

Black, Female

Hispanic, Female

Asian American, Female

Other combinations, missing

000

000

000

000

000

100

010

001

000

Observed Race

by Parents’

Education

All RACE x

PARED

White, < High school

White, = High school

White, > High school

White, Graduated college

White, missing or unknown

Black, < High school

Black, = High school

Black, > High school

Black, Graduated college

Black, missing or unknown

Hispanic, < High school

Hispanic, = High school

Hispanic, > High school

Hispanic, Graduated college

Hispanic, missing or unknown

Asian American, < High school

Asian American, = High school

Asian American, > High school

Asian American, Graduated college

Asian American, missing or

unknown

000000000000

000000000000

000000000000

000000000000

000000000000

000000000000

100000000000

010000000000

001000000000

000100000000

000000000000

000010000000

000001000000

000000100000

000000010000

000000000000

000000001000

000000000100

000000000010

000000000001

School Type All SCHTYPE Public

Private, Catholic, BIA, DoDEA

Missing

0

1

0

Items in the

Home

All HOMEEN2 0-2 items

3 items

4 items

Missing

00

10

01

00
Table C-8 (continued)

Conditioning Variables for the Long-Term Trend Science Assessment in 1996

634

Conditioning

Variable

Age

Classes

Variable

Name(s)

Variable

Coding

Contrast

Coding

Derived Race All DRACE White

Black

Hispanic

Asian American

American Indian, Pacific Islander

Other, missing

000

100

010

001

000

000

Language

Spoken in the

Home (Other

than English)

All LANGHO

M

Never

Sometimes

Always

Missing

00

10

01

00

Observed Race

by Language in

the Home

All RACE x

LANGHO

M

White, Always

White, Sometimes

White, Never

Black, Always

Black, Sometimes

Black, Never

Hispanic, Always

Hispanic, Sometimes

Hispanic, Never

Asian American, Always

Asian American, Sometimes

Asian American, Never

One or both missing

000000

000000

000000

100000

010000

000000

001000

000100

000000

000010

000001

000000

000000

Homework 13, 17 B001701 None assigned

Didn’t do

1/2 hour or less

One hour

Two hours

More than two hours

Missing

100

010

012

013

014

000

000

Highest Level

of Science

Class

17 B005308

B005309

B005310

B005311

General Science

Biology

Chemistry

Physics

Nothing, Something Else

Missing

1000

0100

0010

0001

0000

0000

High School

Program

17 B005001 General

College Preparatory

Vocational, technical

Missing

00

10

01

00
Table C-9

Conditioning Variables for the Long-Term Trend Writing Assessment in 1996

635

Conditioning

Variable Variable

Number of

Contrasts Contrasts

Contrasts

Codes

Intercept 1 Overall 1

Gender DSEX 1 Male

Female

0

1

Race/Ethnicity Ethnic(84)

DRACE (88,90,92,94,96)

2 Black

Hispanic

White+others

00

01

10

Type of

Community/

Location

STOC

(84,88,90,92)

TOL8(94,96)

2

7

STOC:

 Low Metro

 Everyone else

 Hi Metro

TOL8:

 Large City & miss

 Medium City

 Urb. Fring of LC

 Urb. Fring of MC

 Big Town

 Small Town

 Rural-MSA

 Rural-non-MSA

00

01

10

0000000

1000000

0100000

0010000

0001000

0000100

0000010

0000001

Region REGION 3 Northeast

Southeast

Central

West

000

100

010

001

Parents Education PARED 4 Less than H.S.

High School

Greater HS

College

I dont know

0000

1000

0100

0010

0001

Items in the home B000901, B000902,

B000903, B000904

B000905

2 Articles in home:

0-3 or missing

4

5

00

10

01

Modal age DAGE 2 Modal Age

< Modal age

> Modal age

00

10

01
Table C-9 (continued)

Conditioning Variables for the Long-Term Trend Writing Assessment in 1996

636

Conditioning

Variable Variable

Number of

Contrasts Contrasts

Contrasts

Codes

Homework B001701 5 None assigned

None done

< 1 hr

1-2 hrs

2+ hrs

Missing HW

00000

10000

01000

00100

00010

00001

School type SCHTYPE 2 Public School

Non-Public

Missing

0

1

1

TV watching,

Linear

B001801 1 None

1 hour

2 hours

3 hours

Missing

4 hours

5 hours

6 hours

0

1

2

3

3

4

5

6

TV watching,

Quadratic

B001801 2 None

1 hour

2 hours

3 hours

Missing

4 hours

5 hours

6 hours

00

01

04

09

09

16

25

36

Mother Work

Outside the Home

B000801 1 Mother does not work Missing

Mother works

0

0

1

Language Minority

(self speaking at

home)

B000301 2 Not language minority

Language minority

Missing

00

10

01

Percent in Lunch

Program

C001301(84)

PCLUNCH (88,90)

C0320101(92,94,96)

7 None in lunch program

1-5%

6-10%

11-25%

26-50%

51-75%

76-90%

over 90%

0000000

1000000

0100000

0010000

0001000

0000100

0000010

0000001

Minority School PCTWHT 2 Minority (0-49.9%) 00
Table C-9 (continued)

Conditioning Variables for the Long-Term Trend Writing Assessment in 1996

637

Conditioning

Variable Variable

Number of

Contrasts Contrasts

Contrasts

Codes

PCTWHTQ Intergrated (50- 79.9%)

White or Missing (80-100%)

10

01

Grades in School B001901 1 Var Grade

(Missing included at 2.0)

0.0-4.0

Pages Read for HW B001101 3 No pages read

11+ pages read

6-10 pages read

5 or less pages read

000

100

010

001

Number of reports B001201-7 and B001208 1 Number of reports, essays etc. 0-7
Table C-10

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

638

 FEMALE 0.92422

 BLACK 0.93928

 HISPANIC 0.96987

 ASIAN 0.94278

 MEXICAN 0.95600

 PUER RIC 0.92864

 CUBN,OTH 0.96742

 HISP-? 0.74847

 NON MSA 0.92046

 MID CTY5 0.94280

 FR/BTWN5 0.94911

 SML TWN5 0.95067

 RURAL5 0.95097

 URBAN FR 0.90811

 MED CITY 0.89044

 SM PLACE 0.93298

 HS GRAD 0.94671

 POST HS 0.92986

 COL GRAD 0.92976

 PARED-? 0.94023

 S EAST 0.89683

 CENTRAL 0.89537

 WEST 0.90228

 PRIVATE 0.93637

 CATHOLIC 0.91800

 IEP-NO 0.87421

 LEP-NO 0.79025

 CHAP1-N 0.77000

 RED PRIC 0.93333

 FREE 0.73534

 INFO N/A 0.85427

 SCH/REF 0.84591

 HL-SOME 0.85950

 HL-ALWAY 0.85486

 HL-? 0.80373

 B008901N 0.95242

 B008901M 0.70149

 TVLIN-0 0.98291

 TV-QUAD 0.98263

 HW-NO 0.96512

 HW-YES 0.96865

 HWLIN-0 0.97548

 HWQUAD-0 0.96240

 HITEM=3 0.86215

 HITEM=4 0.82846

 MOMHOM-N 0.81244

 MOMHOM-? 0.88256

 DADHOM-N 0.86602

 DADHOM-? 0.92245

 MISS-2< 0.93929

 USA >5 0.93754

 USA 3-5 0.92960

 USA <3 0.92448

 USA-? 0.83469

 STGRD1-2 0.90017

 STGRD3> 0.88466

 SNRM-LIN 0.88196

 SCHCHG-1 0.91473

 SCHCHG-2 0.91001

 SCHCHG-3 0.81984

 DIS@HOM2 0.89416

 DIS@HOM3 0.93408

 DIS@HOM4 0.79632

 PGS>5 0.82911

 PGS>10 0.83882

 G/R 22 0.91016

 G/R 23 0.90786

 G/R 24 0.95898

 G/T 22 0.71798

 G/T 23 0.73468

 G/T 24 0.77888

 G/T 25 0.73060

 G/P 22 0.92747

 G/P 23 0.89300

 G/P 24 0.85988

 G/P 25 0.84783

 G/S 22 0.93396

 G/S 23 0.89897

 R/T 24 0.89235

 R/T 25 0.90331

 R/T 31 0.90034

 R/T 32 0.89554

 R/T 33 0.88687

 R/T 34 0.88052

 R/T 35 0.85371

 R/T 41 0.88419

 R/T 42 0.92435

 R/T 43 0.93164

 R/T 44 0.93998

 R/T 45 0.94866

 R/P 24 0.89509

 R/P 25 0.87983

 R/P 31 0.90064

 R/P 32 0.91138

 R/P 33 0.89862

 R/P 34 0.90463

 R/P 35 0.89655

 R/P 41 0.89841

 R/P 42 0.96949

 R/P 43 0.96354

 R/P 44 0.96174

 R/P 45 0.96426

 R/S 31 0.97978

 R/S 32 0.96311

 R/S 33 0.97170

 R/S 41 0.95920

 R/S 42 0.96764

 R/S 43 0.96940

 T/P 25 0.81392

 T/P 31 0.82723

 T/P 32 0.79005

 T/P 33 0.79097

 T/P 34 0.78458

 T/P 35 0.80476

 T/P 41 0.76880

 T/P 42 0.74497

 T/P 43 0.77272

 T/P 44 0.80053

 T/P 45 0.83715

 T/P 51 0.83797

 T/P 52 0.77254

 T/P 53 0.81563

 T/P 54 0.79556

 T/P 55 0.79026

 T/S 32 0.95336

 T/S 33 0.93082

 T/S 41 0.93927

 T/S 42 0.93128

 T/S 43 0.96264

 T/S 51 0.94403

 T/S 52 0.94797

 P/S 32 0.96888

 P/S 33 0.94519

 P/S 41 0.95888

 P/S 42 0.93031

 P/S 43 0.95356

 P/S 51 0.92917

 P/S 52 0.95848

 P/S 53 0.93447

 SAMP S2 0.87473

 /R 23 0.90811

 /R 24 0.90508

 /R 31 0.96289

 B009301B 0.95836

 B009301C 0.96313

 B009301D 0.96067

 B009301E 0.94948

 B009301M 0.92458

 B009401B 0.94274

 B009401C 0.94860

 B009401D 0.90675

 B009401M 0.91517

 B009501B 0.95808

 B009501C 0.84231

 B009501M 0.89573

 B009502B 0.95167

 B009502C 0.86065

 B009502M 0.87799

 M812701B 0.87446

 M812701C 0.92214

 M812701D 0.84959

 M812701M 0.75474

 M812702B 0.87726

 M812702C 0.90130

 M812701D 0.87129
Table C-10 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

639

M812701M

0.77048

 M812703B 0.94466

 M812703C 0.95361

 M812703D 0.93975

 M812703M 0.71661

 M812704B 0.93276

 M812704C 0.93105

 M812704D 0.92633

 M812704M 0.72479

 M812705B 0.92230

 M812705C 0.92740

 M812705D 0.90764

 M812705M 0.76198

 M812706B 0.92303

 M812706C 0.93229

 M812706D 0.93410

 M812706M 0.72774

 M812707B 0.91670

 M812707C 0.93858

 M812707D 0.88953

 M812707M 0.89200

 M812708B 0.88593

 M812708C 0.89736

 M812708D 0.87920

 M812708M 0.68806

 M812709B 0.91900

 M812709C 0.92927

 M812709D 0.90477

 M812709M 0.72509

 M812710B 0.92641

 M812710C 0.94339

 M812710D 0.90871

 M812710M 0.84702

 M812711B 0.93730

 M812711C 0.95239

 M812711D 0.92672

 M812711M 0.84435

 M811201N 0.85363

 M811201M 0.84968

 M812001B 0.91257

 M812001C 0.93958

 M812001D 0.89292

 M812001M 0.78811

 M812002B 0.93992

 M812001C 0.94042

 M812001D 0.89765

 M812001M 0.81152

 M812003B 0.93589

 M812003C 0.95728

 M812003D 0.93863

 M812003M 0.87599

 M812101B 0.95122

 M812101C 0.98021

 M812101D 0.98464

 M812101E 0.98150

 M812101F 0.97657

 M812101G 0.95779

 M812101M 0.77945

 M811401N 0.83560

 M811401M 0.81227

 M811101B 0.90471

 M811101C 0.85283

 M811101M 0.84889

 M811103B 0.90634

 M811103C 0.85451

 M811103M 0.84217

 M811106B 0.91234

 M811106C 0.90506

 M811106M 0.75855

 M811109B 0.87823

 M811109C 0.85178

 M811109M 0.85089

 M811107B 0.83958

 M811107C 0.81346

 M811107M 0.85408

 M811105B 0.94083

 M811105C 0.92457

 M811105M 0.85277

 M811108B 0.90576

 M811108C 0.88138

 M811108M 0.88722

 M811110B 0.93917

 M811110C 0.94704

 M811110M 0.87218

 MM00101B 0.84889

 MM00101C 0.86647

 MM00101D 0.90934

 MM00101M 0.88111

 MM00201B 0.90505

 MM00201C 0.90559

 MM00201D 0.90476

 MM00201M 0.91399

 MM00301B 0.88918

 MM00301C 0.85485

 MM00301D 0.90776

 MM00301M 0.84424

 MM00401B 0.92763

 MM00401C 0.94108

 MM00401D 0.94031

 MM00401M 0.90467

 MM00501B 0.88779

 MM00501C 0.89769

 MM00501D 0.91306

 MM00501M 0.80670

 C030901B 0.87546

 C030901C 0.89822

 C030901M 0.97693

 C037101B 0.89290

 C037101C 0.90595

 C037101N 0.94737

 C037101M 0.93370

 C031212B 0.89289

 C031212M 0.97017

 C031205B 0.90185

 C031205C 0.89277

 C031205M 0.93941

 C031213M 0.95475

 C031214B 0.90767

 C031214C 0.92306

 C031214D 0.90531

 C031214E 0.90683

 C031214M 0.92600

 C031603N 0.87846

 C031603M 0.92434

 C031607N 0.89303

 C031607M 0.94545

 C031601N 0.88729

 C031601M 0.94662

 C031610N 0.90678

 C031610M 0.95321

 C031606N 0.88964

 C031606M 0.94862

 C035701N 0.88805

 C035701M 0.93004

 C035702N 0.89130

 C035702M 0.92471

 C035703N 0.88402

 C035703M 0.92740

 C037201M 0.94474

 C037202M 0.92251

 C037207M 0.91288

 C037204M 0.90320

 C037205M 0.90800

 C037206M 0.93973

 C037301M 0.97484

 C037302M 0.96190

 C037303M 0.96240

 C037304M 0.89266

 C037305M 0.92732

 C037401M 0.89938

 C037402M 0.89873

 C037403M 0.88274

 C037404M 0.88499

 C037405M 0.88798

 C037406M 0.87276

 C037501M 0.87432

 C037502M 0.88889

 C037503M 0.88022

 C037504M 0.89411

 C037505M 0.91016

 C037601M 0.91900

 C037602M 0.90388

 C037603M 0.93754

 C037604M 0.89697

 C037605M 0.92835

 C036601B 0.90340

 C036601C 0.94582
Table C-10 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

640

 C036601D 0.96106

 C036601M 0.93323

 C032207B 0.90906

 C032207N 0.88328

 C032207M 0.92796

 C032209B 0.88240

 C032209N 0.91232

 C032209M 0.97717

 C032210B 0.89290

 C032210N 0.89959

 C032210M 0.94318

 C032211B 0.89806

 C032211N 0.87633

 C032211M 0.97839

 C037701B 0.88757

 C037701C 0.89656

 C037701D 0.90758

 C037701M 0.98981

 C037702B 0.90859

 C037702C 0.93760

 C037702D 0.94255

 C037703B 0.91776

 C037703C 0.92911

 C037703D 0.95420

 C037704B 0.89812

 C037704C 0.88562

 C037704D 0.87868

 C037704M 0.94449

 C037705B 0.90509

 C037705C 0.89905

 C037705D 0.89193

 C037705M 0.98275

 C032402B 0.91690

 C032402C 0.93603

 C032402D 0.93000

 C032402M 0.95263

 C032401B 0.92205

 C032401C 0.94109

 C032401D 0.94183

 C032401M 0.96538

 C032404B 0.91540

 C032404C 0.94872

 C032404D 0.93668

 C032404M 0.97495

 C032406B 0.89929

 C032406C 0.89908

 C032407B 0.93072

 C032407C 0.90227

 C032408B 0.92336

 C032408C 0.93971

 C032408D 0.94243

 C032409B 0.90573

 C032409C 0.94692

 C032409D 0.93129

 C032410B 0.90989

 C032410C 0.91409

 C032411C 0.92916

 C032411D 0.95921

 C032412C 0.90880

 C032413B 0.87343

 C032413C 0.89923

 C032414B 0.91543

 C032414C 0.95362

 C032414D 0.93445

 C032415B 0.91543

 C032415C 0.88831

 C032502B 0.90525

 C032502C 0.91341

 C032502D 0.91998

 C032503B 0.89030

 C032503C 0.90449

 C032505B 0.89514

 C032505C 0.91082

 C032505D 0.90382

 C032506B 0.88183

 C032506C 0.89775

 C032506D 0.94373

 C033601B 0.88746

 C033601C 0.91444

 C033601M 0.96982

 C036501B 0.88549

 C036501C 0.88945

 C037801B 0.91251

 C037801C 0.91425

 C037801D 0.91956

 C037801E 0.89582

 C037801F 0.88578

 C037801G 0.89624

 C037801H 0.87801

 C037801M 0.93805

 C037901B 0.90039

 C037901C 0.88338

 C037901E 0.89721

 C038001B 0.88267

 C038001C 0.89724

 C038001D 0.90367

 C038001F 0.91884

 C038001G 0.89475

 C038301N 0.87255

 C038801N 0.90074

 C034101M 0.90931

 C034102M 0.86390

 C034103M 0.90034

 C034104M 0.91039

 C034105M 0.88246

 C034106M 0.87927

 C034107M 0.89607

 C034108M 0.88519

 C034109M 0.90284

 T055901B 0.83449

 T055901M 0.95592

 T056001B 0.85919

 T056001C 0.85497

 T056001D 0.88643

 T056001E 0.89026

 T056001F 0.90057

 T056001M 0.93437

 T040301B 0.90023

 T040301C 0.91623

 T040301D 0.92629

 T040301E 0.93485

 T040301M 0.97112

 T056101B 0.91949

 T056101C 0.92407

 T056101D 0.93806

 T056101E 0.94465

 T056101M 0.97687

 T056102B 0.88880

 T056102C 0.89683

 T056102D 0.91719

 T056102E 0.92978

 T056102M 0.90207

 T056201B 0.87996

 T056201C 0.87424

 T056201D 0.90152

 T056201E 0.89091

 T056201F 0.86307

 T056201M 0.89431

 T040501N 0.86646

 T040501C 0.87847

 T040501M 0.88168

 T040506N 0.91930

 T040506C 0.93518

 T040506M 0.92416

 T040504N 0.92875

 T040504C 0.91604

 T040504M 0.93770

 T040507N 0.92414

 T040507C 0.93028

 T040507M 0.93216

 T040508N 0.92610

 T040508C 0.92511

 T040508M 0.95049

 T040505N 0.87425

 T040505C 0.85036

 T040505M 0.88310

 T056301B 0.86863

 T056301C 0.94542

 T056301D 0.94757

 T056301E 0.87991

 T056301F 0.89665

 T056301G 0.92785

 T040701M 0.85850

 T040706M 0.87270

 T040707M 0.88064

 T040703M 0.85761

 T040704M 0.89422

 T040710M 0.86605
Table C-10 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

641

 T040711M 0.88088

 T040712M 0.88793

 T040713M 0.89131

 T040708M 0.86839

 T040709M 0.88842

 T040705M 0.86609

 T040801M 0.86782

 T040807M 0.88407

 T040808M 0.87263

 T040803M 0.85837

 T040804M 0.87242

 T040814M 0.86338

 T040815M 0.89963

 T040816M 0.90897

 T040817M 0.90343

 T040809M 0.85225

 T040810M 0.88623

 T040811M 0.86569

 T040812M 0.86161

 T040813M 0.87130

 T040805M 0.87488

 T040806M 0.87139

 T056401M 0.87217

 T056402M 0.88232

 T056403M 0.86561

 T056404M 0.86030

 T056405M 0.85499

 T056413M 0.84907

 T056414M 0.87428

 T056415M 0.88588

 T056416M 0.88361

 T056406M 0.84066

 T056407M 0.87904

 T056408M 0.86924

 T056409M 0.85604

 T056410M 0.87048

 T056411M 0.88352

 T056412M 0.88974

 T056501B 0.91310

 T056501C 0.91126

 T056501D 0.89844

 T056501E 0.87955

 T056501M 0.94460

 T058101B 0.87513

 T058101C 0.88453

 T058101D 0.88429

 T058101E 0.85574

 T058101M 0.94762

 T056601B 0.88151

 T056601C 0.87588

 T056601D 0.89212

 T056601E 0.87477

 T056601M 0.95900

 T058201B 0.88402

 T058201C 0.87801

 T058201D 0.86875

 T058201E 0.85271

 T058201M 0.93699

 T056701M 0.84442

 T056702M 0.86785

 T056703M 0.84733

 T056704M 0.84667

 T056705M 0.86584

 T056706M 0.88206

 T056707M 0.84208

 T056708M 0.86398

 T056709M 0.86342

 T056710M 0.85226

 T056711M 0.84552

 T056712M 0.86144

 T056713M 0.85136

 T041201B 0.91372

 T041201C 0.91852

 T041201D 0.87742

 T041201M 0.97165

 T041302N 0.92005

 T041302M 0.90450

 T041303N 0.91591

 T041303M 0.89679

 T056801B 0.88290

 T056801C 0.91737

 T056801D 0.93092

 T056801E 0.90584

 T056801F 0.91298

 T056801M 0.92515

 T056901M 0.87090

 T0569A1M 0.88752

 T0569B1M 0.84911

 T0569C1M 0.87488

 T056902M 0.90372

 T0569A2M 0.90806

 T0569B2M 0.86996

 T0569C2M 0.88831

 T056903M 0.88404

 T0569A3M 0.90967

 T0569B3M 0.90105

 T0569C3M 0.87855

 T056904M 0.88116

 T0569A4M 0.89535

 T0569B4M 0.88101

 T0569C4M 0.89052

 T056905M 0.88390

 T0569A5M 0.89367

 T0569B5M 0.86742

 T0569C5M 0.88765

 T056906M 0.91177

 T0569A6M 0.89529

 T0569B6M 0.86272

 T0569C6M 0.91056

 T056907M 0.90241

 T0569A7M 0.89949

 T0569B7M 0.88021

 T0569C7M 0.91690

 T056908M 0.88151

 T0569A8M 0.87210

 T0569B8M 0.87636

 T0569C8M 0.90877

 T057001N 0.86053

 T057001M 0.90995

 T057002N 0.86184

 T057002M 0.93314

 T057003N 0.84736

 T057003M 0.93251

 T057004N 0.84355

 T057004M 0.92756

 T057005N 0.83569

 T057005M 0.93750

 T057006N 0.85345

 T057006M 0.92040

 T057007N 0.86267

 T057007M 0.91579

 T057101B 0.88236

 T057101C 0.92509

 T057101D 0.93133

 T057101M 0.92361

 T057201M 0.87624

 T057211M 0.85779

 T057221M 0.87096

 T057231M 0.87670

 T057241M 0.91534

 T057301B 0.87060

 T057301C 0.87623

 T057301M 0.95025

 T057302B 0.87815

 T057302M 0.95258

 T057303B 0.87830

 T057303C 0.90424

 T057303M 0.95719

 T057304B 0.84929

 T057304C 0.86139

 T057304M 0.92002

 T057401B 0.90239

 T057401C 0.91147

 T057401D 0.91456

 T057401M 0.94497

 T057402B 0.87718

 T057402C 0.86257

 T057402D 0.86972

 T057402M 0.87157

 T057403B 0.89626

 T057403C 0.90086

 T057403D 0.87718

 T057403M 0.95064

 T057404B 0.89499

 T057404C 0.90763

 T057404D 0.89079

 T057404M 0.96581

 T057405B 0.90667
Table C-10 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

642

 T057405C 0.88626

 T057405D 0.89650

 T057405M 0.96396

 T057501B 0.90647

 T057501C 0.90728

 T057501D 0.88542

 T057501E 0.90756

 T057501F 0.88592

 T057501M 0.88922

 T057601B 0.90068

 T057601C 0.87531

 T057601D 0.87089

 T057601E 0.88152

 T057601M 0.88177

 T044002N 0.96057

 T044002M 0.98118

 T057701B 0.88013

 T057701C 0.93030

 T057701D 0.90493

 T057701M 0.93232

 T057801B 0.89626

 T057801C 0.90313

 T057801D 0.88796

 T057801E 0.88363

 T057801M 0.92501

 T044201N 0.84915

 T044201M 0.95404

 T044301B 0.92323

 T044301C 0.92658

 T044301M 0.94428

 T057901B 0.89811

 T057901C 0.92830

 T057901D 0.92205

 T057901M 0.95806

 T044401B 0.93553

 T044401C 0.93908

 T044401D 0.85349

 T044401E 0.84742

 T044401F 0.86032

 T044401M 0.94802

 T044501B 0.86534

 T044501C 0.87699

 T044501D 0.88415

 T044501M 0.98882

 T044502B 0.88635

 T044502C 0.88693

 T044502D 0.87010

 T044502M 0.98776

 T044512B 0.89116

 T044512C 0.90369

 T044512D 0.86574

 T044513B 0.90891

 T044513C 0.91592

 T044513D 0.87999

 T044513M 0.98443

 T044514B 0.88894

 T044514C 0.93670

 T044514D 0.91131

 T044514M 0.94879

 T044505B 0.90978

 T044505C 0.93496

 T044505D 0.90303

 T044505M 0.96128

 T044515B 0.94914

 T044515C 0.95660

 T044515D 0.87350

 T044515M 0.97925

 T044507B 0.90786

 T044507C 0.90997

 T044507D 0.90189

 T044507M 0.94931

 T044516B 0.86404

 T044516C 0.87541

 T044516D 0.86403

 T044516M 0.97424

 T044508B 0.87910

 T044508C 0.94909

 T044508D 0.94682

 T044508M 0.97046

 T044509B 0.88251

 T044509C 0.87048

 T044509D 0.87755

 T044509M 0.98858

 T044510B 0.88589

 T044510C 0.89079

 T044510D 0.87292

 T044510M 0.97816

 T044506B 0.91074

 T044506C 0.91055

 T044506D 0.89806

 T044506M 0.98601

 T058001B 0.87914

 T058001C 0.94285

 T058001D 0.86668

 T058001M 0.98340

 T058002B 0.86930

 T058002C 0.90760

 T058002D 0.88591

 T058002M 0.98362

 T058003B 0.89322

 T058003C 0.90914

 T058003D 0.87573

 T058003M 0.96909

 T058004B 0.91433

 T058004C 0.91976

 T058004D 0.89295

 T058004M 0.94642

 T058005B 0.91308

 T058005C 0.92420

 T058005D 0.90207

 T058005M 0.96135

 T058006B 0.87168

 T058006C 0.87947

 T058007B 0.84998

 T058007C 0.89035

 T058008B 0.86926

 T058008C 0.87852

 T058008D 0.88313

 T058009B 0.87471

 T058009C 0.88656

 T058009D 0.86106

 T058009M 0.97804

 T045401N 0.84916

 T045401M 0.97164

 T044801N 0.85058

 T044801M 0.94684

 T045001N 0.85883

 T045001M 0.97507

 T044901N 0.87216

 T044901M 0.95233

 T045304B 0.89609

 T045304C 0.90295

 T045304M 0.99250

 T045305B 0.90586

 T045305C 0.91145

 T045305M 0.98072

 T045302B 0.90421

 T045302C 0.90917

 T045302D 0.88258

 T045303B 0.89142

 T045303C 0.87448

 T045303D 0.87792

 T045303M 0.98069

 T044000B 0.88559

 T044000C 0.90168

 T044000D 0.88654

 T044000E 0.86796

 T044000M 0.90723

 NATLUNCH 0.88867

 NATLUNCL 0.92919

 REM READ 0.93790

 REMREADL 0.93711

 REM MATH 0.94580

 REMMATHL 0.93332

 NATLUN4 0.91489

 NATLUN4L 0.92700

 REMREAD4 0.93241

 REMREA4L 0.91492

 REMMATH4 0.93880

 REMMAT4L 0.89187
Table C-11

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

643

 FEMALE 0.94701

 BLACK 0.94977

 HISPANIC 0.97892

 ASIAN 0.94296

 MEXICAN 0.95099

 PUER RIC 0.93906

 CUBN,OTH 0.95269

 HISP-? 0.93864

 NON MSA 0.94366

 MID CTY5 0.95565

 FR/BTWN5 0.96058

 SML TWN5 0.96035

 RURAL5 0.96816

 URBAN FR 0.92480

 MED CITY 0.89703

 SM PLACE 0.94156

 HS GRAD 0.96304

 POST HS 0.95738

 COL GRAD 0.95319

 PARED-? 0.96129

 S EAST 0.92396

 CENTRAL 0.91036

 WEST 0.91239

 PRIVATE 0.93569

 CATHOLIC 0.93388

 IEP-NO 0.79397

 LEP-NO 0.77834

 CHAP1-N 0.78653

 RED PRIC 0.89841

 FREE 0.75139

 INFO N/A 0.86504

 SCH/REF 0.90436

 HL-SOME 0.84902

 HL-ALWAY 0.81695

 HL-? 0.83754

 B008901N 0.94253

 B008901M 0.90926

 TVLIN-0 0.98142

 TV-QUAD 0.97999

 HW-NO 0.96362

 HW-YES 0.96561

 HWLIN-0 0.93806

 HWQUAD-0 0.91809

 HITEM=3 0.89183

 HITEM=4 0.86408

 MOMHOM-N 0.84023

 MOMHOM-? 0.87898

 DADHOM-N 0.84817

 DADHOM-? 0.88099

 MISS-2< 0.91536

 USA >5 0.91828

 USA 3-5 0.92784

 USA <3 0.87446

 USA-? 0.79211

 SNRM-LIN 0.89715

 SCHCHG-1 0.92894

 SCHCHG-2 0.92875

 SCHCHG-3 0.86997

 DIS@HOM2 0.89291

 DIS@HOM3 0.91848

 DIS@HOM4 0.82593

 PGS>5 0.81303

 PGS>10 0.81267

 8TH GRD 0.94151

 PREALG 0.93329

 ALGEBRA 0.92511

 INT/SEQ 0.93387

 APPLIED 0.89124

 OTHER 0.93801

 MISSING 0.93013

 G/R 22 0.90755

 G/R 23 0.91042

 G/R 24 0.95593

 G/T 22 0.72733

 G/T 23 0.72892

 G/T 24 0.74182

 G/T 25 0.74657

 G/P 22 0.88531

 G/P 23 0.86071

 G/P 24 0.79421

 G/P 25 0.81323

 G/S 22 0.93783

 G/S 23 0.91741

 R/T 24 0.87374

 R/T 25 0.88210

 R/T 31 0.89950

 R/T 32 0.92058

 R/T 33 0.87576

 R/T 34 0.87096

 R/T 35 0.87505

 R/T 41 0.90554

 R/T 42 0.92224

 R/T 43 0.91507

 R/T 44 0.91470

 R/T 45 0.95898

 R/P 24 0.88729

 R/P 25 0.89254

 R/P 31 0.88749

 R/P 32 0.86224

 R/P 33 0.89687

 R/P 34 0.90150

 R/P 35 0.89567

 R/P 41 0.87712

 R/P 42 0.95636

 R/P 43 0.95964

 R/P 44 0.94623

 R/P 45 0.92191

 R/S 31 0.96606

 R/S 32 0.96618

 R/S 33 0.96676

 R/S 41 0.94469

 R/S 42 0.97132

 R/S 43 0.95780

 T/P 25 0.80094

 T/P 31 0.80134

 T/P 32 0.81521

 T/P 33 0.84046

 T/P 34 0.78677

 T/P 35 0.79144

 T/P 41 0.78907

 T/P 42 0.84007

 T/P 43 0.78337

 T/P 44 0.76401

 T/P 45 0.81980

 T/P 51 0.78691

 T/P 52 0.79266

 T/P 53 0.80917

 T/P 54 0.82290

 T/P 55 0.84808

 T/S 32 0.93164

 T/S 33 0.92003

 T/S 41 0.95479

 T/S 42 0.92872

 T/S 43 0.95063

 T/S 52 0.95197

 P/S 32 0.95164

 P/S 33 0.93344

 P/S 41 0.94493

 P/S 42 0.92261

 P/S 43 0.93863

 P/S 51 0.92295

 P/S 52 0.95461

 P/S 53 0.93576

 G/ 22 0.93592

 G/ 23 0.95116

 G/ 24 0.93179

 G/ 25 0.86323

 G/ 26 0.85715

 G/ 27 0.91950

 G/ 28 0.84763

 R/ 24 0.92793

 R/ 25 0.91838

 R/ 26 0.91243

 R/ 27 0.89314

 R/ 28 0.89928

 R/ 31 0.87915

 R/ 32 0.86721

 R/ 33 0.92894

 R/ 34 0.92274

 R/ 35 0.92208

 R/ 36 0.90532

 R/ 37 0.90052

 R/ 38 0.88622

 R/ 41 0.87288

 R/ 42 0.95356
Table C-11 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

644

 R/ 43 0.95586

 R/ 44 0.94588

 R/ 45 0.89045

 R/ 46 0.87926

 R/ 47 0.92260

 R/ 48 0.93993

 P/ 25 0.91725

 P/ 26 0.89966

 P/ 27 0.88288

 P/ 28 0.81238

 P/ 31 0.82703

 P/ 32 0.78422

 P/ 33 0.81740

 P/ 34 0.88873

 P/ 35 0.86981

 P/ 36 0.92637

 P/ 37 0.82100

 P/ 38 0.84099

 P/ 41 0.77060

 P/ 42 0.81732

 P/ 43 0.87637

 P/ 44 0.85636

 P/ 45 0.83856

 P/ 46 0.79113

 P/ 47 0.87264

 P/ 48 0.76559

 P/ 51 0.80307

 P/ 52 0.87510

 P/ 53 0.83157

 P/ 54 0.89636

 P/ 55 0.80892

 P/ 56 0.80434

 P/ 57 0.82456

 P/ 58 0.81977

 T/ 25 0.86093

 T/ 26 0.84930

 T/ 27 0.82121

 T/ 28 0.87708

 T/ 31 0.85837

 T/ 32 0.80234

 T/ 33 0.80283

 T/ 34 0.85945

 T/ 35 0.81043

 T/ 36 0.82767

 T/ 37 0.86522

 T/ 38 0.87206

 T/ 41 0.77217

 T/ 42 0.82045

 T/ 43 0.87237

 T/ 44 0.84620

 T/ 45 0.86059

 T/ 46 0.95946

 T/ 47 0.88866

 T/ 48 0.81806

 T/ 51 0.85272

 T/ 52 0.88552

 T/ 53 0.83497

 T/ 54 0.84404

 T/ 55 0.87899

 T/ 57 0.81430

 S/ 23 0.95571

 S/ 24 0.94874

 S/ 25 0.93655

 S/ 26 0.87380

 S/ 27 0.88821

 S/ 28 0.94125

 S/ 31 0.88571

 S/ 32 0.94201

 S/ 33 0.93769

 S/ 34 0.92429

 S/ 37 0.96185

 SAMP S2 0.87564

 /R 23 0.90500

 /R 24 0.89817

 /R 31 0.95481

 B009301B 0.94197

 B009301C 0.95406

 B009301D 0.95267

 B009301E 0.93658

 B009301M 0.82239

 B009401B 0.85769

 B009401C 0.89311

 B009401D 0.87783

 B009401M 0.83927

 M812701B 0.90965

 M812701C 0.93011

 M812701D 0.84236

 M812701M 0.82648

 M812702B 0.89768

 M812702C 0.91660

 M812701D 0.90267

 M812701M 0.82565

 M812703B 0.92509

 M812703C 0.93064

 M812703D 0.92087

 M812703M 0.69206

 M812705B 0.94087

 M812705C 0.94574

 M812705D 0.92095

 M812705M 0.78616

 M812706B 0.94530

 M812706C 0.95627

 M812706D 0.90033

 M812706M 0.74296

 M812707B 0.90126

 M812707C 0.91977

 M812707D 0.86700

 M812707M 0.88850

 M812708B 0.87445

 M812708C 0.89525

 M812708D 0.83380

 M812708M 0.74980

 M812709B 0.88785

 M812709C 0.90083

 M812709D 0.86335

 M812709M 0.74053

 M812710B 0.93236

 M812710C 0.93526

 M812710D 0.91073

 M812710M 0.79454

 M812711B 0.86971

 M812711C 0.89593

 M812711D 0.84064

 M812711M 0.85688

 M811201N 0.86266

 M811201M 0.80875

 M812001B 0.84867

 M812001C 0.89470

 M812001D 0.82194

 M812001M 0.81513

 M812002B 0.89876

 M812001C 0.90771

 M812001D 0.86499

 M812001M 0.91680

 M812003B 0.87811

 M812003C 0.92138

 M812003D 0.89682

 M812003M 0.87967

 M812101B 0.88739

 M812101C 0.95249

 M812101D 0.98305

 M812101E 0.96073

 M812101F 0.94860

 M812101G 0.94817

 M812101M 0.76705

 MM00101B 0.90473

 MM00101C 0.90502

 MM00101D 0.92171

 MM00101M 0.84443

 MM00201B 0.95161

 MM00201C 0.95178

 MM00201D 0.94556

 MM00201M 0.88655

 MM00301B 0.92159

 MM00301C 0.92104

 MM00301D 0.92272

 MM00301M 0.83735

 MM00401B 0.90564

 MM00401C 0.90656

 MM00401D 0.90258

 MM00401M 0.85979

 MM00501B 0.91546

 MM00501C 0.91780

 MM00501D 0.91964

 MM00501M 0.86315

 B009701B 0.90685

 B009701C 0.89817

 B009701D 0.93126
Table C-11 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

645

 B009701E 0.92208

 B009701F 0.95144

 B009701M 0.84595

 GRAD HS 0.94110

 ED > HS 0.96475

 GRAD CLG 0.96336

 GRAD SCH 0.94283

 MISSING 0.77302

 B009601B 0.87176

 B009601C 0.84997

 B009601D 0.84804

 B009601M 0.88366

 B009601B 0.94583

 B009601C 0.92200

 B009601D 0.85322

 B009601M 0.86797

 M812712B 0.94893

 M812712C 0.95875

 M812712D 0.95137

 M812712M 0.72729

 M812713B 0.95233

 M812713C 0.97003

 M812713D 0.96448

 M812713M 0.77091

 M812201N 0.89140

 M812201M 0.76400

 M812301N 0.81021

 M812301M 0.83552

 M812401N 0.82627

 M812401M 0.84167

 BASIC 0.93725

 APPLIED 0.96260

 PREALG 0.92809

 ALG 1 0.95522

 GEOMETRY 0.78999

 INT/SEQ 0.81907

 OTHER 0.88223

 IDK 0.95595

 MISSING 0.80969

 M812701B 0.91250

 M812701C 0.92949

 M812701D 0.95011

 M812701E 0.85028

 M812701M 0.85388

 M810701B 0.88582

 M810701C 0.93173

 M810701D 0.94071

 M810701E 0.81516

 M810701M 0.81252

 M810707B 0.87195

 M810707C 0.90383

 M810707D 0.92895

 M810707E 0.80457

 M810707M 0.76770

 M810709B 0.96430

 M810709C 0.97372

 M810709D 0.97616

 M810709E 0.97474

 M810709M 0.87758

 M810708B 0.95046

 M810708C 0.94811

 M810708D 0.95470

 M810708E 0.95704

 M810708M 0.84154

 M810710B 0.90331

 M810710C 0.90910

 M810710D 0.95931

 M810710E 0.91239

 M810710M 0.87203

 M810705B 0.86691

 M810705C 0.89354

 M810705D 0.91930

 M810705E 0.90340

 M810705M 0.92837

 M810706B 0.94797

 M810706C 0.94923

 M810706D 0.96217

 M810706E 0.92694

 M810706M 0.92244

 M810711B 0.88507

 M810711C 0.89834

 M810711D 0.91593

 M810711E 0.90353

 M810711M 0.90112

 C031603N 0.91594

 C031603M 0.96515

 C031607N 0.92005

 C031607M 0.96658

 C031610N 0.90820

 C031610M 0.94926

 C031606N 0.89310

 C031606M 0.95348

 C035701N 0.90366

 C035701M 0.94218

 C035702N 0.88445

 C035702M 0.97011

 C035703N 0.88549

 C035703M 0.92685

 C037201M 0.94973

 C037202M 0.93131

 C037204M 0.90177

 C037205M 0.89256

 C037206M 0.92693

 C037301M 0.97405

 C037302M 0.96113

 C037304M 0.90304

 C037305M 0.93896

 C036601B 0.89148

 C036601C 0.91413

 C036601D 0.94539

 C036601M 0.97685

 C032207B 0.91227

 C032207N 0.93248

 C032207M 0.93447

 C032209B 0.90029

 C032209N 0.90276

 C032209M 0.98585

 C032210B 0.92826

 C032210N 0.92751

 C032210M 0.95062

 C032211B 0.90726

 C032211N 0.88456

 C032211M 0.98648

 C037701B 0.90614

 C037701C 0.91168

 C037701D 0.90296

 C037701M 0.98076

 C037702B 0.92327

 C037702C 0.93673

 C037702D 0.93390

 C037702M 0.97849

 C037703B 0.91672

 C037703C 0.93131

 C037703D 0.93545

 C037703M 0.99105

 C037704B 0.88942

 C037704C 0.89140

 C037704D 0.90698

 C037704M 0.98275

 C037705B 0.90535

 C037705C 0.90174

 C037705D 0.89648

 C037705M 0.98646

 C032402B 0.93467

 C032402C 0.94533

 C032402D 0.93473

 C032401B 0.92963

 C032401C 0.94813

 C032401D 0.93644

 C032404B 0.91664

 C032404C 0.95186

 C032404D 0.93068

 C032406B 0.89857

 C032406C 0.94637

 C032406D 0.94673

 C032406M 0.97844

 C032407B 0.88564

 C032407C 0.90939

 C032408B 0.91070

 C032408C 0.94679

 C032408D 0.94800

 C032409B 0.92746

 C032409C 0.94160

 C032409D 0.92181

 C032410B 0.92980

 C032410C 0.94784

 C032410D 0.95246

 C032411B 0.93020
Table C-11 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

646

 C032411C 0.91582

 C032412B 0.91253

 C032412C 0.93321

 C032413B 0.91076

 C032413C 0.94270

 C032413D 0.94985

 C032414B 0.94363

 C032414C 0.95104

 C032414D 0.88788

 C032415B 0.92749

 C032415C 0.89179

 C032502B 0.90780

 C032502C 0.91345

 C032503B 0.92169

 C032503C 0.90117

 C032503M 0.98531

 C032505B 0.91602

 C032505C 0.90715

 C032506B 0.90674

 C032506C 0.90228

 C032506D 0.90674

 C033601B 0.92819

 C033601C 0.92044

 C033601D 0.90542

 C036501B 0.91363

 C036501C 0.90872

 C037801B 0.91523

 C037801C 0.92636

 C037801D 0.89795

 C037801E 0.89586

 C037801F 0.91907

 C037801G 0.89618

 C037801H 0.93549

 C037801M 0.97520

 C038001B 0.90960

 C038001C 0.90497

 C038001D 0.88774

 C038001E 0.89327

 C038001G 0.93207

 C038301N 0.91318

 C038301M 0.96584

 C038801N 0.90484

 C038801M 0.94554

 C034101M 0.91960

 C034102M 0.85874

 C034103M 0.88720

 C034104M 0.90990

 C034105M 0.91509

 C034106M 0.88247

 C034107M 0.90550

 C034108M 0.89866

 C034109M 0.89096

 T055901B 0.85163

 T055901M 0.98717

 T056001B 0.83897

 T056001C 0.85430

 T056001D 0.87634

 T056001E 0.88429

 T056001F 0.85685

 T056001M 0.98103

 T040301B 0.89927

 T040301C 0.91648

 T040301D 0.93642

 T040301E 0.94637

 T040301M 0.99299

 T056201B 0.90194

 T056201C 0.88094

 T056201D 0.88681

 T056201E 0.88529

 T056201F 0.88692

 T056201M 0.95836

 T040501N 0.89508

 T040501C 0.88914

 T040501M 0.88564

 T040506N 0.92133

 T040506C 0.87177

 T040506M 0.92065

 T040504N 0.87686

 T040504C 0.87640

 T040504M 0.89901

 T040505N 0.89973

 T040505C 0.87897

 T040505M 0.90676

 T056301B 0.88169

 T056301C 0.94810

 T056301D 0.93610

 T056301E 0.88520

 T056301F 0.89411

 T056301G 0.88805

 T056301M 0.97870

 T040701M 0.87808

 T040706M 0.88877

 T040707M 0.86265

 T040703M 0.87527

 T040704M 0.86729

 T040708M 0.86015

 T040709M 0.86686

 T040705M 0.88113

 T040801M 0.87021

 T040807M 0.87467

 T040808M 0.87728

 T040803M 0.87427

 T040804M 0.87398

 T040809M 0.86626

 T040810M 0.86959

 T040811M 0.86418

 T040812M 0.86015

 T040813M 0.86959

 T040805M 0.87448

 T040806M 0.89637

 T056401M 0.88013

 T056402M 0.87195

 T056403M 0.86155

 T056404M 0.89005

 T056405M 0.87917

 T056406M 0.86674

 T056407M 0.89077

 T056408M 0.86793

 T056409M 0.86406

 T056410M 0.87493

 T056411M 0.89169

 T056412M 0.88958

 T056501B 0.89340

 T056501C 0.91637

 T056501D 0.90411

 T056501E 0.89452

 T056601B 0.86582

 T056601C 0.85079

 T056601D 0.87115

 T056601E 0.87349

 T056701M 0.86788

 T056702M 0.87040

 T056703M 0.86560

 T056704M 0.85028

 T056705M 0.86112

 T056706M 0.85983

 T056707M 0.84627

 T056708M 0.87261

 T056709M 0.85872

 T056710M 0.86888

 T056711M 0.86264

 T056712M 0.85995

 T056713M 0.86100

 T041201B 0.88619

 T041201C 0.88360

 T041201D 0.95982

 T056801B 0.88338

 T056801C 0.89619

 T056801D 0.92371

 T056801E 0.93047

 T056801F 0.91406

 T056801M 0.97424

 T056901M 0.90315

 T0569A1M 0.88271

 T0569B1M 0.87323

 T0569C1M 0.89853

 T056902M 0.90492

 T0569A2M 0.91483

 T0569B2M 0.89312

 T0569C2M 0.88269

 T056903M 0.90217

 T0569A3M 0.89857

 T0569B3M 0.89419

 T0569C3M 0.89079

 T056904M 0.89713

 T0569A4M 0.88016

 T0569B4M 0.89325

 T0569C4M 0.87946
Table C-11 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

647

 T056905M 0.90363

 T0569A5M 0.88528

 T0569B5M 0.89336

 T0569C5M 0.87814

 T056906M 0.91481

 T0569A6M 0.88477

 T0569B6M 0.86503

 T0569C6M 0.89837

 T056907M 0.92663

 T0569A7M 0.86584

 T0569B7M 0.86403

 T0569C7M 0.89780

 T056908M 0.91516

 T0569A8M 0.89206

 T0569B8M 0.88691

 T0569C8M 0.89879

 T057001N 0.87328

 T057001M 0.95849

 T057002N 0.87625

 T057002M 0.98859

 T057003N 0.86007

 T057003M 0.97643

 T057004N 0.88587

 T057004M 0.97313

 T057005N 0.86140

 T057005M 0.95776

 T057006N 0.87462

 T057006M 0.94233

 T057007N 0.88237

 T057007M 0.95541

 T057101B 0.88244

 T057101C 0.91693

 T057101D 0.88574

 T057101M 0.94623

 T057201M 0.88932

 T057211M 0.84955

 T057221M 0.88253

 T057231M 0.85572

 T057241M 0.91787

 T057301B 0.86461

 T057301C 0.88359

 T057301M 0.98678

 T057302B 0.87224

 T057302C 0.86300

 T057303B 0.86019

 T057303C 0.86201

 T057303M 0.99279

 T057304B 0.85741

 T057304C 0.86730

 T057304M 0.98213

 T057401B 0.90660

 T057401C 0.90346

 T057401D 0.90436

 T057401M 0.97563

 T057402B 0.83963

 T057402C 0.85985

 T057402D 0.87643

 T057402M 0.91986

 T057403B 0.89899

 T057403C 0.90091

 T057403D 0.90774

 T057403M 0.97727

 T057404B 0.90022

 T057404C 0.91687

 T057404D 0.88966

 T057404M 0.98207

 T057405B 0.88958

 T057405C 0.89392

 T057405D 0.90055

 T057501B 0.87004

 T057501C 0.87899

 T057501D 0.87228

 T057501E 0.89439

 T057501F 0.86958

 T057501M 0.89914

 T057601B 0.87712

 T057601C 0.87547

 T057601D 0.87891

 T057601E 0.89050

 T057601M 0.89058

 T044002N 0.94282

 T044002M 0.98489

 T057701B 0.80117

 T057701C 0.94548

 T057701D 0.83031

 T057701M 0.91628

 T057801B 0.88089

 T057801C 0.91023

 T057801D 0.86027

 T057801E 0.85474

 T057801M 0.93020

 T044201N 0.83582

 T044201M 0.96610

 T044301B 0.89014

 T044301C 0.89524

 T044301M 0.99158

 T057901B 0.90917

 T057901C 0.92372

 T057901D 0.92095

 T057901M 0.98451

 T044401B 0.89582

 T044401C 0.93311

 T044401D 0.89211

 T044401E 0.88988

 T044401F 0.88699

 T044401M 0.98353

 T044501B 0.83880

 T044501C 0.87046

 T044501D 0.85452

 T044501M 0.98985

 T044502B 0.88893

 T044502C 0.89621

 T044502D 0.86751

 T044502M 0.98905

 T044512B 0.87988

 T044512C 0.90848

 T044512D 0.88274

 T044512M 0.99337

 T044513B 0.89689

 T044513C 0.91988

 T044513D 0.89134

 T044513M 0.97943

 T044514B 0.86173

 T044514C 0.92439

 T044514D 0.93344

 T044514M 0.98310

 T044505B 0.86382

 T044505C 0.88513

 T044505D 0.88931

 T044505M 0.98431

 T044515B 0.93339

 T044515C 0.93540

 T044515D 0.88233

 T044515M 0.97711

 T044507B 0.90372

 T044507C 0.91545

 T044507D 0.89482

 T044507M 0.98104

 T044516B 0.88722

 T044516C 0.88880

 T044516D 0.86928

 T044516M 0.97995

 T044508B 0.84325

 T044508C 0.92874

 T044508D 0.94464

 T044509B 0.87212

 T044509C 0.87988

 T044509D 0.85133

 T044509M 0.99242

 T044510B 0.88301

 T044510C 0.90142

 T044510D 0.88009

 T044506B 0.89947

 T044506C 0.90503

 T044506D 0.93084

 T058001B 0.83872

 T058001C 0.83924

 T058001D 0.90580

 T058001M 0.98657

 T058002B 0.88574

 T058002C 0.87935

 T058002D 0.84904

 T058002M 0.97487

 T058003B 0.87332

 T058003C 0.88309

 T058003D 0.85618

 T058003M 0.98302

 T058004B 0.88710
Table C-11 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

648

 T058004C 0.90237

 T058004D 0.86892

 T058004M 0.98773

 T058005B 0.84985

 T058005C 0.84628

 T058005D 0.86918

 T058005M 0.99108

 T058006B 0.85669

 T058006C 0.85238

 T058006D 0.86313

 T058006M 0.99161

 T058007B 0.87286

 T058007C 0.87980

 T058007D 0.91941

 T058007M 0.98868

 T058008B 0.85248

 T058008C 0.83660

 T058008D 0.84409

 T058008M 0.99088

 T058009B 0.86469

 T058009C 0.88677

 T058009D 0.88230

 T058009M 0.98684

 T045401N 0.86512

 T045401M 0.97901

 T044801N 0.88113

 T044801M 0.97237

 T045001N 0.85439

 T045001M 0.98589

 T044901N 0.85653

 T044901M 0.94842

 T045304B 0.90464

 T045304C 0.93677

 T045304D 0.97602

 T045304M 0.98756

 T045305B 0.89497

 T045305C 0.94552

 T045305M 0.97247

 T045302B 0.87919

 T045302C 0.89584

 T045302D 0.90152

 T045302M 0.96954

 T045303B 0.86732

 T045303C 0.87521

 T045303D 0.86656

 T045303M 0.97131

 T044000B 0.86154

 T044000C 0.87006

 T044000D 0.84851

 T044000E 0.88671

 T044000M 0.87980

 C034201B 0.93716

 C034201C 0.95747

 C034201M 0.92975

 C034402N 0.90187

 C034402M 0.97077

 C034403N 0.92593

 C034403M 0.96844

 C034401N 0.90517

 C034401M 0.97825

 C034406N 0.90995

 C034406M 0.95625

 C034510B 0.90143

 C034510C 0.91646

 C034510D 0.90488

 C034510E 0.91600

 C034510M 0.93872

 C034511B 0.92490

 C034512B 0.93015

 C034513B 0.93065

 C034514B 0.90435

 C034514C 0.90408

 C034514D 0.90448

 C034514E 0.87394

 C034514M 0.94466

 C031611N 0.91608

 C031611M 0.95481

 C034601N 0.90328

 C034601M 0.94237

 C037203M 0.93229

 C037306M 0.95528

 C039401M 0.91790

 C039402M 0.91270

 C039403M 0.89748

 C039404M 0.89834

 C039405M 0.90496

 C039406M 0.89760

 C039501M 0.89080

 C039502M 0.89988

 C039503M 0.91084

 C039504M 0.90820

 C039505M 0.90996

 C039601M 0.95724

 C039602M 0.92269

 C039603M 0.95032

 C039604M 0.91266

 C039605M 0.94171

 C041901B 0.89919

 C041901C 0.91499

 C041901D 0.89592

 T063001B 0.90690

 T063001C 0.91075

 T063001D 0.91973

 T063001E 0.93261

 T058301N 0.86272

 T058301M 0.95759

 NATLUNCH 0.93597

 NATLUNCL 0.92837

 REM READ 0.94925

 REMREADL 0.94611

 REM MATH 0.94516

 REMMATHL 0.94199

 NATLUN8 0.93663

 NATLUN8L 0.94041

 REMREAD8 0.94995

 REMREA8L 0.91462

 REMMATH8 0.95261

 REMMAT8L 0.91353
Table C-12

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

649

 FEMALE 0.92889

 BLACK 0.92915

 HISPANIC 0.97024

 ASIAN 0.92696

 MEXICAN 0.94406

 PUER RIC 0.96474

 CUBN,OTH 0.93574

 HISP-? 0.96206

 NON MSA 0.92470

 MID CTY5 0.94452

 FR/BTWN5 0.94650

 SML TWN5 0.95100

 RURAL5 0.95572

 URBAN FR 0.91208

 MED CITY 0.87491

 SM PLACE 0.92893

 HS GRAD 0.94287

 POST HS 0.93664

 COL GRAD 0.93811

 PARED-? 0.94441

 S EAST 0.89733

 CENTRAL 0.91506

 WEST 0.90241

 PRIVATE 0.92047

 CATHOLIC 0.91554

 IEP-NO 0.91519

 LEP-NO 0.83424

 CHAP1-N 0.79997

 RED PRIC 0.96378

 FREE 0.78963

 INFO N/A 0.82443

 SCH/REF 0.84379

 HL-SOME 0.86310

 HL-ALWAY 0.80564

 HL-? 0.88175

 B008901N 0.93293

 B008901M 0.77599

 TVLIN-0 0.97819

 TV-QUAD 0.97739

 HW-NO 0.96859

 HW-YES 0.97042

 HWLIN-0 0.92895

 HWQUAD-0 0.89826

 HITEM=3 0.89943

 HITEM=4 0.87168

 MOMHOM-N 0.81991

 MOMHOM-? 0.82492

 DADHOM-N 0.80223

 DADHOM-? 0.81180

 MISS-2< 0.95102

 USA >5 0.85200

 USA 3-5 0.87304

 USA <3 0.88979

 USA-? 0.88018

 SNRM-LIN 0.84637

 STGRD1-2 0.92029

 STGRD3-5 0.94783

 STGRD6> 0.95254

 SCHCHG-1 0.90937

 SCHCHG-2 0.94307

 SCHCHG-3 0.86335

 DIS@HOM2 0.89647

 DIS@HOM3 0.91935

 DIS@HOM4 0.86792

 PGS>5 0.83637

 PGS>10 0.83038

 ACADEMIC 0.71063

 VOC/TECH 0.91092

 OTHERPGM 0.90838

 HS PGM-? 0.87226

 SEMENG-^ 0.87039

 #ENG-LIN 0.87227

 SEMMAT02 0.94877

 SEMMAT03 0.93763

 SEMMAT04 0.95305

 SEMMAT-? 0.94796

 #MAT-LIN 0.95304

 SEMSCI-^ 0.88033

 #SCI-LIN 0.83029

 SEMHIS-^ 0.87348

 #HIS-LIN 0.80868

 SEMFLG-^ 0.87828

 #FLG-LIN 0.74314

 SEMVOC-^ 0.86063

 #VOC-LIN 0.83484

 SEMART-^ 0.88550

 #ART-LIN 0.82482

 G/R 22 0.91066

 G/R 23 0.91334

 G/R 24 0.94772

 G/T 22 0.72438

 G/T 23 0.73222

 G/T 24 0.74637

 G/T 25 0.74718

 G/P 22 0.86244

 G/P 23 0.90798

 G/P 24 0.83393

 G/P 25 0.88757

 G/S 22 0.93286

 G/S 23 0.91873

 R/T 24 0.88537

 R/T 25 0.86606

 R/T 31 0.87988

 R/T 32 0.90904

 R/T 33 0.88724

 R/T 34 0.89011

 R/T 35 0.90835

 R/T 41 0.90889

 R/T 42 0.91443

 R/T 43 0.91285

 R/T 44 0.92305

 R/T 45 0.95298

 R/P 24 0.88973

 R/P 25 0.89812

 R/P 31 0.90529

 R/P 32 0.84951

 R/P 33 0.88889

 R/P 34 0.89844

 R/P 35 0.90878

 R/P 41 0.85378

 R/P 42 0.93732

 R/P 43 0.95245

 R/P 44 0.94376

 R/P 45 0.87956

 R/S 31 0.97236

 R/S 32 0.95845

 R/S 33 0.96573

 R/S 41 0.93453

 R/S 42 0.96601

 R/S 43 0.96263

 T/P 25 0.80631

 T/P 31 0.78804

 T/P 32 0.79802

 T/P 33 0.87480

 T/P 34 0.80810

 T/P 35 0.78901

 T/P 41 0.79959

 T/P 42 0.87871

 T/P 43 0.81009

 T/P 44 0.79426

 T/P 45 0.81519

 T/P 51 0.83235

 T/P 52 0.77231

 T/P 53 0.78145

 T/P 54 0.81725

 T/P 55 0.85260

 T/S 32 0.94803

 T/S 33 0.91833

 T/S 41 0.93893

 T/S 42 0.92502

 T/S 43 0.94205

 T/S 52 0.93511

 P/S 32 0.95344

 P/S 33 0.93108

 P/S 41 0.94648

 P/S 42 0.92627

 P/S 43 0.93095

 P/S 51 0.91136

 P/S 52 0.95316

 P/S 53 0.95220

 G/S 22 0.91882

 G/S 23 0.92822

 G/S 24 0.89374

 G/S 25 0.91531

 R/S 24 0.90850
Table C-12 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

650

 R/S 25 0.89901

 R/S 31 0.89188

 R/S 32 0.87660

 R/S 33 0.91231

 R/S 34 0.90430

 R/S 35 0.90898

 R/S 41 0.86200

 R/S 42 0.96148

 R/S 43 0.95207

 R/S 44 0.94882

 R/S 45 0.93292

 P/S 25 0.82873

 P/S 31 0.83186

 P/S 32 0.82603

 P/S 33 0.86589

 P/S 34 0.82924

 P/S 35 0.85286

 P/S 41 0.84079

 P/S 42 0.84679

 P/S 43 0.83543

 P/S 44 0.84071

 P/S 45 0.84846

 P/S 51 0.84557

 P/S 52 0.88887

 P/S 53 0.89799

 P/S 54 0.90936

 P/S 55 0.82638

 T/S 25 0.78320

 T/S 31 0.78789

 T/S 32 0.79978

 T/S 33 0.85158

 T/S 34 0.79937

 T/S 35 0.78979

 T/S 41 0.79607

 T/S 42 0.82956

 T/S 43 0.80830

 T/S 44 0.79344

 T/S 45 0.80760

 T/S 51 0.83959

 T/S 52 0.80603

 T/S 53 0.81159

 T/S 54 0.81583

 T/S 55 0.87592

 S/S 23 0.94322

 S/S 24 0.92931

 S/S 25 0.92106

 S/S 31 0.93316

 S/S 32 0.93324

 S/S 33 0.92511

 S/S 34 0.90509

 S/S 35 0.92779

 SAMP S2 0.84496

 /R 23 0.89288

 /R 24 0.91002

 /R 31 0.95448

 B009301B 0.94509

 B009301C 0.96313

 B009301D 0.96473

 B009301E 0.94429

 B009301M 0.85486

 B009401B 0.84742

 B009401C 0.91078

 B009401D 0.89509

 B009401M 0.86608

 M812701B 0.94299

 M812701C 0.93163

 M812701D 0.80006

 M812701M 0.90305

 M812702B 0.91453

 M812702C 0.93249

 M812701D 0.88981

 M812701M 0.89179

 M812703B 0.91990

 M812703C 0.93357

 M812703D 0.89724

 M812703M 0.75234

 M812705B 0.96117

 M812705C 0.95786

 M812705D 0.94391

 M812705M 0.70579

 M812706B 0.94209

 M812706C 0.94233

 M812706D 0.83293

 M812706M 0.69415

 M812707B 0.91333

 M812707C 0.92045

 M812707D 0.86804

 M812707M 0.72257

 M812708B 0.91510

 M812708C 0.92602

 M812708D 0.80625

 M812708M 0.84040

 M812709B 0.93844

 M812709C 0.93740

 M812709D 0.81655

 M812709M 0.79137

 M812710B 0.93004

 M812710C 0.93262

 M812710D 0.90046

 M812710M 0.73863

 M812711B 0.89780

 M812711C 0.92079

 M812711D 0.83461

 M812711M 0.80236

 M811201N 0.84007

 M811201M 0.83676

 M812001B 0.83237

 M812001C 0.87232

 M812001D 0.85506

 M812001M 0.90125

 M812002B 0.83705

 M812001C 0.87673

 M812001D 0.82854

 M812001M 0.92458

 M812003B 0.86440

 M812003C 0.85592

 M812003D 0.85387

 M812003M 0.91317

 M812101B 0.92796

 M812101C 0.92880

 M812101D 0.95261

 M812101E 0.94985

 M812101F 0.94558

 M812101G 0.93445

 M812101M 0.72041

 MM00101B 0.94768

 MM00101C 0.93958

 MM00101D 0.93427

 MM00101M 0.84862

 MM00201B 0.95098

 MM00201C 0.94926

 MM00201D 0.93590

 MM00201M 0.88012

 MM00301B 0.95948

 MM00301C 0.96544

 MM00301D 0.95838

 MM00301M 0.89161

 MM00401B 0.94518

 MM00401C 0.95820

 MM00401D 0.94797

 MM00401M 0.88277

 MM00501B 0.94211

 MM00501C 0.94736

 MM00501D 0.92707

 MM00501M 0.86945

 B009601B 0.96300

 B009601C 0.82502

 B009601D 0.82014

 B009601M 0.90689

 B009601B 0.96814

 B009601C 0.94405

 B009601D 0.81806

 B009601M 0.88549

 M812712B 0.94634

 M812712C 0.94978

 M812712D 0.93745

 M812712M 0.76943

 M812713B 0.97058

 M812713C 0.98047

 M812713D 0.97780

 M812713M 0.83470

 M812201N 0.92856

 M812201M 0.75873

 M812301N 0.90158

 M812301M 0.88514

 M812401N 0.75286

 M812401M 0.76428

 M812701B 0.89155
Table C-12 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

651

 M812701C 0.94357

 M812701D 0.94946

 M812701E 0.85459

 M812701M 0.89191

 M810701B 0.88919

 M810701C 0.94654

 M810701D 0.93144

 M810701E 0.82369

 M810701M 0.85934

 M810707B 0.89511

 M810707C 0.93724

 M810707D 0.93885

 M810707E 0.80012

 M810707M 0.78312

 M810709B 0.97172

 M810709C 0.97283

 M810709D 0.98478

 M810709E 0.98017

 M810709M 0.81868

 M810708B 0.96101

 M810708C 0.97105

 M810708D 0.96817

 M810708E 0.95129

 M810708M 0.77785

 M810710B 0.90819

 M810710C 0.90931

 M810710D 0.95848

 M810710E 0.84153

 M810710M 0.83574

 M810705B 0.89777

 M810705C 0.91620

 M810705D 0.93308

 M810705E 0.87146

 M810705M 0.82100

 M810706B 0.93812

 M810706C 0.94644

 M810706D 0.95249

 M810706E 0.93335

 M810706M 0.88532

 M810711B 0.93470

 M810711C 0.94493

 M810711D 0.94052

 M810711E 0.91279

 M810711M 0.88385

 B008301D 0.93996

 B008301M 0.78877

 B009901B 0.91112

 B009901C 0.89614

 B009901D 0.95457

 B009901E 0.86926

 B009901F 0.90848

 B009901M 0.84227

 VOC COLL 0.95667

 2 YR COL 0.95115

 4 YR COL 0.90968

 MILITARY 0.96617

 OTHER 0.95012

 MISSING 0.89138

 OFFICE W 0.94234

 TECHNIC 0.97251

 PROT SRV 0.96321

 SALES 0.95678

 OWNER 0.96696

 SERV WRK 0.95346

 SKILL T 0.97118

 LABORER 0.95498

 OPERATOR 0.95170

 FARMER 0.96317

 HOMEMAKE 0.85064

 MANAGER 0.97064

 MILITARY 0.97445

 TEACHER 0.93819

 PROFESS1 0.95860

 PROFESS2 0.97452

 B011A01N 0.84863

 OFFICE 0.97488

 TECHNIC 0.96419

 PROT SRV 0.96755

 SALES 0.96678

 OWNER 0.95490

 SERV WRK 0.97190

 SKILL TR 0.95445

 LABORER 0.94934

 OPERATOR 0.94201

 FARMER 0.94369

 HOMEMAKE 0.95674

 MANAGER 0.96598

 MILITARY 0.97771

 TEACHER 0.95096

 PROFESS1 0.94708

 PROFESS2 0.91694

 B012A02N 0.81736

 M810901N 0.80953

 M810901M 0.82090

 M811801B 0.91589

 M811801C 0.97455

 M811801D 0.93853

 M811801E 0.91037

 M811801M 0.77042

 M811001B 0.87459

 M811001C 0.93298

 M811001D 0.81116

 M811001M 0.75042

 M811002B 0.95531

 M811002C 0.96639

 M811002D 0.95585

 M811002M 0.83289

 M811014B 0.95058

 M811014C 0.95387

 M811014D 0.93483

 M811014M 0.80040

 M811014B 0.92280

 M811014C 0.92952

 M811014D 0.91739

 M811014M 0.72346

 M811004B 0.86588

 M811004C 0.87451

 M811004D 0.85508

 M811004M 0.83351

 M811005B 0.88832

 M811005C 0.92215

 M811005D 0.84635

 M811005M 0.84901

 M811006B 0.91509

 M811006C 0.94481

 M811006D 0.86813

 M811006M 0.82563

 M811007B 0.94606

 M811007C 0.89898

 M811007D 0.91453

 M811007M 0.80535

 M811008B 0.93516

 M811008C 0.90089

 M811008D 0.92676

 M811008M 0.86758

 M811012B 0.94521

 M811012C 0.95911

 M811012D 0.92003

 M811012M 0.90186

 M811009B 0.96115

 M811009C 0.96577

 M811009D 0.96424

 M811009M 0.91124

 M811011B 0.90893

 M811011C 0.97182

 M811011D 0.93567

 M811011M 0.90579

 M812801N 0.84653

 M812801M 0.83905

 C035701N 0.86686

 C035701M 0.93081

 C035702N 0.89168

 C035702M 0.97529

 C035703N 0.87631

 C035703M 0.93179

 C037201M 0.94402

 C037202M 0.92866

 C037204M 0.89740

 C037205M 0.89736

 C037206M 0.90217

 C037301M 0.96296

 C037302M 0.96132

 C037304M 0.90405

 C037305M 0.91941

 C036601B 0.89703

 C036601C 0.91500

 C036601D 0.95387

 C036601M 0.97024
Table C-12 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

652

 C032207B 0.92302

 C032207N 0.94402

 C032207M 0.92083

 C032209B 0.91337

 C032209N 0.93446

 C032209M 0.94787

 C032210B 0.93101

 C032210N 0.94633

 C032210M 0.93765

 C032211B 0.88637

 C032211N 0.90622

 C032211M 0.94126

 C037701B 0.90046

 C037701C 0.89540

 C037701D 0.89470

 C037701M 0.95870

 C037702B 0.91199

 C037702C 0.92385

 C037702D 0.89213

 C037702M 0.96807

 C037703B 0.89290

 C037703C 0.90844

 C037703D 0.91385

 C037703M 0.98568

 C037704B 0.88396

 C037704C 0.88768

 C037704D 0.91019

 C037704M 0.93188

 C037705B 0.89396

 C037705C 0.89019

 C037705D 0.86953

 C037705M 0.94772

 C032402B 0.92107

 C032402C 0.92528

 C032402D 0.90811

 C032402M 0.99466

 C032401B 0.89473

 C032401C 0.93144

 C032401D 0.89418

 C032404B 0.88648

 C032404C 0.94295

 C032404D 0.93133

 C032406B 0.88220

 C032406C 0.94367

 C032406D 0.94071

 C032407B 0.85721

 C032407C 0.90257

 C032408B 0.89199

 C032408C 0.87317

 C032409B 0.92022

 C032409C 0.93469

 C032409D 0.90807

 C032410B 0.90849

 C032410C 0.93557

 C032410D 0.92060

 C032411B 0.92589

 C032411C 0.93713

 C032411D 0.90612

 C032412B 0.91793

 C032412C 0.94392

 C032412D 0.90855

 C032413B 0.90748

 C032413C 0.94639

 C032413D 0.94378

 C032413M 0.97446

 C032414B 0.91207

 C032414C 0.94107

 C032414D 0.88274

 C032415B 0.92605

 C032415C 0.92522

 C032502B 0.90450

 C032502C 0.90350

 C032502D 0.94285

 C032502M 0.97611

 C032503B 0.90781

 C032503C 0.90315

 C032503M 0.97893

 C032505B 0.88865

 C032505C 0.89919

 C032506B 0.89021

 C032506C 0.88718

 C032506D 0.92530

 C033601B 0.91818

 C033601C 0.92725

 C033601D 0.88997

 C033601M 0.96468

 C036501B 0.89604

 C036501C 0.90429

 C037801B 0.89556

 C037801C 0.90588

 C037801D 0.91885

 C037801E 0.91122

 C037801F 0.90982

 C037801G 0.91465

 C037801M 0.98176

 C038001B 0.84995

 C038001C 0.90179

 C038001D 0.93271

 C038001M 0.98025

 C038301N 0.88718

 C038301M 0.93248

 C038801N 0.90293

 C038801M 0.92243

 C034101M 0.88453

 C034102M 0.88179

 C034103M 0.88071

 C034104M 0.90456

 C034105M 0.89147

 C034106M 0.87657

 C034107M 0.90138

 C034108M 0.89179

 C034109M 0.89172

 C037203M 0.88792

 C037306M 0.95734

 C035002N 0.93828

 C035002M 0.98503

 C035003N 0.92750

 C035003M 0.97758

 C035006N 0.90580

 C035006M 0.96848

 C035007N 0.87812

 C035007M 0.93619

 C040201C 0.92029

 C040201D 0.95054

 C040201E 0.95411

 C040201F 0.92232

 C040201M 0.98935

 C040202C 0.91346

 C040202D 0.95124

 C040202E 0.92503

 C040203E 0.92372

 C040203F 0.94662

 C040204B 0.89422

 C040204C 0.88909

 C040204D 0.88218

 C040204M 0.95160

 C040301M 0.92441

 C040302M 0.87495

 C040303M 0.88201

 C040304M 0.91946

 C040305M 0.87201

 C040306M 0.88846

 C040307M 0.86896

 C040308M 0.86455

 C040309M 0.89038

 C040310M 0.89854

 C040311M 0.90704

 C040401N 0.95301

 C040401M 0.98325

 C040402N 0.92195

 C040402M 0.94504

 C040403N 0.95216

 C040404N 0.94319

 C040501M 0.89832

 C040502M 0.85814

 C040503M 0.87339

 C040504M 0.88199

 C040505M 0.88898

 C040506M 0.87669

 C040601M 0.89355

 C040602M 0.89762

 C040603M 0.88420

 C040604M 0.87274

 C040605M 0.85977

 C040701M 0.90518

 C040702M 0.89943

 C040703M 0.90987

 C040704M 0.87241
Table C-12 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Mathematics Main Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

653

 C040705M 0.91477

 C040801B 0.89997

 C040801C 0.88676

 C040801D 0.90481

 C040801E 0.89002

 C040801F 0.90652

 C040801G 0.91176

 C040801M 0.91525

 C040802B 0.90037

 C040802C 0.90687

 C040802D 0.91291

 C040802E 0.88286

 C040802F 0.90444

 C040802G 0.90068

 C040802M 0.92939

 C040803B 0.87355

 C040803C 0.87565

 C040803D 0.87617

 C040803G 0.90965

 C040803M 0.88149

 C040804B 0.86226

 C040804C 0.91401

 C040804D 0.91207

 C040804E 0.89213

 C040804F 0.87585

 C040804G 0.92287

 C040804M 0.94197

 C040901M 0.91375

 C040902M 0.89865

 C040903M 0.91475

 C040904M 0.90002

 C040905M 0.93204

 C041001B 0.91548

 C041001C 0.87835

 C041001D 0.88912

 C041101B 0.91241

 C041101C 0.89830

 C036001B 0.89703

 C036001C 0.88469

 C036001D 0.87517

 NATLUNCH 0.89506

 NATLUNCL 0.89650

 REM READ 0.87890

 REMREADL 0.90861

 REMMATHL 0.90472
Table C-13

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

654

 FEMALE 0.91651

 BLACK 0.93312

 HISPANIC 0.96527

 ASIAN 0.95176

 MEXICAN 0.93869

 PUER RIC 0.92524

 CUBN,OTH 0.96503

 HISP-? 0.81924

 NON MSA 0.95741

 MID CTY5 0.95016

 FR/BTWN5 0.94663

 SML TWN5 0.95003

 RURAL5 0.96741

 URBAN FR 0.90792

 MED CITY 0.91599

 SM PLACE 0.93846

 HS GRAD 0.93658

 POST HS 0.92025

 COL GRAD 0.92303

 PARED-? 0.93244

 S EAST 0.89914

 CENTRAL 0.91367

 WEST 0.89828

 PRIVATE 0.92836

 CATHOLIC 0.92022

 IEP-NO 0.93526

 LEP-NO 0.79947

 CHAP1-N 0.75390

 REDUCED 0.91147

 FREE 0.72822

 INFO NA 0.84735

 SCH REF 0.89453

 HL-SOME 0.84839

 HL-ALWAY 0.79850

 HL-? 0.73669

 TVLIN-0 0.98029

 TV-QUAD 0.98000

 HW-NO 0.96073

 HW-YES 0.96470

 HWLIN-0 0.98137

 HWQUAD-0 0.97281

 1-2 HRS 0.91902

 3-4 HRS 0.94019

 5-6 HRS 0.95904

 7-8 HRS 0.96322

 9-10 HRS 0.96691

 > 10 HRS 0.94955

 B009101M 0.77222

 HITEM=3 0.86009

 HITEM=4 0.81060

 MISS-2< 0.92655

 USA >5 0.90593

 USA 3-5 0.92725

 USA <3 0.92754

 USA-? 0.78635

 STGRD1-2 0.87161

 STGRD3> 0.84855

 SNRM-LIN 0.89875

 B009201B 0.85946

 B009201C 0.89060

 B009201D 0.84523

 B009201M 0.81101

 DIS@HOM2 0.86885

 DIS@HOM3 0.91087

 DIS@HOM4 0.78205

 1-2 WEEK 0.94751

 1-2 MTH 0.95215

 NEVER 0.94409

 NO COMP 0.93048

 MISSING 0.86738

 G/R 22 0.91396

 G/R 23 0.90912

 G/R 24 0.96268

 G/T 22 0.69418

 G/T 23 0.66997

 G/T 24 0.77415

 G/T 25 0.73587

 G/P 22 0.91395

 G/P 23 0.88694

 G/P 24 0.84063

 G/P 25 0.84875

 G/S 22 0.92396

 G/S 23 0.90171

 R/T 24 0.86023

 R/T 25 0.89305

 R/T 31 0.87802

 R/T 32 0.89389

 R/T 33 0.86112

 R/T 34 0.86129

 R/T 35 0.88236

 R/T 41 0.88877

 R/T 42 0.91771

 R/T 43 0.92809

 R/T 44 0.93044

 R/T 45 0.95100

 R/P 24 0.89904

 R/P 25 0.88076

 R/P 31 0.90960

 R/P 32 0.91029

 R/P 33 0.89805

 R/P 34 0.89852

 R/P 35 0.89951

 R/P 41 0.89486

 R/P 42 0.97268

 R/P 43 0.96015

 R/P 44 0.96464

 R/P 45 0.96335

 R/S 31 0.95642

 R/S 32 0.96868

 R/S 33 0.96432

 R/S 41 0.94664

 R/S 42 0.96346

 R/S 43 0.97550

 T/P 25 0.78665

 T/P 31 0.83830

 T/P 32 0.78928

 T/P 33 0.76732

 T/P 34 0.76387

 T/P 35 0.78267

 T/P 41 0.76155

 T/P 42 0.75914

 T/P 43 0.79689

 T/P 44 0.80321

 T/P 45 0.83598

 T/P 51 0.81841

 T/P 52 0.79478

 T/P 53 0.81808

 T/P 54 0.79771

 T/P 55 0.80328

 T/S 32 0.95828

 T/S 33 0.93286

 T/S 41 0.94266

 T/S 42 0.93954

 T/S 43 0.95438

 T/S 51 0.94557

 T/S 52 0.97040

 T/S 53 0.96619

 P/S 32 0.95468

 P/S 33 0.92734

 P/S 41 0.94834

 P/S 42 0.90951

 P/S 43 0.94043

 P/S 51 0.92263

 P/S 52 0.94462

 P/S 53 0.92977

 B008901N 0.95010

 B008901M 0.64425

 SAFE 0.87260

 UNSAFE 0.90710

 VRUNSAFE 0.88994

 MISSING 0.84004

 MOMHOM-N 0.80557

 MOMHOM-? 0.79943

 DADHOM-N 0.83554

 DADHOM-? 0.82552

 K811001B 0.82197

 K811001C 0.85117

 K811001M 0.78778

 K811002B 0.83114

 K811002C 0.84120

 K811002M 0.74258

 K811003B 0.80011

 K811003C 0.79434

 K811003M 0.64436

 K811004B 0.80713
Table C-13 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

655

 K811004C 0.80115

 K811004M 0.67661

 K811005B 0.88234

 K811005C 0.86102

 K811005M 0.72124

 K811006B 0.94116

 K811006C 0.92431

 K811006M 0.77614

 K811007B 0.88338

 K811007C 0.87999

 K811007M 0.74157

 K811008B 0.84335

 K811008C 0.83097

 K811008M 0.83133

 K811101M 0.85314

 K811102M 0.75917

 K811103M 0.74925

 K811104M 0.76326

 K811105M 0.74473

 K811106M 0.70566

 K811107M 0.74751

 K811108M 0.92666

 K811201B 0.88957

 K811201C 0.91390

 K811201D 0.92028

 K811201E 0.93299

 K811201M 0.87365

 K811301B 0.92638

 K811301C 0.93837

 K811301D 0.94938

 K811301E 0.96340

 K811301F 0.96269

 K811301M 0.88178

 K811401N 0.84490

 K811401M 0.89888

 K811501N 0.75563

 K811501M 0.90556

 K811601B 0.90579

 K811601C 0.93331

 K811601D 0.86144

 K811601M 0.78641

 K811602B 0.93446

 K811602C 0.93522

 K811602D 0.91488

 K811602M 0.80978

 K811603B 0.94079

 K811603C 0.94416

 K811603D 0.91219

 K811603M 0.77565

 K811604B 0.93590

 K811604C 0.91399

 K811604D 0.91171

 K811604M 0.76089

 K811605B 0.94002

 K811605C 0.94156

 K811605D 0.92541

 K811605M 0.81252

 K811606B 0.92744

 K811606C 0.93016

 K811606D 0.90992

 K811606M 0.73070

 K811607B 0.91407

 K811607C 0.91450

 K811607D 0.90192

 K811607M 0.77303

 K811608B 0.91591

 K811608C 0.92996

 K811608D 0.90372

 K811608M 0.82830

 K811609B 0.89975

 K811609C 0.93252

 K811609D 0.88360

 K811609M 0.83978

 K811610B 0.93662

 K811610C 0.92396

 K811610D 0.92722

 K811610M 0.80885

 K811611B 0.94193

 K811611C 0.93725

 K811611D 0.93167

 K811611M 0.83030

 K811612B 0.94534

 K811612C 0.93793

 K811612D 0.91572

 K811612M 0.86702

 K811701B 0.90475

 K811701C 0.91033

 K811701D 0.84629

 K811701M 0.87126

 K811702B 0.94339

 K811702C 0.94654

 K811702D 0.94051

 K811702M 0.88966

 K811703B 0.93095

 K811703C 0.94496

 K811703D 0.92911

 K811703M 0.87002

 K811704B 0.89118

 K811704C 0.93560

 K811704D 0.89198

 K811704M 0.91996

 K811705B 0.93027

 K811705C 0.93831

 K811705D 0.90084

 K811705M 0.92413

 K811801B 0.92272

 K811801C 0.90026

 K811801M 0.89963

 K811901B 0.91691

 K811901C 0.90711

 K811901M 0.88535

 SM00101B 0.85426

 SM00101C 0.85860

 SM00101D 0.90776

 SM00101M 0.86142

 SM00201B 0.90458

 SM00201C 0.90363

 SM00201D 0.90657

 SM00201M 0.88572

 SM00301B 0.87162

 SM00301C 0.85440

 SM00301D 0.90224

 SM00301M 0.83821

 SM00401B 0.88524

 SM00401C 0.89177

 SM00401D 0.92217

 SM00401M 0.90223

 SM00501B 0.85850

 SM00501C 0.87820

 SM00501D 0.86234

 SM00501M 0.88279

 B009501N 0.93679

 B009501C 0.81293

 B009501M 0.85361

 B009502N 0.95137

 B009502C 0.84209

 B009502M 0.85508

 C030901B 0.89403

 C030901C 0.90221

 C030901M 0.97962

 C037101B 0.92384

 C037101C 0.92028

 C037101N 0.94104

 C037101M 0.94257

 C031212B 0.87125

 C031212M 0.96859

 C031205B 0.89767

 C031205C 0.90190

 C031205M 0.94785

 C031213M 0.95871

 C031214B 0.92465

 C031214C 0.93150

 C031214D 0.89904

 C031214E 0.91092

 C031214M 0.94845

 C031603N 0.89974

 C031603M 0.94317

 C031607N 0.89448

 C031607M 0.96400

 C031601N 0.87885

 C031601M 0.96929

 C031610N 0.91936

 C031610M 0.96212

 C031606N 0.90036

 C031606M 0.95365

 C035701N 0.89605

 C035701M 0.91746

 C035702N 0.90389
Table C-13 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

656

 C035702M 0.93199

 C035703N 0.88916

 C035703M 0.93495

 C037201M 0.95553

 C037202M 0.92137

 C037207M 0.94860

 C037204M 0.90422

 C037205M 0.89754

 C037206M 0.93927

 C037301M 0.96994

 C037302M 0.95988

 C037303M 0.94894

 C037304M 0.90594

 C037305M 0.95441

 C037401M 0.89597

 C037402M 0.90113

 C037403M 0.92514

 C037404M 0.89399

 C037405M 0.91465

 C037406M 0.87946

 C037501M 0.90453

 C037502M 0.89548

 C037503M 0.90802

 C037504M 0.92563

 C037505M 0.92290

 C037601M 0.93328

 C037602M 0.92709

 C037603M 0.95951

 C037604M 0.92443

 C037605M 0.95223

 C036601B 0.88504

 C036601C 0.93206

 C036601D 0.95477

 C036601M 0.95288

 C032207B 0.92922

 C032207N 0.91089

 C032207M 0.93632

 C032209B 0.89828

 C032209N 0.87527

 C032209M 0.98238

 C032210B 0.90543

 C032210N 0.91405

 C032210M 0.94207

 C032211B 0.89516

 C032211N 0.90357

 C032211M 0.99172

 C037701B 0.91761

 C037701C 0.90801

 C037701D 0.92439

 C037702B 0.91405

 C037702C 0.93501

 C037702D 0.93940

 C037703B 0.93418

 C037703C 0.94161

 C037703D 0.94797

 C037704B 0.92282

 C037704C 0.91613

 C037704D 0.90235

 C037704M 0.94786

 C037705B 0.92164

 C037705C 0.90676

 C037705D 0.89117

 C032402B 0.90070

 C032402C 0.94641

 C032402D 0.94378

 C032402M 0.95157

 C032401B 0.93303

 C032401C 0.94412

 C032401D 0.94804

 C032401M 0.98947

 C032404B 0.92340

 C032404C 0.95081

 C032404D 0.94137

 C032404M 0.96651

 C032406B 0.91596

 C032406C 0.90435

 C032407B 0.91061

 C032407C 0.95076

 C032407D 0.95552

 C032407M 0.98151

 C032408B 0.90945

 C032408C 0.94772

 C032408D 0.94886

 C032408M 0.96953

 C032409B 0.93628

 C032409C 0.93266

 C032409D 0.94444

 C032410C 0.93272

 C032411B 0.90528

 C032411C 0.92930

 C032412C 0.93921

 C032413B 0.88499

 C032413C 0.89413

 C032414B 0.92372

 C032414C 0.91823

 C032415B 0.93497

 C032415C 0.90676

 C032502B 0.90056

 C032502C 0.89916

 C032502D 0.93295

 C032503B 0.91273

 C032503C 0.93271

 C032505B 0.90764

 C032505C 0.90822

 C032505D 0.91176

 C032506B 0.89234

 C032506C 0.91856

 C033601B 0.91617

 C033601C 0.90915

 C033601D 0.91403

 C033601M 0.95663

 C036501B 0.91852

 C036501C 0.93032

 C036501M 0.97935

 C037801B 0.91450

 C037801C 0.89298

 C037801D 0.92829

 C037801E 0.91023

 C037801F 0.90574

 C037801G 0.90165

 C037801H 0.90540

 C037801M 0.93753

 C037901B 0.90514

 C037901C 0.90170

 C037901D 0.90960

 C038001B 0.89636

 C038001C 0.90399

 C038001D 0.90048

 C038001F 0.88251

 C038301N 0.90020

 C038801N 0.90155

 C038801M 0.97431

 C034101M 0.92230

 C034102M 0.89611

 C034103M 0.94551

 C034104M 0.91301

 C034105M 0.89582

 C034106M 0.91470

 C034107M 0.90153

 C034108M 0.90872

 C034109M 0.90109

 T055901B 0.87479

 T055901M 0.96866

 T056001B 0.85043

 T056001C 0.86327

 T056001D 0.86537

 T056001E 0.90289

 T056001F 0.86833

 T056001M 0.94320

 T040301B 0.90532

 T040301C 0.93146

 T040301D 0.93576

 T040301E 0.94781

 T040301M 0.98027

 T056101B 0.91259

 T056101C 0.93748

 T056101D 0.94770

 T056101E 0.96263

 T056101M 0.96167

 T056102B 0.95393

 T056102C 0.93711

 T056102D 0.94738

 T056102E 0.95662

 T056102M 0.94851

 T056201B 0.88672

 T056201C 0.87479

 T056201D 0.88203

 T056201E 0.84446
Table C-13 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

657

 T056201F 0.85825

 T056201M 0.91983

 T040501N 0.83560

 T040501C 0.90736

 T040501M 0.89047

 T040506N 0.94737

 T040506C 0.94707

 T040506M 0.94102

 T040504N 0.94207

 T040504C 0.92334

 T040504M 0.95367

 T040507N 0.94296

 T040507C 0.94279

 T040507M 0.94702

 T040508N 0.94706

 T040508C 0.93476

 T040508M 0.95486

 T040505N 0.88670

 T040505C 0.87901

 T040505M 0.88696

 T056301B 0.86576

 T056301C 0.93282

 T056301D 0.93167

 T056301E 0.88357

 T056301G 0.89492

 T056301M 0.95311

 T040701M 0.87840

 T040706M 0.86770

 T040707M 0.86427

 T040703M 0.89060

 T040704M 0.89093

 T040710M 0.88417

 T040711M 0.92342

 T040712M 0.91839

 T040713M 0.93532

 T040708M 0.86960

 T040709M 0.88466

 T040705M 0.88844

 T040801M 0.88669

 T040807M 0.86361

 T040808M 0.86571

 T040803M 0.89119

 T040804M 0.87594

 T040814M 0.87928

 T040815M 0.95483

 T040816M 0.94150

 T040817M 0.91605

 T040809M 0.86332

 T040810M 0.88005

 T040811M 0.87776

 T040812M 0.87941

 T040813M 0.86773

 T040805M 0.86343

 T040806M 0.87073

 T056401M 0.88847

 T056402M 0.87790

 T056403M 0.85589

 T056404M 0.86367

 T056405M 0.87848

 T056413M 0.86716

 T056414M 0.90389

 T056415M 0.90062

 T056416M 0.90838

 T056406M 0.87295

 T056407M 0.87562

 T056408M 0.87219

 T056409M 0.86749

 T056410M 0.87411

 T056411M 0.90319

 T056412M 0.89205

 T056501B 0.88647

 T056501C 0.88938

 T056501D 0.89411

 T056501E 0.89143

 T056501M 0.94306

 T058101B 0.88634

 T058101C 0.89152

 T058101D 0.87735

 T058101E 0.88157

 T058101M 0.96791

 T056601B 0.88791

 T056601C 0.87160

 T056601D 0.89490

 T056601E 0.89456

 T056601M 0.96538

 T058201B 0.88404

 T058201C 0.85907

 T058201D 0.88962

 T058201E 0.86467

 T058201M 0.96544

 T056701M 0.86285

 T056702M 0.87917

 T056703M 0.85590

 T056704M 0.85823

 T056705M 0.84873

 T056706M 0.85463

 T056707M 0.85042

 T056708M 0.86160

 T056709M 0.86150

 T056710M 0.86055

 T056711M 0.87227

 T056712M 0.86345

 T056713M 0.85920

 T041201B 0.90893

 T041201C 0.92565

 T041201D 0.88467

 T041201M 0.98547

 T041302N 0.93022

 T041302M 0.90144

 T041303N 0.91305

 T041303M 0.91998

 T056801B 0.86554

 T056801C 0.89230

 T056801D 0.92794

 T056801E 0.89607

 T056801F 0.88949

 T056801M 0.94958

 T060201B 0.92000

 T060201C 0.94579

 T060201D 0.92003

 T060201E 0.91478

 T060201F 0.90732

 T060201M 0.95858

 T060301M 0.87421

 T060311M 0.88509

 T060321M 0.87497

 T060331M 0.88300

 T060341M 0.86623

 T060302M 0.90604

 T060312M 0.85498

 T060322M 0.91112

 T060332M 0.88119

 T060342M 0.88255

 T060303M 0.88005

 T060313M 0.83306

 T060323M 0.92645

 T060333M 0.88680

 T060343M 0.88312

 T060304M 0.89420

 T060314M 0.86209

 T060324M 0.93375

 T060334M 0.88551

 T060344M 0.89704

 T060305M 0.92229

 T060315M 0.88554

 T060325M 0.90188

 T060335M 0.89047

 T060345M 0.87521

 T060306M 0.87953

 T060316M 0.85377

 T060326M 0.88992

 T060336M 0.84977

 T060346M 0.85970

 T060307M 0.88846

 T060317M 0.96011

 T060327M 0.95994

 T060337M 0.90362

 T060347M 0.90776

 T060401M 0.84814

 T060402M 0.86197

 T060403M 0.86825

 T060404M 0.86779

 T060405M 0.86266

 T060501N 0.85967

 T060501M 0.93416

 T060601B 0.87529

 T060601C 0.88302

 T060601D 0.89945
Table C-13 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

658

 T060601M 0.98632

 T060602B 0.89151

 T060602C 0.93844

 T060602D 0.90940

 T060602M 0.94668

 T060603B 0.90724

 T060603C 0.93143

 T060603D 0.90517

 T060603M 0.98055

 T060604B 0.89483

 T060604C 0.92361

 T060604D 0.88300

 T060604M 0.94484

 T060605B 0.88991

 T060605C 0.92415

 T060605D 0.93407

 T060605M 0.98883

 T060606B 0.87899

 T060606C 0.94199

 T060606D 0.94163

 T060606M 0.97172

 T060607B 0.91855

 T060607C 0.91617

 T060607D 0.86690

 T060607M 0.96799

 T060608B 0.91945

 T060608C 0.92808

 T060608D 0.88940

 T060608M 0.97707

 T060609B 0.91789

 T060609C 0.95337

 T060609D 0.90651

 T060609M 0.98558

 T060610B 0.89244

 T060610C 0.93467

 T060610D 0.90222

 T060610M 0.96781

 T060611B 0.88331

 T060611C 0.90780

 T060611D 0.93529

 T060611M 0.97020

 T060701B 0.88527

 T060701C 0.86155

 T060701D 0.89594

 T060702B 0.92325

 T060702C 0.93876

 T060702D 0.88673

 T060703B 0.89777

 T060703C 0.95750

 T060703D 0.92713

 T060704B 0.89563

 T060704C 0.92015

 T060704D 0.95093

 T060705B 0.88473

 T060705C 0.92378

 T060705D 0.94478

 T060801B 0.92551

 T060801C 0.93240

 T060801M 0.98743

 T060901B 0.93158

 T060901C 0.93591

 T061001N 0.91202

 T061001M 0.98196

 T061101B 0.88422

 T061101C 0.86186

 T061101M 0.97686

 T061102B 0.85567

 T061102C 0.96220

 T061102M 0.98334

 T061103B 0.87809

 T061103C 0.88451

 T061103M 0.97978

 T061104B 0.89582

 T061104C 0.89609

 T061104M 0.97552

 T061105B 0.89677

 T061105C 0.90152

 T061105M 0.95800

 T061106B 0.91773

 T061106C 0.91940

 T061107B 0.85928

 T061107C 0.88080

 T061107M 0.95604

 T061108B 0.90244

 T061108C 0.91920

 T061108M 0.96316

 T061109B 0.90216

 T061109C 0.91645

 T061109M 0.96285

 T061201N 0.85080

 T061201M 0.96119

 T061301B 0.91510

 T061301C 0.90641

 T061301D 0.88445

 T061301E 0.90907

 T061301M 0.97112

 T061302B 0.88943

 T061302C 0.89580

 T061302D 0.86364

 T061302E 0.86412

 T061302M 0.97288

 T061303B 0.90705

 T061303C 0.92973

 T061303D 0.91196

 T061303E 0.87885

 T061304B 0.89552

 T061304C 0.91273

 T061304D 0.90015

 T061304E 0.90184

 T061305B 0.89481

 T061305C 0.90307

 T061305D 0.88751

 T061305E 0.94363

 T061306B 0.88254

 T061306C 0.90312

 T061306D 0.90891

 T061306E 0.93034

 T061307B 0.88521

 T061307C 0.86756

 T061307D 0.90171

 T061307E 0.91337

 T061307M 0.96368

 T061308B 0.89370

 T061308C 0.88299

 T061308D 0.87586

 T061308E 0.91111

 T061308M 0.97867

 T061309B 0.88476

 T061309C 0.88316

 T061309D 0.87785

 T061309E 0.89188

 T061309M 0.97721

 T061310B 0.88162

 T061310C 0.85340

 T061310D 0.86935

 T061310E 0.89975

 T061310M 0.97042

 T061401B 0.89801

 T061401C 0.91559

 T061401D 0.89104

 T061401M 0.97890

 T061501B 0.90299

 T061501C 0.90394

 T061501D 0.88877

 T061501E 0.88255

 T061501F 0.88173

 T061501M 0.88715

 T061601M 0.88212

 T061611M 0.87828

 T061621M 0.86907

 T061631M 0.86113

 T061641M 0.87562

 T061651M 0.89760

 T061701C 0.88626

 T061701E 0.88636

 T061701M 0.94247

 T061801B 0.87449

 T061801C 0.87469

 T061801D 0.85703

 T061801E 0.88894

 T061801M 0.84784

 T061901M 0.95569

 T062001B 0.88898

 T062001C 0.85320

 T062001M 0.94992

 T062002B 0.89213

 T062002C 0.88992

 T062002M 0.95118
Table C-13 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 4

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

659

 T062003B 0.91950

 T062003C 0.89596

 T062003D 0.90014

 T062003M 0.93908

 T062101B 0.87928

 T062101C 0.87528

 T062101D 0.89825

 T062101M 0.95016

 T062201M 0.86708

 T062202M 0.86001

 T062203M 0.84448

 T062204M 0.86846

 T062205M 0.85517

 T062206M 0.86636

 T062207M 0.86048

 T062208M 0.86099

 T062209M 0.86118

 T062210M 0.84056

 T062301B 0.91500

 T062301C 0.89514

 T062301D 0.89244

 T062301E 0.86790

 T062401B 0.86494

 T062401C 0.87876

 T062401D 0.85596

 T062401M 0.90603

 NATLUNCH 0.88877

 NATLUNCL 0.94512

 REMREADL 0.94139

 REMMATHL 0.94325

 NATLUN4L 0.94128

 REMREA4L 0.92554

 REMMAT4L 0.91214
Table C-14

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

660

 FEMALE 0.91614

 BLACK 0.92990

 HISPANIC 0.96684

 ASIAN 0.94591

 MEXICAN 0.92894

 PUER RIC 0.93935

 CUBN,OTH 0.93707

 HISP-? 0.91996

 NON MSA 0.94360

 MID CTY5 0.93750

 FR/BTWN5 0.95233

 SML TWN5 0.96189

 RURAL5 0.96124

 URBAN FR 0.92243

 MED CITY 0.91865

 SM PLACE 0.94745

 HS GRAD 0.94717

 POST HS 0.93099

 COL GRAD 0.93104

 PARED-? 0.94345

 S EAST 0.92674

 CENTRAL 0.91034

 WEST 0.92103

 PRIVATE 0.92875

 CATHOLIC 0.92907

 IEP-NO 0.93018

 LEP-NO 0.83100

 CHAP1-N 0.75194

 REDUCED 0.91650

 FREE 0.72745

 INFO NA 0.87544

 SCH REF 0.90281

 HL-SOME 0.85339

 HL-ALWAY 0.79809

 HL-? 0.80979

 TVLIN-0 0.98037

 TV-QUAD 0.98013

 HW-NO 0.95915

 HW-YES 0.96126

 HWLIN-0 0.95941

 HWQUAD-0 0.94546

 1-2 HRS 0.87252

 3-4 HRS 0.90544

 5-6 HRS 0.94207

 7-8 HRS 0.96277

 9-10 HRS 0.96101

 > 10 HRS 0.94668

 B009101M 0.75282

 HITEM=3 0.88494

 HITEM=4 0.85134

 MISS-2< 0.91161

 1-2 GRDS 0.94676

 3-5 GRDS 0.92313

 > 5 GRDS 0.91742

 MISSING 0.75547

 B010101B 0.92025

 B010101C 0.93096

 B010101D 0.91930

 B010101E 0.92367

 B010101F 0.93928

 B010101G 0.95512

 B010101H 0.93420

 B010101M 0.87041

 USA >5 0.89993

 USA 3-5 0.86697

 USA <3 0.88452

 USA-? 0.79048

 SNRM-LIN 0.90986

 DIS@HOM2 0.89041

 DIS@HOM3 0.92742

 DIS@HOM4 0.81911

 1-2 WEEK 0.93788

 1-2 MTH 0.95331

 NEVER 0.94437

 NO COMP 0.92631

 MISSING 0.85984

 LIFESCI 0.92125

 PHYSSCI 0.93355

 EATHSCI 0.92607

 GEN SCI 0.93232

 INTESCI 0.92713

 MISSING 0.92856

 G/R 22 0.89847

 G/R 23 0.90431

 G/R 24 0.95406

 G/T 22 0.72773

 G/T 23 0.69148

 G/T 24 0.72904

 G/T 25 0.71284

 G/P 22 0.87625

 G/P 23 0.88131

 G/P 24 0.78673

 G/P 25 0.82195

 G/S 22 0.93547

 G/S 23 0.91639

 R/T 24 0.88414

 R/T 25 0.86662

 R/T 31 0.88462

 R/T 32 0.89752

 R/T 33 0.86054

 R/T 34 0.87969

 R/T 35 0.89570

 R/T 41 0.87819

 R/T 42 0.90709

 R/T 43 0.92014

 R/T 44 0.93513

 R/T 45 0.94157

 R/P 24 0.88239

 R/P 25 0.87230

 R/P 31 0.89516

 R/P 32 0.84568

 R/P 33 0.88235

 R/P 34 0.87741

 R/P 35 0.89102

 R/P 41 0.86287

 R/P 42 0.96401

 R/P 43 0.94772

 R/P 44 0.95160

 R/P 45 0.92827

 R/S 31 0.96624

 R/S 32 0.96493

 R/S 33 0.96959

 R/S 41 0.95100

 R/S 42 0.96508

 R/S 43 0.95516

 T/P 25 0.83367

 T/P 31 0.83020

 T/P 32 0.79386

 T/P 33 0.83018

 T/P 34 0.77702

 T/P 35 0.76449

 T/P 41 0.74285

 T/P 42 0.82335

 T/P 43 0.82211

 T/P 44 0.82045

 T/P 45 0.80729

 T/P 51 0.86164

 T/P 52 0.78413

 T/P 53 0.79276

 T/P 54 0.76487

 T/P 55 0.83446

 T/S 32 0.93520

 T/S 33 0.93512

 T/S 41 0.94594

 T/S 42 0.93812

 T/S 43 0.95391

 T/S 51 0.96133

 T/S 52 0.94122

 T/S 53 0.95970

 P/S 32 0.95706

 P/S 33 0.94346

 P/S 41 0.93082

 P/S 42 0.91747

 P/S 43 0.92843

 P/S 51 0.91823

 P/S 52 0.95568

 P/S 53 0.95059

 G/ 22 0.91993

 G/ 23 0.88869

 G/ 24 0.89713

 G/ 25 0.92382

 G/ 26 0.92393

 G/ 27 0.89480

 R/ 24 0.88571

 R/ 25 0.88127
Table C-14 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

661

 R/ 26 0.89114

 R/ 27 0.89301

 R/ 31 0.87985

 R/ 32 0.88022

 R/ 33 0.88771

 R/ 34 0.88885

 R/ 35 0.89104

 R/ 36 0.89094

 R/ 37 0.88799

 R/ 41 0.88203

 R/ 42 0.95859

 R/ 43 0.94254

 R/ 44 0.94717

 R/ 45 0.94641

 R/ 46 0.94642

 R/ 47 0.95513

 P/ 25 0.77627

 P/ 26 0.72857

 P/ 27 0.77004

 P/ 31 0.77586

 P/ 32 0.76597

 P/ 33 0.77922

 P/ 34 0.79899

 P/ 35 0.76460

 P/ 36 0.80671

 P/ 37 0.81010

 P/ 41 0.80712

 P/ 42 0.82500

 P/ 43 0.75814

 P/ 44 0.73239

 P/ 45 0.75056

 P/ 46 0.72796

 P/ 47 0.77206

 P/ 51 0.76185

 P/ 52 0.81723

 P/ 53 0.81640

 P/ 54 0.81093

 P/ 55 0.82528

 P/ 56 0.82416

 P/ 57 0.77089

 T/ 25 0.79324

 T/ 26 0.82756

 T/ 27 0.82149

 T/ 31 0.77091

 T/ 32 0.77801

 T/ 33 0.76151

 T/ 34 0.75718

 T/ 35 0.77436

 T/ 36 0.79196

 T/ 37 0.73520

 T/ 41 0.72689

 T/ 42 0.75406

 T/ 43 0.80738

 T/ 44 0.83782

 T/ 45 0.85254

 T/ 46 0.79455

 T/ 47 0.77813

 T/ 51 0.82041

 T/ 52 0.80377

 T/ 53 0.83346

 T/ 54 0.82488

 T/ 55 0.77455

 T/ 56 0.75942

 T/ 57 0.76203

 S/ 23 0.95122

 S/ 24 0.92274

 S/ 25 0.94121

 S/ 26 0.93674

 S/ 27 0.93221

 S/ 31 0.93462

 S/ 32 0.92677

 S/ 33 0.91612

 S/ 34 0.92593

 S/ 35 0.91833

 S/ 36 0.92749

 S/ 37 0.92679

 B008901N 0.91402

 B008901M 0.66302

 SAFE 0.82633

 UNSAFE 0.87104

 VRUNSAFE 0.88342

 MISSING 0.87016

 MOMHOM-N 0.82638

 MOMHOM-? 0.89993

 DADHOM-N 0.83372

 DADHOM-? 0.88324

 K811001B 0.86472

 K811001C 0.86809

 K811001M 0.90216

 K811002B 0.87834

 K811002C 0.83391

 K811002M 0.87099

 K811003B 0.80714

 K811003C 0.79994

 K811003M 0.83722

 K811004B 0.83708

 K811004C 0.78722

 K811004M 0.83116

 K811005B 0.88820

 K811005C 0.86514

 K811005M 0.87145

 K811006B 0.87376

 K811006C 0.82705

 K811006M 0.85432

 K811007B 0.86851

 K811007C 0.87603

 K811007M 0.81372

 K811008B 0.84769

 K811008C 0.81309

 K811008M 0.88759

 K811101M 0.72975

 K811102M 0.71801

 K811103M 0.70933

 K811104M 0.75621

 K811105M 0.68818

 K811106M 0.71854

 K811107M 0.69382

 K811108M 0.93086

 K811201B 0.84019

 K811201C 0.90405

 K811201D 0.92124

 K811201E 0.80101

 K811201M 0.79853

 K811401N 0.84590

 K811401M 0.88161

 K811501N 0.78834

 K811501M 0.89711

 K811601B 0.89197

 K811601C 0.91423

 K811601D 0.85585

 K811601M 0.86567

 K811602B 0.94775

 K811602C 0.95656

 K811602D 0.94647

 K811602M 0.85469

 K811603B 0.93454

 K811603C 0.94630

 K811603D 0.92746

 K811603M 0.72829

 K811604B 0.91165

 K811604C 0.91959

 K811604D 0.90370

 K811604M 0.80173

 K811605B 0.92286

 K811605C 0.96265

 K811605D 0.95109

 K811605M 0.80892

 K811606B 0.92998

 K811606C 0.95950

 K811606D 0.93549

 K811606M 0.77265

 K811609B 0.88827

 K811609C 0.93241

 K811609D 0.91500

 K811609M 0.78244

 K811610B 0.91748

 K811610C 0.92316

 K811610D 0.87468

 K811610M 0.80046

 K811611B 0.93837

 K811611C 0.96312

 K811611D 0.95086

 K811611M 0.85240

 K811612B 0.95105

 K811612C 0.95529

 K811612D 0.94703

 K811612M 0.84598

 K811701B 0.92557
Table C-14 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

662

 K811701C 0.92739

 K811701D 0.81938

 K811701M 0.91797

 K811702B 0.88975

 K811702C 0.93695

 K811702D 0.91557

 K811702M 0.88220

 K811703B 0.91571

 K811703C 0.95734

 K811703D 0.91535

 K811703M 0.91578

 K811704B 0.84774

 K811704C 0.92386

 K811704D 0.88198

 K811704M 0.91560

 K811705B 0.88071

 K811705C 0.95244

 K811705D 0.91098

 K811705M 0.91713

 K811801B 0.93819

 K811801C 0.94240

 K811801M 0.87096

 K811901B 0.93512

 K811901C 0.92890

 K811901M 0.87312

 SM00101B 0.90525

 SM00101C 0.91770

 SM00101D 0.93306

 SM00101M 0.95285

 SM00201B 0.93476

 SM00201C 0.93711

 SM00201D 0.93611

 SM00201M 0.95767

 SM00301B 0.90366

 SM00301C 0.90169

 SM00301D 0.90118

 SM00301M 0.94894

 SM00401B 0.88398

 SM00401C 0.90140

 SM00401D 0.87922

 SM00401M 0.95275

 SM00501B 0.85855

 SM00501C 0.87566

 SM00501D 0.86682

 SM00501M 0.95177

 B009701B 0.90354

 B009701C 0.89062

 B009701D 0.93025

 B009701E 0.94423

 B009701F 0.88957

 B009701M 0.80372

 B009801B 0.94022

 B009801C 0.96009

 B009801D 0.95302

 B009801E 0.93544

 B009801M 0.75458

 B009601B 0.87602

 B009601N 0.86451

 B009601D 0.83509

 B009601E 0.87279

 B009602B 0.93300

 B009602N 0.93749

 B009602D 0.84613

 B009602M 0.85463

 K812101N 0.88273

 K812101M 0.68304

 K812201B 0.90121

 K812201C 0.97806

 K812201D 0.97145

 K812201E 0.96189

 K812201F 0.95603

 K812201G 0.94666

 K812201M 0.86237

 K811613B 0.84497

 K811613C 0.88635

 K811613D 0.81316

 K811613M 0.80864

 K811614B 0.84704

 K811614C 0.90033

 K811614D 0.82387

 K811614M 0.84082

 K811615B 0.95950

 K811615C 0.96202

 K811615D 0.95256

 K811615M 0.89083

 C031603N 0.90975

 C031603M 0.98092

 C031607N 0.92014

 C031607M 0.97281

 C031610N 0.93440

 C031610M 0.96670

 C031606N 0.91719

 C031606M 0.95685

 C035701N 0.91734

 C035701M 0.95147

 C035702N 0.90181

 C035702M 0.96955

 C035703N 0.90879

 C035703M 0.95440

 C037201M 0.92725

 C037202M 0.94087

 C037204M 0.91201

 C037205M 0.91814

 C037206M 0.94486

 C037301M 0.96584

 C037302M 0.95274

 C037304M 0.92484

 C037305M 0.93716

 C036601B 0.88832

 C036601C 0.93494

 C036601D 0.95643

 C036601M 0.98130

 C032207B 0.92927

 C032207N 0.94546

 C032207M 0.97639

 C032209B 0.92106

 C032209N 0.91132

 C032209M 0.98314

 C032210B 0.92488

 C032210N 0.92843

 C032210M 0.97860

 C032211B 0.91891

 C032211N 0.92395

 C032211M 0.98808

 C037701B 0.90725

 C037701C 0.93778

 C037701D 0.89634

 C037701M 0.97900

 C037702B 0.94639

 C037702C 0.93730

 C037702D 0.94056

 C037702M 0.98515

 C037703B 0.92938

 C037703C 0.93699

 C037703D 0.94122

 C037704B 0.91366

 C037704C 0.92668

 C037704D 0.94508

 C037705B 0.91635

 C037705C 0.92357

 C037705D 0.90238

 C032402B 0.93579

 C032402C 0.94391

 C032402D 0.94077

 C032401B 0.92283

 C032401C 0.94059

 C032401D 0.93269

 C032404B 0.92135

 C032404C 0.94445

 C032404D 0.92768

 C032406B 0.90498

 C032406C 0.94860

 C032406D 0.95961

 C032407B 0.91251

 C032407C 0.92244

 C032408B 0.90869

 C032408C 0.94606

 C032408D 0.95375

 C032409B 0.93567

 C032409C 0.95068

 C032409D 0.91318

 C032410B 0.95175

 C032410C 0.96093

 C032410D 0.96021

 C032411B 0.95345

 C032411C 0.95621

 C032411D 0.95241

 C032412B 0.93736
Table C-14 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

663

 C032412C 0.94614

 C032413B 0.91707

 C032413C 0.94331

 C032413D 0.95711

 C032414B 0.94610

 C032414C 0.95588

 C032414D 0.91607

 C032415B 0.94094

 C032415C 0.92816

 C032502B 0.91340

 C032502C 0.93605

 C032503B 0.93127

 C032503C 0.93640

 C032505B 0.92988

 C032505C 0.93405

 C032506B 0.93706

 C032506C 0.90986

 C032506D 0.93664

 C033601B 0.93383

 C033601C 0.94904

 C033601D 0.91813

 C036501B 0.93041

 C036501C 0.89186

 C037801B 0.92049

 C037801C 0.94056

 C037801D 0.92005

 C037801E 0.89936

 C037801F 0.90621

 C037801G 0.89982

 C037801H 0.97713

 C038001B 0.92061

 C038001C 0.93432

 C038001D 0.91998

 C038001E 0.92972

 C038001F 0.95733

 C038301N 0.89979

 C038301M 0.97806

 C038801N 0.90879

 C038801M 0.96647

 C034101M 0.92820

 C034102M 0.87312

 C034103M 0.91047

 C034104M 0.90529

 C034105M 0.92680

 C034106M 0.91053

 C034107M 0.90289

 C034108M 0.92767

 C034109M 0.90994

 T055901B 0.88802

 T055901M 0.98465

 T056001B 0.89223

 T056001C 0.86428

 T056001D 0.88436

 T056001E 0.87119

 T056001M 0.96663

 T040301B 0.88967

 T040301C 0.91439

 T040301D 0.92707

 T040301E 0.93078

 T040301M 0.98985

 T056201B 0.92043

 T056201C 0.90183

 T056201D 0.88880

 T056201E 0.88756

 T056201F 0.90292

 T056201M 0.96554

 T040501N 0.89775

 T040501C 0.89466

 T040501M 0.92269

 T040507N 0.93038

 T040507C 0.90690

 T040507M 0.94888

 T040508N 0.89248

 T040508C 0.89449

 T040508M 0.91750

 T040505N 0.89959

 T040505C 0.91096

 T040505M 0.90737

 T056301C 0.95510

 T056301D 0.94411

 T056301E 0.93838

 T056301F 0.88421

 T056301G 0.93448

 T040701M 0.89623

 T040706M 0.90710

 T040707M 0.89193

 T040710M 0.89837

 T040711M 0.89568

 T040712M 0.89824

 T040713M 0.91887

 T040708M 0.87922

 T040709M 0.89708

 T040705M 0.88932

 T040801M 0.87570

 T040807M 0.89148

 T040808M 0.88910

 T040814M 0.88568

 T040815M 0.90948

 T040816M 0.89814

 T040817M 0.91779

 T040809M 0.87783

 T040810M 0.91710

 T040811M 0.89671

 T040812M 0.88915

 T040813M 0.91308

 T040805M 0.88688

 T040806M 0.90519

 T056401M 0.90580

 T056402M 0.90364

 T056403M 0.91130

 T056405M 0.89011

 T056413M 0.90415

 T056414M 0.89906

 T056415M 0.88978

 T056406M 0.90689

 T056407M 0.83465

 T056408M 0.89545

 T056409M 0.89098

 T056410M 0.89696

 T056411M 0.91324

 T056412M 0.91635

 T056701M 0.90719

 T056702M 0.90765

 T056703M 0.88362

 T056704M 0.89440

 T056705M 0.88548

 T056706M 0.89563

 T056707M 0.89044

 T056708M 0.89673

 T056709M 0.89234

 T056710M 0.89310

 T056711M 0.89591

 T056712M 0.88572

 T056713M 0.90095

 T041201B 0.92948

 T041201C 0.92377

 T041201D 0.90187

 T041201M 0.97482

 T056801B 0.90269

 T056801C 0.89685

 T056801D 0.93572

 T056801E 0.92502

 T056801F 0.92079

 T056801M 0.97882

 T060301M 0.89871

 T060311M 0.88713

 T060321M 0.91015

 T060331M 0.90785

 T060341M 0.89674

 T060302M 0.91440

 T060312M 0.88867

 T060322M 0.90793

 T060332M 0.91563

 T060342M 0.89156

 T060303M 0.90771

 T060313M 0.91891

 T060323M 0.91814

 T060333M 0.93180

 T060343M 0.88995

 T060304M 0.90672

 T060314M 0.88527

 T060324M 0.92512

 T060334M 0.89411

 T060344M 0.87520

 T060305M 0.89475

 T060315M 0.91378

 T060325M 0.90676

 T060335M 0.89944
Table C-14 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

664

 T060345M 0.90127

 T060306M 0.91021

 T060316M 0.89986

 T060326M 0.89358

 T060336M 0.90101

 T060346M 0.88824

 T060307M 0.93028

 T060317M 0.95089

 T060327M 0.95911

 T060337M 0.93385

 T060347M 0.94106

 T060401M 0.89735

 T060402M 0.90556

 T060403M 0.90690

 T060404M 0.90239

 T060405M 0.88399

 T060501N 0.88255

 T060501M 0.96558

 T060601B 0.89817

 T060601C 0.90725

 T060601D 0.91362

 T060601M 0.98229

 T060602B 0.91039

 T060602C 0.94621

 T060602D 0.93498

 T060602M 0.97138

 T060603B 0.92096

 T060603C 0.92704

 T060603D 0.90280

 T060603M 0.99358

 T060604B 0.92797

 T060604C 0.91529

 T060604D 0.88505

 T060604M 0.98204

 T060605B 0.87990

 T060605C 0.94006

 T060605D 0.94583

 T060606B 0.91430

 T060606C 0.93240

 T060606D 0.93459

 T060606M 0.96897

 T060607B 0.93251

 T060607C 0.91844

 T060607D 0.89907

 T060607M 0.98348

 T060608B 0.92186

 T060608C 0.92566

 T060608D 0.89098

 T060608M 0.98679

 T060609B 0.94387

 T060609C 0.94775

 T060609D 0.90733

 T060609M 0.98983

 T060610B 0.91969

 T060610C 0.93803

 T060610D 0.92345

 T060611B 0.92131

 T060611C 0.94551

 T060611D 0.95990

 T060611M 0.97921

 T060701B 0.87986

 T060701C 0.91720

 T060701D 0.89237

 T060701M 0.98628

 T060702B 0.92521

 T060702C 0.92786

 T060702D 0.88744

 T060703B 0.93667

 T060703C 0.96062

 T060703D 0.92577

 T060704B 0.92499

 T060704C 0.94650

 T060704D 0.96097

 T060704M 0.98845

 T060705B 0.91501

 T060705C 0.91963

 T060705D 0.94025

 T060801B 0.92565

 T060801C 0.94636

 T060801M 0.98460

 T060901B 0.92840

 T060901C 0.94011

 T061001N 0.92429

 T061101B 0.90209

 T061101C 0.89796

 T061102B 0.90173

 T061102C 0.88588

 T061103B 0.89462

 T061103C 0.88815

 T061104B 0.89675

 T061104C 0.90901

 T061105B 0.90915

 T061105C 0.89809

 T061106B 0.89719

 T061106C 0.90879

 T061106M 0.99140

 T061107B 0.89008

 T061107C 0.88150

 T061108B 0.91325

 T061108C 0.91792

 T061109B 0.91804

 T061109C 0.93857

 T061109M 0.97808

 T061201N 0.89944

 T061201M 0.97288

 T061301B 0.92181

 T061301C 0.92146

 T061301D 0.88518

 T061301E 0.87885

 T061302B 0.89660

 T061302C 0.89068

 T061302D 0.89932

 T061302E 0.90358

 T061302M 0.98735

 T061303B 0.92772

 T061303C 0.91672

 T061303D 0.92102

 T061303E 0.89028

 T061303M 0.97771

 T061304B 0.90561

 T061304C 0.92062

 T061304D 0.91873

 T061304E 0.90233

 T061305B 0.91260

 T061305C 0.90751

 T061305D 0.91166

 T061305E 0.94292

 T061306B 0.90959

 T061306C 0.90314

 T061306D 0.91100

 T061306E 0.92821

 T061307B 0.91269

 T061307C 0.89753

 T061307D 0.92324

 T061307E 0.92166

 T061308B 0.90178

 T061308C 0.89947

 T061308D 0.87906

 T061308E 0.92257

 T061309B 0.88125

 T061309C 0.90314

 T061309D 0.88450

 T061309E 0.88346

 T061310B 0.91339

 T061310C 0.89561

 T061310D 0.88164

 T061310E 0.87982

 T061401B 0.93338

 T061401C 0.93784

 T061401D 0.87774

 T061501B 0.89898

 T061501C 0.87609

 T061501D 0.93697

 T061501E 0.91172

 T061501F 0.90437

 T061501M 0.92621

 T061601M 0.88164

 T061611M 0.90275

 T061621M 0.90213

 T061631M 0.88394

 T061641M 0.90154

 T061651M 0.91183

 T061701B 0.87908

 T061701C 0.87702

 T061701D 0.87003

 T061701E 0.86902

 T061701M 0.93701

 T061801B 0.89685
Table C-14 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 8

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

665

 T061801C 0.82298

 T061801D 0.82136

 T061801E 0.87751

 T061801M 0.88743

 T061901B 0.97672

 T061901C 0.89348

 T061901M 0.95575

 T062001B 0.92394

 T062001C 0.91700

 T062001D 0.90789

 T062001M 0.92574

 T062002B 0.91343

 T062002C 0.88801

 T062002D 0.89329

 T062002M 0.92914

 T062003B 0.91379

 T062003C 0.91518

 T062003D 0.89893

 T062003M 0.92391

 T062101B 0.88537

 T062101C 0.91298

 T062101D 0.91918

 T062101M 0.94411

 T062201M 0.88523

 T062202M 0.90225

 T062203M 0.87435

 T062204M 0.87554

 T062205M 0.91436

 T062206M 0.90197

 T062207M 0.90015

 T062208M 0.88440

 T062209M 0.87189

 T062210M 0.88852

 T062301B 0.87946

 T062301C 0.91774

 T062301D 0.92148

 T062301E 0.89088

 T062301M 0.96040

 T062401B 0.84979

 T062401C 0.87496

 T062401D 0.83843

 T062401E 0.87456

 T062401M 0.89555

 C034201B 0.93437

 C034201C 0.95508

 C034402N 0.91768

 C034401N 0.92105

 C034510D 0.88851

 C034514B 0.91214

 C039401M 0.90840

 C039501M 0.88955

 T062501E 0.93702

 T062601B 0.88934

 T062601C 0.90431

 T062801N 0.90888

 NATLUNCL 0.93080

 REMREADL 0.95036

 REMMATHL 0.93477

 NATLUN8L 0.95035

 REMREA8L 0.94179

 REMMAT8L 0.91575
Table C-15

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

666

 FEMALE 0.92370

 BLACK 0.91671

 HISPANIC 0.96148

 ASIAN 0.93361

 MEXICAN 0.92939

 PUER RIC 0.97662

 CUBN,OTH 0.94184

 HISP-? 0.92808

 NON MSA 0.91456

 MID CTY5 0.93434

 FR/BTWN5 0.94074

 SML TWN5 0.95555

 RURAL5 0.94797

 URBAN FR 0.90644

 MED CITY 0.85799

 SM PLACE 0.91587

 HS GRAD 0.93832

 POST HS 0.94431

 COL GRAD 0.94447

 PARED-? 0.95349

 S EAST 0.87537

 CENTRAL 0.87869

 WEST 0.89809

 PRIVATE 0.90600

 CATHOLIC 0.91113

 IEP-NO 0.96858

 LEP-NO 0.84055

 CHAP1-N 0.80994

 REDUCED 0.96889

 FREE 0.81843

 INFO NA 0.84872

 SCH REF 0.88282

 HL-SOME 0.87173

 HL-ALWAY 0.79104

 HL-? 0.74483

 TVLIN-0 0.97823

 TV-QUAD 0.97682

 HW-NO 0.97714

 HW-YES 0.97841

 HWLIN-0 0.95670

 HWQUAD-0 0.93911

 HITEM=3 0.89323

 HITEM=4 0.85631

 MISS-2< 0.95667

 USA >5 0.85566

 USA 3-5 0.85481

 USA <3 0.87402

 USA-? 0.85582

 SNRM-LIN 0.85074

 B007302B 0.93970

 B007302C 0.94259

 B007302D 0.92964

 B007302E 0.94782

 B007302F 0.96231

 B007302G 0.95909

 B007302H 0.93841

 B007302M 0.89368

 STGRD1-2 0.93694

 STGRD3-5 0.93985

 STGRD6> 0.93795

 DIS@HOM2 0.89580

 DIS@HOM3 0.92398

 DIS@HOM4 0.85025

 1-2 WEEK 0.95179

 1-2 MTH 0.95974

 NEVER 0.96102

 NO COMP 0.93694

 MISSING 0.86829

 PGS>5 0.83576

 PGS>10 0.83543

 ACADEMIC 0.74011

 VOC/TECH 0.88858

 OTHERPGM 0.93125

 HS PGM-? 0.87703

 SEMENG-^ 0.85785

 #ENG-LIN 0.86187

 SEMMAT-^ 0.89980

 #MAT-LIN 0.85762

 SEMSCI-^ 0.94353

 SEMSCI-^ 0.94391

 SEMSCI-^ 0.92937

 SEMSCI-^ 0.94424

 #SCI-LIN 0.94928

 SEMHIS-^ 0.87145

 #HIS-LIN 0.81282

 SEMFLG-^ 0.87461

 #FLG-LIN 0.73498

 SEMVOC-^ 0.83514

 #VOC-LIN 0.83192

 SEMART-^ 0.85524

 #ART-LIN 0.84026

 G/R 22 0.90404

 G/R 23 0.90937

 G/R 24 0.94198

 G/T 22 0.72103

 G/T 23 0.73591

 G/T 24 0.73911

 G/T 25 0.75204

 G/P 22 0.88244

 G/P 23 0.93367

 G/P 24 0.84716

 G/P 25 0.87283

 G/S 22 0.93957

 G/S 23 0.93157

 R/T 24 0.86645

 R/T 25 0.88815

 R/T 31 0.88006

 R/T 32 0.90237

 R/T 33 0.87748

 R/T 34 0.88682

 R/T 35 0.89641

 R/T 41 0.91799

 R/T 42 0.90707

 R/T 43 0.90569

 R/T 44 0.89637

 R/T 45 0.94571

 R/P 24 0.89406

 R/P 25 0.90413

 R/P 31 0.89725

 R/P 32 0.84060

 R/P 33 0.89209

 R/P 34 0.90020

 R/P 35 0.90331

 R/P 41 0.85040

 R/P 42 0.93317

 R/P 43 0.94337

 R/P 44 0.93201

 R/P 45 0.85683

 R/S 31 0.96840

 R/S 32 0.94832

 R/S 33 0.96108

 R/S 41 0.91717

 R/S 42 0.97077

 R/S 43 0.96034

 T/P 25 0.78555

 T/P 31 0.77884

 T/P 32 0.80087

 T/P 33 0.85271

 T/P 34 0.80212

 T/P 35 0.77866

 T/P 41 0.79150

 T/P 42 0.87999

 T/P 43 0.76779

 T/P 44 0.78170

 T/P 45 0.81952

 T/P 51 0.83260

 T/P 52 0.78216

 T/P 53 0.77941

 T/P 54 0.82295

 T/P 55 0.86631

 T/S 32 0.93111

 T/S 33 0.92985

 T/S 41 0.93703

 T/S 42 0.91521

 T/S 43 0.95788

 T/S 51 0.95006

 P/S 32 0.95860

 P/S 33 0.93211

 P/S 41 0.94838

 P/S 42 0.92213

 P/S 43 0.93854

 P/S 51 0.91858

 P/S 52 0.96626

 P/S 53 0.94966

 G/S 22 0.87359
Table C-15 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

667

 G/S 23 0.85199

 G/S 24 0.80521

 R/S 24 0.84972

 R/S 25 0.89377

 R/S 31 0.88767

 R/S 32 0.91324

 R/S 33 0.83528

 R/S 34 0.88995

 R/S 35 0.90079

 R/S 41 0.91950

 R/S 42 0.92783

 R/S 43 0.95621

 R/S 44 0.94445

 R/S 45 0.94598

 P/S 25 0.84160

 P/S 31 0.81919

 P/S 32 0.82187

 P/S 33 0.86127

 P/S 34 0.80452

 P/S 35 0.85549

 P/S 41 0.86807

 P/S 42 0.87586

 P/S 43 0.84197

 P/S 44 0.82647

 P/S 45 0.82360

 P/S 51 0.82931

 P/S 52 0.88706

 P/S 53 0.92145

 P/S 54 0.92535

 P/S 55 0.91986

 T/S 25 0.82508

 T/S 31 0.82858

 T/S 32 0.80646

 T/S 33 0.85515

 T/S 34 0.82833

 T/S 35 0.83691

 T/S 41 0.76720

 T/S 42 0.79785

 T/S 43 0.78394

 T/S 44 0.81266

 T/S 45 0.81070

 T/S 51 0.81065

 T/S 52 0.84731

 T/S 53 0.84975

 T/S 54 0.85432

 T/S 55 0.84521

 S/S 23 0.94833

 S/S 24 0.93453

 S/S 25 0.91953

 S/S 31 0.90401

 S/S 32 0.92777

 S/S 33 0.90733

 S/S 34 0.90341

 S/S 35 0.89551

 B008901N 0.96035

 B008901M 0.73193

 SAFE 0.83758

 UNSAFE 0.91634

 VRUNSAFE 0.89039

 MISSING 0.86977

 MOMHOM-N 0.79818

 MOMHOM-? 0.84249

 DADHOM-N 0.80065

 DADHOM-? 0.83309

 K811001B 0.87640

 K811001C 0.86949

 K811001M 0.96926

 K811002B 0.89254

 K811002C 0.85703

 K811002M 0.95790

 K811003B 0.81128

 K811003C 0.80332

 K811003M 0.91841

 K811004B 0.83191

 K811004C 0.79424

 K811004M 0.87589

 K811005B 0.89915

 K811005C 0.84462

 K811005M 0.93623

 K811006B 0.83533

 K811006C 0.80193

 K811006M 0.93241

 K811007B 0.88119

 K811007C 0.88794

 K811007M 0.92876

 K811008B 0.84982

 K811008C 0.81528

 K811008M 0.93534

 K811501N 0.83972

 K811501M 0.88107

 K811601B 0.91457

 K811601C 0.93836

 K811601D 0.86253

 K811601M 0.90577

 K811602B 0.96573

 K811602C 0.97587

 K811602D 0.96610

 K811602M 0.90736

 K811603B 0.95509

 K811603C 0.96497

 K811603D 0.94843

 K811603M 0.86568

 K811604B 0.86856

 K811604C 0.92230

 K811604D 0.82110

 K811604M 0.80114

 K811605B 0.96389

 K811605C 0.97974

 K811605D 0.97665

 K811605M 0.90457

 K811606B 0.95720

 K811606C 0.97895

 K811606D 0.96824

 K811606M 0.88656

 K811609B 0.88984

 K811609C 0.94882

 K811609D 0.90147

 K811609M 0.85823

 K811610B 0.90043

 K811610C 0.92427

 K811610D 0.79174

 K811610M 0.88371

 K811611B 0.97180

 K811611C 0.97730

 K811611D 0.97093

 K811611M 0.89673

 K811612B 0.94892

 K811612C 0.97547

 K811612D 0.96428

 K811612M 0.91000

 K811701B 0.96680

 K811701C 0.96952

 K811701D 0.81164

 K811701M 0.93645

 K811702B 0.87026

 K811702C 0.90906

 K811702D 0.87970

 K811702M 0.92286

 K811703B 0.92065

 K811703C 0.96463

 K811703D 0.89293

 K811703M 0.93180

 K811704B 0.86213

 K811704C 0.94034

 K811704D 0.88453

 K811704M 0.92997

 K811705B 0.92549

 K811705C 0.96879

 K811705D 0.94646

 K811705M 0.94342

 K811801B 0.95538

 K811801C 0.95294

 K811801M 0.91146

 K811901B 0.95389

 K811901C 0.94725

 K811901M 0.90236

 SM00101B 0.93541

 SM00101C 0.94607

 SM00101D 0.94310

 SM00101M 0.93669

 SM00201B 0.94988

 SM00201C 0.94372

 SM00201D 0.93606

 SM00201M 0.92801

 SM00301B 0.96646

 SM00301C 0.95724

 SM00301D 0.94827

 SM00301M 0.95431
Table C-15 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

668

 SM00401B 0.93849

 SM00401C 0.95563

 SM00401D 0.94106

 SM00401M 0.95862

 SM00501B 0.89908

 SM00501C 0.91708

 SM00501D 0.86213

 SM00501M 0.94235

 B009601B 0.96681

 B009601N 0.82588

 B009601D 0.82668

 B009601E 0.91039

 B009602B 0.97020

 B009602N 0.91670

 B009602D 0.81088

 B009602M 0.89009

 K812201B 0.95074

 K812201C 0.95852

 K812201D 0.96820

 K812201E 0.95870

 K812201F 0.96426

 K812201G 0.92004

 K812201M 0.76695

 K811613B 0.78967

 K811613C 0.81480

 K811613D 0.81977

 K811613M 0.90693

 K811614B 0.80434

 K811614C 0.83632

 K811614D 0.81649

 K811614M 0.92128

 K811615B 0.95715

 K811615C 0.96735

 K811615D 0.95802

 K811615M 0.90318

 B008301D 0.91047

 B008301M 0.85479

 B009901B 0.90169

 B009901C 0.88406

 B009901D 0.95840

 B009901E 0.96738

 B009901F 0.93242

 B009901M 0.82910

 B005501B 0.95315

 B005501C 0.95268

 B005501D 0.90958

 B005501E 0.96631

 B005501F 0.96104

 B005501M 0.89563

 OFFICE W 0.93957

 TECHNIC 0.96790

 PROT SRV 0.97745

 SALES 0.97623

 OWNER 0.96605

 SERV WRK 0.96570

 SKILL T 0.96972

 LABORER 0.97106

 OPERATOR 0.97540

 FARMER 0.97340

 HOMEMAKE 0.84191

 MANAGER 0.97479

 MILITARY 0.95518

 TEACHER 0.93503

 PROFESS1 0.94833

 PROFESS2 0.96191

 B011A01N 0.83488

 OFFICE 0.97489

 TECHNIC 0.96958

 PROT SRV 0.97817

 SALES 0.97251

 OWNER 0.93699

 SERV WRK 0.97771

 SKILL TR 0.95835

 LABORER 0.96193

 OPERATOR 0.96862

 FARMER 0.96916

 HOMEMAKE 0.92559

 MANAGER 0.95432

 MILITARY 0.97175

 TEACHER 0.96287

 PROFESS1 0.95270

 PROFESS2 0.95053

 B012A02N 0.80717

 K812301M 0.68403

 K812302M 0.72750

 K812303M 0.70247

 K812304M 0.77563

 K812305M 0.70288

 K812306M 0.67864

 K812307M 0.70809

 K812308M 0.73819

 K812309M 0.90994

 K812401N 0.87571

 K812401M 0.73201

 K812501B 0.93090

 K812501C 0.94349

 K812501D 0.87892

 K812501M 0.79450

 K812502B 0.95817

 K812502C 0.93269

 K812502D 0.92930

 K812502M 0.77542

 K812503B 0.83251

 K812503C 0.90384

 K812503D 0.89370

 K812503M 0.73458

 K812504B 0.96720

 K812504C 0.93237

 K812504D 0.90946

 K812504M 0.75624

 K812505B 0.93971

 K812505C 0.96015

 K812505D 0.91924

 K812505M 0.81853

 K812506B 0.93685

 K812506C 0.96164

 K812506D 0.91692

 K812506M 0.81541

 K812507B 0.95634

 K812507C 0.93825

 K812507D 0.92636

 K812507M 0.75786

 K812508B 0.95876

 K812508C 0.94943

 K812508D 0.92637

 K812508M 0.92681

 K812509B 0.96423

 K812509C 0.94068

 K812509D 0.93203

 K812509M 0.92891

 K812510B 0.95701

 K812510C 0.97071

 K812510D 0.90151

 K812510M 0.81960

 K812601N 0.76508

 K812601M 0.85884

 K812701N 0.94118

 K812701M 0.72392

 K811616B 0.88195

 K811616C 0.95726

 K811616D 0.86971

 K811616M 0.90145

 C035701N 0.87089

 C035701M 0.93561

 C035702N 0.88353

 C035702M 0.96946

 C035703N 0.88092

 C035703M 0.90409

 C037201M 0.92494

 C037202M 0.91093

 C037204M 0.88947

 C037205M 0.88924

 C037206M 0.88132

 C037301M 0.97333

 C037302M 0.96547

 C037304M 0.86899

 C037305M 0.91604

 C036601B 0.89973

 C036601C 0.90271

 C036601D 0.94276

 C036601M 0.97157

 C032207B 0.92786

 C032207N 0.92407

 C032207M 0.92511

 C032209B 0.91965

 C032209N 0.92544

 C032209M 0.95453

 C032210B 0.94360
Table C-15 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

669

 C032210N 0.95198

 C032210M 0.93908

 C032211B 0.88500

 C032211N 0.89029

 C032211M 0.95735

 C037701B 0.86986

 C037701C 0.90194

 C037701D 0.88184

 C037701M 0.96339

 C037702B 0.89756

 C037702C 0.91838

 C037702D 0.90530

 C037702M 0.98220

 C037703B 0.89745

 C037703C 0.92208

 C037703D 0.90642

 C037704B 0.88329

 C037704C 0.87431

 C037704D 0.90198

 C037704M 0.96324

 C037705B 0.89701

 C037705C 0.89818

 C037705D 0.90113

 C032402B 0.93144

 C032402C 0.93392

 C032402D 0.91518

 C032402M 0.99407

 C032401B 0.91072

 C032401C 0.94322

 C032401D 0.89027

 C032404B 0.89397

 C032404C 0.94549

 C032404D 0.91661

 C032406B 0.89388

 C032406C 0.95032

 C032406D 0.94029

 C032407B 0.88643

 C032407C 0.86886

 C032408B 0.89443

 C032408C 0.96359

 C032408D 0.95832

 C032409B 0.92367

 C032409C 0.93134

 C032409D 0.91390

 C032410B 0.91959

 C032410C 0.93374

 C032410D 0.90272

 C032410M 0.98139

 C032411B 0.93842

 C032411C 0.94358

 C032411D 0.88421

 C032412B 0.93187

 C032412C 0.93695

 C032412D 0.91771

 C032413B 0.90251

 C032413C 0.94961

 C032413D 0.95037

 C032414B 0.91331

 C032414C 0.94297

 C032414D 0.88909

 C032415B 0.92753

 C032415C 0.95380

 C032415D 0.92709

 C032502B 0.90890

 C032502C 0.90013

 C032502D 0.90145

 C032502M 0.96764

 C032503B 0.89695

 C032503C 0.92592

 C032505B 0.90278

 C032505C 0.90078

 C032505D 0.91001

 C032506B 0.89881

 C032506C 0.87640

 C032506D 0.86800

 C033601B 0.92369

 C033601C 0.93328

 C033601D 0.91139

 C033601M 0.97710

 C036501B 0.90250

 C036501C 0.89498

 C036501M 0.97172

 C037801B 0.89545

 C037801C 0.90352

 C037801D 0.90447

 C037801E 0.91240

 C037801F 0.90460

 C037801G 0.89732

 C038001B 0.85284

 C038001C 0.87525

 C038001D 0.91860

 C038001M 0.98010

 C038301N 0.87718

 C038301M 0.91358

 C038801N 0.90183

 C038801M 0.89507

 C034101M 0.88584

 C034102M 0.90191

 C034103M 0.87030

 C034104M 0.88278

 C034105M 0.87628

 C034106M 0.87212

 C034107M 0.88182

 C034108M 0.90287

 C034109M 0.89353

 C037203M 0.90966

 C037306M 0.96368

 C035002N 0.93660

 C035002M 0.97875

 C035003N 0.92974

 C035003M 0.97412

 C035006N 0.87744

 C035006M 0.98034

 C035007N 0.88632

 C035007M 0.95328

 C040201C 0.88300

 C040201D 0.94551

 C040201E 0.94396

 C040201F 0.89375

 C040202C 0.89626

 C040202D 0.95294

 C040202E 0.93950

 C040202F 0.87251

 C040203E 0.93462

 C040203F 0.94699

 C040204B 0.87484

 C040204C 0.88743

 C040204D 0.88296

 C040204F 0.91571

 C040204M 0.95036

 C040301M 0.92424

 C040302M 0.88503

 C040303M 0.88238

 C040304M 0.91496

 C040305M 0.86109

 C040306M 0.89956

 C040307M 0.84836

 C040308M 0.88209

 C040309M 0.86589

 C040310M 0.88911

 C040311M 0.90474

 C040401N 0.94208

 C040402N 0.92615

 C040402M 0.94019

 C040403N 0.94880

 C040404N 0.93593

 C040501M 0.88498

 C040502M 0.87825

 C040503M 0.88078

 C040504M 0.90028

 C040505M 0.84857

 C040506M 0.89174

 C040601M 0.87994

 C040602M 0.86801

 C040603M 0.88405

 C040604M 0.88632

 C040605M 0.85646

 C040701M 0.93398

 C040702M 0.90968

 C040703M 0.89486

 C040704M 0.88840

 C040705M 0.91671

 C040801B 0.89025

 C040801C 0.88122
Table C-15 (continued)

Proportion of Variance of the Conditioning Variable Contrasts Accounted for

by the Principal Components Used in the Conditioning Model for

Science Main Conditioning Variables, Grade 12

Proportion of Proportion of Proportion of

Contrast Variance Contrast Variance Contrast Variance

670

 C040801D 0.89810

 C040801E 0.89719

 C040801M 0.88578

 C040802B 0.89384

 C040802C 0.88248

 NATLUNCH 0.89537

 NATLUNCL 0.88462

 REM READ 0.86228
671

Appendix D

IRT PARAMETERS

This appendix contains tables of IRT (item response theory) parameters for NAEP items

that were scaled in each subject area and study (main and long-term trend) for which IRT scales

were created.

For each of the binary scored items used in scaling (i.e., multiple-choice items and short

constructed-response items), the tables provide estimates of the IRT parameters (which

correspond to aj, bj, and cj in Equation 11.1 in Chapter 11) and their associated standard errors

(s.e.) of the estimates. For each of the polytomously scored items (i.e., the extended constructedresponse items), the tables also show the estimates of the djv parameters (see Equation 11.3) and

their associated standard errors.

For the main assessment items, the tables also show the block in which each item appears

for each age class (Block) and the position of each item within its block (Item).

Note that item parameters shown in this appendix are in the metrics used for the original

calibration of the scales. The transformations needed to represent these parameters in terms of

the metric of the final reporting scales are given in Chapters 12 through 18.
672
Table D-1

IRT Parameters for the 1996 Mathematics Main Samples

Number Sense, Properties, and Operations, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

1M010131 MH 2A 0.461 (0.037) -1.996 (0.220) 0.210 (0.058)1M010231 MH 3A 0.556 (0.044) -0.586 (0.155) 0.197 (0.048)1M010431 MH 5A 0.711 (0.065) 0.194 (0.106) 0.228 (0.036)1M010531 MH 6A 0.569 (0.078) 1.387 (0.130) 0.128 (0.029)1M010631 MH 7A 0.295 (0.025) -2.058 (0.177) 0.000 (0.000)1M010831 MH 9A 1.005 (0.068) 0.095 (0.058) 0.172 (0.024)1M011131 MH 13A 0.872 (0.062) -0.053 (0.075) 0.171 (0.030)1M017401 MD 1 0.359 (0.037) -3.271 (0.365) 0.209 (0.058)1M017701 MD 4 0.961 (0.074) 0.438 (0.056) 0.170 (0.022)1M017901 MD 6 1.383 (0.087) 0.539 (0.032) 0.115 (0.013)1M018201 MD 9 1.052 (0.102) 1.297 (0.056) 0.163 (0.014)1M018401 MD 11 1.414 (0.113) 0.767 (0.039) 0.223 (0.015)1M018501 MD 12 0.738 (0.210) 3.443 (0.488) 0.231 (0.012)1M018601 MD 13 0.740 (0.120) 2.359 (0.172) 0.145 (0.014)1M020001 MF 4A 1.159 (0.042) -0.185 (0.024) 0.000 (0.000)1M020101 MF 5A 0.934 (0.044) 1.301 (0.046) 0.000 (0.000)1M020501 MF 9A 0.678 (0.034) 1.030 (0.051) 0.000 (0.000)1M039001 MC 1 0.766 (0.050) -1.617 (0.130) 0.209 (0.054)1M039201 MC 3A 0.792 (0.032) -0.217 (0.032) 0.000 (0.000)1M039901 MC 10 0.899 (0.095) 1.694 (0.076) 0.134 (0.013)1M040201 MC 13A 1.005 (0.055) 1.632 (0.057) 0.000 (0.000)1M040301 MI 1A 0.647 (0.029) 0.116 (0.037) 0.000 (0.000)1M040701 MI 5 0.671 (0.069) 0.629 (0.101) 0.231 (0.032)1M040901 MI 7A 1.412 (0.053) 0.422 (0.022) 0.000 (0.000)1M042601 MM 1 0.830 (0.060) -0.445 (0.100) 0.221 (0.040)1M042901 MM 4 0.603 (0.068) 0.831 (0.109) 0.211 (0.033)1M043001 MM 5 0.690 (0.047) -0.870 (0.126) 0.204 (0.048)1M043301 MM 8A 0.781 (0.034) 0.738 (0.038) 0.000 (0.000)1M046001 MK 1A 0.658 (0.031) -1.104 (0.053) 0.000 (0.000)1M046301 MK 4 1.175 (0.089) 0.609 (0.044) 0.196 (0.017)1M046501 MK 6 0.684 (0.100) 1.454 (0.107) 0.303 (0.025)1M046801 MK 9A 1.035 (0.041) 0.616 (0.029) 0.000 (0.000)1M046901 MK 10A 0.919 (0.037) 0.425 (0.030) 0.000 (0.000)1M047501 MK 16 0.511 (0.043) -0.750 (0.183) 0.210 (0.053)1M066101 ME 2 0.791 (0.065) -0.877 (0.123) 0.190 (0.048)1M066401 ME 5 0.793 (0.120) 1.508 (0.110) 0.194 (0.024)1M066701 ME 8A 0.731 (0.031) -0.114 (0.032) 0.000 (0.000) -0.187 (0.064) 0.187 (0.061)1M066801 ME 9A 0.470 (0.018) 0.108 (0.042) 0.000 (0.000) -2.287 (0.149) 2.287 (0.149)1M067801 MG 3 1.937 (0.167) 1.001 (0.040) 0.186 (0.014)1M068001 MG 5A 0.731 (0.046) 0.713 (0.055) 0.000 (0.000)1M068002 MG 6A 0.710 (0.032) 0.544 (0.034) 0.000 (0.000) -0.140 (0.061) 0.140 (0.067)1M068301 ML 1A 1.192 (0.101) -0.173 (0.071) 0.185 (0.032)1M068401 ML 2A 0.889 (0.085) 0.191 (0.086) 0.173 (0.033)1M068501 ML 3A 1.177 (0.103) 0.369 (0.055) 0.143 (0.023)
Table D-1 (continued)

IRT Parameters for the 1996 Mathematics Main Samples

Number Sense, Properties, and Operations, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

6741M068701 ML 5A 0.484 (0.029) -1.148 (0.064) 0.000 (0.000) 0.878 (0.110) -0.878 (0.071)1M068801 ML 6A 1.547 (0.174) 0.698 (0.054) 0.287 (0.021)1M068901 ML 7A 0.737 (0.037) 0.974 (0.041) 0.000 (0.000) -0.058 (0.059) 0.058 (0.075)1M071901 MN 1A 0.824 (0.103) 0.481 (0.109) 0.247 (0.038)1M072001 MN 2A 0.590 (0.062) -0.140 (0.155) 0.191 (0.048)1M072401 MN 7A 0.746 (0.038) 0.455 (0.034) 0.000 (0.000) 0.253 (0.054) -0.253 (0.061)1M072501 MN 8A 0.744 (0.043) 1.431 (0.058) 0.000 (0.000) -0.145 (0.070) 0.145 (0.099)1M072601 MN 9A 0.548 (0.031) 1.550 (0.077) 0.000 (0.000) -1.186 (0.129) 1.186 (0.158)1M074601 MO 5 0.793 (0.130) 0.962 (0.126) 0.359 (0.034)1M074701 MO 6A 1.014 (0.054) -0.387 (0.040) 0.000 (0.000)1M075001 MO 9A 0.693 (0.033) 1.463 (0.055) 0.000 (0.000) -0.952 (0.096) 0.952 (0.117)1M075101 MO 10A 0.563 (0.024) 0.693 (0.035) 0.000 (0.000) 1.946 (0.079) -0.362 (0.079) -1.01N202831 MH 12A 0.749 (0.068) -0.345 (0.140) 0.289 (0.048)1N240031 MH 14A 1.413 (0.094) 0.368 (0.037) 0.146 (0.017)1N277903 MF 10A 0.529 (0.031) -1.535 (0.086) 0.000 (0.000)
Table D-2

IRT Parameters for the 1996 Mathematics Main Samples

Measurement, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

2M010731 MH 8A 1.316 (0.104) 0.695 (0.043) 0.241 (0.017)2M010931 MH 10A 0.737 (0.081) 1.176 (0.079) 0.191 (0.023)2M017501 MD 2 0.545 (0.037) -1.365 (0.144) 0.143 (0.045)2M018101 MD 8 0.799 (0.068) 0.777 (0.064) 0.150 (0.022)2M020301 MF 7A 1.002 (0.044) 1.054 (0.037) 0.000 (0.000)2M039101 MC 2 0.493 (0.036) -1.277 (0.173) 0.162 (0.050)2M039301 MC 4A 0.824 (0.036) 0.778 (0.037) 0.000 (0.000)2M039401 MC 5 0.821 (0.056) -0.194 (0.079) 0.164 (0.032)2M039501 MC 6 0.826 (0.052) -0.285 (0.074) 0.139 (0.030)2M039601 MC 7 0.634 (0.049) 0.689 (0.072) 0.093 (0.023)2M040461 MI 2D 0.444 (0.011) -1.026 (0.031) 0.000 (0.000) -0.822 (0.101) -1.911 (0.147) 2.72M040801 MI 6 1.123 (0.143) 1.277 (0.075) 0.192 (0.020)2M041001 MI 8 1.951 (0.164) 1.142 (0.042) 0.132 (0.012)2M042701 MM 2 0.962 (0.055) -1.511 (0.085) 0.135 (0.040)2M042801 MM 3 0.818 (0.048) -1.649 (0.104) 0.143 (0.044)2M047101 MK 12 1.341 (0.130) 2.046 (0.086) 0.197 (0.009)2M047201 MK 13 1.086 (0.088) 0.840 (0.048) 0.182 (0.017)2M061906 MJ 6A 1.113 (0.077) 2.228 (0.091) 0.000 (0.000)2M066001 ME 1 0.698 (0.088) 1.070 (0.100) 0.145 (0.028)2M067601 MG 1 0.599 (0.079) 1.256 (0.119) 0.118 (0.028)2M069001 ML 8A 0.730 (0.043) 1.453 (0.054) 0.000 (0.000) 0.881 (0.053) -0.881 (0.106)2M072101 MN 3A 0.779 (0.065) -1.898 (0.151) 0.157 (0.050)2M074201 MO 1 1.037 (0.077) -0.018 (0.057) 0.090 (0.023)2M074301 MO 2A 1.026 (0.056) -0.098 (0.037) 0.000 (0.000)2M074801 MO 7A 0.598 (0.030) 0.431 (0.038) 0.000 (0.000) -0.012 (0.067) 0.012 (0.073)
Table D-3

IRT Parameters for the 1996 Mathematics Main Samples

Geometry and Spatial Sense, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

6763M011231 MH 15A 0.916 (0.191) 2.496 (0.207) 0.248 (0.014)3M017601 MD 3 0.307 (0.032) -0.719 (0.269) 0.254 (0.048)3M018001 MD 7 1.056 (0.123) 1.472 (0.068) 0.235 (0.015)3M019801 MF 2A 0.443 (0.037) 3.009 (0.220) 0.000 (0.000)3M019901 MF 3A 0.625 (0.029) -0.812 (0.048) 0.000 (0.000)3M020701 MF 11A 0.452 (0.031) 1.382 (0.093) 0.000 (0.000)3M039801 MC 9 0.928 (0.152) 1.817 (0.115) 0.366 (0.016)3M041201 MI 10A 0.382 (0.021) 0.699 (0.048) 0.000 (0.000) 1.078 (0.107) 0.351 (0.105) -1.43M043401 MM 9A 2.082 (0.077) 0.260 (0.016) 0.000 (0.000)3M043402 MM 10A 1.838 (0.069) 0.442 (0.018) 0.000 (0.000)3M043403 MM 11A 1.125 (0.069) 1.970 (0.073) 0.000 (0.000)3M046101 MK 2 0.478 (0.040) -1.317 (0.222) 0.242 (0.060)3M046201 MK 3 1.058 (0.079) 0.571 (0.047) 0.171 (0.019)3M046401 MK 5 0.837 (0.062) 0.226 (0.070) 0.178 (0.027)3M046701 MK 8 0.569 (0.078) 1.564 (0.114) 0.186 (0.027)3M047401 MK 15 0.932 (0.066) -1.921 (0.123) 0.219 (0.054)3M061901 MJ 1A 0.594 (0.029) -0.292 (0.042) 0.000 (0.000)3M061902 MJ 2A 1.657 (0.060) -0.144 (0.019) 0.000 (0.000)3M061903 MJ 3A 1.781 (0.067) -0.516 (0.020) 0.000 (0.000)3M061904 MJ 4A 0.870 (0.046) 1.581 (0.061) 0.000 (0.000)3M068003 MG 7A 1.015 (0.069) 0.978 (0.053) 0.000 (0.000)3M068004 MG 8A 0.200 (0.012) 1.696 (0.105) 0.000 (0.000) -2.093 (0.252) 1.957 (0.305) -0.23M074501 MO 4A 0.505 (0.030) 0.719 (0.051) 0.000 (0.000) 0.690 (0.070) -0.690 (0.089)3M074901 MO 8A 0.580 (0.030) 0.906 (0.048) 0.000 (0.000) -0.233 (0.072) 0.233 (0.089)3N214331 MH 1A 0.789 (0.057) -2.195 (0.148) 0.223 (0.057)
Table D-4

IRT Parameters for the 1996 Mathematics Main Samples

Data Analysis, Statistics, and Probabilties, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

4M017801 MD 5 0.726 (0.071) 0.720 (0.078) 0.196 (0.026)4M020201 MF 6A 0.910 (0.036) -0.029 (0.028) 0.000 (0.000)4M040001 MC 11A 0.992 (0.039) 0.290 (0.026) 0.000 (0.000)4M040101 MC 12 0.777 (0.082) 0.448 (0.097) 0.322 (0.031)4M040601 MI 4 0.999 (0.063) -0.038 (0.055) 0.147 (0.023)4M041101 MI 9 2.551 (0.132) 1.408 (0.033) 0.221 (0.009)4M043101 MM 6 1.533 (0.124) 0.654 (0.038) 0.299 (0.016)4M043201 MM 7A 0.672 (0.030) -0.740 (0.044) 0.000 (0.000)4M046601 MK 7A 1.228 (0.049) 0.559 (0.024) 0.000 (0.000)4M047001 MK 11 1.756 (0.114) 1.449 (0.041) 0.154 (0.009)4M047301 MK 14A 1.178 (0.045) -0.285 (0.026) 0.000 (0.000)4M061905 MJ 5A 0.890 (0.041) 0.811 (0.036) 0.000 (0.000)4M066901 ME 10A 0.282 (0.011) 0.785 (0.045) 0.000 (0.000) -1.522 (0.168) -2.244 (0.325) 2.74M068601 ML 4A 0.731 (0.071) 0.047 (0.107) 0.175 (0.037)4M069101 ML 9A 0.427 (0.018) 1.071 (0.040) 0.000 (0.000) -1.993 (0.196) 3.473 (0.200) -0.54M072301 MN 6A 0.627 (0.077) 0.022 (0.166) 0.248 (0.050)4N250231 MH 11A 1.083 (0.080) 0.234 (0.054) 0.209 (0.023)
Table D-5

IRT Parameters for the 1996 Mathematics Main Samples

Algebra and Functions, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

6785M010331 MH 4A 0.741 (0.076) 0.295 (0.113) 0.310 (0.035)5M018301 MD 10 1.390 (0.119) 1.102 (0.038) 0.174 (0.012)5M018701 MD 14 1.052 (0.215) 2.723 (0.270) 0.184 (0.009)5M019701 MF 1A 0.653 (0.028) -0.391 (0.039) 0.000 (0.000)5M020401 MF 8A 0.681 (0.030) 0.292 (0.036) 0.000 (0.000)5M039701 MC 8 1.396 (0.111) 1.245 (0.041) 0.204 (0.011)5M040501 MI 3 0.740 (0.064) -0.125 (0.121) 0.274 (0.040)5M043501 MM 12A 0.343 (0.013) 1.058 (0.037) 0.000 (0.000) 0.903 (0.079) -0.596 (0.106) 0.35M066201 ME 3 1.626 (0.164) 0.478 (0.054) 0.305 (0.023)5M066301 ME 4A 0.443 (0.016) -0.045 (0.042) 0.000 (0.000) -2.434 (0.149) 2.434 (0.147)5M066501 ME 6A 0.678 (0.032) 0.997 (0.039) 0.000 (0.000) -0.196 (0.063) 0.196 (0.075)5M066601 ME 7A 0.609 (0.032) 0.986 (0.045) 0.000 (0.000) 0.044 (0.064) -0.044 (0.080)5M067701 MG 2 1.158 (0.111) 0.218 (0.075) 0.240 (0.031)5M067901 MG 4A 0.689 (0.029) 1.169 (0.039) 0.000 (0.000) -1.324 (0.099) 1.324 (0.109)5M072201 MN 4A 0.417 (0.018) -0.407 (0.047) 0.000 (0.000) -5.308 (0.387) 5.308 (0.385)5M072202 MN 5A 0.763 (0.034) 0.803 (0.033) 0.000 (0.000) -0.197 (0.058) 0.197 (0.066)5M072701 MN 10A 0.393 (0.030) 1.801 (0.101) 0.000 (0.000) 0.343 (0.116) 0.157 (0.168) -0.55M074401 MO 3 1.241 (0.146) 0.761 (0.065) 0.287 (0.024)
Table D-6

IRT Parameters for the 1996 Mathematics Main Samples

Number Sense, Properties, and Operations, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

1M011131 MH 13A 0.783 (0.057) -1.197 (0.132) 0.251 (0.051)1M012431 MH 3A 0.960 (0.057) -0.179 (0.056) 0.130 (0.025)1M012531 MH 4A 0.688 (0.049) 0.540 (0.063) 0.087 (0.022)1M012931 MH 8A 1.187 (0.104) 0.983 (0.046) 0.222 (0.016)1M013431 MH 15A 1.360 (0.087) 0.159 (0.041) 0.182 (0.020)1M013531 MH 16A 1.041 (0.121) 1.147 (0.070) 0.128 (0.020)1M013631 MH 17A 1.381 (0.086) 0.722 (0.032) 0.068 (0.011)1M017401 MD 1 0.252 (0.034) -5.173 (0.727) 0.243 (0.063)1M017701 MD 4 1.016 (0.062) -0.896 (0.078) 0.211 (0.036)1M017901 MD 6 1.386 (0.086) -0.695 (0.057) 0.265 (0.030)1M018201 MD 9 0.595 (0.053) -0.561 (0.141) 0.179 (0.043)1M018401 MD 11 1.003 (0.063) -0.940 (0.084) 0.221 (0.038)1M018501 MD 12 1.538 (0.110) 0.550 (0.037) 0.273 (0.016)1M018601 MD 13 0.801 (0.064) 0.932 (0.056) 0.125 (0.018)1M020001 MF 4A 0.619 (0.028) -0.446 (0.040) 0.000 (0.000)1M020101 MF 5A 1.258 (0.046) -0.449 (0.024) 0.000 (0.000)1M020501 MF 9A 0.809 (0.032) -0.314 (0.031) 0.000 (0.000)1M046001 MK 1A 0.364 (0.029) -3.134 (0.237) 0.000 (0.000)1M046301 MK 4 0.870 (0.057) -1.258 (0.113) 0.233 (0.047)1M046501 MK 6 0.933 (0.063) -0.328 (0.078) 0.243 (0.032)1M046801 MK 9A 0.615 (0.030) -1.250 (0.062) 0.000 (0.000)1M046901 MK 10A 0.948 (0.043) -1.518 (0.052) 0.000 (0.000)1M049901 MC 1 0.806 (0.049) -0.507 (0.079) 0.135 (0.032)1M050001 MC 2 1.422 (0.070) -0.316 (0.034) 0.094 (0.018)1M050101 MC 3 0.947 (0.053) 0.310 (0.042) 0.073 (0.017)1M050301 MC 5 0.939 (0.088) 1.130 (0.055) 0.176 (0.017)1M051101 MC 13A 0.305 (0.019) 0.824 (0.061) 0.000 (0.000) 0.704 (0.122) -0.724 (0.150) 0.01M051201 MM 1A 0.636 (0.044) -1.496 (0.096) 0.000 (0.000)1M051501 MM 4 1.037 (0.139) 1.726 (0.082) 0.211 (0.013)1M051601 MM 5A 0.676 (0.031) -0.997 (0.049) 0.000 (0.000)1M051901 MM 8 1.970 (0.114) 0.709 (0.023) 0.097 (0.009)1M052401 MI 2A 0.861 (0.036) 0.738 (0.033) 0.000 (0.000)1M052901 MI 7A 0.608 (0.028) -0.001 (0.038) 0.000 (0.000)1M053001 MI 8A 0.920 (0.040) 1.015 (0.037) 0.000 (0.000)1M066401 ME 5 0.535 (0.055) -0.216 (0.165) 0.198 (0.046)1M067801 MG 3 0.851 (0.069) -1.204 (0.126) 0.207 (0.048)1M069201 ML 1A 0.779 (0.093) 1.099 (0.085) 0.134 (0.025)1M069501 ML 4A 0.758 (0.073) -0.270 (0.119) 0.182 (0.043)1M069601 ML 5A 0.626 (0.047) 1.477 (0.096) 0.000 (0.000)1M069901 ML 8A 0.784 (0.039) 1.238 (0.047) 0.000 (0.000) -0.327 (0.068) 0.327 (0.088)1M073001 MN 3A 1.183 (0.093) 0.027 (0.057) 0.143 (0.027)1M073101 MN 4A 0.552 (0.082) 0.665 (0.170) 0.229 (0.048)1M073402 MN 7A 1.644 (0.145) 1.821 (0.072) 0.151 (0.010)1M073601 MN 9B 0.465 (0.023) 1.406 (0.057) 0.000 (0.000) -0.148 (0.087) -0.328 (0.140) 0.4
Table D-6 (continued)

IRT Parameters for the 1996 Mathematics Main Samples

Number Sense, Properties, and Operations, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

6801M073602 MN 9A 0.559 (0.120) 1.140 (0.229) 0.422 (0.044)1M075901 MO 8A 0.561 (0.080) 0.916 (0.140) 0.153 (0.039)1N202831 MH 12A 0.671 (0.049) -2.037 (0.173) 0.238 (0.059)
Table D-7

IRT Parameters for the Mathematics Main Samples

Measurement, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

2M012331 MH 2A 0.670 (0.043) -1.615 (0.132) 0.171 (0.051)2M013331 MH 14A 0.931 (0.058) -1.268 (0.097) 0.177 (0.048)2M017501 MD 2 0.348 (0.035) -2.932 (0.353) 0.251 (0.062)2M018101 MD 8 0.543 (0.045) -0.356 (0.160) 0.225 (0.047)2M019101 MD 18 1.147 (0.111) 1.695 (0.063) 0.152 (0.010)2M019201 MD 19 1.122 (0.113) 1.700 (0.063) 0.134 (0.010)2M020301 MF 7A 1.042 (0.038) -0.406 (0.026) 0.000 (0.000)2M047101 MK 12 1.068 (0.068) 0.288 (0.048) 0.163 (0.020)2M047201 MK 13 0.831 (0.055) -0.739 (0.104) 0.229 (0.043)2M047901 MK 18A 1.162 (0.057) 1.562 (0.045) 0.000 (0.000)2M050501 MC 7 0.766 (0.070) 0.331 (0.095) 0.261 (0.032)2M050901 MC 11A 0.795 (0.038) 1.353 (0.052) 0.000 (0.000)2M051301 MM 2A 0.464 (0.033) -2.624 (0.164) 0.000 (0.000)2M052201 MM 11A 0.635 (0.022) 1.080 (0.028) 0.000 (0.000) 0.784 (0.040) -0.502 (0.061) -0.22M052301 MI 1 1.528 (0.104) 0.761 (0.033) 0.181 (0.013)2M061907 MJ 5A 0.812 (0.036) 1.067 (0.043) 0.000 (0.000)2M061908 MJ 6A 0.753 (0.057) 2.720 (0.152) 0.000 (0.000)2M068008 MG 9A 0.331 (0.032) 2.700 (0.206) 0.000 (0.000) 0.611 (0.118) -0.611 (0.236)2M069401 ML 3A 1.073 (0.081) -0.849 (0.085) 0.160 (0.042)2M069701 ML 6A 0.745 (0.036) 1.228 (0.045) 0.000 (0.000) -0.088 (0.059) 0.088 (0.079)2M069801 ML 7A 0.747 (0.099) 1.540 (0.102) 0.101 (0.021)2M072801 MN 1A 0.411 (0.049) 0.048 (0.219) 0.178 (0.052)2M073301 MN 6A 0.939 (0.121) 1.236 (0.078) 0.181 (0.022)2M075201 MO 1A 0.471 (0.050) -0.523 (0.211) 0.190 (0.056)2M075401 MO 3A 0.627 (0.030) 0.222 (0.039) 0.000 (0.000) 0.944 (0.059) -0.944 (0.064)2M075601 MO 5A 0.489 (0.019) 0.295 (0.039) 0.000 (0.000) -1.164 (0.096) 1.164 (0.099)2M076001 MO 9A 0.435 (0.018) 1.556 (0.059) 0.000 (0.000) -2.284 (0.172) 1.740 (0.208) 0.5
Table D-8

IRT Parameters for the 1996 Mathematics Main Samples

Geometry and Spatial Sense, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

6823M012731 MH 6A 0.553 (0.075) 1.612 (0.115) 0.160 (0.026)3M012831 MH 7A 1.100 (0.073) 0.509 (0.041) 0.133 (0.017)3M017601 MD 3 0.558 (0.046) -1.228 (0.209) 0.313 (0.059)3M018001 MD 7 0.824 (0.072) 0.207 (0.090) 0.307 (0.030)3M019001 MD 17 0.789 (0.068) 0.853 (0.064) 0.164 (0.022)3M019601 MD 21 0.632 (0.071) 1.503 (0.091) 0.117 (0.021)3M019801 MF 2A 0.891 (0.035) -0.760 (0.035) 0.000 (0.000)3M019901 MF 3A 0.636 (0.032) -1.627 (0.073) 0.000 (0.000)3M020901 MF 11A 0.566 (0.028) 0.608 (0.045) 0.000 (0.000)3M021001 MF 12A 0.827 (0.033) 0.067 (0.029) 0.000 (0.000)3M021301 MF 15A 1.705 (0.062) -0.075 (0.018) 0.000 (0.000)3M021302 MF 16A 1.454 (0.054) -0.224 (0.021) 0.000 (0.000)3M046101 MK 2 0.750 (0.057) -2.147 (0.183) 0.323 (0.066)3M046201 MK 3 1.211 (0.089) -0.308 (0.069) 0.365 (0.029)3M046401 MK 5 0.814 (0.062) -0.760 (0.125) 0.316 (0.045)3M046701 MK 8 1.276 (0.096) 0.271 (0.051) 0.328 (0.021)3M048001 MK 19A 1.124 (0.098) 1.487 (0.054) 0.081 (0.010)3M051001 MC 12A 0.660 (0.050) 1.422 (0.089) 0.000 (0.000)3M051801 MM 7 0.993 (0.128) 1.694 (0.081) 0.242 (0.014)3M052001 MM 9 0.857 (0.089) 1.417 (0.068) 0.172 (0.017)3M052601 MI 4 0.768 (0.062) 0.474 (0.072) 0.173 (0.026)3M061901 MJ 1A 0.619 (0.029) -1.232 (0.059) 0.000 (0.000)3M061902 MJ 4A 1.366 (0.054) -1.040 (0.030) 0.000 (0.000)3M061903 MJ 2A 1.247 (0.054) -1.469 (0.040) 0.000 (0.000)3M061904 MJ 3A 0.848 (0.034) 0.148 (0.029) 0.000 (0.000)3M068003 MG 5A 0.552 (0.019) -0.150 (0.034) 0.000 (0.000) -1.993 (0.121) 1.993 (0.118)3M068005 MG 6A 0.426 (0.017) -0.474 (0.047) 0.000 (0.000) -1.640 (0.122) 1.640 (0.114)3M068006 MG 7A 0.479 (0.019) 0.717 (0.042) 0.000 (0.000) -1.432 (0.103) 1.432 (0.111)3M068007 MG 8 0.758 (0.070) 0.490 (0.079) 0.110 (0.028)3M068201 MG 10A 0.248 (0.035) 1.970 (0.280) 0.000 (0.000)3M075801 MO 7A 0.982 (0.050) 0.988 (0.034) 0.000 (0.000) 0.287 (0.042) -0.287 (0.058)
Table D-9

IRT Parameters for the 1996 Mathematics Main Samples

Data Analysis, Statistics, and Probability, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

4M012631 MH 5A 1.294 (0.088) 0.484 (0.041) 0.209 (0.017)4M013031 MH 9A 1.127 (0.053) 1.368 (0.041) 0.000 (0.000)4M013131 MH 10A 0.900 (0.042) 1.259 (0.045) 0.000 (0.000)4M017801 MD 5 1.009 (0.062) -0.620 (0.076) 0.209 (0.035)4M018901 MD 16 0.578 (0.114) 1.846 (0.172) 0.165 (0.030)4M020201 MF 6A 0.585 (0.033) -2.087 (0.099) 0.000 (0.000)4M020801 MF 10A 1.006 (0.050) 1.553 (0.050) 0.000 (0.000)4M021101 MF 13A 0.928 (0.034) 0.034 (0.027) 0.000 (0.000)4M046601 MK 7A 1.002 (0.040) -1.063 (0.036) 0.000 (0.000)4M047001 MK 11 0.968 (0.059) 0.049 (0.056) 0.147 (0.024)4M047301 MK 14A 0.976 (0.049) -1.844 (0.061) 0.000 (0.000)4M047801 MK 17 1.486 (0.095) 0.770 (0.030) 0.117 (0.012)4M050261 MC 4E 0.763 (0.024) -1.294 (0.033) 0.000 (0.000) -0.389 (0.062) 0.389 (0.047)4M050401 MC 6 0.730 (0.056) -0.145 (0.105) 0.208 (0.038)4M051401 MM 3 0.807 (0.046) -1.074 (0.094) 0.155 (0.041)4M052701 MI 5 1.135 (0.078) 0.496 (0.046) 0.189 (0.018)4M052801 MI 6 0.545 (0.196) 4.612 (1.191) 0.179 (0.011)4M053101 MI 9A 0.664 (0.023) 1.289 (0.029) 0.000 (0.000) 0.221 (0.044) 0.151 (0.061) -0.34M061905 MJ 7A 0.484 (0.026) -0.466 (0.052) 0.000 (0.000)4M067101 ME 2 1.092 (0.087) -0.051 (0.069) 0.165 (0.029)4M067501 ME 11A 0.584 (0.030) 1.847 (0.058) 0.000 (0.000) 1.410 (0.057) -1.410 (0.146)4M070001 ML 9A 0.227 (0.023) 1.216 (0.134) 0.000 (0.000) 0.412 (0.164) -0.412 (0.201)4M072901 MN 2A 0.438 (0.019) -0.542 (0.049) 0.000 (0.000) -0.766 (0.105) 0.766 (0.093)4M0732CL MN 5F 0.354 (0.020) -0.326 (0.047) 0.000 (0.000) 1.065 (0.134) -0.139 (0.112) -0.9M073501 MN 8A 0.153 (0.029) 4.118 (0.770) 0.000 (0.000)
Table D-10

IRT Parameters for the 1996 Mathematics Main Samples

Algebra and Functions, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

6845M012231 MH 1A 0.611 (0.048) -2.789 (0.198) 0.164 (0.053)5M013231 MH 11A 1.018 (0.099) 1.583 (0.062) 0.116 (0.012)5M013731 MH 18A 1.082 (0.104) 1.289 (0.056) 0.120 (0.014)5M018301 MD 10 0.928 (0.058) -0.502 (0.078) 0.201 (0.034)5M018701 MD 14 1.535 (0.102) 0.231 (0.038) 0.265 (0.018)5M018801 MD 15 1.052 (0.093) 0.719 (0.058) 0.292 (0.020)5M019301 MD 20 1.413 (0.117) 1.048 (0.040) 0.195 (0.013)5M019701 MF 1A 0.435 (0.028) -2.147 (0.129) 0.000 (0.000)5M020401 MF 8A 0.643 (0.028) -0.287 (0.037) 0.000 (0.000)5M021201 MF 14A 0.960 (0.037) 0.355 (0.027) 0.000 (0.000)5M047601 MK 15 1.333 (0.075) 0.307 (0.033) 0.108 (0.015)5M047701 MK 16 0.977 (0.124) 1.682 (0.081) 0.239 (0.015)5M050601 MC 8 1.172 (0.068) 0.535 (0.034) 0.085 (0.013)5M050701 MC 9 1.655 (0.091) 0.032 (0.030) 0.141 (0.016)5M050801 MC 10A 0.792 (0.032) -0.056 (0.031) 0.000 (0.000)5M051701 MM 6 1.448 (0.094) 0.460 (0.035) 0.196 (0.016)5M052101 MM 10A 0.879 (0.035) 0.374 (0.029) 0.000 (0.000)5M052501 MI 3 0.732 (0.067) 1.224 (0.066) 0.119 (0.019)5M066201 ME 3 1.058 (0.091) -0.975 (0.109) 0.238 (0.050)5M066301 ME 4A 0.621 (0.052) -2.180 (0.156) 0.000 (0.000)5M066501 ME 6A 0.566 (0.026) -0.426 (0.040) 0.000 (0.000) -0.195 (0.078) 0.195 (0.068)5M066601 ME 7A 0.381 (0.020) -0.912 (0.068) 0.000 (0.000) -0.594 (0.122) 0.594 (0.101)5M067001 ME 1 1.079 (0.113) 0.558 (0.075) 0.261 (0.028)5M067201 ME 8B 0.494 (0.019) 1.030 (0.047) 0.000 (0.000) -1.990 (0.127) 1.990 (0.137)5M067202 ME 8A 1.361 (0.190) 0.924 (0.076) 0.422 (0.023)5M067301 ME 9 1.017 (0.092) 0.640 (0.062) 0.132 (0.024)5M067401 ME 10 0.887 (0.117) 1.333 (0.086) 0.163 (0.023)5M067701 MG 2 0.786 (0.065) -1.318 (0.134) 0.206 (0.050)5M067901 MG 4A 0.637 (0.023) -0.511 (0.034) 0.000 (0.000) -1.335 (0.097) 1.335 (0.091)5M068101 MG 1 0.968 (0.073) -0.601 (0.084) 0.144 (0.038)5M069301 ML 2A 0.683 (0.046) 1.256 (0.075) 0.000 (0.000)5M075301 MO 2A 1.196 (0.044) -0.182 (0.029) 0.000 (0.000) 1.314 (0.047) -1.314 (0.038)5M075501 MO 4A 1.139 (0.097) 0.310 (0.061) 0.166 (0.026)5M0757CL MO 6E 0.541 (0.030) 0.282 (0.041) 0.000 (0.000) 0.473 (0.069) -0.473 (0.074)
Table D-11

IRT Parameters for the 1996 Mathematics Main Samples

Number Sense, Properties, and Operations, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

1M011531 MH 15A 2.240 (0.111) 1.065 (0.024) 0.080 (0.007)1M012431 MH 3A 0.485 (0.038) -2.457 (0.213) 0.168 (0.051)1M012531 MH 4A 0.617 (0.044) -0.340 (0.107) 0.150 (0.037)1M012931 MH 8A 1.088 (0.077) 0.265 (0.054) 0.213 (0.023)1M017401 MD 1 0.307 (0.038) -4.679 (0.589) 0.213 (0.059)1M017701 MD 4 0.933 (0.062) -1.773 (0.110) 0.187 (0.046)1M017901 MD 6 1.197 (0.082) -1.718 (0.091) 0.199 (0.044)1M018201 MD 9 0.583 (0.041) -1.957 (0.163) 0.181 (0.049)1M018401 MD 11 0.994 (0.070) -1.676 (0.114) 0.236 (0.050)1M018501 MD 12 1.441 (0.088) -0.263 (0.046) 0.230 (0.025)1M018601 MD 13 0.507 (0.052) 0.631 (0.130) 0.184 (0.037)1M020501 MF 9A 0.915 (0.040) -1.034 (0.041) 0.000 (0.000)1M021401 MF 1A 0.817 (0.034) -0.367 (0.032) 0.000 (0.000)1M021701 MF 15A 1.205 (0.047) 0.357 (0.024) 0.000 (0.000)1M052401 MI 2A 0.522 (0.027) -0.032 (0.044) 0.000 (0.000)1M056501 MC 8 1.314 (0.128) 1.215 (0.046) 0.216 (0.013)1M056601 MC 9 0.826 (0.074) 0.733 (0.069) 0.092 (0.022)1M056801 MC 10A 1.654 (0.066) 0.886 (0.023) 0.000 (0.000)1M057101 MC 14A 1.638 (0.111) 1.992 (0.063) 0.000 (0.000)1M058601 MK 2 1.165 (0.067) -0.330 (0.052) 0.167 (0.026)1M059001 MK 5 0.925 (0.052) 0.151 (0.048) 0.095 (0.020)1M059201 MK 7 0.688 (0.043) -1.359 (0.112) 0.146 (0.041)1M059601 MK 11 1.138 (0.085) 1.167 (0.042) 0.092 (0.011)1M060101 MM 3 0.579 (0.060) 0.680 (0.114) 0.194 (0.035)1M062101 MJ 7A 0.606 (0.031) 0.575 (0.045) 0.000 (0.000)1M069501 ML 4A 0.671 (0.059) -0.978 (0.145) 0.179 (0.050)1M069601 ML 5A 0.759 (0.046) 1.905 (0.064) 0.000 (0.000) 0.796 (0.050) -0.796 (0.125)1M070301 ML 7A 0.541 (0.102) 1.565 (0.176) 0.220 (0.038)1M070401 ML 8A 0.788 (0.039) 0.597 (0.034) 0.000 (0.000) 0.152 (0.053) -0.152 (0.062)1M071201 ME 8A 0.436 (0.020) 0.938 (0.057) 0.000 (0.000) -1.230 (0.113) 1.230 (0.128)1M071401 ME 10A 0.545 (0.032) 1.724 (0.088) 0.000 (0.000) -0.996 (0.120) 0.996 (0.158)1M071503 MG 3 0.793 (0.079) 0.190 (0.104) 0.195 (0.038)1M073001 MN 3A 0.884 (0.074) -1.189 (0.121) 0.185 (0.050)1M073101 MN 4A 0.469 (0.055) -0.039 (0.202) 0.191 (0.053)1M073401 MN 7B 0.714 (0.032) 0.701 (0.035) 0.000 (0.000) -0.096 (0.057) 0.096 (0.067)1M073701 MN 1A 0.797 (0.074) -0.096 (0.110) 0.185 (0.042)1M074001 MN 9B 0.899 (0.048) 0.419 (0.032) 0.000 (0.000) 0.514 (0.049) -0.514 (0.054)1M076101 MO 1A 0.500 (0.033) -1.863 (0.091) 0.000 (0.000) 0.845 (0.148) -0.845 (0.071)1M076201 MO 2A 0.526 (0.052) -0.447 (0.165) 0.166 (0.048)1N202831 MH 12A 0.722 (0.054) -2.341 (0.169) 0.205 (0.054)
Table D-12

IRT Parameters for the 1996 Mathematics Main Samples

Measurement, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

6862M011331 MH 13A 1.130 (0.078) 0.325 (0.048) 0.186 (0.020)2M011931 MH 19A 2.135 (0.118) 1.929 (0.048) 0.000 (0.000)2M012331 MH 2A 0.787 (0.052) -2.078 (0.131) 0.176 (0.052)2M017501 MD 2 0.396 (0.040) -3.260 (0.353) 0.260 (0.063)2M018101 MD 8 0.461 (0.038) -1.102 (0.211) 0.240 (0.055)2M019101 MD 18 1.004 (0.086) 0.945 (0.051) 0.174 (0.017)2M019201 MD 19 1.164 (0.101) 1.100 (0.046) 0.173 (0.014)2M020301 MF 7A 1.014 (0.044) -1.325 (0.043) 0.000 (0.000)2M052301 MI 1 1.619 (0.091) 0.086 (0.032) 0.152 (0.016)2M055801 MC 1 0.623 (0.042) -1.952 (0.153) 0.174 (0.052)2M058501 MK 1 0.815 (0.050) 0.092 (0.060) 0.105 (0.023)2M059801 MK 14A 1.511 (0.102) 1.900 (0.060) 0.000 (0.000)2M060201 MM 4A 0.645 (0.032) 1.001 (0.052) 0.000 (0.000)2M060501 MM 7A 0.922 (0.041) 0.938 (0.037) 0.000 (0.000)2M061907 MJ 3A 0.850 (0.036) 0.668 (0.034) 0.000 (0.000)2M061908 MJ 4A 0.933 (0.050) 1.685 (0.062) 0.000 (0.000)2M069401 ML 3A 1.012 (0.079) -1.446 (0.111) 0.173 (0.048)2M069701 ML 6A 0.494 (0.026) 0.440 (0.043) 0.000 (0.000) -0.135 (0.078) 0.135 (0.085)2M071501 MG 1 1.100 (0.083) -0.946 (0.087) 0.158 (0.039)2M071502 MG 2A 0.648 (0.024) 0.399 (0.031) 0.000 (0.000) -1.168 (0.085) 1.168 (0.088)2M073301 MN 6A 0.692 (0.076) -0.043 (0.136) 0.209 (0.045)2M073801 MN 2A 0.581 (0.021) 0.575 (0.034) 0.000 (0.000) -1.724 (0.109) 1.724 (0.113)2M076901 MO 9A 0.537 (0.031) 1.968 (0.097) 0.000 (0.000) -1.704 (0.157) 1.704 (0.192)
Table D-13

IRT Parameters for the 1996 Mathematics Main Samples

Geometry and Spatial Sense, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

3M011731 MH 17A 1.435 (0.103) 0.357 (0.042) 0.234 (0.019)3M012731 MH 6A 1.015 (0.088) 0.100 (0.070) 0.172 (0.029)3M012831 MH 7A 1.569 (0.093) -0.743 (0.048) 0.203 (0.027)3M017601 MD 3 0.676 (0.051) -1.656 (0.185) 0.325 (0.062)3M018001 MD 7 0.963 (0.079) -0.261 (0.096) 0.366 (0.034)3M019001 MD 17 0.897 (0.065) -0.165 (0.081) 0.220 (0.032)3M020901 MF 11A 0.647 (0.029) 0.087 (0.037) 0.000 (0.000)3M021001 MF 12A 0.797 (0.034) -1.083 (0.046) 0.000 (0.000)3M021801 MF 17A 1.359 (0.075) 1.387 (0.040) 0.000 (0.000)3M052601 MI 4 0.635 (0.051) -0.073 (0.113) 0.188 (0.037)3M053301 MI 8A 1.368 (0.063) 1.214 (0.032) 0.000 (0.000)3M056001 MC 3 1.062 (0.059) -0.672 (0.063) 0.155 (0.029)3M056701 MC 11 1.478 (0.106) 1.465 (0.044) 0.134 (0.009)3M058701 MK 3 1.317 (0.113) -0.862 (0.088) 0.224 (0.042)3M058901 MK 4A 0.886 (0.035) 0.167 (0.029) 0.000 (0.000)3M059101 MK 6 1.073 (0.085) 0.211 (0.064) 0.308 (0.025)3M060001 MM 2A 1.465 (0.063) 1.071 (0.027) 0.000 (0.000)3M060601 MM 8A 1.124 (0.051) 1.096 (0.035) 0.000 (0.000)3M061901 MJ 1A 0.533 (0.031) -2.017 (0.104) 0.000 (0.000)3M061904 MJ 2A 0.618 (0.028) -0.316 (0.040) 0.000 (0.000)3M062401 MJ 10A 0.261 (0.016) 1.288 (0.090) 0.000 (0.000) -0.319 (0.148) -1.419 (0.239) 1.737 3M070101 ML 1A 0.623 (0.111) 1.562 (0.147) 0.207 (0.032)3M071101 ME 7A 0.503 (0.021) 0.563 (0.040) 0.000 (0.000) -1.284 (0.100) 1.284 (0.105)3M071301 ME 9A 1.178 (0.079) 1.495 (0.049) 0.000 (0.000) 0.119 (0.050) -0.119 (0.083)3M073901 MN 8A 1.286 (0.083) 1.212 (0.046) 0.000 (0.000)3M076701 MO 7A 2.145 (0.124) 1.325 (0.023) 0.000 (0.000) -0.012 (0.033) 0.012 (0.043)3M076801 MO 8A 0.989 (0.054) 1.512 (0.046) 0.000 (0.000) -0.293 (0.060) 0.293 (0.084)
Table D-14

IRT Parameters for the 1996 Mathematics Main Samples

Data Analysis, Statistics, and Probability, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

6884M011631 MH 16A 0.618 (0.049) -0.257 (0.112) 0.154 (0.037)4M012631 MH 5A 0.942 (0.063) -0.875 (0.093) 0.215 (0.039)4M013031 MH 9A 0.821 (0.038) 0.673 (0.035) 0.000 (0.000)4M013131 MH 10A 0.604 (0.045) 0.667 (0.061) 0.000 (0.000)4M017801 MD 5 1.367 (0.093) -1.082 (0.075) 0.298 (0.037)4M018901 MD 16 0.648 (0.116) 1.738 (0.166) 0.192 (0.028)4M020201 MF 6A 0.561 (0.036) -2.785 (0.149) 0.000 (0.000)4M020801 MF 10A 0.874 (0.042) 0.930 (0.039) 0.000 (0.000)4M021101 MF 13A 0.783 (0.034) -0.867 (0.044) 0.000 (0.000)4M021501 MF 2A 0.925 (0.052) -0.412 (0.044) 0.000 (0.000)4M021502 MF 3A 0.887 (0.039) -1.416 (0.051) 0.000 (0.000)4M052701 MI 5 0.915 (0.056) -0.608 (0.073) 0.150 (0.031)4M052801 MI 6 0.196 (0.042) 5.485 (1.033) 0.156 (0.023)4M053401 MI 9A 0.278 (0.017) 0.678 (0.049) 0.000 (0.000) 1.029 (0.161) 0.260 (0.163) 0.245 4M055901 MC 2 0.572 (0.041) -0.954 (0.144) 0.169 (0.045)4M057001 MC 13A 1.341 (0.082) 1.625 (0.057) 0.000 (0.000)4M059701 MK 12 0.292 (0.058) 2.962 (0.415) 0.228 (0.032)4M059702 MK 13A 0.806 (0.046) 1.335 (0.059) 0.000 (0.000)4M059901 MM 1 0.702 (0.050) -0.906 (0.123) 0.185 (0.044)4M060301 MM 5 0.505 (0.040) -0.921 (0.169) 0.172 (0.048)4M060401 MM 6A 0.823 (0.035) -0.481 (0.035) 0.000 (0.000)4M061905 MJ 5A 0.357 (0.034) -1.159 (0.138) 0.000 (0.000)4M070201 ML 2A 0.477 (0.027) 0.988 (0.056) 0.000 (0.000) 1.018 (0.070) -1.018 (0.100)4M0705CL ML 9Z 0.564 (0.033) 1.037 (0.044) 0.000 (0.000) 0.413 (0.077) 0.377 (0.091) -0.790 4M070601 ML 10A 0.209 (0.022) 2.982 (0.263) 0.000 (0.000) 0.865 (0.184) -0.217 (0.291) -0.648 4M070801 ME 4A 0.475 (0.023) 0.052 (0.041) 0.000 (0.000) -0.436 (0.087) 0.436 (0.084)4M070901 ME 5A 1.038 (0.115) 0.844 (0.057) 0.125 (0.021)4M071601 MG 4 0.801 (0.084) 0.057 (0.100) 0.179 (0.037)4M071602 MG 5A 0.700 (0.033) 0.454 (0.031) 0.000 (0.000) -0.221 (0.059) 0.221 (0.063)4M071603 MG 6A 2.084 (0.096) 0.197 (0.015) 0.000 (0.000) 0.271 (0.025) -0.271 (0.022)4M071604 MG 7A 1.429 (0.057) 0.176 (0.019) 0.000 (0.000) -0.177 (0.038) 0.177 (0.036)4M071801 MG 10A 0.295 (0.016) 1.118 (0.078) 0.000 (0.000) 1.791 (0.126) -2.096 (0.187) 0.305 4M0732CL MN 5F 0.347 (0.022) -1.385 (0.079) 0.000 (0.000) 0.789 (0.206) 0.024 (0.138) -0.813 4M076301 MO 3A 0.645 (0.081) 0.734 (0.106) 0.154 (0.034)
Table D-15

IRT Parameters for the 1996 Mathematics Main Samples

Algebra and Functions, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) 

5M011431 MH 14A 1.382 (0.108) 0.714 (0.041) 0.247 (0.016)5M011831 MH 18A 1.786 (0.121) 1.446 (0.044) 0.201 (0.010)5M012031 MH 20A 1.136 (0.057) 0.859 (0.036) 0.000 (0.000)5M012131 MH 21A 1.654 (0.145) 1.275 (0.049) 0.152 (0.012)5M012231 MH 1A 0.532 (0.050) -3.569 (0.298) 0.203 (0.057)5M013231 MH 11A 0.951 (0.074) 0.745 (0.051) 0.160 (0.019)5M018301 MD 10 0.720 (0.045) -1.444 (0.122) 0.174 (0.047)5M018701 MD 14 1.191 (0.069) -0.923 (0.065) 0.169 (0.033)5M018801 MD 15 0.997 (0.064) -0.651 (0.080) 0.220 (0.035)5M019301 MD 20 2.071 (0.136) 0.183 (0.030) 0.242 (0.017)5M019401 MD 21 2.151 (0.136) 1.561 (0.045) 0.282 (0.010)5M019501 MD 22 1.562 (0.106) 1.346 (0.038) 0.090 (0.009)5M020401 MF 8A 0.609 (0.029) -0.782 (0.049) 0.000 (0.000)5M021201 MF 14A 0.617 (0.030) -0.796 (0.051) 0.000 (0.000)5M021601 MF 4A 1.518 (0.148) 2.105 (0.102) 0.000 (0.000)5M021602 MF 5A 1.833 (0.126) 1.514 (0.047) 0.000 (0.000)5M021702 MF 16A 0.748 (0.035) -0.112 (0.036) 0.000 (0.000)5M052501 MI 3 1.273 (0.080) 0.015 (0.046) 0.213 (0.021)5M053201 MI 7 1.292 (0.101) 0.679 (0.044) 0.261 (0.017)5M056101 MC 4 1.101 (0.088) 0.551 (0.052) 0.253 (0.020)5M056201 MC 5 1.213 (0.104) 0.791 (0.048) 0.274 (0.017)5M056301 MC 6 1.374 (0.069) 0.003 (0.031) 0.091 (0.015)5M056401 MC 7 1.123 (0.087) 0.447 (0.053) 0.270 (0.021)5M056901 MC 12A 1.906 (0.090) 1.336 (0.028) 0.000 (0.000)5M059301 MK 8 1.100 (0.078) 0.581 (0.045) 0.163 (0.018)5M059401 MK 9 2.647 (0.138) 0.831 (0.022) 0.200 (0.010)5M059501 MK 10 1.703 (0.151) 1.685 (0.067) 0.390 (0.010)5M060701 MM 9A 0.511 (0.029) 1.472 (0.052) 0.000 (0.000) 0.823 (0.075) 0.391 (0.094) -1.214 5M062001 MJ 6 0.994 (0.064) -1.374 (0.102) 0.211 (0.047)5M062201 MJ 8 1.140 (0.088) 1.088 (0.042) 0.095 (0.012)5M062301 MJ 9 1.951 (0.108) 1.271 (0.031) 0.095 (0.007)5M070701 ME 1A 1.981 (0.186) 0.026 (0.047) 0.319 (0.026)5M070702 ME 2A 1.588 (0.130) 0.710 (0.038) 0.111 (0.015)5M070703 ME 3A 2.058 (0.198) 0.199 (0.044) 0.329 (0.024)5M071001 ME 6A 0.707 (0.032) 1.035 (0.041) 0.000 (0.000) -0.429 (0.064) 0.429 (0.080)5M0717CL MG 9 0.751 (0.033) 1.552 (0.043) 0.000 (0.000) -0.848 (0.093) -0.702 (0.220) 1.551 5M074101 MN 10A 0.499 (0.024) 1.689 (0.066) 0.000 (0.000) -1.272 (0.126) 0.274 (0.199) 0.998 5M076401 MO 4A 1.384 (0.165) 1.481 (0.074) 0.223 (0.016)5M076501 MO 5A 0.841 (0.150) 1.822 (0.138) 0.169 (0.022)5M076601 MO 6A 1.057 (0.043) 0.920 (0.027) 0.000 (0.000) -0.669 (0.062) 0.669 (0.068)5M077001 MO 10A 0.441 (0.043) 1.231 (0.122) 0.000 (0.000)
Table D-16

IRT Parameters for the 1996 Science Main Samples

Physical Science, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 

6901K031101 SD 1A 0.722 (0.044) -1.320 (0.079) 0.000 (0.000)1K031102 SD 2A 0.469 (0.035) -0.744 (0.084) 0.000 (0.000)1K031103 SD 3A 0.525 (0.038) -0.012 (0.060) 0.000 (0.000)1K031104 SD 4A 0.788 (0.052) -1.791 (0.098) 0.000 (0.000)1K031105 SD 5A 0.524 (0.080) 3.427 (0.450) 0.000 (0.000)1K031106 SD 6 0.649 (0.183) 2.495 (0.374) 0.172 (0.023)1K031107 SD 7A 0.576 (0.036) 1.527 (0.075) 0.000 (0.000) -0.446 (0.081) 0.446 (0.114)1K031201 SE 1A 0.239 (0.011) -0.516 (0.093) 0.000 (0.000) 2.650 (0.161) -2.650 (0.140)1K031204 SE 4A 0.357 (0.019) 1.126 (0.053) 0.000 (0.000) 0.631 (0.097) -0.456 (0.132) -0.486 1K031210 SE 10A 0.686 (0.039) 0.143 (0.040) 0.000 (0.000) 0.744 (0.066) -0.744 (0.063)1K031302 SF 7A 0.403 (0.059) 3.936 (0.467) 0.000 (0.000) 0.347 (0.137) -0.347 (0.450)1K031303 SF 8A 0.700 (0.042) -0.449 (0.046) 0.000 (0.000) 0.678 (0.082) -0.678 (0.057)1K031304 SF 9A 0.476 (0.037) -1.076 (0.094) 0.000 (0.000) 0.380 (0.142) -0.380 (0.092)1K031309 SF 6G 0.366 (0.012) -0.704 (0.046) 0.000 (0.000) -3.345 (0.255) 1.243 (0.273) 2.102 1K032001 SJ 4A 0.396 (0.028) 0.426 (0.063) 0.000 (0.000) 0.838 (0.100) -0.838 (0.112)1K032101 SJ 5 0.575 (0.077) 0.226 (0.153) 0.194 (0.045)1K032701 SK 1 0.487 (0.057) -1.384 (0.257) 0.226 (0.059)1K032901 SK 3 0.660 (0.086) 0.222 (0.134) 0.211 (0.042)1K034001 SL 5A 0.509 (0.044) 1.949 (0.136) 0.000 (0.000) -0.080 (0.094) 0.080 (0.164)1K034201 SL 7 0.724 (0.084) 0.114 (0.113) 0.181 (0.038)1K034301 SL 8 0.593 (0.081) 0.343 (0.141) 0.189 (0.042)1K034801 SM 3 0.826 (0.118) 0.620 (0.103) 0.246 (0.035)1K034802 SM 4A 0.706 (0.050) 0.060 (0.054) 0.000 (0.000)1K034901 SM 5A 0.295 (0.023) -1.779 (0.147) 0.000 (0.000) -0.251 (0.200) 0.251 (0.141)1K034902 SM 6A 0.751 (0.038) 0.626 (0.035) 0.000 (0.000) -0.244 (0.064) 0.244 (0.072)1K035401 SN 1 0.235 (0.053) 2.588 (0.549) 0.221 (0.042)1K035601 SN 3A 0.309 (0.026) 3.479 (0.198) 0.000 (0.000) 1.672 (0.110) -1.672 (0.326)1K035701 SN 4 0.877 (0.126) 1.072 (0.085) 0.196 (0.026)1K035801 SN 5A 0.523 (0.028) 0.696 (0.045) 0.000 (0.000) -0.179 (0.074) 0.179 (0.086)1K036201 SN 9 0.683 (0.122) 1.842 (0.177) 0.100 (0.021)1K036301 SN 10A 0.473 (0.028) 1.695 (0.093) 0.000 (0.000) -1.372 (0.131) 1.372 (0.164)1K037301 SO 4A 0.389 (0.026) 2.200 (0.146) 0.000 (0.000) -2.479 (0.220) 2.479 (0.267)1K037501 SO 6A 0.374 (0.069) 5.414 (1.108) 0.000 (0.000) -1.985 (0.464) 1.985 (1.184)1K037701 SO 8A 0.373 (0.021) -1.318 (0.077) 0.000 (0.000) 1.291 (0.268) 0.983 (0.156) -2.273 1K037702 SO 9A 0.646 (0.057) 1.566 (0.096) 0.000 (0.000) 0.237 (0.081) -0.237 (0.139)1K038401 ST 1 0.642 (0.070) -1.578 (0.214) 0.222 (0.058)1K039201 ST 9A 0.346 (0.027) 2.426 (0.197) 0.000 (0.000) -1.804 (0.199) 1.804 (0.271)1K039301 ST 10A 0.495 (0.039) 0.754 (0.063) 0.000 (0.000) 0.410 (0.090) -0.410 (0.110)1K039501 SU 1 0.888 (0.096) -0.600 (0.131) 0.233 (0.048)1K039701 SU 3 0.687 (0.166) 1.731 (0.208) 0.232 (0.032)1K040001 SU 6A 0.419 (0.027) 0.852 (0.068) 0.000 (0.000) -0.633 (0.116) 0.633 (0.134)1K040101 SU 7 0.640 (0.082) -0.389 (0.179) 0.230 (0.053)1K040301 SU 9A 0.356 (0.038) 1.928 (0.167) 0.000 (0.000) 0.560 (0.123) -0.560 (0.206)
Table D-17

IRT Parameters for the 1996 Science Main Samples

Earth Science, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 

2K031202 SE 2A 0.278 (0.033) 4.490 (0.431) 0.000 (0.000) 0.963 (0.137) -0.963 (0.420)2K031205 SE 5 0.566 (0.056) -0.834 (0.187) 0.218 (0.054)2K031208 SE 8 0.194 (0.042) 0.360 (0.557) 0.409 (0.051)2K031211 SE 11A 0.586 (0.047) 1.154 (0.067) 0.000 (0.000) 0.300 (0.078) -0.300 (0.108)2K031401 SG 1A 0.272 (0.017) 3.767 (0.122) 0.000 (0.000) 3.259 (0.117) 0.161 (0.202) -3.420 2K031402 SG 2A 0.727 (0.051) 1.012 (0.064) 0.000 (0.000)2K031403 SG 3A 0.623 (0.032) -0.520 (0.042) 0.000 (0.000) 0.656 (0.073) -0.656 (0.057)2K031404 SG 4A 0.907 (0.049) -0.438 (0.045) 0.000 (0.000)2K031405 SG 5 0.722 (0.212) 2.785 (0.446) 0.183 (0.018)2K031406 SG 6 0.432 (0.059) -0.327 (0.269) 0.253 (0.060)2K031407 SG 7A 0.461 (0.036) -1.038 (0.100) 0.000 (0.000)2K031408 SG 8A 0.600 (0.032) -0.043 (0.040) 0.000 (0.000) 0.660 (0.069) -0.660 (0.063)2K031409 SG 9A 0.488 (0.022) 0.381 (0.041) 0.000 (0.000) -0.904 (0.095) 0.904 (0.098)2K031410 SG 10A 0.419 (0.028) 0.599 (0.058) 0.000 (0.000) 0.654 (0.089) -0.654 (0.103)2K031801 SJ 2 0.799 (0.121) 0.935 (0.094) 0.187 (0.030)2K031901 SJ 3A 0.504 (0.031) -0.085 (0.049) 0.000 (0.000) 0.603 (0.088) -0.603 (0.081)2K032201 SJ 6 1.011 (0.109) 0.418 (0.069) 0.150 (0.027)2K032301 SJ 7 1.006 (0.185) 1.507 (0.118) 0.183 (0.021)2K032401 SJ 8 0.777 (0.131) 1.065 (0.104) 0.204 (0.031)2K032801 SK 2 0.590 (0.068) -0.250 (0.163) 0.195 (0.047)2K033001 SK 4A 0.705 (0.048) -1.156 (0.080) 0.000 (0.000)2K033101 SK 5A 0.523 (0.059) 3.268 (0.240) 0.000 (0.000) 0.859 (0.098) -0.859 (0.395)2K033201 SK 6 0.733 (0.072) -0.933 (0.160) 0.222 (0.052)2K033301 SK 7 0.547 (0.064) -0.575 (0.200) 0.218 (0.054)2K033401 SK 8 0.505 (0.072) 0.313 (0.178) 0.197 (0.046)2K033601 SL 1 0.397 (0.053) -2.489 (0.370) 0.222 (0.059)2K033701 SL 2 0.380 (0.050) -1.101 (0.281) 0.211 (0.056)2K033801 SL 3 0.618 (0.080) -0.169 (0.173) 0.221 (0.052)2K034101 SL 6A 1.438 (0.079) 0.834 (0.026) 0.000 (0.000) 0.478 (0.032) -0.478 (0.046)2K034102 SM 9 0.600 (0.061) -1.723 (0.209) 0.211 (0.056)2KW34101 SL 6B 1.871 (0.101) 0.497 (0.019) 0.000 (0.000) 0.322 (0.029) -0.322 (0.030)2KX34101 SL 6C 1.583 (0.072) 0.766 (0.027) 0.000 (0.000) 0.886 (0.032) -0.886 (0.052)2KY34101 SL 6D 1.200 (0.073) 0.870 (0.030) 0.000 (0.000) 0.068 (0.043) -0.068 (0.054)2K035001 SM 7 0.703 (0.079) -0.510 (0.165) 0.235 (0.052)2K035101 SM 8 0.726 (0.138) 1.189 (0.126) 0.233 (0.034)2K035201 SM 10A 0.414 (0.021) 0.712 (0.056) 0.000 (0.000) -1.607 (0.138) 1.607 (0.147)2K035301 SM 11A 0.510 (0.021) 0.697 (0.058) 0.000 (0.000) 2.790 (0.109) -0.996 (0.094) -1.794 2K036001 SN 7 0.460 (0.142) 2.736 (0.506) 0.255 (0.032)2K036101 SN 8A 0.659 (0.034) 1.367 (0.050) 0.000 (0.000) 1.157 (0.054) -1.157 (0.110)2K037401 SO 5A 0.981 (0.113) 2.473 (0.183) 0.000 (0.000) -0.129 (0.106) 0.129 (0.265)2K037601 SO 7A 0.515 (0.048) 2.100 (0.135) 0.000 (0.000) 0.600 (0.088) -0.600 (0.189)2K038601 ST 3 0.570 (0.225) 3.520 (0.987) 0.265 (0.022)2K038901 ST 6A 0.543 (0.035) 1.549 (0.071) 0.000 (0.000) 0.923 (0.075) -1.864 (0.213) 0.940 2K039101 ST 8 0.588 (0.102) 0.622 (0.163) 0.229 (0.046)
Table D-17 (continued)

IRT Parameters for the 1996 Science Main Samples

Earth Science, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 

6922K039401 ST 11A 0.539 (0.025) 0.164 (0.068) 0.000 (0.000) 3.785 (0.186) -0.795 (0.084) -2.990 2K039601 SU 2 0.718 (0.072) -1.094 (0.165) 0.211 (0.053)2K039801 SU 4A 0.522 (0.034) -1.432 (0.073) 0.000 (0.000) 1.306 (0.145) -1.306 (0.076)2K040401 SU 10A 0.523 (0.032) 1.604 (0.075) 0.000 (0.000) -1.822 (0.188) 1.718 (0.223) 0.104 2K040501 SU 11A 0.686 (0.070) 0.795 (0.087) 0.000 (0.000)
Table D-18

IRT Parameters for the 1996 Science Main Samples

Life Science, Grade 4

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 

3K031003 SC 3A 0.584 (0.030) 2.472 (0.054) 0.000 (0.000) 1.980 (0.051) -1.980 (0.236)3K031004 SC 4A 0.715 (0.043) -0.911 (0.065) 0.000 (0.000)3K031005 SC 5A 0.485 (0.028) 1.227 (0.059) 0.000 (0.000) 0.914 (0.066) -0.914 (0.102)3K031006 SC 6A 0.618 (0.033) -0.107 (0.038) 0.000 (0.000) 0.251 (0.069) -0.251 (0.061)3K031007 SC 7A 0.414 (0.028) 0.647 (0.056) 0.000 (0.000) -0.095 (0.097) 0.095 (0.108)3K031501 SH 1A 0.105 (0.005) -2.818 (0.216) 0.000 (0.000) -14.563 (0.824) 14.563 (0.796)3K031502 SH 2A 0.311 (0.012) 1.906 (0.090) 0.000 (0.000) 3.369 (0.105) -3.369 (0.225)3K031503 SH 3A 0.303 (0.022) 2.325 (0.134) 0.000 (0.000) 1.373 (0.105) -1.373 (0.205)3K031504 SH 4 0.721 (0.071) -2.361 (0.219) 0.258 (0.062)3K031505 SH 5A 0.598 (0.032) -0.989 (0.050) 0.000 (0.000) 0.801 (0.088) -0.801 (0.057)3K031506 SH 6A 0.584 (0.025) 1.297 (0.032) 0.000 (0.000) 2.323 (0.079) 1.320 (0.066) -0.015 3K031507 SH 7A 0.262 (0.011) 0.744 (0.108) 0.000 (0.000) 3.178 (0.154) -3.178 (0.191)3K031508 SH 8A 0.820 (0.074) 1.437 (0.095) 0.000 (0.000)3K031509 SH 3G 0.377 (0.019) 2.083 (0.077) 0.000 (0.000) 2.166 (0.083) -2.166 (0.196)3K031601 SI 1 0.792 (0.079) -0.192 (0.116) 0.222 (0.041)3K031602 SI 2A 0.515 (0.031) 0.799 (0.051) 0.000 (0.000) 0.185 (0.072) -0.185 (0.090)3K031603 SI 3A 0.477 (0.027) -1.307 (0.074) 0.000 (0.000) -0.144 (0.118) 0.144 (0.084)3K031604 SI 4A 0.343 (0.030) 3.601 (0.197) 0.000 (0.000) 1.692 (0.104) -1.692 (0.360)3K031606 SI 6A 0.396 (0.026) 2.774 (0.105) 0.000 (0.000) 1.902 (0.084) -1.902 (0.266)3K031607 SI 7A 0.464 (0.025) -0.567 (0.054) 0.000 (0.000) -0.161 (0.100) 0.161 (0.085)3K031608 SI 8A 0.457 (0.033) 3.458 (0.100) 0.000 (0.000) 2.482 (0.078) -2.482 (0.470)3K031609 SI 9A 0.603 (0.050) 2.105 (0.133) 0.000 (0.000) -0.209 (0.092) 0.209 (0.168)3K031701 SJ 1 0.801 (0.100) 0.163 (0.122) 0.259 (0.042)3K032601 SJ 10A 0.680 (0.044) 1.378 (0.058) 0.000 (0.000) 0.858 (0.058) -0.858 (0.110)3K032602 SJ 11A 0.449 (0.030) 0.586 (0.063) 0.000 (0.000) 1.083 (0.094) -1.083 (0.111)3K033501 SK 9A 1.201 (0.103) 1.485 (0.077) 0.000 (0.000)3K033502 SK 10A 0.500 (0.035) 0.806 (0.060) 0.000 (0.000) 0.646 (0.081) -0.646 (0.104)3K033503 SK 11A 0.527 (0.031) 0.445 (0.057) 0.000 (0.000) 1.228 (0.083) -1.228 (0.097)3K033901 SL 4 0.665 (0.089) 0.347 (0.133) 0.217 (0.041)3K034601 SM 1 0.304 (0.055) 0.407 (0.343) 0.243 (0.057)3K034701 SM 2 0.842 (0.184) 1.460 (0.146) 0.306 (0.029)3K035501 SN 2 0.386 (0.050) -0.804 (0.292) 0.250 (0.059)3K035901 SN 6A 0.391 (0.045) 3.310 (0.324) 0.000 (0.000) 0.064 (0.123) -0.064 (0.301)3K037001 SO 1 0.833 (0.139) 1.215 (0.108) 0.177 (0.027)3K037101 SO 2 0.511 (0.063) -0.985 (0.238) 0.251 (0.058)3K037201 SO 3 0.578 (0.117) 1.135 (0.175) 0.262 (0.042)3K038501 ST 2 0.836 (0.089) -1.281 (0.169) 0.272 (0.058)3K038701 ST 4 0.873 (0.108) 0.207 (0.107) 0.231 (0.038)3K038801 ST 5A 0.530 (0.022) 0.454 (0.063) 0.000 (0.000) 3.658 (0.145) -0.710 (0.081) -2.949 3K039001 ST 7 0.598 (0.076) -0.427 (0.192) 0.252 (0.054)3K039901 SU 5A 0.465 (0.022) -1.054 (0.040) 0.000 (0.000) 3.220 (0.273) -0.844 (0.113) -1.665 3K040201 SU 8 0.755 (0.201) 2.280 (0.314) 0.155 (0.022)3K0310CL SC 0.214 (0.006) 0.178 (0.040) 0.000 (0.000) -9.983 (0.660) 7.280 (0.678) -0.633 3K0325CL SJ 0.296 (0.013) -0.946 (0.059) 0.000 (0.000) -0.659 (0.253) 0.486 (0.237) -1.851 
Table D-19

IRT Parameters for 1996 Science Items

Physical Science, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 

6941K040601 SC 1A 0.573 (0.036) -0.130 (0.051) 0.000 (0.000)1K0406CL SC 0.493 (0.020) -0.488 (0.026) 0.000 (0.000) 1.772 (0.150) 0.777 (0.101) -0.201 (0.082) -0.1K040603 SC 3A 0.440 (0.016) -0.288 (0.041) 0.000 (0.000) -2.167 (0.132) 2.167 (0.127)1K040604 SC 4A 0.585 (0.023) 0.365 (0.033) 0.000 (0.000) -0.831 (0.077) 0.831 (0.080)1K040605 SC 5A 0.505 (0.032) 0.523 (0.045) 0.000 (0.000) 0.353 (0.073) -0.353 (0.082)1K040606 SC 6A 0.497 (0.019) -0.241 (0.043) 0.000 (0.000) 2.683 (0.137) -1.683 (0.086) -0.999 (0.107)1K040702 SD 3A 0.669 (0.031) -0.416 (0.034) 0.000 (0.000) 0.579 (0.061) -0.579 (0.049)1K040704 SD 5A 0.804 (0.034) -0.606 (0.032) 0.000 (0.000) 0.062 (0.057) -0.062 (0.045)1K040705 SD 6A 0.237 (0.029) 1.830 (0.243) 0.000 (0.000)1K031309 SF 6G 0.307 (0.013) -2.835 (0.131) 0.000 (0.000) -4.062 (0.545) 1.697 (0.513) 2.365 (0.183)1K031302 SF 7A 0.472 (0.034) 2.436 (0.119) 0.000 (0.000) 0.861 (0.071) -0.861 (0.179)1K031305 SF 8A 0.718 (0.053) 2.437 (0.105) 0.000 (0.000) 0.659 (0.058) -0.659 (0.186)1K031306 SF 9A 0.576 (0.072) 3.146 (0.323) 0.000 (0.000)1K041901 SJ 6A 0.439 (0.035) 2.847 (0.134) 0.000 (0.000) 1.478 (0.088) -1.478 (0.284)1K042301 SJ 11 0.292 (0.072) 2.113 (0.458) 0.261 (0.053)1K042401 SJ 12 1.399 (0.197) 1.622 (0.098) 0.259 (0.016)1K042501 SJ 13 0.965 (0.211) 1.984 (0.173) 0.195 (0.019)1K042801 SK 4 0.673 (0.108) 1.325 (0.125) 0.156 (0.030)1K042901 SK 5 0.810 (0.253) 2.577 (0.390) 0.215 (0.020)1K043201 SK 10 0.323 (0.056) 0.158 (0.341) 0.254 (0.060)1K043501 SK 13A 0.586 (0.029) 0.010 (0.049) 0.000 (0.000) 1.297 (0.079) -1.297 (0.074)1K043601 SK 14A 0.510 (0.032) 1.405 (0.067) 0.000 (0.000) 1.233 (0.075) -1.233 (0.134)1K043602 SK 15A 0.551 (0.063) 2.951 (0.275) 0.000 (0.000) -0.405 (0.139) 0.405 (0.306)1K043603 SK 16A 0.647 (0.052) -1.052 (0.098) 0.000 (0.000)1K046801 SL 1 0.407 (0.068) 0.147 (0.273) 0.260 (0.058)1K046901 SL 2 0.789 (0.099) 0.278 (0.112) 0.229 (0.038)1K047101 SL 4 1.109 (0.108) 0.166 (0.069) 0.177 (0.029)1K047201 SL 5A 0.547 (0.024) 0.365 (0.033) 0.000 (0.000) 0.219 (0.080) -1.441 (0.136) 1.221 (0.138)1K047401 SL 7A 0.630 (0.034) 0.940 (0.050) 0.000 (0.000) -0.427 (0.078) 0.427 (0.095)1K047901 SL 12A 0.759 (0.046) 0.873 (0.044) 0.000 (0.000) 0.303 (0.056) -0.303 (0.076)1K048201 SM 1 1.049 (0.146) 1.002 (0.079) 0.209 (0.025)1K048501 SM 4 1.455 (0.216) 1.014 (0.068) 0.301 (0.022)1K048601 SM 5A 0.428 (0.027) 1.176 (0.072) 0.000 (0.000) 1.261 (0.086) -1.261 (0.134)1K049001 SM 9A 0.450 (0.088) 5.270 (1.109) 0.000 (0.000) -1.817 (0.491) 1.817 (1.367)1K049101 SM 10 0.472 (0.057) -0.337 (0.195) 0.202 (0.049)1K035401 SN 1 0.261 (0.049) 1.318 (0.366) 0.263 (0.048)1K035601 SN 3A 0.283 (0.023) 2.225 (0.146) 0.000 (0.000) 1.140 (0.112) -1.140 (0.197)1K035701 SN 4 0.975 (0.109) 0.680 (0.074) 0.216 (0.027)1K035801 SN 5A 0.545 (0.025) -0.516 (0.042) 0.000 (0.000) -0.218 (0.081) 0.218 (0.070)1K036201 SN 8 0.883 (0.078) 0.477 (0.066) 0.121 (0.024)1K036301 SN 9A 0.505 (0.019) 0.553 (0.038) 0.000 (0.000) -1.522 (0.103) 1.522 (0.108)1K036601 SN 13 0.575 (0.060) -0.207 (0.151) 0.199 (0.045)1K036901 SN 16 0.843 (0.078) -0.176 (0.099) 0.201 (0.038)1K037301 SO 4A 0.418 (0.020) 0.987 (0.060) 0.000 (0.000) -2.174 (0.156) 2.174 (0.168)1K037501 SO 6A 0.611 (0.048) 2.226 (0.138) 0.000 (0.000) -0.439 (0.104) 0.439 (0.183)1K037701 SO 8A 0.324 (0.024) -3.165 (0.149) 0.000 (0.000) 0.225 (0.625) 1.852 (0.316) -2.077 (0.112)1K037703 SO 9A 0.532 (0.035) 0.673 (0.052) 0.000 (0.000) 0.473 (0.075) -0.473 (0.091)1K037901 SO 11 0.729 (0.160) 1.996 (0.212) 0.185 (0.025)1K038301 SO 16A 0.744 (0.035) 1.394 (0.036) 0.000 (0.000) 2.374 (0.074) 0.932 (0.059) -1.084 (0.122) -2.
Table D-19 (continued)

IRT Parameters for the 1996 Science Main Samples

Physical Science, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 1K043901 ST 3 0.698 (0.090) 0.076 (0.141) 0.248 (0.045)1K044201 ST 6A 0.463 (0.024) -0.282 (0.060) 0.000 (0.000) 1.543 (0.101) -1.543 (0.089)1K044301 ST 7A 0.484 (0.031) -0.142 (0.050) 0.000 (0.000) 0.598 (0.090) -0.598 (0.084)1K044701 ST 11 0.616 (0.102) 1.530 (0.157) 0.136 (0.027)1K045101 ST 15A 0.523 (0.029) 1.285 (0.056) 0.000 (0.000) -0.419 (0.101) 0.754 (0.128) -0.334 (0.155)1K045102 ST 16A 0.450 (0.032) 1.496 (0.083) 0.000 (0.000) 0.652 (0.093) -1.132 (0.178) 0.481 (0.235)1K045401 SU 3 0.761 (0.102) -0.369 (0.182) 0.344 (0.057)1K045501 SU 4 0.824 (0.161) 1.200 (0.132) 0.342 (0.033)1K045801 SU 7A 0.522 (0.027) 1.562 (0.063) 0.000 (0.000) 1.849 (0.070) -1.849 (0.162)1K045901 SU 8 0.401 (0.080) 0.753 (0.294) 0.293 (0.059)1K046201 SU 11 0.462 (0.123) 2.139 (0.333) 0.282 (0.041)1K046501 SU 14A 0.699 (0.044) 1.280 (0.056) 0.000 (0.000) 0.775 (0.057) -0.775 (0.102)1K046601 SU 15A 0.554 (0.035) 0.535 (0.048) 0.000 (0.000) 0.211 (0.077) -0.211 (0.088)
Table D-20

IRT Parameters for the 1996 Science Main Samples

Earth Science, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 6962K040701 SD 1A 0.361 (0.015) 1.169 (0.055) 0.000 (0.000) 1.346 (0.083) -3.294 (0.206) 1.948 (0.234)2K040706 SD 7 0.644 (0.075) -1.906 (0.289) 0.505 (0.068)2K040708 SD 9A 0.503 (0.024) 1.446 (0.063) 0.000 (0.000) -0.662 (0.083) 0.662 (0.109)2K040709 SD 10 0.694 (0.088) -0.624 (0.233) 0.475 (0.058)2K040713 SD 14A 0.631 (0.024) 0.810 (0.029) 0.000 (0.000) -0.385 (0.072) -0.322 (0.106) 0.707 (0.102)2K040801 SE 0A 0.498 (0.025) -0.763 (0.048) 0.000 (0.000) 0.125 (0.085) -0.125 (0.067)2K040802 SE 1A 0.565 (0.025) -1.561 (0.063) 0.000 (0.000) -0.868 (0.118) 0.868 (0.090)2K040808 SE 1G 0.570 (0.020) 0.032 (0.033) 0.000 (0.000) -1.497 (0.099) 1.497 (0.098)2K040809 SE 1M 0.621 (0.023) 0.639 (0.034) 0.000 (0.000) -1.304 (0.092) 1.304 (0.098)2K040803 SE 2A 0.737 (0.045) -0.238 (0.049) 0.000 (0.000)2K040804 SE 3A 0.774 (0.051) 1.779 (0.076) 0.000 (0.000) -0.494 (0.091) 0.494 (0.129)2K040805 SE 4A 0.799 (0.046) 1.489 (0.057) 0.000 (0.000) -0.918 (0.106) 0.918 (0.127)2K040806 SE 5A 1.037 (0.104) 1.612 (0.095) 0.000 (0.000)2K040901 SG 1A 0.831 (0.046) -0.777 (0.052) 0.000 (0.000)2K040902 SG 2A 0.479 (0.029) 1.311 (0.066) 0.000 (0.000) 0.820 (0.069) -0.820 (0.110)2K040903 SG 3A 0.542 (0.030) 0.627 (0.045) 0.000 (0.000) 0.635 (0.064) -0.635 (0.078)2K040904 SG 4A 0.667 (0.045) 0.994 (0.067) 0.000 (0.000)2K040905 SG 5A 1.106 (0.142) 2.681 (0.212) 0.000 (0.000)2K041001 SG 6 0.552 (0.053) -0.485 (0.161) 0.186 (0.047)2K041002 SG 7A 0.589 (0.026) -0.085 (0.035) 0.000 (0.000) -0.121 (0.070) 0.121 (0.066)2K041003 SG 8 1.185 (0.094) -1.184 (0.093) 0.202 (0.045)2K041004 SG 9A 0.559 (0.028) 0.714 (0.043) 0.000 (0.000) 0.002 (0.067) -0.002 (0.079)2K041101 SG 10A 0.578 (0.044) 1.299 (0.092) 0.000 (0.000)2K041201 SG 11A 0.433 (0.020) 1.048 (0.057) 0.000 (0.000) -1.027 (0.101) 1.027 (0.118)2K041202 SG 12A 0.548 (0.033) 1.784 (0.088) 0.000 (0.000) -0.472 (0.088) 0.472 (0.130)2K041501 SJ 1 0.309 (0.046) -2.282 (0.433) 0.225 (0.061)2K041601 SJ 2 0.420 (0.057) -0.287 (0.253) 0.224 (0.058)2K041701 SJ 3 1.071 (0.284) 2.745 (0.330) 0.188 (0.013)2K041801 SJ 4 0.806 (0.084) -0.527 (0.138) 0.230 (0.051)2K041802 SJ 5 1.032 (0.157) 1.130 (0.090) 0.251 (0.025)2K042201 SJ 10A 0.736 (0.073) 2.752 (0.187) 0.000 (0.000) -0.008 (0.106) 0.008 (0.266)2K042701 SK 1 0.616 (0.083) 0.298 (0.160) 0.227 (0.048)2K042702 SK 2 1.097 (0.127) 0.530 (0.076) 0.229 (0.030)2K042703 SK 3 1.099 (0.135) 0.492 (0.084) 0.278 (0.032)2K043101 SK 7A 0.589 (0.029) 0.120 (0.039) 0.000 (0.000) -0.199 (0.078) 0.199 (0.078)2K043102 SK 8A 0.349 (0.019) 0.116 (0.047) 0.000 (0.000) -0.438 (0.149) 1.001 (0.148) -0.563 (0.130)2K043103 SK 9A 0.614 (0.049) 2.194 (0.125) 0.000 (0.000) 0.107 (0.084) -0.107 (0.168)2K043301 SK 11 0.690 (0.093) 0.806 (0.116) 0.186 (0.035)2K047001 SL 3 0.393 (0.071) 1.090 (0.246) 0.204 (0.049)2K047301 SL 6A 0.567 (0.038) 1.185 (0.065) 0.000 (0.000) 0.451 (0.071) -0.451 (0.106)2K047601 SL 9 1.091 (0.191) 2.004 (0.151) 0.096 (0.013)2K048301 SM 2 0.876 (0.088) -0.300 (0.112) 0.209 (0.042)2K048701 SM 6 0.367 (0.060) 0.493 (0.272) 0.218 (0.055)2K048801 SM 7 0.639 (0.153) 1.995 (0.232) 0.243 (0.030)2K049401 SM 13A 0.608 (0.037) 1.718 (0.065) 0.000 (0.000) 1.254 (0.063) -1.254 (0.149)2K049402 SM 14A 0.411 (0.029) 0.244 (0.058) 0.000 (0.000) 0.525 (0.101) -0.525 (0.106)2K036101 SN 7A 0.327 (0.013) 0.555 (0.078) 0.000 (0.000) 2.610 (0.110) -2.610 (0.135)2K0364CL SN 0.441 (0.029) 3.001 (0.093) 0.000 (0.000) 1.827 (0.077) 0.678 (0.134) -2.505 (0.601)2K036402 SN 11A 0.286 (0.025) 2.341 (0.177) 0.000 (0.000) 0.629 (0.117) -0.629 (0.200)
Table D-20 (continued)

IRT Parameters for the 1996 Science Main Samples

Earth Science, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 2K037401 SO 5A 0.623 (0.052) 2.286 (0.135) 0.000 (0.000) 0.175 (0.085) -0.175 (0.184)2K037601 SO 7A 0.566 (0.035) 0.692 (0.051) 0.000 (0.000) 0.659 (0.070) -0.659 (0.089)2K037801 SO 10 0.838 (0.108) 0.823 (0.092) 0.181 (0.030)2K038201 SO 15A 0.362 (0.035) 4.344 (0.219) 0.000 (0.000) 2.264 (0.114) -2.264 (0.607)2K043801 ST 2 0.845 (0.101) 0.119 (0.117) 0.244 (0.041)2K044001 ST 4 1.391 (0.228) 2.461 (0.187) 0.076 (0.008)2K044101 ST 5A 0.852 (0.061) 1.114 (0.069) 0.000 (0.000)2K044401 ST 8A 0.668 (0.049) -1.267 (0.087) 0.000 (0.000)2K044501 ST 9 1.029 (0.195) 1.577 (0.120) 0.253 (0.021)2K044801 ST 12 0.296 (0.050) 0.199 (0.361) 0.232 (0.060)2K045701 SU 6A 0.451 (0.030) 0.070 (0.053) 0.000 (0.000) 0.603 (0.092) -0.603 (0.094)2K046101 SU 10 0.700 (0.077) 0.173 (0.117) 0.168 (0.039)2K0407CL SD 0.836 (0.032) -0.003 (0.022) 0.000 (0.000) 0.948 (0.052) -0.602 (0.053) -0.346 (0.055)2K0494CL SM 0.164 (0.008) 2.326 (0.103) 0.000 (0.000) -11.081 (0.930) 9.529 (0.977) 3.780 (0.423) 0.
Table D-21

IRT Parameters for the 1996 Science Main Samples

Life Science, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 6983K031307 SF 10A 1.053 (0.057) 1.643 (0.040) 0.000 (0.000) 0.813 (0.036) -0.813 (0.099)3K031308 SF 11A 0.433 (0.034) 2.539 (0.181) 0.000 (0.000) -0.898 (0.128) 0.898 (0.207)3K041301 SH 1 0.714 (0.067) -2.541 (0.206) 0.220 (0.057)3K041302 SH 2 0.816 (0.069) -1.692 (0.163) 0.242 (0.059)3K041303 SH 3 0.693 (0.061) -1.058 (0.158) 0.221 (0.051)3K041304 SH 4 0.756 (0.077) 0.272 (0.091) 0.167 (0.032)3K041305 SH 5 1.045 (0.081) -0.804 (0.095) 0.201 (0.039)3K041306 SH 6A 0.526 (0.036) -0.709 (0.075) 0.000 (0.000)3K041307 SH 7A 0.463 (0.031) 1.594 (0.088) 0.000 (0.000) -0.141 (0.081) 0.141 (0.120)3K041401 SH 8A 0.771 (0.034) 0.831 (0.035) 0.000 (0.000) 0.988 (0.045) -0.988 (0.066)3K041402 SH 9A 0.986 (0.047) 1.588 (0.035) 0.000 (0.000) 1.124 (0.035) -1.124 (0.110)3K041403 SH 10A 0.268 (0.022) 4.408 (0.206) 0.000 (0.000) 2.471 (0.120) -2.471 (0.436)3K031601 SI 1 0.658 (0.059) -2.226 (0.197) 0.218 (0.057)3K031602 SI 2A 0.362 (0.023) -0.443 (0.060) 0.000 (0.000) 0.742 (0.108) -0.742 (0.094)3K031603 SI 3A 0.651 (0.042) -1.551 (0.091) 0.000 (0.000)3K031604 SI 4A 0.429 (0.030) 1.949 (0.102) 0.000 (0.000) 0.888 (0.077) -0.888 (0.155)3K031605 SI 5 0.409 (0.044) -1.641 (0.266) 0.218 (0.057)3K031606 SI 6A 0.491 (0.025) 1.040 (0.055) 0.000 (0.000) 1.227 (0.067) -1.227 (0.103)3K031610 SI 7A 0.601 (0.024) 0.226 (0.044) 0.000 (0.000) 1.453 (0.064) -1.453 (0.070)3K031607 SI 8A 0.504 (0.023) -0.711 (0.040) 0.000 (0.000) 0.382 (0.120) 0.863 (0.094) -1.246 (0.069)3K031608 SI 9A 0.501 (0.023) 1.861 (0.059) 0.000 (0.000) 2.048 (0.063) -2.048 (0.175)3K031609 SI 10A 0.364 (0.022) 1.184 (0.079) 0.000 (0.000) -0.521 (0.104) 0.521 (0.130)3K031611 SI 11A 0.336 (0.042) 2.751 (0.328) 0.000 (0.000)3K031612 SI 12 0.814 (0.084) -0.435 (0.136) 0.271 (0.046)3K031613 SI 13A 0.653 (0.043) 2.217 (0.118) 0.000 (0.000) -1.268 (0.136) 1.268 (0.189)3K042001 SJ 7A 0.549 (0.040) 1.939 (0.090) 0.000 (0.000) 0.940 (0.071) -0.940 (0.163)3K042101 SJ 8A 0.217 (0.031) 9.143 (0.634) 0.000 (0.000) 4.783 (0.204) -4.783 (2.220)3K042102 SJ 9A 0.745 (0.039) 1.368 (0.048) 0.000 (0.000) 1.250 (0.052) -1.250 (0.115)3K042601 SJ 14A 0.588 (0.028) 0.458 (0.039) 0.000 (0.000) -0.704 (0.087) 0.704 (0.091)3K042602 SJ 15A 0.628 (0.042) 1.094 (0.056) 0.000 (0.000) 0.619 (0.065) -0.619 (0.096)3K042603 SJ 16A 0.525 (0.061) 2.088 (0.204) 0.000 (0.000)3K043001 SK 6A 0.267 (0.033) 3.195 (0.337) 0.000 (0.000) 0.525 (0.151) -0.525 (0.303)3K043401 SK 12 0.201 (0.040) -0.641 (0.533) 0.242 (0.061)3K047501 SL 8 0.252 (0.075) 3.689 (0.881) 0.224 (0.042)3K047701 SL 10 0.837 (0.291) 2.676 (0.482) 0.159 (0.017)3K047801 SL 11 0.991 (0.163) 1.269 (0.097) 0.204 (0.023)3K048001 SL 13A 0.835 (0.035) 1.080 (0.050) 0.000 (0.000) 1.955 (0.054) -1.955 (0.135)3K048101 SL 14A 0.397 (0.028) 1.967 (0.124) 0.000 (0.000) -0.260 (0.118) -1.932 (0.335) 2.191 (0.385)3K048102 SL 15A 0.704 (0.103) 3.792 (0.310) 0.000 (0.000) 1.081 (0.111) -1.081 (1.015)3K048103 SL 16A 0.555 (0.052) 0.871 (0.093) 0.000 (0.000)3K048401 SM 3 0.496 (0.097) 1.015 (0.191) 0.246 (0.046)3K048901 SM 8A 0.519 (0.033) 0.577 (0.051) 0.000 (0.000) 0.645 (0.076) -0.645 (0.091)3K049201 SM 11 0.152 (0.035) 0.427 (0.650) 0.251 (0.057)3K049301 SM 12A 0.303 (0.057) 4.416 (0.785) 0.000 (0.000)3K035501 SN 2 0.528 (0.050) -1.750 (0.218) 0.220 (0.057)3K035901 SN 6A 0.523 (0.032) 1.386 (0.071) 0.000 (0.000) -0.090 (0.073) 0.090 (0.104)3K036501 SN 12 0.539 (0.097) 0.826 (0.171) 0.277 (0.046)3K036701 SN 14A 0.478 (0.040) 2.558 (0.189) 0.000 (0.000) -0.510 (0.109) 0.510 (0.202)3K036801 SN 15A 0.679 (0.036) 1.240 (0.052) 0.000 (0.000) -0.217 (0.062) 0.217 (0.085)
Table D-21 (continued)

IRT Parameters for the 1996 Science Main Samples

Life Science, Grade 8

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 3K037101 SO 2 0.220 (0.041) -2.919 (0.665) 0.240 (0.062)3K037201 SO 3 0.644 (0.066) -1.386 (0.196) 0.231 (0.058)3K038001 SO 12 0.927 (0.213) 1.796 (0.174) 0.232 (0.023)3K038002 SO 13 1.433 (0.192) 1.318 (0.072) 0.176 (0.017)3K038101 SO 14A 0.637 (0.081) 2.529 (0.259) 0.000 (0.000)3K043701 ST 1 0.510 (0.063) -0.666 (0.230) 0.240 (0.058)3K044601 ST 10 0.329 (0.110) 4.057 (1.009) 0.199 (0.032)3K044901 ST 13A 0.590 (0.026) 0.747 (0.054) 0.000 (0.000) 1.629 (0.070) -1.629 (0.101)3K045001 ST 14A 0.433 (0.027) 1.302 (0.074) 0.000 (0.000) 1.319 (0.086) -1.319 (0.140)3K045201 SU 1 0.508 (0.057) -1.736 (0.261) 0.240 (0.062)3K045301 SU 2A 0.317 (0.027) 1.120 (0.102) 0.000 (0.000) 0.648 (0.119) -0.648 (0.158)3K045601 SU 5A 0.414 (0.024) 1.528 (0.071) 0.000 (0.000) -0.538 (0.120) 1.288 (0.146) -0.750 (0.197)3K046001 SU 9 0.444 (0.064) 0.465 (0.184) 0.177 (0.044)3K046301 SU 12A 0.253 (0.011) 2.279 (0.130) 0.000 (0.000) 4.350 (0.150) -4.350 (0.329)3K046401 SU 13A 0.610 (0.056) 2.127 (0.143) 0.000 (0.000) 0.139 (0.084) -0.139 (0.177)3K046701 SU 16A 0.464 (0.020) 1.332 (0.065) 0.000 (0.000) 2.472 (0.092) -1.658 (0.145) -0.815 (0.266)
Table D-22

IRT Parameters for 1996 Science Items

Physical Science, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 7001K049501 SC 2A 0.234 (0.011) -3.558 (0.198) 0.000 (0.000) -2.892 (0.528) 0.080 (0.444) 2.811 (0.221)1K049502 SC 4A 0.342 (0.020) 2.539 (0.147) 0.000 (0.000) -1.006 (0.124) -2.028 (0.381) 3.033 (0.437)1K049503 SC 5A 0.941 (0.047) -0.019 (0.035) 0.000 (0.000)1K049601 SD 1A 0.645 (0.035) 1.651 (0.050) 0.000 (0.000) 0.924 (0.052) -0.021 (0.083) -0.903 (0.175)1K049602 SD 2A 0.182 (0.008) -1.686 (0.124) 0.000 (0.000) -4.953 (0.307) 4.953 (0.281)1K049603 SD 3A 0.390 (0.015) -0.697 (0.043) 0.000 (0.000) 0.110 (0.125) -3.289 (0.213) 3.180 (0.203)1K049604 SD 4A 0.630 (0.031) 0.582 (0.037) 0.000 (0.000) -0.385 (0.072) 0.385 (0.078)1K049701 SF 1A 0.412 (0.016) -0.645 (0.048) 0.000 (0.000) -1.663 (0.121) 1.663 (0.111)1K049702 SF 2A 0.941 (0.057) -1.321 (0.066) 0.000 (0.000)1K049708 SF 3A 1.172 (0.056) -0.055 (0.030) 0.000 (0.000)1K049703 SF 4A 1.373 (0.091) 1.450 (0.053) 0.000 (0.000)1K049704 SF 5A 1.241 (0.052) -0.707 (0.025) 0.000 (0.000) 0.769 (0.049) -0.769 (0.029)1K049705 SF 6A 0.652 (0.025) 0.325 (0.025) 0.000 (0.000) -0.416 (0.077) 0.874 (0.079) -0.459 (0.064)1K049706 SF 7A 1.383 (0.069) 0.146 (0.028) 0.000 (0.000)1K049707 SF 8A 0.585 (0.026) 0.665 (0.028) 0.000 (0.000) 0.525 (0.088) 1.138 (0.084) -0.328 (0.078) -1.1K049806 SG 6 0.947 (0.081) -0.275 (0.091) 0.205 (0.038)1K049808 SG 8 0.651 (0.069) -0.120 (0.141) 0.219 (0.045)1K049914 SI 12A 0.740 (0.029) 0.687 (0.032) 0.000 (0.000) -1.121 (0.086) 1.121 (0.092)1K049912 SI 13A 0.631 (0.033) 1.332 (0.041) 0.000 (0.000) 0.409 (0.067) 0.285 (0.093) -0.335 (0.133) -0.1K049913 SI 14 1.369 (0.164) 0.922 (0.056) 0.178 (0.020)1K050001 SJ 1 0.624 (0.067) -0.936 (0.176) 0.224 (0.051)1K050101 SJ 2 0.679 (0.087) 0.347 (0.130) 0.203 (0.042)1K050201 SJ 3 1.609 (0.220) 0.904 (0.056) 0.268 (0.021)1K050401 SJ 5A 0.323 (0.042) 3.855 (0.404) 0.000 (0.000) 0.767 (0.143) -0.767 (0.415)1K050701 SJ 8 1.711 (0.208) 1.457 (0.078) 0.261 (0.015)1K050801 SJ 9 1.658 (0.163) 0.320 (0.049) 0.190 (0.024)1K050802 SJ 10 1.430 (0.135) 0.262 (0.055) 0.175 (0.025)1K051401 SK 2 0.874 (0.117) 0.335 (0.118) 0.292 (0.041)1K051501 SK 3 1.095 (0.206) 1.310 (0.103) 0.312 (0.025)1K051901 SK 7 1.165 (0.234) 1.671 (0.134) 0.298 (0.020)1K052301 SK 11A 0.333 (0.022) 3.806 (0.106) 0.000 (0.000) 3.617 (0.106) -3.617 (0.532)1K052401 SK 12A 0.584 (0.032) 1.107 (0.047) 0.000 (0.000) 0.056 (0.077) -0.281 (0.117) 0.226 (0.137)1K052402 SK 13A 0.517 (0.037) 3.407 (0.074) 0.000 (0.000) 2.987 (0.071) -2.987 (0.619)1K046801 SL 1 0.576 (0.065) -0.683 (0.183) 0.220 (0.052)1K046901 SL 2 0.907 (0.083) -0.323 (0.097) 0.180 (0.038)1K047101 SL 4 1.061 (0.092) -0.592 (0.090) 0.190 (0.039)1K047201 SL 5A 0.542 (0.023) -0.444 (0.037) 0.000 (0.000) 0.485 (0.101) -1.391 (0.122) 0.906 (0.113)1K047401 SL 7A 0.521 (0.027) 0.501 (0.046) 0.000 (0.000) -0.561 (0.091) 0.561 (0.099)1K047901 SL 12A 0.706 (0.041) 0.133 (0.038) 0.000 (0.000) 0.490 (0.064) -0.490 (0.064)1K048201 SM 1 1.725 (0.166) -0.047 (0.058) 0.240 (0.031)1K048501 SM 4 1.046 (0.130) 0.258 (0.099) 0.292 (0.038)1K048601 SM 5A 0.572 (0.035) 0.177 (0.046) 0.000 (0.000) 0.775 (0.075) -0.775 (0.078)1K049001 SM 9A 0.989 (0.074) 1.683 (0.062) 0.000 (0.000) -2.047 (0.277) 2.047 (0.288)1K049101 SM 10 0.410 (0.053) -0.985 (0.258) 0.214 (0.055)1K052601 SN 1 0.623 (0.064) -0.201 (0.141) 0.209 (0.045)1K052701 SN 2 0.918 (0.106) 0.835 (0.073) 0.174 (0.025)1K053101 SN 7A 0.846 (0.041) 1.021 (0.037) 0.000 (0.000) -0.074 (0.050) 0.074 (0.066)1K053102 SN 8A 1.061 (0.049) 0.914 (0.030) 0.000 (0.000) 0.762 (0.034) -0.762 (0.057)1K053201 SN 9 1.463 (0.149) 1.035 (0.045) 0.105 (0.014)
Table D-22 (continued)

IRT Parameters for Science Items

Physical Science, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 1K053301 SN 10 2.541 (0.207) 0.679 (0.029) 0.201 (0.014)1K053401 SN 11 0.521 (0.071) 0.309 (0.179) 0.238 (0.048)1K057501 ST 6A 0.349 (0.033) 1.896 (0.160) 0.000 (0.000) 0.167 (0.116) -0.167 (0.184)1K058001 ST 11A 0.296 (0.030) 5.914 (0.206) 0.000 (0.000) 4.222 (0.125) -4.222 (1.164)1K058101 ST 12 1.897 (0.206) 1.106 (0.055) 0.293 (0.017)1K058301 ST 14A 0.731 (0.044) 1.362 (0.050) 0.000 (0.000) 0.101 (0.067) 0.027 (0.104) -0.128 (0.139)1K058601 SU 1 0.664 (0.076) -0.497 (0.167) 0.245 (0.052)1K059001 SU 5A 0.203 (0.025) -3.684 (0.420) 0.000 (0.000) 0.516 (0.372) -0.516 (0.199)1K059101 SU 6A 0.545 (0.039) 2.643 (0.093) 0.000 (0.000) 1.643 (0.071) -1.643 (0.293)1K059501 SU 10 1.988 (0.249) 2.063 (0.117) 0.091 (0.009)
Table D-23

IRT Parameters for 1996 Science Items

Earth Science, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 7022K040801 SE 0A 0.432 (0.034) -1.567 (0.130) 0.000 (0.000)2K040802 SE 1A 0.545 (0.027) -2.271 (0.097) 0.000 (0.000) -1.361 (0.179) 1.361 (0.133)2K040808 SE 1G 0.652 (0.024) -0.914 (0.040) 0.000 (0.000) -1.273 (0.103) 1.273 (0.092)2K040809 SE 1M 0.644 (0.022) -0.356 (0.032) 0.000 (0.000) -1.363 (0.094) 1.363 (0.089)2K040803 SE 2A 0.540 (0.025) -0.529 (0.043) 0.000 (0.000) -0.239 (0.083) 0.239 (0.070)2K040804 SE 3A 0.952 (0.040) 0.598 (0.025) 0.000 (0.000) -0.343 (0.052) 0.343 (0.055)2K040805 SE 4A 0.854 (0.035) 0.498 (0.027) 0.000 (0.000) -0.627 (0.064) 0.627 (0.066)2K040806 SE 5A 1.020 (0.072) 0.681 (0.042) 0.000 (0.000)2K049801 SG 1 1.192 (0.105) -1.823 (0.113) 0.218 (0.049)2K049802 SG 2A 0.446 (0.017) -0.409 (0.045) 0.000 (0.000) -1.443 (0.115) 1.443 (0.108)2K049803 SG 3 1.517 (0.152) 0.341 (0.054) 0.282 (0.024)2K049804 SG 4A 0.487 (0.037) 0.229 (0.065) 0.000 (0.000)2K049805 SG 5 0.653 (0.058) -1.394 (0.170) 0.232 (0.053)2K049807 SG 7A 0.280 (0.009) 0.229 (0.043) 0.000 (0.000) -4.008 (0.234) 0.649 (0.294) 3.360 (0.212)2K049809 SG 9A 0.974 (0.056) 1.239 (0.042) 0.000 (0.000) 0.203 (0.043) -0.203 (0.068)2K049810 SG 10A 0.699 (0.028) -0.668 (0.037) 0.000 (0.000) -0.476 (0.077) 0.476 (0.066)2K089811 SG 11A 0.701 (0.049) 0.900 (0.063) 0.000 (0.000)2K049812 SG 12A 0.697 (0.064) 1.887 (0.136) 0.000 (0.000)2K049813 SG 13A 0.586 (0.034) 0.676 (0.043) 0.000 (0.000) 0.568 (0.061) -0.568 (0.076)2K049814 SG 14A 0.712 (0.034) 0.985 (0.039) 0.000 (0.000) -0.566 (0.069) 0.566 (0.081)2K049815 SG 15A 0.482 (0.033) 1.469 (0.070) 0.000 (0.000) 0.410 (0.084) -0.022 (0.121) -0.387 (0.180)2K049901 SI 1A 0.559 (0.029) -0.675 (0.047) 0.000 (0.000) 0.128 (0.083) -0.128 (0.066)2K049902 SI 2A 0.541 (0.030) 0.055 (0.041) 0.000 (0.000) 0.515 (0.071) -0.515 (0.070)2K049903 SI 3A 0.632 (0.042) 0.413 (0.055) 0.000 (0.000)2K049904 SI 4A 0.855 (0.051) 0.440 (0.042) 0.000 (0.000)2K049905 SI 5 0.443 (0.051) -2.088 (0.303) 0.264 (0.062)2K049906 SI 6 1.527 (0.214) 1.420 (0.087) 0.343 (0.016)2K049907 SI 7A 0.881 (0.050) -0.702 (0.050) 0.000 (0.000)2K049908 SI 8A 0.991 (0.046) 0.554 (0.029) 0.000 (0.000) 0.699 (0.039) -0.699 (0.049)2K049909 SI 9A 1.081 (0.057) 1.321 (0.034) 0.000 (0.000) 0.163 (0.044) -0.073 (0.073) -0.091 (0.098)2K049910 SI 10 0.890 (0.157) 1.431 (0.115) 0.255 (0.025)2K049911 SI 11A 0.561 (0.028) 0.752 (0.044) 0.000 (0.000) -0.361 (0.075) 0.361 (0.087)2K050501 SJ 6A 0.563 (0.028) 1.227 (0.043) 0.000 (0.000) 0.536 (0.081) 0.980 (0.089) -1.516 (0.161)2K050601 SJ 7 0.827 (0.117) 0.711 (0.095) 0.212 (0.032)2K050901 SJ 11A 1.055 (0.075) 2.872 (0.053) 0.000 (0.000) 1.897 (0.043) -1.897 (0.808)2K051601 SK 4 1.134 (0.158) 0.789 (0.070) 0.226 (0.027)2K051701 SK 5A 0.847 (0.056) 0.295 (0.047) 0.000 (0.000)2K051801 SK 6A 0.244 (0.024) 1.352 (0.141) 0.000 (0.000) 0.814 (0.155) -0.814 (0.205)2K047001 SL 3 0.649 (0.103) 0.177 (0.187) 0.325 (0.052)2K047301 SL 6A 0.533 (0.035) 0.341 (0.048) 0.000 (0.000) 0.565 (0.078) -0.565 (0.085)2K047601 SL 9 1.176 (0.180) 1.463 (0.096) 0.108 (0.016)2K048301 SM 2 0.731 (0.076) -0.854 (0.163) 0.248 (0.053)2K048701 SM 6 0.601 (0.080) -0.224 (0.198) 0.279 (0.055)2K048801 SM 7 1.320 (0.215) 1.266 (0.084) 0.284 (0.021)2K049401 SM 13A 0.661 (0.037) 1.284 (0.053) 0.000 (0.000) 1.222 (0.060) -1.222 (0.115)2K049402 SM 14A 0.460 (0.032) 0.111 (0.055) 0.000 (0.000) 0.722 (0.096) -0.722 (0.094)2K052801 SN 3 0.452 (0.092) 1.318 (0.225) 0.302 (0.046)2K053001 SN 6A 0.621 (0.039) 2.494 (0.080) 0.000 (0.000) 1.332 (0.057) -1.332 (0.219)2K053501 SN 12 2.342 (0.177) 1.001 (0.036) 0.267 (0.014)
Table D-23 (continued)

IRT Parameters for Science Items

Earth Science, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 2K057101 ST 1 0.522 (0.063) -1.263 (0.262) 0.286 (0.064)2K057401 ST 5A 0.524 (0.033) -0.072 (0.048) 0.000 (0.000) 0.668 (0.084) -0.668 (0.079)2K057901 ST 10 0.868 (0.108) 0.272 (0.108) 0.248 (0.038)2K058401 ST 15A 0.636 (0.036) 0.587 (0.042) 0.000 (0.000) -0.236 (0.077) 0.236 (0.085)2K058701 SU 2 0.411 (0.057) -0.831 (0.303) 0.277 (0.062)2K058901 SU 4 0.923 (0.134) 0.573 (0.095) 0.259 (0.033)2K059301 SU 8A 0.489 (0.030) 2.210 (0.114) 0.000 (0.000) -2.005 (0.199) 2.077 (0.250) -0.073 (0.304)2K059701 SU 12 0.710 (0.153) 1.596 (0.178) 0.232 (0.029)2K059801 SU 13A 0.824 (0.065) 1.688 (0.081) 0.000 (0.000) 0.363 (0.059) -0.363 (0.126)2K059901 SU 14A 0.414 (0.016) 1.793 (0.050) 0.000 (0.000) -1.452 (0.200) 4.458 (0.202) -3.006 (0.293)2K060001 SU 15A 0.353 (0.029) 0.402 (0.071) 0.000 (0.000) 0.396 (0.118) -0.396 (0.130)2K060101 SU 16A 0.278 (0.015) 1.403 (0.096) 0.000 (0.000) 2.841 (0.153) -1.424 (0.191) -1.416 (0.332)2K0494CL SM 0.161 (0.008) 1.565 (0.087) 0.000 (0.000) -14.767 (1.457) 11.377 (1.496) 3.015 (0.472) 1.
Table D-24

IRT Parameters for 1996 Science Items

Life Science, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 7043K049504 SC 6A 0.644 (0.022) -0.012 (0.029) 0.000 (0.000) -0.938 (0.074) 0.938 (0.072)3K049505 SC 7A 0.699 (0.028) -0.486 (0.033) 0.000 (0.000) -0.252 (0.066) 0.252 (0.056)3K049506 SC 8A 0.699 (0.045) 2.357 (0.075) 0.000 (0.000) 1.147 (0.053) -1.147 (0.199)3K041301 SH 1 0.646 (0.072) -3.292 (0.309) 0.209 (0.057)3K041302 SH 2 0.889 (0.093) -2.611 (0.209) 0.211 (0.055)3K041303 SH 3 0.650 (0.060) -2.123 (0.194) 0.189 (0.051)3K041304 SH 4 0.470 (0.047) -0.480 (0.163) 0.160 (0.042)3K041305 SH 5 1.436 (0.115) -1.208 (0.076) 0.196 (0.040)3K041307 SH 7A 0.398 (0.027) 1.084 (0.073) 0.000 (0.000) 0.464 (0.084) -0.464 (0.113)3K041401 SH 8A 0.494 (0.049) 2.128 (0.188) 0.000 (0.000)3K041402 SH 9A 0.947 (0.039) 0.999 (0.033) 0.000 (0.000) 1.180 (0.037) -1.180 (0.073)3K041403 SH 10A 0.215 (0.017) 4.245 (0.226) 0.000 (0.000) 2.506 (0.145) -2.506 (0.391)3K041404 SH 11A 0.395 (0.034) -1.272 (0.127) 0.000 (0.000)3K041405 SH 12 0.636 (0.073) 0.275 (0.123) 0.188 (0.039)3K041406 SH 13A 0.459 (0.039) 2.313 (0.160) 0.000 (0.000) 0.093 (0.091) -0.093 (0.177)3K050301 SJ 4 0.602 (0.117) 1.668 (0.184) 0.169 (0.031)3K051001 SJ 12A 0.393 (0.021) -0.581 (0.061) 0.000 (0.000) -0.975 (0.133) 0.975 (0.121)3K051002 SJ 13A 1.608 (0.096) 0.411 (0.030) 0.000 (0.000)3K051003 SJ 13G 1.252 (0.079) 0.568 (0.038) 0.000 (0.000)3K051004 SJ 13M 1.115 (0.078) 0.932 (0.051) 0.000 (0.000)3K051101 SJ 14A 0.260 (0.015) 1.386 (0.109) 0.000 (0.000) -3.032 (0.229) 3.032 (0.255)3K051102 SJ 15A 0.854 (0.070) 1.942 (0.093) 0.000 (0.000) 0.364 (0.064) -0.364 (0.149)3K051201 SJ 16A 0.388 (0.041) 2.413 (0.206) 0.000 (0.000) 0.438 (0.116) -0.438 (0.233)3K051301 SK 1 0.831 (0.095) 0.004 (0.118) 0.236 (0.042)3K052001 SK 8 0.575 (0.157) 2.363 (0.343) 0.225 (0.030)3K052101 SK 9 0.779 (0.093) 0.367 (0.106) 0.194 (0.037)3K052201 SK 10 0.733 (0.160) 1.762 (0.182) 0.232 (0.028)3K052501 SK 14A 0.470 (0.026) -0.356 (0.043) 0.000 (0.000) 0.150 (0.128) 0.503 (0.113) -0.653 (0.093)3K052502 SK 15A 0.248 (0.018) -3.430 (0.139) 0.000 (0.000) 3.542 (0.744) -0.263 (0.229) -3.279 (0.151)3K052503 SK 16A 0.213 (0.027) -4.819 (0.484) 0.000 (0.000) 1.473 (0.526) -1.473 (0.199)3K047501 SL 8 0.521 (0.110) 1.078 (0.212) 0.298 (0.049)3K047701 SL 10 1.007 (0.174) 1.606 (0.118) 0.154 (0.020)3K047801 SL 11 0.705 (0.095) 0.470 (0.124) 0.208 (0.040)3K048001 SL 13A 0.923 (0.038) 0.099 (0.043) 0.000 (0.000) 1.548 (0.061) -1.548 (0.062)3K048101 SL 14A 0.382 (0.028) 1.480 (0.092) 0.000 (0.000) 0.567 (0.109) -1.071 (0.189) 0.503 (0.245)3K048102 SL 15A 0.462 (0.057) 4.895 (0.279) 0.000 (0.000) 2.398 (0.115) -2.398 (1.304)3K048103 SL 16A 0.580 (0.051) -0.016 (0.071) 0.000 (0.000)3K048401 SM 3 0.703 (0.082) -0.109 (0.141) 0.213 (0.047)3K048901 SM 8A 0.618 (0.037) -0.750 (0.052) 0.000 (0.000) 0.303 (0.090) -0.303 (0.068)3K049201 SM 11 0.354 (0.055) -0.322 (0.314) 0.237 (0.060)3K049301 SM 12A 0.506 (0.044) 2.411 (0.178) 0.000 (0.000) -0.387 (0.117) 0.387 (0.214)3K052901 SN 4A 0.441 (0018) -0.644 (0.040) 0.000 (0.000) -1.409 (0.156) 1.456 (0.148) -0.047 (0.082)3K052902 SN 5 0.795 (0.070) -1.586 (0.150) 0.211 (0.051)3K053601 SN 13A 0.422 (0.017) 1.005 (0.053) 0.000 (0.000) 2.328 (0.088) -0.979 (0.100) -1.349 (0.179)3K053701 SN 14A 0.472 (0.039) 2.441 (0.145) 0.000 (0.000) 0.682 (0.080) -0.682 (0.194)3K053801 SN 15A 0.510 (0.021) 0.699 (0.042) 0.000 (0.000) -1.335 (0.101) 1.335 (0.109)3K053901 SN 16A 0.652 (0.050) 2.129 (0.099) 0.000 (0.000) 0.664 (0.063) -0.664 (0.163)3K054001 SO 1A 0.534 (0.032) 2.286 (0.050) 0.000 (0.000) 2.564 (0.078) 1.152 (0.088) -3.715 (0.687)3K054002 SO 2A 0.930 (0.056) 1.548 (0.048) 0.000 (0.000) 0.948 (0.045) -0.948 (0.116)
Table D-24 (continued)

IRT Parameters for Science Items

Life Science, Grade 12

NAEP ID Block Item aj (s.e.) bj (s.e.) cj (s.e.) dj1 (s.e.) dj2 (s.e.) dj3 (s.e.) 3K054003 SO 3A 1.429 (0.148) 2.213 (0.097) 0.000 (0.000) -0.342 (0.110) 0.342 (0.175)3K054004 SO 4A 0.714 (0.081) 2.246 (0.194) 0.000 (0.000)3K054005 SO 5A 0.438 (0.037) 2.566 (0.134) 0.000 (0.000) 1.292 (0.092) -1.292 (0.252)3K054006 SO 6A 0.522 (0.071) 2.729 (0.311) 0.000 (0.000)3K054007 SO 7A 0.564 (0.046) 1.597 (0.094) 0.000 (0.000) 0.249 (0.081) -0.249 (0.134)3K054008 SO 8A 0.415 (0.027) 1.641 (0.068) 0.000 (0.000) 0.518 (0.123) 1.334 (0.138) -1.852 (0.255)3K057301 ST 3 0.444 (0.114) 2.372 (0.366) 0.199 (0.037)3K057302 ST 4 0.500 (0.091) 1.206 (0.185) 0.197 (0.043)3K057601 ST 7A 0.310 (0.025) -2.374 (0.143) 0.000 (0.000) 1.578 (0.244) -1.578 (0.117)3K057701 ST 8A 0.573 (0.051) 2.194 (0.139) 0.000 (0.000) 0.269 (0.084) -0.269 (0.180)3K057801 ST 9 0.661 (0.071) -0.205 (0.136) 0.189 (0.044)3K058501 ST 16A 0.582 (0.036) 1.336 (0.054) 0.000 (0.000) 1.499 (0.077) -0.195 (0.099) -1.304 (0.204)3K058801 SU 3 1.453 (0.144) 0.274 (0.054) 0.178 (0.026)3K059201 SU 7A 0.805 (0.043) 0.850 (0.033) 0.000 (0.000) 0.957 (0.054) -0.067 (0.063) -0.890 (0.100)3K059401 SU 9 0.755 (0.074) -1.267 (0.154) 0.203 (0.051)3K059601 SU 11 1.184 (0.165) 0.834 (0.073) 0.265 (0.026)
Table D-25

IRT Parameters for the Reading Long-Term Trend Samples, Age 9

NAEP ID A S.E. B S.E. C S.E.

706

N001101 0.594 (0.152) 1.629 (0.236) 0.358 (0.035)

N008601 1.728 (0.142) -0.227 (0.053) 0.245 (0.027)

N008602 1.362 (0.119) 0.124 (0.053) 0.218 (0.026)

N008603 1.226 (0.105) -0.259 (0.070) 0.223 (0.029)

N008701 0.548 (0.056) -3.106 (0.311) 0.281 (0.065)

N001501 2.488 (0.243) -0.508 (0.055) 0.329 (0.029)

N001502 2.331 (0.191) 0.319 (0.031) 0.222 (0.021)

N001503 2.140 (0.192) -0.080 (0.047) 0.329 (0.027)

N001504 2.368 (0.204) 0.262 (0.034) 0.279 (0.023)

N001507 1.043 (0.138) 2.112 (0.194) 0.000 (0.000)

N001601 1.213 (0.123) 0.186 (0.065) 0.292 (0.028)

N001602 1.934 (0.193) 0.470 (0.040) 0.304 (0.022)

N001603 1.335 (0.201) 0.929 (0.062) 0.319 (0.023)

N001604 1.505 (0.176) 0.775 (0.045) 0.233 (0.021)

N008801 1.461 (0.137) -0.839 (0.093) 0.318 (0.036)

N001801 0.334 (0.097) 6.427 (1.925) 0.144 (0.014)

N001802 2.334 (0.248) 1.461 (0.070) 0.203 (0.012)

N008901 1.835 (0.160) -0.140 (0.051) 0.276 (0.027)

N008902 1.281 (0.123) -0.204 (0.075) 0.304 (0.031)

N008905 0.156 (0.000) 15.358 (0.000) 0.000 (0.000)

N002001 2.113 (0.173) 0.884 (0.033) 0.197 (0.015)

N002002 1.582 (0.166) 0.675 (0.041) 0.220 (0.021)

N002003 1.619 (0.176) 0.568 (0.045) 0.289 (0.023)

N009001 1.762 (0.162) 0.518 (0.036) 0.190 (0.020)

N009002 1.645 (0.175) 0.679 (0.041) 0.221 (0.021)

N009003 1.659 (0.191) 1.105 (0.055) 0.240 (0.017)

N009004 2.304 (0.233) 0.471 (0.036) 0.313 (0.022)

N009101 0.933 (0.095) -0.722 (0.132) 0.286 (0.042)

N009201 1.480 (0.145) -0.594 (0.086) 0.330 (0.034)

N002101 1.519 (0.259) 1.569 (0.123) 0.236 (0.016)

N002102 2.003 (0.298) 1.679 (0.116) 0.163 (0.012)

N002702 1.702 (0.190) 0.809 (0.039) 0.190 (0.019)

N009601 0.863 (0.085) -1.126 (0.165) 0.272 (0.052)

N002401 1.747 (0.170) 0.730 (0.035) 0.146 (0.017)

N009401 1.920 (0.162) -0.257 (0.050) 0.245 (0.027)

N002801 3.165 (0.260) 0.261 (0.025) 0.179 (0.019)

N002802 2.501 (0.215) 0.182 (0.033) 0.220 (0.023)

N002804 0.662 (0.076) 1.665 (0.147) 0.000 (0.000)

N009701 1.231 (0.122) 0.246 (0.060) 0.256 (0.027)

N009702 1.867 (0.171) 0.286 (0.042) 0.273 (0.024)

N009703 2.056 (0.205) 0.673 (0.035) 0.274 (0.019)

N009704 1.936 (0.194) 0.683 (0.034) 0.201 (0.018)

N009705 1.755 (0.155) 0.193 (0.044) 0.264 (0.024)

N003001 0.970 (0.257) 1.971 (0.262) 0.195 (0.019)

N003002 0.525 (0.085) 0.663 (0.150) 0.219 (0.041)

N003003 0.081 (0.018) 14.651 (3.865) 0.032 (0.010)

N014001 1.414 (0.138) 0.083 (0.060) 0.283 (0.028)

N003101 1.392 (0.146) 0.149 (0.062) 0.300 (0.028)

N003102 3.430 (0.222) 0.686 (0.023) 0.199 (0.015)

N003104 0.869 (0.138) 2.355 (0.280) 0.000 (0.000)

N009801 1.224 (0.121) -1.584 (0.151) 0.337 (0.054)

N009901 1.091 (0.115) 0.043 (0.079) 0.296 (0.032)

N014301 3.051 (0.257) 0.310 (0.026) 0.243 (0.019)

N014302 1.658 (0.161) 0.464 (0.042) 0.248 (0.023)

N014303 2.697 (0.229) 0.067 (0.033) 0.271 (0.023)
Table D-25 (continued)

IRT Parameters for the Reading Long-Term Trend Samples, Age 9

NAEP ID A S.E. B S.E. C S.E.

707

N010002 1.783 (0.173) 0.191 (0.049) 0.326 (0.026)

N010003 1.940 (0.165) 0.210 (0.039) 0.234 (0.023)

N010102 1.881 (0.219) 0.661 (0.042) 0.292 (0.022)

N010103 2.171 (0.188) 0.072 (0.040) 0.261 (0.025)

N003701 1.266 (0.135) -0.042 (0.078) 0.327 (0.032)

N003702 1.990 (0.190) 0.403 (0.038) 0.240 (0.023)

N003704 0.731 (0.073) 1.196 (0.091) 0.000 (0.000)

N003801 1.287 (0.395) 1.892 (0.267) 0.345 (0.018)

N003802 0.511 (0.086) 0.499 (0.176) 0.259 (0.046)

N003803 1.130 (0.341) 2.034 (0.299) 0.228 (0.017)

N010301 0.656 (0.070) -1.167 (0.215) 0.291 (0.057)

N010201 0.954 (0.092) -1.367 (0.166) 0.303 (0.053)

N004101 1.292 (0.127) -0.134 (0.075) 0.316 (0.032)

N004201 1.546 (0.205) 0.926 (0.052) 0.263 (0.021)

N004202 1.113 (0.190) 1.015 (0.082) 0.314 (0.026)

N010401 0.703 (0.077) -0.817 (0.175) 0.262 (0.049)

N010402 1.642 (0.207) 0.899 (0.049) 0.255 (0.020)

N010403 1.612 (0.205) 1.119 (0.061) 0.233 (0.018)

N004701 2.232 (0.185) 0.400 (0.030) 0.190 (0.019)

N004702 1.016 (0.121) 0.167 (0.089) 0.339 (0.033)

N004703 1.977 (0.165) 0.318 (0.035) 0.216 (0.021)

N004801 1.232 (0.123) -0.314 (0.087) 0.340 (0.034)

N004901 2.143 (0.185) 0.980 (0.039) 0.250 (0.015)

N005101 0.743 (0.068) -1.734 (0.202) 0.281 (0.060)

N010801 1.208 (0.131) 0.407 (0.060) 0.252 (0.027)

N010902 2.336 (0.232) 0.531 (0.033) 0.258 (0.020)

N010903 2.951 (0.249) 0.322 (0.027) 0.198 (0.019)

N010904 2.415 (0.212) 0.653 (0.031) 0.257 (0.019)

N014101 0.980 (0.108) -0.014 (0.092) 0.257 (0.034)

N011001 1.653 (0.112) 0.277 (0.034) 0.293 (0.018)

N011002 2.323 (0.153) 0.574 (0.022) 0.267 (0.014)

N011003 2.619 (0.158) 0.019 (0.025) 0.287 (0.017)

N011004 2.453 (0.145) 0.341 (0.021) 0.228 (0.015)

N011101 2.248 (0.136) 0.405 (0.022) 0.218 (0.014)

N011201 1.441 (0.116) 0.604 (0.035) 0.264 (0.017)

N011301 2.159 (0.137) 0.292 (0.026) 0.279 (0.016)

N011302 1.528 (0.142) 0.647 (0.038) 0.347 (0.018)

N011401 2.891 (0.180) 1.102 (0.032) 0.392 (0.011)

N011402 1.010 (0.126) 0.944 (0.061) 0.297 (0.021)

N011403 2.055 (0.156) 1.132 (0.039) 0.297 (0.011)

N011404 1.869 (0.139) 1.051 (0.034) 0.214 (0.012)

N013201 2.445 (0.196) 0.203 (0.031) 0.234 (0.021)

N013301 1.657 (0.164) -0.361 (0.072) 0.416 (0.030)

N013401 1.676 (0.154) 0.581 (0.036) 0.183 (0.019)

N013402 2.405 (0.225) 0.240 (0.037) 0.355 (0.023)

N013403 2.556 (0.219) 0.553 (0.027) 0.234 (0.017)

N014201 1.325 (0.130) 0.042 (0.065) 0.334 (0.028)

N014501 0.687 (0.041) -0.625 (0.062) 0.000 (0.000)

N014502 0.635 (0.036) -0.341 (0.065) 0.000 (0.000)

N014502 0.718 (0.058) -0.738 (0.088) 0.000 (0.000)

N014503 0.824 (0.045) -1.253 (0.071) 0.000 (0.000)
Table D-26

IRT Parameters for the Reading Long-Term Trend Samples, Age 13

NAEP ID A S.E. B S.E. C S.E.

708

N001101 0.221 (0.038) 0.647 (0.397) 0.290 (0.048)

N001201 0.537 (0.108) 1.467 (0.191) 0.336 (0.037)

N001202 1.548 (0.162) 0.812 (0.046) 0.241 (0.018)

N001301 0.633 (0.091) 0.151 (0.192) 0.433 (0.045)

N001302 0.598 (0.075) -2.213 (0.345) 0.528 (0.070)

N001303 0.894 (0.100) 0.588 (0.082) 0.246 (0.029)

N001401 0.919 (0.087) -0.173 (0.100) 0.273 (0.038)

N001501 2.067 (0.183) -1.655 (0.077) 0.271 (0.048)

N001502 1.414 (0.108) -0.684 (0.065) 0.214 (0.033)

N001503 1.124 (0.089) -1.196 (0.101) 0.233 (0.045)

N001504 1.260 (0.098) -0.746 (0.077) 0.226 (0.036)

N001507 0.536 (0.048) 2.054 (0.158) 0.000 (0.000)

N001601 0.336 (0.044) -1.237 (0.371) 0.286 (0.066)

N001602 0.865 (0.076) -1.473 (0.160) 0.286 (0.060)

N001603 0.749 (0.091) -0.010 (0.146) 0.343 (0.045)

N001604 0.766 (0.069) -0.560 (0.127) 0.235 (0.045)

N001701 0.692 (0.066) -0.819 (0.162) 0.265 (0.052)

N001702 0.807 (0.187) 2.399 (0.253) 0.249 (0.018)

N001703 0.693 (0.067) -0.283 (0.130) 0.227 (0.043)

N001801 0.128 (0.035) 11.783 (3.143) 0.050 (0.013)

N001802 1.001 (0.117) 0.753 (0.073) 0.264 (0.026)

N001901 0.781 (0.080) 0.007 (0.112) 0.235 (0.039)

N001904 0.805 (0.000) 0.242 (0.000) 0.000 (0.000)

N002001 0.995 (0.078) -0.167 (0.071) 0.170 (0.030)

N002002 1.095 (0.100) -0.150 (0.082) 0.284 (0.034)

N002003 0.943 (0.081) -0.674 (0.108) 0.254 (0.043)

N002101 0.703 (0.110) 1.336 (0.116) 0.223 (0.029)

N002102 1.227 (0.117) 0.824 (0.049) 0.144 (0.018)

N002201 0.958 (0.000) -0.393 (0.061) 0.195 (0.029)

N002202 0.980 (0.099) -0.560 (0.125) 0.347 (0.046)

N002203 0.483 (0.049) -1.986 (0.267) 0.265 (0.062)

N002401 0.800 (0.060) -0.904 (0.108) 0.153 (0.041)

N002501 0.533 (0.059) 0.026 (0.160) 0.211 (0.045)

N002701 0.706 (0.085) 0.462 (0.112) 0.235 (0.037)

N002801 1.255 (0.098) -1.143 (0.089) 0.217 (0.042)

N002802 1.276 (0.104) -1.412 (0.099) 0.230 (0.047)

N002804 0.340 (0.000) 2.103 (0.000) 0.000 (0.000)

N002902 0.564 (0.056) -1.336 (0.222) 0.274 (0.061)

N002903 1.325 (0.112) -0.745 (0.083) 0.285 (0.039)

N002904 0.891 (0.081) -0.272 (0.100) 0.238 (0.038)

N002905 0.570 (0.070) 0.426 (0.145) 0.223 (0.041)

N002906 1.186 (0.094) -0.829 (0.085) 0.228 (0.039)

N003001 0.669 (0.095) 1.313 (0.111) 0.170 (0.028)

N003002 0.292 (0.039) -0.126 (0.291) 0.179 (0.052)

N003003 1.841 (0.209) 2.355 (0.118) 0.091 (0.007)

N003101 1.080 (0.091) -1.113 (0.111) 0.256 (0.048)

N003102 1.489 (0.121) -0.418 (0.062) 0.258 (0.031)

N003104 0.569 (0.047) 1.758 (0.125) 0.000 (0.000)

N003201 0.876 (0.075) -0.919 (0.125) 0.246 (0.047)

N003202 0.973 (0.087) 0.155 (0.073) 0.194 (0.029)

N003203 1.243 (0.125) 0.308 (0.066) 0.306 (0.027)

N003204 1.049 (0.103) 0.408 (0.068) 0.228 (0.027)

N003301 0.772 (0.070) -0.573 (0.126) 0.233 (0.045)

N003401 0.999 (0.085) -0.251 (0.082) 0.198 (0.034)

N003501 0.809 (0.078) -0.530 (0.131) 0.268 (0.046)
Table D-26 (continued)

IRT Parameters for the Reading Long-Term Trend Samples, Age 13

NAEP ID A S.E. B S.E. C S.E.

709

N003601 0.934 (0.081) -1.243 (0.134) 0.261 (0.052)

N003602 0.977 (0.082) -0.350 (0.086) 0.205 (0.035)

N003701 0.814 (0.073) -0.885 (0.135) 0.245 (0.049)

N003702 1.107 (0.112) -0.107 (0.089) 0.315 (0.036)

N003704 0.596 (0.043) 0.105 (0.059) 0.000 (0.000)

N003801 0.329 (0.059) 1.324 (0.279) 0.199 (0.051)

N003802 0.248 (0.035) -1.477 (0.413) 0.198 (0.059)

N003803 0.419 (0.113) 3.034 (0.489) 0.243 (0.031)

N004002 0.508 (0.051) -2.289 (0.268) 0.265 (0.063)

N003901 1.095 (0.112) -2.541 (0.162) 0.256 (0.061)

N004101 0.947 (0.081) -1.588 (0.145) 0.272 (0.058)

N004201 0.865 (0.081) -0.079 (0.097) 0.237 (0.037)

N004202 0.638 (0.074) -0.078 (0.158) 0.274 (0.047)

N004301 1.058 (0.106) 0.288 (0.075) 0.276 (0.030)

N004303 0.859 (0.051) 0.222 (0.041) 0.000 (0.000)

N004401 1.505 (0.146) -2.244 (0.116) 0.265 (0.060)

N004402 0.810 (0.073) -0.247 (0.106) 0.215 (0.039)

N004403 1.269 (0.110) -1.673 (0.116) 0.268 (0.055)

N004501 0.600 (0.083) 0.307 (0.170) 0.316 (0.046)

N004502 0.628 (0.059) -0.957 (0.174) 0.250 (0.053)

N004601 0.844 (0.082) 0.224 (0.087) 0.204 (0.032)

N004602 1.020 (0.087) -0.184 (0.080) 0.231 (0.033)

N004603 1.279 (0.104) -0.539 (0.074) 0.263 (0.035)

N004605 0.655 (0.042) -1.096 (0.078) 0.000 (0.000)

N004701 1.585 (0.118) -0.800 (0.059) 0.187 (0.031)

N004702 0.752 (0.062) -1.296 (0.144) 0.228 (0.051)

N004703 0.853 (0.063) -1.176 (0.108) 0.183 (0.041)

N004801 1.131 (0.093) -1.351 (0.112) 0.250 (0.049)

N004901 0.795 (0.077) -0.077 (0.107) 0.239 (0.038)

N005101 0.697 (0.066) -2.475 (0.220) 0.265 (0.063)

N005001 1.312 (0.310) 2.234 (0.221) 0.259 (0.016)

N005002 0.664 (0.155) 2.083 (0.233) 0.331 (0.026)

N005003 0.950 (0.174) 2.096 (0.163) 0.170 (0.016)

N005201 0.676 (0.164) 1.391 (0.212) 0.578 (0.030)

N005202 0.459 (0.065) 0.432 (0.205) 0.245 (0.050)

N005203 1.074 (0.232) 1.989 (0.166) 0.329 (0.017)

N005301 0.903 (0.091) -0.092 (0.100) 0.265 (0.037)

N005302 1.792 (0.172) 0.630 (0.039) 0.190 (0.018)

N005303 0.860 (0.129) 0.966 (0.101) 0.294 (0.029)

N005304 1.766 (0.162) 0.185 (0.046) 0.236 (0.023)

N005305 0.989 (0.099) -0.716 (0.123) 0.293 (0.047)

N005403 1.119 (0.102) -0.629 (0.102) 0.330 (0.042)

N005404 1.101 (0.098) -1.432 (0.131) 0.293 (0.056)

N005405 1.633 (0.143) 0.092 (0.051) 0.299 (0.025)

N005406 0.902 (0.081) -0.312 (0.101) 0.248 (0.039)

N005407 1.339 (0.110) -0.483 (0.071) 0.271 (0.034)

N005503 0.832 (0.100) 0.425 (0.104) 0.308 (0.035)

N005504 1.157 (0.129) 0.998 (0.059) 0.201 (0.019)

N005505 0.952 (0.092) -0.874 (0.139) 0.334 (0.052)

N005601 1.405 (0.128) -0.519 (0.079) 0.350 (0.036)

N005602 1.352 (0.129) 0.536 (0.052) 0.237 (0.022)

N005603 1.475 (0.132) -0.385 (0.071) 0.339 (0.033)
Table D-27

IRT Parameters for the Reading Long-Term Trend Samples, Age 17

NAEP ID A S.E. B S.E. C S.E.

710

N017001 1.474 (0.141) 0.014 (0.063) 0.331 (0.029)

N017002 1.717 (0.160) 0.465 (0.044) 0.239 (0.021)

N017003 1.698 (0.170) 1.348 (0.059) 0.230 (0.014)

N001301 1.071 (0.147) -0.065 (0.133) 0.569 (0.036)

N001302 0.663 (0.093) -2.474 (0.384) 0.592 (0.068)

N001303 0.910 (0.089) -0.237 (0.103) 0.268 (0.039)

N001401 0.709 (0.076) -1.101 (0.196) 0.318 (0.058)

N001501 1.527 (0.165) -2.116 (0.118) 0.261 (0.053)

N001502 1.643 (0.139) -1.079 (0.070) 0.194 (0.034)

N001503 1.173 (0.108) -1.675 (0.129) 0.265 (0.052)

N001504 1.339 (0.117) -1.155 (0.091) 0.239 (0.040)

N001507 0.358 (0.042) 2.182 (0.241) 0.000 (0.000)

N001701 0.621 (0.064) -1.558 (0.223) 0.294 (0.061)

N001702 0.588 (0.169) 3.082 (0.514) 0.308 (0.022)

N001703 0.940 (0.098) -0.612 (0.131) 0.348 (0.046)

N001901 0.981 (0.093) -0.927 (0.124) 0.288 (0.047)

N001904 0.748 (0.049) -1.258 (0.081) 0.000 (0.000)

N002001 1.268 (0.111) -0.471 (0.075) 0.250 (0.035)

N002002 0.983 (0.087) -0.794 (0.105) 0.231 (0.042)

N002003 1.183 (0.113) -1.174 (0.116) 0.294 (0.047)

N002101 0.614 (0.070) 0.190 (0.129) 0.173 (0.041)

N002102 1.538 (0.130) 0.113 (0.049) 0.197 (0.025)

N002201 1.181 (0.000) -0.908 (0.077) 0.389 (0.041)

N002202 1.657 (0.202) -0.718 (0.099) 0.516 (0.040)

N002203 0.431 (0.057) -3.349 (0.441) 0.291 (0.065)

N002501 0.451 (0.064) -0.441 (0.288) 0.328 (0.062)

N002701 0.736 (0.070) -0.369 (0.118) 0.185 (0.042)

N002702 0.819 (0.072) -0.973 (0.129) 0.194 (0.047)

N002801 1.488 (0.150) -1.764 (0.115) 0.272 (0.052)

N002802 1.152 (0.115) -2.047 (0.154) 0.291 (0.059)

N002804 0.202 (0.033) 2.784 (0.450) 0.000 (0.000)

N002902 0.648 (0.071) -1.386 (0.227) 0.320 (0.063)

N002903 1.604 (0.162) -1.199 (0.095) 0.305 (0.044)

N002904 1.136 (0.103) -0.828 (0.100) 0.267 (0.042)

N002905 0.658 (0.079) 0.118 (0.141) 0.249 (0.043)

N002906 1.747 (0.181) -1.081 (0.087) 0.327 (0.042)

N003001 1.000 (0.095) 0.414 (0.066) 0.159 (0.027)

N003002 0.301 (0.041) -0.708 (0.312) 0.185 (0.055)

N003003 1.528 (0.154) 1.291 (0.052) 0.089 (0.011)

N003101 0.943 (0.090) -1.727 (0.162) 0.284 (0.057)

N003102 1.270 (0.117) -1.172 (0.104) 0.254 (0.045)

N003104 0.754 (0.054) 1.057 (0.068) 0.000 (0.000)

N003201 0.934 (0.092) -1.821 (0.175) 0.298 (0.060)

N003202 1.172 (0.117) -0.722 (0.106) 0.345 (0.043)

N003203 0.910 (0.082) -0.615 (0.107) 0.225 (0.041)

N003204 1.069 (0.095) -0.988 (0.107) 0.244 (0.043)

N003301 1.153 (0.100) -1.165 (0.103) 0.224 (0.043)

N015201 0.707 (0.070) -2.188 (0.215) 0.274 (0.060)

N003501 0.688 (0.067) -1.041 (0.163) 0.248 (0.051)

N003601 1.087 (0.105) -1.839 (0.150) 0.275 (0.056)

N003602 1.226 (0.114) -0.815 (0.095) 0.270 (0.042)

N003701 0.663 (0.073) -1.445 (0.226) 0.315 (0.063)

N003702 1.573 (0.151) -0.535 (0.072) 0.326 (0.036)

N003704 0.664 (0.047) -0.441 (0.063) 0.000 (0.000)

N003801 0.570 (0.084) 0.576 (0.156) 0.228 (0.045)
Table D-27 (continued)

IRT Parameters for the Reading Long-Term Trend Samples, Age 17

NAEP ID A S.E. B S.E. C S.E.

711

N003802 0.184 (0.033) -2.639 (0.654) 0.199 (0.059)

N003803 0.652 (0.155) 1.842 (0.217) 0.313 (0.031)

N016001 0.978 (0.095) -0.877 (0.124) 0.285 (0.047)

N016002 0.885 (0.119) 0.495 (0.109) 0.340 (0.036)

N016003 0.870 (0.093) -0.109 (0.111) 0.284 (0.040)

N016004 1.309 (0.126) -0.428 (0.082) 0.322 (0.037)

N016005 1.416 (0.128) -0.410 (0.071) 0.275 (0.035)

N016006 0.881 (0.086) -0.088 (0.096) 0.224 (0.037)

N004201 0.895 (0.093) -0.477 (0.123) 0.288 (0.045)

N004202 0.685 (0.090) -0.043 (0.164) 0.319 (0.048)

N004301 0.891 (0.096) -0.339 (0.119) 0.270 (0.043)

N004303 0.673 (0.054) -0.092 (0.065) 0.000 (0.000)

N015502 1.250 (0.110) -0.310 (0.074) 0.275 (0.034)

N015503 1.218 (0.113) 0.128 (0.066) 0.257 (0.029)

N015504 1.220 (0.107) -0.403 (0.078) 0.268 (0.035)

N015505 0.654 (0.064) -0.944 (0.167) 0.252 (0.050)

N004501 0.668 (0.072) -0.631 (0.168) 0.280 (0.051)

N004502 0.493 (0.054) -2.025 (0.285) 0.288 (0.063)

N004601 0.826 (0.077) -0.149 (0.097) 0.206 (0.036)

N004602 1.502 (0.129) -0.553 (0.066) 0.247 (0.034)

N004603 1.335 (0.121) -0.883 (0.089) 0.273 (0.040)

N004605 0.749 (0.054) -1.265 (0.093) 0.000 (0.000)

N005001 1.980 (0.209) 0.743 (0.041) 0.251 (0.018)

N005002 0.812 (0.100) 0.613 (0.098) 0.252 (0.033)

N005003 0.895 (0.120) 1.264 (0.085) 0.150 (0.022)

N004901 0.850 (0.094) -0.552 (0.147) 0.359 (0.049)

N005201 0.769 (0.158) 0.585 (0.202) 0.635 (0.035)

N005202 0.533 (0.077) 0.327 (0.189) 0.284 (0.049)

N005203 0.757 (0.142) 1.296 (0.134) 0.338 (0.031)

N015901 1.270 (0.135) 0.118 (0.078) 0.363 (0.032)

N015902 1.198 (0.123) 0.205 (0.074) 0.299 (0.031)

N015903 2.076 (0.196) 0.518 (0.037) 0.201 (0.019)

N015905 0.728 (0.000) 0.247 (0.000) 0.000 (0.000)

N005503 0.742 (0.094) 0.032 (0.146) 0.350 (0.044)

N005504 1.381 (0.137) 0.345 (0.060) 0.294 (0.026)

N005505 0.786 (0.078) -1.926 (0.207) 0.309 (0.064)

N015101 1.022 (0.117) 0.150 (0.097) 0.369 (0.035)

N015102 2.671 (0.226) 0.004 (0.032) 0.227 (0.021)

N015103 2.693 (0.229) 0.110 (0.030) 0.210 (0.020)

N015104 2.168 (0.191) -0.027 (0.041) 0.275 (0.025)
Table D-28

IRT Parameters for the Mathematics Long-Term Trend Samples, Age 9

NAEP ID A S.E. B S.E. C S.E.

712

N2709011 0.59950 0.03694 -2.84371 0.14206 0.00000 0.00000

N2774011 0.91059 0.05279 -1.73828 0.09895 0.17174 0.04417

N2676011 1.16597 0.06386 -0.69400 0.06026 0.22100 0.02978

N2768011 0.69953 0.04740 -3.21047 0.16225 0.00000 0.00000

N2768021 0.58482 0.03237 -2.33512 0.10902 0.00000 0.00000

N2768031 0.54214 0.02503 -0.05625 0.03920 0.00000 0.00000

N2507011 0.64744 0.03759 -1.41456 0.11571 0.14224 0.04151

N2507021 1.13106 0.07160 0.48798 0.03863 0.15349 0.01635

N2507031 1.01455 0.05179 -0.49314 0.05496 0.11757 0.02566

N2622011 0.72079 0.05486 -0.72016 0.13738 0.31391 0.04583

N2572011 0.93291 0.05708 -0.71528 0.08508 0.24537 0.03633

N2761011 1.03200 0.03885 -1.02631 0.03378 0.00000 0.00000

N2861011 0.88563 0.03357 -0.87547 0.03512 0.00000 0.00000

N2700011 0.57662 0.02578 -0.66154 0.04489 0.00000 0.00000

N2721021 0.81734 0.04941 -0.47932 0.08147 0.17124 0.03313

N2840011 0.79395 0.03102 -0.84005 0.03760 0.00000 0.00000

N2840021 0.75296 0.04204 1.87866 0.07965 0.00000 0.00000

N2676021 1.00823 0.05478 -0.13182 0.05033 0.15327 0.02205

N2625011 0.46201 0.05964 0.29971 0.21852 0.34233 0.04846

N2625021 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2654011 0.47952 0.13882 3.46971 0.57466 0.28410 0.01985

N2661011 0.57237 0.07755 1.35985 0.10758 0.24159 0.02796

N2691011 0.50012 0.07249 1.52585 0.12337 0.19987 0.03081

N2682011 0.95788 0.07476 0.62152 0.05230 0.21573 0.01992

N2521011 0.68677 0.09376 1.62183 0.09947 0.23094 0.02074

N2526011 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2723012 0.77379 0.05260 -2.44300 0.14599 0.17087 0.05028

N2766012 1.16928 0.06842 -1.14011 0.07632 0.22568 0.04110

N2578012 0.64050 0.04461 -1.39460 0.16037 0.26842 0.05431

N2634012 0.77485 0.05443 -1.25946 0.14323 0.30576 0.05417

N2634022 0.90833 0.06814 -0.42377 0.10049 0.32971 0.03773

N2735012 0.62232 0.05145 -0.93564 0.18471 0.32768 0.05576

N2754012 0.98503 0.03665 -0.85998 0.03191 0.00000 0.00000

N2775012 0.77063 0.03025 -0.68815 0.03564 0.00000 0.00000

N2776012 0.84989 0.03313 -0.91905 0.03691 0.00000 0.00000

N2776022 0.83584 0.03107 -0.03557 0.02749 0.00000 0.00000

N2776032 0.79308 0.03006 -0.21771 0.02949 0.00000 0.00000

N2614012 0.41818 0.03423 -0.73342 0.19602 0.21652 0.04786

N2506012 0.93796 0.05858 -1.97101 0.10891 0.17798 0.04849

N2506022 0.57587 0.03736 -1.88859 0.15604 0.17040 0.05013

N2506032 0.95856 0.05149 -0.25672 0.05505 0.12352 0.02461

N2514012 0.70587 0.04081 -0.80890 0.09514 0.14658 0.03691

N2509012 0.48879 0.03297 -1.81552 0.17006 0.15974 0.04796

N2509022 1.09953 0.06793 0.42731 0.04018 0.14315 0.01701

N2509032 1.18622 0.06456 -0.06436 0.04272 0.13661 0.02045

N2503012 0.72341 0.08481 1.05275 0.08474 0.29008 0.02497

N2760012 0.95855 0.03648 -0.95870 0.03430 0.00000 0.00000

N2760022 0.83677 0.03543 1.01588 0.03830 0.00000 0.00000

N2861022 0.90443 0.03271 0.01609 0.02577 0.00000 0.00000

N2711012 0.73684 0.02885 -0.23597 0.03139 0.00000 0.00000

N2520012 1.13023 0.12554 1.77819 0.07829 0.22868 0.01093

N2690012 0.59892 0.10854 2.99976 0.29445 0.08071 0.01225

N2768213 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2768223 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000
Table D-28 (continued)

IRT Parameters for the Mathematics Long-Term Trend Samples, Age 9

NAEP ID A S.E. B S.E. C S.E.

713

N2768233 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2840213 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2840223 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2760213 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2760223 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2776213 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2776223 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2776233 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2682213 0.00000 0.00000 -0.10734 0.00000 0.00000 0.00000

N2728013 0.89213 0.05271 -1.60990 0.10228 0.18525 0.04618

N2670013 0.94882 0.05918 -1.37676 0.10407 0.26707 0.04720

N2721013 0.76836 0.05370 -0.95164 0.12834 0.28977 0.04748

N2624013 0.80162 0.07726 0.52062 0.08431 0.31390 0.02743

N2585013 0.56937 0.08584 1.68734 0.12176 0.22902 0.02593
Table D-29

IRT Parameters for the Mathematics Long-Term Trend Samples, Age 13

NAEP ID A S.E. B S.E. C S.E.

714

N2819011 1.14166 0.08519 -2.21322 0.11766 0.22337 0.04755

N2546011 0.84076 0.06375 -2.06860 0.15938 0.32730 0.05643

N2768011 0.50147 0.04175 -4.32401 0.30550 0.00000 0.00000

N2768021 0.50356 0.03968 -4.07409 0.27292 0.00000 0.00000

N2768031 0.40041 0.02485 -2.04391 0.12367 0.00000 0.00000

N2776011 0.68977 0.03973 -2.61117 0.11894 0.00000 0.00000

N2776021 0.68283 0.03180 -1.49505 0.06217 0.00000 0.00000

N2776031 0.61362 0.03160 -1.91585 0.08601 0.00000 0.00000

N2672011 1.01261 0.07773 -0.80989 0.11129 0.43931 0.04191

N2862011 0.96660 0.05721 -0.84701 0.08097 0.25278 0.03707

N2509011 0.35161 0.03194 -3.57131 0.34026 0.18338 0.05197

N2509021 0.86649 0.04567 -0.89285 0.07248 0.13528 0.03282

N2509031 0.82089 0.05250 -2.08095 0.12354 0.15468 0.04439

N2624011 1.15737 0.07332 -0.47616 0.06661 0.30845 0.03123

N2748011 1.36812 0.11261 0.27123 0.05497 0.45473 0.02029

N2652021 0.71187 0.05409 -0.67179 0.13437 0.31397 0.04560

N2668011 0.68302 0.04711 -1.17426 0.13837 0.28919 0.04765

N2529011 1.21576 0.06143 -0.09054 0.03787 0.12643 0.01933

N2625011 0.54813 0.04319 -1.15266 0.18669 0.31206 0.05227

N2625021 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2576011 1.26554 0.04386 -0.58253 0.02262 0.00000 0.00000

N2652011 0.67316 0.05615 -2.35874 0.22164 0.35331 0.06308

N2739011 1.58900 0.08805 -0.08173 0.03413 0.21782 0.01910

N2588011 1.18663 0.10308 0.67807 0.05399 0.37648 0.01843

N2631011 0.64190 0.02742 -0.56523 0.03824 0.00000 0.00000

N2659011 0.76359 0.06972 0.94217 0.06987 0.22197 0.02236

N2521011 0.98805 0.08716 0.46445 0.07017 0.36614 0.02419

N2750011 0.86228 0.03310 0.69449 0.03058 0.00000 0.00000

N2601011 1.51040 0.08872 -0.09785 0.03903 0.23718 0.02132

N2690011 1.18736 0.06881 0.02047 0.04434 0.18630 0.02171

N2863011 1.30556 0.07778 0.34034 0.03658 0.20346 0.01687

N2546021 0.94949 0.08998 1.16354 0.05776 0.24036 0.01713

N2610011 0.68078 0.05390 0.32732 0.08902 0.21317 0.03023

N2865011 0.92314 0.06419 0.78062 0.04668 0.13451 0.01724

N2789041 0.65447 0.07568 1.46561 0.08759 0.19367 0.02217

N2557011 1.02120 0.06648 0.76744 0.04130 0.13699 0.01524

N2831011 1.98595 0.11576 0.93254 0.02435 0.14242 0.00858

N2774012 0.55830 0.04494 -3.39203 0.25203 0.19018 0.05347

N2779012 0.69697 0.04398 -3.06832 0.14847 0.00000 0.00000

N2779022 0.70057 0.04447 -3.09463 0.15011 0.00000 0.00000

N2779032 0.67984 0.03835 -2.53712 0.11356 0.00000 0.00000

N2634012 0.72108 0.05722 -2.71701 0.20373 0.28011 0.05922

N2634022 0.64681 0.04723 -2.10054 0.18187 0.28513 0.05652

N2507012 0.49809 0.04120 -3.73340 0.28311 0.13432 0.04782

N2507022 0.80945 0.04387 -1.36855 0.08799 0.14301 0.03659

N2507032 0.55562 0.03849 -2.83807 0.18703 0.12102 0.04294

N2561012 0.90472 0.03943 -1.51745 0.05110 0.00000 0.00000

N2622012 0.48839 0.04072 -1.69723 0.24206 0.32053 0.06053

N2502012 0.57688 0.04537 -1.48973 0.20035 0.31995 0.05791

N2703012 0.39105 0.03095 -2.32292 0.25011 0.15751 0.05437

N2703022 1.35410 0.09816 1.71969 0.05047 0.08018 0.00648

N2537012 0.29752 0.03037 -0.93216 0.26762 0.31463 0.04325

N2866012 0.92129 0.03326 -0.27410 0.02565 0.00000 0.00000

N2866022 0.91481 0.03328 -0.38613 0.02677 0.00000 0.00000
Table D-29 (continued)

IRT Parameters for the Mathematics Long-Term Trend Samples, Age 13

NAEP ID A S.E. B S.E. C S.E.

715

N2866032 1.09784 0.03926 0.60353 0.02417 0.00000 0.00000

N2691012 0.93222 0.06083 -0.26860 0.07405 0.24378 0.03130

N2857012 0.86784 0.07362 0.12916 0.08873 0.35170 0.03047

N2822012 1.08773 0.07855 0.48265 0.05013 0.27691 0.01954

N2789022 0.92484 0.08581 0.91405 0.06106 0.27922 0.02003

N2635012 0.92296 0.04735 -0.11892 0.04787 0.09969 0.02117

N2588022 1.40917 0.08526 0.29574 0.03525 0.22132 0.01693

N2789012 1.32488 0.08361 0.13825 0.04266 0.26789 0.02004

N2647012 1.07150 0.06803 0.32213 0.04605 0.20016 0.01983

N2615012 0.63407 0.04418 -0.86457 0.13403 0.24133 0.04502

N2618012 0.59496 0.04898 -0.12107 0.12959 0.25620 0.03943

N2616012 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2613012 0.52460 0.05706 1.13531 0.10523 0.17028 0.02948

N2612012 0.62164 0.10539 2.17167 0.16058 0.24185 0.02005

N2814012 0.72502 0.10315 2.14540 0.13231 0.16744 0.01496

N2520012 0.98749 0.07661 0.58809 0.05498 0.25292 0.02083

N2588032 1.46701 0.10537 0.95767 0.03386 0.19433 0.01194

N2789032 1.88578 0.12786 0.80740 0.02750 0.22164 0.01115

N2865022 1.01183 0.06869 1.01892 0.04089 0.10150 0.01280

N2692012 0.90312 0.04330 1.68434 0.05771 0.00000 0.00000

N2768213 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2768223 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2768233 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2806213 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2806223 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2806233 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2806243 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2806253 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2806263 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2789213 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2645213 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2599213 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2789233 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2789223 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2789253 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2789243 0.00000 0.00000 -0.11006 0.00000 0.00000 0.00000

N2753013 0.32096 0.02984 -2.49410 0.32055 0.20248 0.05623

N2822023 1.16158 0.08450 -0.31459 0.07300 0.37877 0.03067

N2661013 0.83754 0.05917 -0.66475 0.10688 0.30429 0.04122

N2540013 0.89809 0.05678 -0.52647 0.08167 0.22012 0.03549

N2699013 0.77154 0.05560 -0.43978 0.10526 0.27645 0.03869

N2565013 1.28677 0.09122 0.32289 0.04610 0.29533 0.01980

N2659023 0.76494 0.09719 1.39971 0.08804 0.30157 0.02119

N2568013 1.31502 0.09495 0.46865 0.04327 0.28610 0.01807
Table D-30

IRT Parameters for the Mathematics Long-Term Trend Samples, Age 17

NAEP ID A S.E. B S.E. C S.E.

716

N2561011 0.79491 0.04311 -2.20982 0.08772 0.00000 0.00000

N2606011 1.34667 0.06412 -1.70690 0.04401 0.00000 0.00000

N2624011 0.99863 0.06306 -1.49109 0.10305 0.25117 0.05003

N2588041 0.77466 0.05655 -2.23284 0.16806 0.26626 0.06341

N2860011 0.77406 0.04238 -1.25574 0.09080 0.14979 0.03848

N2860021 0.98452 0.05540 -1.72376 0.08688 0.13072 0.04205

N2857011 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2863021 0.93379 0.05727 -0.95066 0.09019 0.24177 0.04096

N2785011 0.89587 0.03431 -0.75799 0.03220 0.00000 0.00000

N2785021 0.91341 0.03334 -0.32812 0.02674 0.00000 0.00000

N2785031 0.73752 0.03016 -0.77553 0.03796 0.00000 0.00000

N2588021 1.61427 0.09376 -0.47732 0.04245 0.24547 0.02510

N2546021 1.23038 0.06548 -0.43454 0.04751 0.16243 0.02526

N2599011 0.88206 0.05936 -0.23800 0.08061 0.23007 0.03306

N2871011 1.09599 0.06619 -0.59405 0.06787 0.24571 0.03281

N2703011 0.79357 0.05233 -2.31065 0.13649 0.15273 0.05089

N2703021 1.23893 0.05782 -0.22615 0.03493 0.08123 0.01757

N2557011 1.20628 0.06659 -1.06349 0.06334 0.19375 0.03513

N2543011 0.94198 0.06977 0.02816 0.07569 0.27980 0.02958

N2865021 1.35117 0.06894 -0.49347 0.04182 0.14970 0.02354

N2609011 1.67439 0.08737 -0.21545 0.03166 0.17280 0.01842

N2568011 1.19200 0.07693 -0.48209 0.06609 0.30551 0.03125

N2588031 1.16183 0.06826 -0.11321 0.04989 0.19059 0.02408

N2626011 0.71586 0.05636 0.23733 0.09052 0.21434 0.03145

N2539011 1.27296 0.07183 -0.50724 0.05195 0.21597 0.02767

N2539021 0.68407 0.07353 0.33672 0.13250 0.37471 0.03661

N2539031 0.97734 0.07560 0.38247 0.06450 0.29387 0.02356

N2539041 1.60610 0.12066 0.45194 0.04055 0.37398 0.01608

N2630011 0.63484 0.02811 0.85767 0.04419 0.00000 0.00000

N2789051 0.56862 0.06637 1.05364 0.11226 0.22750 0.03187

N2873011 0.67899 0.02795 0.39321 0.03417 0.00000 0.00000

N2873021 0.78625 0.03117 0.69872 0.03406 0.00000 0.00000

N2643011 0.81348 0.03464 1.21058 0.04413 0.00000 0.00000

N2828011 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2511011 1.19012 0.04410 0.91059 0.02739 0.00000 0.00000

N2546012 1.01954 0.07885 -2.47842 0.14197 0.23942 0.05824

N2668012 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2623012 0.59455 0.04634 -1.38437 0.19974 0.28418 0.06366

N2632012 0.74713 0.05476 -1.66485 0.16535 0.34067 0.05986

N2632022 0.79125 0.06335 -0.77153 0.14160 0.38347 0.04787

N2601012 1.23905 0.07225 -1.39281 0.07222 0.20711 0.04085

N2540012 0.85834 0.05622 -1.11846 0.11377 0.24266 0.04961

N2690012 1.72750 0.10979 -0.25555 0.04099 0.30837 0.02264

N2789012 0.95891 0.05614 -0.73878 0.07703 0.20365 0.03609

N2615012 0.66473 0.04366 -2.09292 0.14828 0.19419 0.05189

N2618012 0.54161 0.03672 -1.47640 0.16169 0.20938 0.05067

N2612012 0.49101 0.04376 0.02825 0.15874 0.21235 0.04344

N2616012 0.77994 0.12183 1.78405 0.10980 0.34878 0.01845

N2613012 0.48799 0.03928 0.14487 0.13050 0.15626 0.03714

N2814012 0.48840 0.06089 1.62588 0.12025 0.15176 0.02779

N2804012 0.57974 0.02681 -0.93122 0.05039 0.00000 0.00000

N2590012 1.00534 0.03568 -0.30252 0.02469 0.00000 0.00000

N2871022 0.99647 0.05467 -0.90878 0.07107 0.18005 0.03498

N2863012 0.96169 0.05614 -0.97244 0.08183 0.21935 0.03854
Table D-30 (continued)

IRT Parameters for the Mathematics Long-Term Trend Samples, Age 17

NAEP ID A S.E. B S.E. C S.E.

717

N2865012 1.20266 0.07136 -1.01036 0.07055 0.21977 0.03858

N2625012 0.52646 0.04288 -1.55908 0.22589 0.34264 0.06068

N2625022 1.16549 0.11963 1.40031 0.05547 0.27011 0.01300

N2631012 0.70471 0.02979 -0.90810 0.04225 0.00000 0.00000

N2588012 1.30308 0.08950 -0.39955 0.06397 0.34481 0.03014

N2647012 1.22357 0.06770 -0.39538 0.04980 0.19564 0.02577

N2610012 0.79100 0.05004 -0.46191 0.08873 0.20633 0.03544

N2517012 0.83140 0.05163 -0.45041 0.08232 0.16674 0.03513

N2789022 1.01373 0.06788 -0.33627 0.07507 0.26713 0.03280

N2608012 1.38183 0.04597 0.04373 0.01893 0.00000 0.00000

N2789032 1.22774 0.07459 -0.17532 0.05132 0.23410 0.02498

N2556012 1.88440 0.12169 1.47987 0.04369 0.34624 0.00923

N2553012 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2689012 1.65811 0.09139 0.31535 0.02827 0.17930 0.01355

N2688012 1.29277 0.08393 1.14930 0.03461 0.08848 0.00897

N2558012 0.78131 0.03773 1.68142 0.06260 0.00000 0.00000

N2768213 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2768223 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2768233 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2806213 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2806223 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2806233 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2806243 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2806253 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2806263 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2789213 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2599213 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2645213 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2789223 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2853213 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2789233 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2789253 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2789243 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2643213 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2679213 0.00000 0.00000 -0.08216 0.00000 0.00000 0.00000

N2665013 0.84796 0.05965 -0.49696 0.09867 0.26261 0.03943

N2713013 1.18593 0.07920 -0.08152 0.05709 0.27857 0.02573

N2555013 0.84856 0.06714 0.20441 0.08097 0.26643 0.02942

N2560013 1.06035 0.03815 -0.23860 0.02407 0.00000 0.00000

N2571013 0.55500 0.10663 2.24410 0.18757 0.28961 0.02426
Table D-31

IRT Parameters for the Science Long-Term Trend Samples, Age 9

NAEP ID A S.E. B S.E. C S.E.

718

N4000011 0.56311 0.04528 -1.64856 0.21563 0.33365 0.05239

N4002011 0.00000 0.00000 -0.13604 0.00000 0.00000 0.00000

N4003011 0.73474 0.07142 -0.04423 0.10970 0.44352 0.02722

N4004011 0.91803 0.07765 -1.40966 0.14896 0.50831 0.03655

N4004021 1.87737 0.12423 -0.78651 0.04697 0.35748 0.02081

N4004031 0.59447 0.05294 -2.05406 0.26837 0.49484 0.05512

N4004041 1.32191 0.09209 -0.66275 0.06400 0.40014 0.02282

N4004051 0.68599 0.05778 -0.98284 0.15213 0.40990 0.03660

N4005011 0.49172 0.06112 0.44668 0.15774 0.37533 0.03338

N4001011 1.04274 0.16668 1.50077 0.09544 0.51612 0.01490

N4001021 0.80641 0.10779 1.25531 0.09690 0.45069 0.01892

N4006011 0.81545 0.06826 0.02681 0.08091 0.35528 0.02445

N4007011 1.17303 0.08376 0.43991 0.04365 0.29307 0.01686

N4009011 0.28543 0.05136 1.82967 0.28756 0.34420 0.03575

N4010011 0.54419 0.05162 0.52351 0.09799 0.21851 0.02659

N4011011 0.31592 0.06357 1.67353 0.28317 0.39704 0.03904

N4012011 0.86721 0.14000 2.06877 0.13179 0.25864 0.01377

N4013011 0.50247 0.06336 0.73835 0.13570 0.33332 0.03104

N4015012 0.60132 0.11386 1.65079 0.16694 0.50029 0.02241

N4016012 0.63757 0.05684 -1.07525 0.17994 0.38394 0.04347

N4017012 0.00000 0.00000 -0.13604 0.00000 0.00000 0.00000

N4017022 0.24744 0.05649 2.48785 0.53721 0.56905 0.02796

N4017032 0.33532 0.08283 1.93238 0.35540 0.53545 0.03489

N4018012 1.25686 0.12124 0.02366 0.06987 0.57864 0.01960

N4018022 1.41914 0.14693 -0.13923 0.07391 0.64604 0.01901

N4018032 0.99172 0.11152 0.23069 0.09425 0.61889 0.02030

N4018042 0.79316 0.10869 1.14505 0.10423 0.50150 0.01971

N4019012 0.45144 0.09207 2.08817 0.22929 0.37678 0.02702

N4020012 0.75743 0.06220 -1.10947 0.14555 0.41553 0.03714

N4020022 0.74744 0.06182 -1.30157 0.16076 0.43260 0.03994

N4020032 0.00000 0.00000 -0.13604 0.00000 0.00000 0.00000

N4020042 0.00000 0.00000 -0.13604 0.00000 0.00000 0.00000

N4020052 0.75333 0.07617 -0.24187 0.12563 0.50283 0.02860

N4021012 0.79414 0.06765 -0.12558 0.09179 0.36789 0.02683

N4022012 0.28829 0.03549 0.00296 0.28100 0.31156 0.04313

N4024012 0.57455 0.14920 2.78002 0.36401 0.36424 0.01848

N4025012 1.12771 0.12103 1.54349 0.06302 0.22499 0.01247

N4026012 0.00000 0.00000 -0.13604 0.00000 0.00000 0.00000

N4026022 0.42732 0.00000 0.20889 0.21005 0.57496 0.02669

N4026032 0.00000 0.00000 -0.13604 0.00000 0.00000 0.00000

N4027012 0.67834 0.07849 1.50002 0.09391 0.22729 0.01861

N4028012 1.56082 0.10903 1.59177 0.04767 0.18587 0.00827

N4029012 0.42670 0.12550 4.05322 0.75787 0.20262 0.01740

N4030013 0.37776 0.05055 -6.77791 0.78455 0.30681 0.06678

N4031013 0.43931 0.04297 -5.26217 0.44242 0.30033 0.06564

N4032013 0.46770 0.03235 -2.74544 0.24330 0.25768 0.05764

N4032023 0.33750 0.03072 -1.34363 0.28629 0.25354 0.05200

N4033013 0.54311 0.04199 -1.32947 0.18259 0.27452 0.04526

N4034013 0.43217 0.05913 0.50716 0.19391 0.37455 0.03766

N4035013 0.48962 0.05891 0.11979 0.18035 0.39951 0.03640

N4035023 0.53233 0.04765 -2.33567 0.30546 0.48334 0.05966

N4035033 0.34337 0.05306 0.31740 0.30122 0.46663 0.04182

N4036013 0.80150 0.07147 0.72634 0.06735 0.28030 0.02053

N4037013 3.50307 0.00000 -0.31270 0.02007 0.39230 0.01571
Table D-31 (continued)

IRT Parameters for the Science Long-Term Trend Samples, Age 9

NAEP ID A S.E. B S.E. C S.E.

719

N4037023 3.35457 0.00000 -0.35879 0.02368 0.48618 0.01644

N4037033 3.75914 0.29325 -0.24694 0.02539 0.39095 0.01599

N4038013 0.46127 0.09380 1.89190 0.21792 0.43910 0.02715

N4038023 0.00000 0.00000 -0.13604 0.00000 0.00000 0.00000

N4038033 0.48779 0.05494 -0.90377 0.26483 0.47288 0.04632

N4038043 0.40332 0.05095 -0.35973 0.27889 0.42061 0.04644

N4039013 0.60614 0.04973 -0.46523 0.12440 0.26422 0.03363

N4040013 0.23197 0.03052 1.09308 0.26634 0.24042 0.03499

N4042013 0.49011 0.05656 1.24827 0.10759 0.18109 0.02513
Table D-32

IRT Parameters for the Science Long-Term Trend Samples, Age 13

NAEP ID A S.E. B S.E. C S.E.

720

N4045011 0.85565 0.05351 -2.31702 0.13739 0.25274 0.05922

N4046011 0.48238 0.04925 -0.18754 0.19754 0.28356 0.04856

N4047011 0.76049 0.05044 -1.56840 0.15228 0.29493 0.05606

N4047021 0.63359 0.06098 0.24217 0.11615 0.28737 0.03444

N4002011 0.44659 0.03463 -1.83850 0.24816 0.28313 0.06071

N4049011 0.76399 0.04971 -0.59825 0.09727 0.22707 0.03502

N4049021 0.00000 0.00000 -0.01344 0.00000 0.00000 0.00000

N4049031 0.00000 0.00000 -0.01344 0.00000 0.00000 0.00000

N4048011 0.98001 0.08233 -1.68271 0.17682 0.52180 0.05700

N4048021 1.72682 0.10676 -0.29858 0.03993 0.33931 0.02046

N4048031 1.44376 0.10451 0.47822 0.04162 0.37175 0.01667

N4050011 0.79237 0.08030 0.63614 0.08311 0.36227 0.02502

N4051011 0.84401 0.07120 0.92283 0.05519 0.19584 0.01849

N4052011 0.57024 0.08018 0.85935 0.13560 0.37345 0.03399

N4053011 0.86595 0.09949 1.20749 0.06828 0.30316 0.01896

N4054011 1.11274 0.10370 1.21835 0.04767 0.21978 0.01388

N4012011 1.05210 0.07084 0.11518 0.05553 0.28190 0.02237

N4055011 1.19018 0.09090 0.48914 0.04989 0.35287 0.01874

N4056011 0.75155 0.12998 1.57379 0.11259 0.40943 0.02075

N4057011 0.91242 0.06034 0.39984 0.05042 0.17925 0.01980

N4058011 1.40936 0.11938 1.08156 0.03773 0.24713 0.01264

N4059011 1.23781 0.11235 1.51164 0.05485 0.21477 0.01054

N4060011 0.95940 0.23629 2.92186 0.37582 0.14820 0.00821

N4061011 1.45895 0.21427 2.38895 0.17170 0.20091 0.00728

N4062011 1.42854 0.13555 2.13437 0.09698 0.11224 0.00615

N4063012 0.56698 0.12571 1.23073 0.20393 0.63128 0.02801

N4063022 0.33217 0.03749 -1.02640 0.30926 0.47909 0.04298

N4063032 0.87581 0.08037 0.15703 0.09174 0.41952 0.02785

N4063042 0.61305 0.11679 1.45169 0.14544 0.47125 0.02676

N4064012 1.01093 0.11089 0.26178 0.09749 0.59402 0.02311

N4064022 1.25122 0.10057 0.26081 0.05741 0.43891 0.02068

N4064032 1.05340 0.11479 -0.65178 0.14583 0.66547 0.03147

N4064042 1.24445 0.10721 -0.19117 0.07711 0.53874 0.02447

N4064052 1.12918 0.09414 -0.26500 0.08321 0.49278 0.02701

N4065012 0.77587 0.07101 0.74893 0.06857 0.23083 0.02331

N4066012 0.49248 0.05082 -0.74990 0.25283 0.33634 0.05996

N4067012 1.08533 0.09700 0.60897 0.05964 0.38486 0.02013

N4068012 0.80242 0.06583 -1.58961 0.19118 0.46833 0.05768

N4068022 0.45650 0.08969 1.05124 0.23627 0.51134 0.03882

N4068032 0.67738 0.05493 -0.79811 0.15474 0.36254 0.04490

N4068042 0.72057 0.05593 -1.06937 0.15734 0.37972 0.04753

N4068052 1.56880 0.15222 1.23170 0.05582 0.54283 0.01132

N4068062 0.38666 0.04667 0.11822 0.23608 0.41368 0.04088

N4069012 0.70690 0.07138 0.21753 0.11532 0.35851 0.03352

N4070012 0.42021 0.06919 1.04549 0.19917 0.30418 0.04493

N4071012 0.82574 0.11961 1.97283 0.12033 0.14310 0.01384

N4072012 0.80020 0.08809 0.85388 0.07954 0.34212 0.02395

N4073012 0.44487 0.05355 0.81371 0.14998 0.21455 0.03776

N4073022 0.89255 0.17304 1.78441 0.12771 0.40733 0.01704

N4080012 1.43693 0.10586 0.91884 0.03441 0.20692 0.01292

N4076012 0.73803 0.09443 1.33781 0.08332 0.26370 0.02176

N4075012 0.00000 0.00000 -0.01344 0.00000 0.00000 0.00000

N4077012 0.76949 0.08684 1.28984 0.07242 0.21377 0.01990

N4078012 1.36883 0.14211 1.82767 0.08538 0.30380 0.00966
Table D-32 (continued)

IRT Parameters for the Science Long-Term Trend Samples, Age 13

NAEP ID A S.E. B S.E. C S.E.

721

N4079012 0.52343 0.07264 1.07908 0.12737 0.27277 0.03386

N4082012 1.03478 0.13878 1.93198 0.10689 0.20136 0.01125

N4083013 1.23038 0.10140 0.75319 0.04690 0.34605 0.01656

N4083023 0.69316 0.05783 -1.22132 0.19190 0.44143 0.05245

N4083033 0.68206 0.07183 -0.97399 0.23439 0.55134 0.05057

N4083043 0.97872 0.07871 -1.03037 0.13047 0.47376 0.04153

N4084013 0.38108 0.03850 -0.29704 0.22413 0.31768 0.04454

N4085013 0.78391 0.05808 -0.70055 0.12049 0.30595 0.04111

N4085023 0.54303 0.06703 1.10800 0.10546 0.21805 0.03013

N4086013 0.45875 0.03460 -1.12937 0.19781 0.18206 0.05332

N4087013 0.31063 0.03907 0.68616 0.22743 0.35566 0.03556

N4088013 0.28896 0.03712 0.93120 0.23154 0.33292 0.03391

N4089013 1.37529 0.15554 0.81009 0.06066 0.56976 0.01544

N4089023 0.87551 0.07293 -1.75233 0.19560 0.50879 0.06031

N4089033 0.92821 0.08601 0.52585 0.07281 0.40329 0.02255

N4089043 0.78548 0.13029 1.39005 0.10563 0.47551 0.02051

N4090013 0.47751 0.03562 -0.41157 0.14261 0.17657 0.03902

N4091013 0.79425 0.05241 -1.35261 0.13641 0.28649 0.05089

N4091023 0.75910 0.07188 0.36214 0.09339 0.34899 0.02856

N4091033 0.85745 0.15687 1.83667 0.12733 0.35671 0.01667

N4092013 0.76617 0.09905 1.01019 0.09027 0.40541 0.02393

N4093013 0.88858 0.06127 -0.07622 0.07132 0.24851 0.02722

N4094013 0.00000 0.00000 -0.01344 0.00000 0.00000 0.00000

N4094023 0.00000 0.00000 -0.01344 0.00000 0.00000 0.00000

N4094033 0.00000 0.00000 -0.01344 0.00000 0.00000 0.00000

N4095013 0.98168 0.11349 1.67834 0.07436 0.14850 0.01220

N4096013 1.34898 0.12250 1.15736 0.04273 0.28859 0.01297

N4097013 1.33892 0.11657 1.75686 0.06749 0.20768 0.00874

N4098013 0.00000 0.00000 -0.01344 0.00000 0.00000 0.00000
Table D-33

IRT Parameters for the Science Long-Term Trend Samples, Age 17

NAEP ID A S.E. B S.E. C S.E.

722

N4002011 0.48961 0.04025 -2.80462 0.25554 0.24478 0.05804

N4046011 0.36941 0.03406 -1.65536 0.28311 0.25768 0.05745

N4100011 0.00000 0.00000 -0.05246 0.00000 0.00000 0.00000

N4100021 0.00000 0.00000 -0.05246 0.00000 0.00000 0.00000

N4100031 0.24892 0.03601 -4.23989 0.73728 0.46308 0.06551

N4100041 0.52243 0.06168 -0.86993 0.28723 0.56137 0.05027

N4099011 0.84051 0.06004 -0.88321 0.11818 0.29998 0.04496

N4086011 0.33388 0.03027 -2.69910 0.31234 0.18504 0.05437

N4093011 0.81322 0.05208 -1.39173 0.11897 0.22096 0.04809

N4063011 1.15843 0.12861 0.24633 0.08882 0.60036 0.02252

N4063021 0.30049 0.03490 -1.40400 0.34079 0.43034 0.04610

N4063031 0.89335 0.08335 -0.17972 0.11360 0.47337 0.03321

N4063041 0.48559 0.05506 -0.40659 0.24253 0.43123 0.04958

N4101011 0.70132 0.07329 -0.91459 0.21985 0.54313 0.05078

N4101021 0.32474 0.03823 -1.24174 0.33313 0.47067 0.04557

N4101031 0.35253 0.03945 -1.52347 0.33548 0.46857 0.04938

N4066011 0.37313 0.03196 -2.20500 0.27163 0.20399 0.05572

N4050011 0.44223 0.04925 -0.25826 0.23159 0.32492 0.05137

N4012011 0.89779 0.06517 -0.42039 0.09336 0.30911 0.03533

N4052011 0.62688 0.08429 0.70083 0.13071 0.35856 0.03581

N4102011 0.80127 0.10396 1.53016 0.08655 0.22590 0.01826

N4060011 0.75794 0.12904 2.00914 0.14467 0.21074 0.01721

N4095011 0.70233 0.06980 1.11159 0.06951 0.14791 0.02139

N4061011 1.40936 0.11459 1.48896 0.05064 0.18525 0.00961

N4103011 0.00000 0.00000 -0.05246 0.00000 0.00000 0.00000

N4062011 1.19643 0.11440 1.54744 0.05618 0.10057 0.00949

N4081011 0.65487 0.08758 1.55297 0.09689 0.18004 0.02221

N4064012 0.97961 0.08256 -0.50930 0.11104 0.47105 0.03587

N4064022 1.01256 0.07403 -0.70145 0.09797 0.40396 0.03663

N4064032 1.13816 0.09419 -1.44138 0.12954 0.50314 0.04808

N4064042 1.17028 0.08626 -1.07980 0.10004 0.44047 0.04070

N4064052 1.01346 0.07506 -1.24200 0.11888 0.42634 0.04533

N4104012 0.22079 0.03114 0.25036 0.30688 0.34747 0.03659

N4068012 0.77902 0.06708 -2.11823 0.20115 0.45913 0.05877

N4068022 0.28668 0.04220 1.03988 0.27208 0.41353 0.03338

N4068032 0.76462 0.06229 -1.19911 0.16460 0.43715 0.05001

N4068042 0.64284 0.05192 -1.84614 0.20103 0.40048 0.05618

N4068052 0.55830 0.08251 0.55399 0.18465 0.46707 0.03855

N4068062 0.37129 0.04441 -0.13503 0.25009 0.43904 0.04098

N4105012 0.34812 0.02922 -0.84915 0.19391 0.15372 0.04173

N4106012 2.22456 0.10967 1.13627 0.02537 0.15519 0.00800

N4106022 0.50555 0.04949 -2.75735 0.32733 0.43501 0.06644

N4106032 1.47739 0.15215 1.01873 0.04812 0.42845 0.01415

N4106042 0.46553 0.04541 -2.39292 0.32611 0.43134 0.06484

N4069012 0.53496 0.04910 -0.60142 0.19111 0.27961 0.05223

N4074012 0.42679 0.04499 -0.78513 0.25679 0.43222 0.04798

N4074022 0.00000 0.00000 -0.05246 0.00000 0.00000 0.00000

N4074032 0.57936 0.06571 -0.20359 0.20025 0.43983 0.04578

N4074042 0.62038 0.05274 -2.12497 0.22815 0.41582 0.05960

N4072012 0.64225 0.07151 0.24853 0.13791 0.32865 0.03981

N4070012 0.34630 0.03289 -0.74863 0.23552 0.20844 0.04850

N4107012 0.92935 0.08270 0.71901 0.05909 0.25772 0.02116

N4077012 0.74941 0.06744 0.75671 0.06816 0.18867 0.02360

N4073012 0.29677 0.03616 0.78352 0.21737 0.24971 0.03662
Table D-33 (continued)

IRT Parameters for the Science Long-Term Trend Samples, Age 17

NAEP ID A S.E. B S.E. C S.E.

723

N4073022 1.12228 0.15435 1.24211 0.06961 0.43463 0.01659

N4071012 1.09467 0.10061 1.11265 0.04738 0.19397 0.01508

N4108012 0.36698 0.05452 1.25953 0.19833 0.24071 0.04055

N4109012 0.99808 0.07345 1.02010 0.04294 0.10964 0.01313

N4110012 1.15590 0.10938 1.31821 0.04953 0.16157 0.01247

N4083013 1.04277 0.07523 -0.56072 0.08889 0.39743 0.03349

N4083023 0.72177 0.05706 -1.87014 0.18998 0.42565 0.05679

N4083033 0.81542 0.06677 -1.82696 0.18344 0.46994 0.05657

N4083043 0.99883 0.07591 -1.72258 0.13779 0.43647 0.05176

N4051013 1.07687 0.07790 0.17801 0.05752 0.29439 0.02324

N4089013 0.85016 0.06708 -1.06585 0.14279 0.45301 0.04537

N4089023 1.31265 0.11069 -1.80596 0.11932 0.47345 0.05059

N4089033 0.75710 0.07022 0.07180 0.11031 0.38320 0.03274

N4089043 0.52642 0.05685 -0.06095 0.18789 0.37462 0.04400

N4054013 0.85491 0.06394 0.57100 0.05480 0.16494 0.02102

N4113013 0.61534 0.17737 3.49267 0.58316 0.14062 0.01258

N4055013 0.66249 0.05133 -0.50176 0.12988 0.27312 0.04182

N4111013 0.51607 0.04614 -0.19342 0.15665 0.21546 0.04430

N4112013 0.73900 0.06302 0.20167 0.09177 0.25667 0.03143

N4088013 0.90586 0.08028 -0.01962 0.09598 0.40462 0.03125

N4114013 1.66500 0.10158 0.32959 0.03048 0.23200 0.01539

N4115013 1.13488 0.09250 1.08760 0.04262 0.16938 0.01344

N4115023 0.81636 0.05719 -1.06192 0.12631 0.31610 0.04668

N4116013 1.45694 0.10553 0.85804 0.03283 0.20082 0.01280

N4117013 1.24097 0.09496 0.92225 0.03803 0.18924 0.01388

N4118013 1.70366 0.10004 0.35988 0.02810 0.20134 0.01434

N4119013 1.48805 0.12300 1.06898 0.03600 0.22053 0.01207

N4120013 1.24929 0.12664 1.56534 0.06351 0.25507 0.01119
Table D-34

IRT Parameters for the Writing Long-Term Trend Samples, Age 9

724

Table D-35

IRT Parameters for the Writing Long-Term Trend Samples, Age 13

Table D-36

IRT Parameters for the Writing Long-Term Trend Samples, Age 17

[Tables D-34, D-35, and D-36 are intended to provide data from the 1996

long-term trend assessment in writing; however, the data from this assessment

are currently under review. After additional examination and analyses, these

tables will be included in a revised web version of the complete report.]
725

Appendix E

ESTIMATION ERROR VARIANCE

BY GENDER AND RACE/ETHNICITY

Table E-1

Estimation Error Variance for the Main Mathematics

Number Sense, Properties, and Operations Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

GRADE 4

Total 0.88 0.91 0.09

Male 1.33 0.89 0.11

Female 1.10 0.85 0.15

White 1.13 0.89 0.11

Black 7.31 0.93 0.07

Hispanic 4.49 0.89 0.11

Asian American 17.64 0.89 0.11

American Indian 6.49 0.78 0.22

Other 53.12 0.76 0.24

GRADE 8

Total 1.18 0.91 0.09

Male 1.89 0.88 0.12

Female 1.30 0.85 0.15

White 1.48 0.91 0.09

Black 4.99 0.83 0.17

Hispanic 3.58 0.76 0.24

Asian American 15.24 0.79 0.21

American Indian 11.79 0.79 0.21

Other 505.74 0.94 0.06

GRADE 12

Total 1.27 0.93 0.07

Male 1.73 0.88 0.12

Female 1.53 0.91 0.09

White 1.12 0.91 0.09

Black 5.82 0.87 0.13

Hispanic 3.51 0.77 0.23

Asian American 27.19 0.92 0.08

American Indian 111.35 0.96 0.04

Other 141.65 0.89 0.11
726

Table E-2

Estimation Error Variance for the Main Mathematics

Measurement Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

GRADE 4

Total 1.25 0.84 0.16

Male 1.75 0.78 0.22

Female 1.61 0.77 0.23

White 1.63 0.84 0.16

Black 5.59 0.75 0.25

Hispanic 6.39 0.77 0.23

Asian American 24.93 0.77 0.23

American Indian 7.36 0.75 0.25

Other 128.73 0.87 0.13

GRADE 8

Total 2.06 0.87 0.13

Male 3.30 0.86 0.14

Female 2.52 0.78 0.22

White 2.58 0.87 0.13

Black 7.81 0.77 0.23

Hispanic 9.43 0.74 0.26

Asian American 29.03 0.72 0.28

American Indian 20.90 0.79 0.21

Other 249.71 0.78 0.22

GRADE 12

Total 1.25 0.84 0.16

Male 1.78 0.69 0.31

Female 1.78 0.82 0.18

White 1.39 0.80 0.20

Black 5.34 0.72 0.28

Hispanic 5.25 0.65 0.35

Asian American 38.41 0.85 0.15

American Indian 138.38 0.94 0.06

Other 101.64 0.84 0.16
727

Table E-3

Estimation Error Variance for the Main Mathematics

Geometry and Spatial Sense Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

GRADE 4

Total 0.77 0.82 0.18

Male 1.15 0.79 0.21

Female 1.02 0.74 0.26

White 0.92 0.79 0.21

Black 3.21 0.66 0.34

Hispanic 4.26 0.84 0.16

Asian American 20.33 0.82 0.18

American Indian 6.45 0.73 0.27

Other 72.49 0.75 0.25

GRADE 8

Total 1.10 0.91 0.09

Male 1.73 0.85 0.15

Female 1.39 0.82 0.18

White 1.32 0.88 0.12

Black 4.73 0.84 0.16

Hispanic 6.10 0.85 0.15

Asian American 15.43 0.78 0.22

American Indian 13.65 0.76 0.24

Other 221.87 0.86 0.14

GRADE 12

Total 1.12 0.88 0.12

Male 1.44 0.82 0.18

Female 1.51 0.87 0.13

White 1.38 0.86 0.14

Black 5.92 0.83 0.17

Hispanic 5.85 0.84 0.16

Asian American 20.32 0.82 0.18

American Indian 67.86 0.91 0.09

Other 140.13 0.91 0.09
728

Table E-4

Estimation Error Variance for the Main Mathematics

Data Analysis, Statistics, and Probability Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

GRADE 4

Total 1.28 0.87 0.13

Male 1.61 0.79 0.21

Female 1.78 0.82 0.18

White 1.41 0.84 0.16

Black 10.38 0.87 0.13

Hispanic 7.58 0.76 0.24

Asian American 25.26 0.77 0.23

American Indian 7.61 0.72 0.28

Other 36.14 0.44 0.56

GRADE 8

Total 2.31 0.92 0.08

Male 3.56 0.89 0.11

Female 2.41 0.83 0.17

White 3.36 0.93 0.07

Black 5.95 0.75 0.25

Hispanic 7.69 0.76 0.24

Asian American 27.17 0.73 0.27

American Indian 23.20 0.76 0.24

Other 1034.92 0.98 0.02

GRADE 12

Total 0.99 0.91 0.09

Male 1.45 0.85 0.15

Female 1.30 0.84 0.16

White 0.96 0.85 0.15

Black 6.01 0.89 0.11

Hispanic 4.11 0.78 0.22

Asian American 30.67 0.91 0.09

American Indian 70.57 0.94 0.06

Other 162.78 0.89 0.11
729

Table E-5

Estimation Error Variance for the Main Mathematics

Algebra and Functions Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

GRADE 4

Total 0.99 0.83 0.17

Male 1.28 0.78 0.22

Female 1.38 0.76 0.24

White 1.09 0.80 0.20

Black 5.73 0.75 0.25

Hispanic 4.88 0.73 0.27

Asian American 16.94 0.69 0.31

American Indian 6.10 0.71 0.29

Other 49.25 0.72 0.28

GRADE 8

Total 1.19 0.89 0.11

Male 2.16 0.91 0.09

Female 1.08 0.79 0.21

White 1.61 0.87 0.13

Black 3.90 0.78 0.22

Hispanic 4.33 0.71 0.29

Asian American 18.11 0.80 0.20

American Indian 12.34 0.78 0.22

Other 441.38 0.94 0.06

GRADE 12

Total 1.33 0.93 0.07

Male 1.70 0.87 0.13

Female 1.64 0.88 0.12

White 1.40 0.92 0.08

Black 8.34 0.89 0.11

Hispanic 4.33 0.76 0.24

Asian American 23.64 0.88 0.12

American Indian 63.71 0.94 0.06

Other 136.92 0.92 0.08
730

Table E-6

Estimation Error Variance for the Main Mathematics

Composite Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

GRADE 4

Total 0.78 0.95 0.05

Male 1.06 0.95 0.05

Female 0.96 0.93 0.07

White 0.92 0.95 0.05

Black 5.07 0.94 0.06

Hispanic 4.15 0.93 0.07

Asian American 16.38 0.94 0.06

American Indian 5.14 0.82 0.18

Other 54.25 0.83 0.17

GRADE 8

Total 1.13 0.96 0.04

Male 1.88 0.96 0.04

Female 1.14 0.94 0.06

White 1.46 0.96 0.04

Black 3.73 0.94 0.06

Hispanic 4.20 0.89 0.11

Asian American 14.81 0.93 0.07

American Indian 8.67 0.85 0.15

Other 421.34 0.95 0.05

GRADE 12

Total 0.99 0.98 0.02

Male 1.24 0.95 0.05

Female 1.25 0.96 0.04

White 0.97 0.96 0.04

Black 5.14 0.95 0.05

Hispanic 3.35 0.92 0.08

Asian American 23.64 0.96 0.04

American Indian 79.94 0.98 0.02

Other 127.08 0.94 0.06
731

Table E-7

Estimation Error Variance for the Main Science

Earth Science Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

GRADE 4

Total 0.72 0.85 0.15

Male 0.98 0.75 0.25

Female 1.08 0.80 0.20

White 1.02 0.85 0.15

Black 5.53 0.83 0.17

Hispanic 3.69 0.77 0.23

Asian American 19.31 0.83 0.17

American Indian 20.13 0.89 0.11

Other 37.63 0.38 0.62

GRADE 8

Total 0.89 0.91 0.09

Male 1.24 0.86 0.14

Female 1.46 0.89 0.11

White 1.30 0.88 0.12

Black 3.08 0.78 0.22

Hispanic 2.91 0.79 0.21

Asian American 12.67 0.83 0.17

American Indian 19.60 0.91 0.09

Other 28.44 0.53 0.47

GRADE 12

Total 0.91 0.93 0.07

Male 1.82 0.91 0.09

Female 0.95 0.87 0.13

White 1.04 0.92 0.08

Black 3.69 0.79 0.21

Hispanic 5.87 0.88 0.12

Asian American 9.18 0.84 0.16

American Indian 19.57 0.84 0.16

Other 91.69 0.84 0.16
732

Table E-8

Estimation Error Variance for the Main Science

Physical Science Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

GRADE 4

Total 1.16 0.82 0.18

Male 1.62 0.76 0.24

Female 1.40 0.73 0.27

White 1.53 0.79 0.21

Black 4.45 0.72 0.28

Hispanic 4.29 0.69 0.31

Asian American 16.42 0.73 0.27

American Indian 13.11 0.84 0.16

Other 45.78 0.45 0.55

GRADE 8

Total 0.91 0.92 0.08

Male 1.54 0.88 0.12

Female 1.26 0.87 0.13

White 1.41 0.91 0.09

Black 1.59 0.73 0.27

Hispanic 3.51 0.84 0.16

Asian American 11.13 0.77 0.23

American Indian 24.17 0.92 0.08

Other 40.24 0.58 0.42

GRADE 12

Total 1.03 0.92 0.08

Male 1.81 0.92 0.08

Female 1.16 0.88 0.12

White 1.34 0.92 0.08

Black 3.60 0.84 0.16

Hispanic 6.16 0.89 0.11

Asian American 9.17 0.86 0.14

American Indian 39.80 0.89 0.11

Other 122.19 0.89 0.11
733

Table E-9

Estimation Error Variance for the Main Science

Life Science Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

GRADE 4

Total 0.86 0.78 0.22

Male 1.10 0.77 0.23

Female 1.24 0.67 0.33

White 1.06 0.76 0.24

Black 4.07 0.80 0.20

Hispanic 3.53 0.72 0.28

Asian American 17.33 0.74 0.26

American Indian 15.78 0.86 0.14

Other 53.07 0.59 0.41

GRADE 8

Total 1.07 0.86 0.14

Male 1.40 0.79 0.21

Female 1.77 0.85 0.15

White 1.40 0.86 0.14

Black 2.11 0.68 0.32

Hispanic 4.90 0.82 0.18

Asian American 11.07 0.73 0.27

American Indian 16.00 0.85 0.15

Other 66.28 0.77 0.23

GRADE 12

Total 0.80 0.89 0.11

Male 1.35 0.87 0.13

Female 1.17 0.85 0.15

White 1.07 0.89 0.11

Black 1.95 0.69 0.31

Hispanic 7.38 0.86 0.14

Asian American 11.73 0.82 0.18

American Indian 27.92 0.86 0.14

Other 59.92 0.76 0.24
734

Table E-10

Estimation Error Variance for the Main Science

Composite Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

GRADE 4

Total 0.64 0.89 0.11

Male 0.87 0.86 0.14

Female 0.83 0.84 0.16

White 0.81 0.87 0.13

Black 3.74 0.88 0.12

Hispanic 2.80 0.86 0.14

Asian American 13.54 0.91 0.09

American Indian 14.21 0.89 0.11

Other 20.64 0.29 0.71

GRADE 8

Total 0.78 0.94 0.06

Male 1.12 0.92 0.08

Female 1.27 0.93 0.07

White 1.10 0.93 0.07

Black 1.47 0.84 0.16

Hispanic 3.11 0.91 0.09

Asian American 9.11 0.92 0.08

American Indian 17.45 0.92 0.08

Other 29.61 0.67 0.33

GRADE 12

Total 0.76 0.96 0.04

Male 1.44 0.95 0.05

Female 0.89 0.94 0.06

White 0.96 0.95 0.05

Black 2.37 0.87 0.13

Hispanic 5.50 0.95 0.05

Asian American 8.39 0.94 0.06

American Indian 23.45 0.89 0.11

Other 82.38 0.87 0.13
735

Table E-11

Estimation Error Variance for the

Reading Long-Term Trend Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

AGE 9

Total 1.01 0.86 0.14

Male 1.88 0.84 0.16

Female 1.26 0.79 0.21

White 1.37 0.85 0.15

Black 5.21 0.76 0.24

Hispanic 12.36 0.92 0.08

Asian American 22.11 0.87 0.13

American Indian 200.67 0.98 0.02

Other 88.41 0.66 0.34

AGE 13

Total 0.93 0.85 0.15

Male 1.51 0.82 0.18

Female 1.40 0.78 0.22

White 1.12 0.79 0.21

Black 5.83 0.84 0.16

Hispanic 8.81 0.88 0.12

Asian American 16.83 0.82 0.18

American Indian 69.59 0.95 0.05

Other 197.80 0.69 0.31

AGE 17

Total 1.08 0.82 0.18

Male 1.92 0.80 0.20

Female 1.34 0.67 0.33

White 1.50 0.80 0.20

Black 5.44 0.72 0.28

Hispanic 13.50 0.85 0.15

Asian American 35.31 0.85 0.15

American Indian 123.66 0.92 0.08

Other 174.14 0.68 0.32
736

Table E-12

Estimation Error Variance for the

Mathematics Long-Term Trend Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

AGE 9

Total 0.66 0.85 0.15

Male 1.48 0.82 0.18

Female 0.66 0.68 0.32

White 1.12 0.88 0.12

Black 2.35 0.72 0.28

Hispanic 3.13 0.83 0.17

Asian American 20.38 0.82 0.18

American Indian 16.40 0.79 0.21

Other 60.95 0.40 0.60

AGE 13

Total 0.66 0.89 0.11

Male 0.99 0.86 0.14

Female 0.84 0.86 0.14

White 0.84 0.89 0.11

Black 1.77 0.74 0.26

Hispanic 2.94 0.82 0.18

Asian American 14.03 0.92 0.08

American Indian 73.20 0.96 0.04

Other 81.97 0.57 0.43

AGE 17

Total 1.24 0.93 0.07

Male 1.68 0.85 0.15

Female 1.79 0.93 0.07

White 1.86 0.93 0.07

Black 3.13 0.83 0.17

Hispanic 4.66 0.78 0.22

Asian American 45.49 0.95 0.05

American Indian 98.09 0.95 0.05

Other 85.23 0.67 0.33
737

Table E-13

Estimation Error Variance for the

Science Long-Term Trend Scale

Proportion of Variance Due to . . .

Total Student Latency

Variance Sampling %

AGE 9

Total 1.13 0.81 0.19

Male 2.22 0.84 0.16

Female 1.33 0.60 0.40

White 1.60 0.79 0.21

Black 7.21 0.66 0.34

Hispanic 8.58 0.82 0.18

Asian American 34.07 0.83 0.17

American Indian 47.26 0.86 0.14

Other 98.41 0.24 0.76

AGE 13

Total 1.04 0.87 0.13

Male 1.23 0.75 0.25

Female 1.70 0.82 0.18

White 1.35 0.83 0.17

Black 4.12 0.69 0.31

Hispanic 6.18 0.73 0.27

Asian American 13.17 0.75 0.25

American Indian 47.52 0.93 0.07

Other 67.10 0.56 0.44

AGE 17

Total 1.40 0.86 0.14

Male 2.92 0.84 0.16

Female 1.84 0.80 0.20

White 1.69 0.85 0.15

Black 6.16 0.74 0.26

Hispanic 11.81 0.83 0.17

Asian American 75.19 0.94 0.06

American Indian 282.35 0.97 0.03

Other 181.29 0.76 0.24
738

Table E-14

Estimation Error Variance for the

Writing Long-Term Trend Scale

[This table is intended to provide data from the 1996 long-term trend

assessment in writing; however, the data from this assessment are

currently under review. After additional examination and analyses, this

table will be included in a revised web version of the complete report.]
739

Appendix F

SETTING THE NAEP ACHIEVEMENT LEVELS FOR

THE 1996 MATHEMATICS ASSESSMENT

Mary Lyn Bourque

National Assessment Governing Board

Introduction

Since 1984, NAEP has reported the performance of students in the nation and for specific

subpopulations on a 0-to-500 proficiency scale. The history and development of the scale and the

anchoring procedure used to interpret specific points on that scale is described elsewhere in this report.

The achievement levels reported in 1996 were first developed in 1992 through a process

described in more detail in the following sections of this chapter. The levels were based on the

mathematics assessment framework and item pools developed for the 1990 and 1992 assessments.

Although the National Assessment Governing Board (NAGB) updated this framework in preparation for

the 1996 assessment, the differences in test and item specifications were not large enough to warrant a

new mathematics scale. Therefore, a decision was made by the Board to retain the same levels as

reported for the 1990 and 1992 assessments, thus allowing a third point of comparison for the short-term

mathematics trend.

History of the Achievement Levels Development 1990 - 1992

The 1988 legislation1 created an independent board, the National Assessment Governing Board

(NAGB), responsible for setting policy for the NAEP program. The Board has a statutory mandate to

identify “appropriate achievement goals for each . . . grade in each subject area to be tested under the

National Assessment.” Consistent with this directive, and striving to achieve one of the primary mandates

of the statute “to improve the form and use of NAEP results,” the Board set performance standards

(called achievement levels by NAGB) for the National Assessment in 1990 and again in 1992. The 1994

legislation (Public Law 103-382) continued the policy Board with slightly increased membership, and a

continued mandate to set student performance standards on each age and grade tested.

The 1990 trial, initiated in December 1989 with the dissemination of a draft policy statement

(NAGB, 1989) and culminating 22 months later in the publication of the NAGB report, The Levels of

Mathematics Achievement (Bourque & Garrison, 1991), consisted of two phases: the main study and a

replication-validation study. Although there were slight differences between the two phases, there were

many common elements. Both phases used a modified (iterative/empirical) Angoff (1971) procedure for

arriving at the levels; both focused on estimating performance levels based on a review of the 1990

NAEP mathematics item pool; and both phases employed a set of policy definitions for Basic, Proficient,

and Advanced (NAGB, 1990) as the criteria for making the item ratings. However, the 1990 process was

evaluated by a number of different groups (see Hambleton & Bourque, 1991) who identified technical

flaws in the 1990 process. These evaluations influenced NAGB’s decision to set the levels again in 1992

 

1

 Public Law 100-297. (1988). National assessment of educational progress improvement act (Article No. USC 1221).

Washington, DC.
740

and to not use the 1990 levels as benchmarks for progress toward the national goals during the coming

decade. However, it is interesting to note that the 1990 and 1992 processes produced remarkably similar

results.

In September 1991 NAGB contracted with American College Testing (ACT) to convene the

panels of judges that would recommend the levels on the 1992 NAEP assessments in reading, writing,

and mathematics. While the 1992 level-setting activities were not unlike those undertaken by NAGB in

1990, there were significant improvements made in the process for 1992. There was a concerted effort to

bring greater technical expertise to the process: The contractor selected by NAGB has a national

reputation for setting standards in a large number of certification and licensure exams; an internal and

external advisory team monitored all the technical decisions made by the contractor throughout the

process; and state assessment directors periodically provided their expertise and technical assistance at

key stages in the project.

Setting achievement levels is a method for setting standards on the NAEP assessment that

identifies what students should know and be able to do at various points along the proficiency scale. The

initial policy definitions of the achievement levels were presented to panelists along with an illustrative

framework for more indepth development and operationalization of the levels. Panelists were asked to

determine descriptions/definitions of the three levels from the specific framework developed for the

NAEP assessment with respect to the content and skills to be assessed. The operationalized definitions

were refined throughout the level-setting process, as well as validated with a supplementary group of

judges subsequent to the level-setting meetings. Panelists were also asked to develop a list of illustrative

tasks associated with each of the levels, after which sample items from the NAEP item pool were

identified to exemplify the full range of performance of the intervals between levels. The emphasis in

operationalizing the definitions and in identifying and selecting exemplar items and papers was to

represent the full range of performance from the lower level to the next higher level. The details of the

implementation procedures are outlined in the remainder of this appendix.

Preparing for the 1992 Mathematics Level-Setting Meeting

It is important for the planning of any standard-setting effort to know how various process

elements interact with each other. For example, panelists interact with pre-meeting materials, the meeting

materials (i.e., the assessment questions, rating forms, rater feedback, and so forth), each other, and the

project staff. All of these elements combine to promote or degrade what has been called intrajudge

consistency and interjudge consensus (Friedman & Ho, 1990).

Previous research has conceptualized the effects of two major kinds of interaction: people

interacting with text (Smith & Smith, 1988) and people interacting with each other (Curry, 1987;

Fitzpatrick, 1989). To assess the effects of textual and social interaction and adjust the standard setting

procedures accordingly, a pilot study was conducted as the first phase of the 1992 initiative.

Reading was chosen as the single content area to be pilot-tested since it combined all of the

various features found in the other NAEP assessments, including multiple-choice, short constructedresponse, and extended constructed-response items. The pilot study provided the opportunity to

implement and evaluate all aspects of the operational planbackground materials, meeting materials,

study design, meeting logistics, staff function, and participant function.
741

The overall pilot effort was quite successful. The level-setting process worked well, and the pilot

allowed the contractor to make improvements in the design before implementation activities began. For

example, schedule changes were made that allowed the panelists more time to operationalize the policy

definitions before beginning the item-rating task. Also, the feedback mechanisms used to inform panelists

about interjudge and intrajudge consistency data were improved for clarity and utility to the entire

process.

The Mathematics Level-Setting Panel

Sixty-nine panelists representing 32 jurisdictions (31 states and the District of Columbia) from

the 424 nominees were invited to participate in the level-setting process. They represented mathematics

teachers at grades 4, 8, and 12, nonteacher educators, and members of the noneducator (general public)

community. The group was balanced by gender, race/ethnicity, NAEP regions of the country, community

type (low SES, not low SES), district size, and school type (public/private). One panelist was unable to

attend due to a family emergency, resulting in 68 participants: 24 at grade 4 and 22 at grades 8 and 12.

Process for Developing the Achievement Levels

The four-and-one-half day session began with a brief overview of NAEP and NAGB, a

presentation on the policy definitions of the achievement levels, a review of the NAEP mathematics

assessment framework, and a discussion of factors that influence item difficulty. The purpose of the

presentation was to focus panelists’ attention on the mathematics framework and to emphasize the fact

that panelists’ work was directly related to the NAEP assessment, not to the whole domain of

mathematics.

All panelists completed and self-scored an appropriate grade-level form of the NAEP assessment.

The purpose of this exercise was to familiarize panelists with the test content and scoring protocols

before beginning to develop the preliminary operationalized descriptions of the three levels.

Working in small groups of five or six, panelists expanded and operationalized the policy

definitions of Basic, Proficient, and Advanced in terms of specific mathematical skills, knowledge, and

behaviors that were judged to be appropriate expectations for students in each grade, and were in

accordance with the current mathematics assessment framework.

The policy definitions are as follows:

Basic This level, below proficient, denotes partial mastery of the knowledge and

skills that are fundamental for proficient work at each grade4, 8, and 12.

Proficient This central level represents solid academic performance for each grade

tested4, 8, and 12. Students reaching this level have demonstrated

competency over challenging subject matter and are well prepared for the

next level of schooling.

Advanced This higher level signifies superior performance beyond proficient gradelevel mastery at grades 4, 8, and 12.
742

The small groups were allowed to brainstorm about what student performance should be, using

the framework and their experience in completing the NAEP assessment as guides2

. A comprehensive

listing of grade-level descriptors was developed, and panelists were asked to identify the five that best

described what students should be able to do at each of the levels. Those descriptors appearing with the

greatest frequency were compiled into a discussion list for the grade-level groups. Additions, deletions,

and modifications were made as a result of discussions, and the groups reached general agreement that

the final list of descriptors represented what students should be able to do at each achievement level.

Panelists next received two hours of training in the Angoff method. Training was customized to

reflect the unique item formats of the particular subject area assessment. Once a conceptual consensus

was reached about the characteristics of marginally acceptable examinees at each of the three levels,

practice items from the released pool were rated by the panelists according to the process defined in the

contractor’s plan. For multiple-choice and short constructed-response items, panelists were asked to rate

each item for the expected probability of a correct response for a group of marginally acceptable

examinees at the Basic, Proficient, and Advanced levels. For extended constructed-response items,

panelists were asked to review 20 to 25 student response papers and select three papers, one for each

achievement level, that typified marginally acceptable examinee performance for that level.

Following training in the Angoff method, the judges began the rating process, inspecting and

rating each item in the pool for the expected probabilities of answering the item correctly at each level.

Panelists completed three rounds of item ratings. For Round 1, panelists first answered the items in each

section, then reviewed their answers using scoring keys and protocols. This process helped ensure that

panelists would be thoroughly familiar with each item, including the foils and scoring rubrics, before

rating the items. Panelists provided item ratings/paper selections for all three achievement levels, one

item at a time, for all the items in a section, then proceeded to the next set of items, for which the process

was repeated. During Round 1, panelists used their lists of descriptors and other training materials for

guidance in the rating process.

Following Round 1, item response theory (IRT) was used to convert the rating results3

 for each

rater to a latent ability scale represented by the Greek letter theta (). This  scale was the same scale

used to calibrate the NAEP items evaluated by each panelist. In order to provide meaningful feedback

about item ratings, a special relative scale was constructed, which was a linear transformation of the

theta scale having a mean of 75 and standard deviation of 15. Before Round 2 of the rating process,

panelists were given interjudge consistency information using this relative scale. This information

allowed panelists to see on the scale where their individual mean item ratings were, relative to the mean

for the group and to the means for other panelists. Reasons for extreme mean ratings, including the

possibility that some panelists misinterpreted the item rating task, were discussed briefly.

Before Round 2, panelists were also given item difficulty data. This information was presented as

the percentage of students who answered each item correctly during the actual NAEP administration, for

items scored “correct” or “incorrect” (i.e., multiple-choice and short constructed-response items), and as

the percentage of students receiving scores of 1, 2, 3, and 4 for the extended constructed-response items4

.

Panelists were told that this item difficulty information should be used as a reality check. For items on

which item ratings differed substantially from the item difficulty value, panelists were asked to

 

2

 The panelists also reviewed about half the item pool (the half they would not be rating later) so the descriptors could be further

modified if appropriate.

3

 Because the IRT item parameters were not available for the polytomously scored (extended constructed-response) items, these items

(five at grade 4, six each at grades 8 and 12) were not included in the following discussion of results.

4

 The percentages presented to the raters summed to 100 percent, but this excluded the percentages—around 80 percent, in some

cases—of students who wrote responses that were judged to be “off-task,” those who “skipped” that question and continued beyond

that question, and those who, apparently, “never reached” that question.
743

reexamine the item to determine if they had misinterpreted the item or misjudged its difficulty. Results of

the data analysis, and panelists’ own evaluations, indicated that the item difficulty information was

perceived as very useful but had little impact on panelists’ ratings.

For Round 2, panelists reviewed the same set of items they had rated in Round 1 and, using the

interjudge consistency information, the item difficulty information, and the information provided prior to

Round 1, they either confirmed their initial item ratings or adjusted their ratings to reflect the additional

information. About one-third of Round 1 item ratings were adjusted during Round 2.

Following Round 2, panelists’ ratings were reanalyzed and additional information was presented

to panelists concerning intrajudge variability prior to Round 3. For each panelist, the intrajudge

variability information consisted of those items that they had rated differently than items having similar

difficulty, taking into consideration the panelist’s aggregated item ratings. That is, the panelists’

aggregated item ratings were converted to the theta () scale. All items rated by the panelists were then

analyzed in terms of the panelist’s achievement level () in comparison to actual student performance on

the items. The observed item rating from each panelist was contrasted to an expected item rating. Those

items with large differences between observed and expected ratings were identified. Panelists were given

this information and asked to review each of these items and decide if their Round 2 ratings still

accurately reflected their best judgments of the items. The intrajudge consistency data was to be used to

flag items for reconsideration in the final round of rating.

For Round 3, panelists reviewed the same set of items they rated in Rounds 1 and 2 using both

the new intrajudge variability information and the information made available during Rounds 1 and 2. In

addition, panelists could discuss, within their small groups, ratings of specific items about which they

were unsure. About 20 percent of the item ratings were adjusted during Round 3. 

Process of Selecting Exemplar Items

Following the standard-setting meeting, a series of procedures was implemented to select

exemplar items. First, expected and empirical p-values were computed for each item in the released item

pool. Expected p-values were based on predicted performance at the cut-off score for each achievement

level and empirical p-values were based on the average performance of all students responding to the

item. Items that did not have expected p-values  0.51 for any of the levels were deleted from the item

pool. Second, items were compared to the operationalized descriptions of the levels. Items that did not

match the content of the descriptions were deleted from the item pool. Third, the remaining items were

classified as possible Basic, Proficient, or Advanced exemplars based on content match. Fourth, the

validation panel reviewed the items and recommended a set of items to serve as exemplars for the levels.

The final set of items was reviewed and approved by NAGB at their May 1992 meeting. These

procedures are described in detail below.

Using the standard-setting ratings, expected p-values were computed for each item at the cut

point for each achievement level. The criteria described below were applied to the scale-level results and

an analysis was conducted to delineate items that could serve as exemplars for each achievement level

(Basic, Proficient, Advanced).
744

More specifically, for an item to be chosen as a possible exemplar for the Basic achievement level:

1. The expected p-value for students at the cut point for the Basic level of achievement

had to be greater than 0.51;

2. The content of the item had to match the content of the operationalized description

of Basic; and

3. The empirical p-value for the item had to be higher than empirical p-values for items

selected as exemplars for the Proficient level.

As an example:

Grade 4 Basic Level Item M022801

Level Basic Proficient Advanced

 Scale point 211 248 280

 Expected p-value 0.70 0.82 0.94

 Empirical p-value = 0.52

For an item to be chosen as a possible exemplar for the Proficient achievement level:

1. The expected p-value for students at the cut-off score for the Proficient level of

achievement had to be greater than 0.51;

2. The content of the item had to match the content of the operationalized description

of Proficient; and

3. The empirical p-value for the item had to be lower than empirical p-values for Basic

exemplar items, but higher than student p-values for Advanced exemplar items.

As an example:

Grade 4 Proficient Level Item M022001

Level Basic Proficient Advanced

 Scale point 211 248 280

 Expected p-value 0.37 0.58 0.76

 Empirical p-value = 0.35
745

For an item to be chosen as a possible exemplar for the Advanced achievement level:

1. The expected p-value for students at the cut-point for the Advanced level of

achievement had to be greater than 0.51;

2. The content of the item had to match the content of the operationalized description

of Advanced; and

3. The empirical p-value for the item had to be lower than empirical p-values for

Proficient exemplar items.

As an example:

Grade 4 Advanced Level Item M023101

Level Basic Proficient Advanced

 Scale point 211 248 280

 Expected p-value 0.29 0.43 0.61

 Empirical p-value = 0.22

The analysis procedures described above yielded 31 items as possible grade 4 exemplars, 43 items as

possible grade 8 exemplars, and 37 items as possible grade 12 exemplars, as follows:

Possible Exemplar Items by Grade and Achievement Level

Grade Basic Proficient Advanced

 4 9 14 8

 8 23 15 5

12 14 16 7

For grade 4, the possible exemplars represented 49 percent of the released item pool. For grades 8 and

12, the possible exemplars represented 54 percent of the released item pool for each grade.

Process for Validating the Levels

Eighteen mathematics educators participated in the item selection and content validation process.

Ten of the panelists were mathematics teachers who had participated in the original achievement levelssetting process and who had been identified as outstanding panelists by grade group facilitators during

this meeting. The other eight panelists represented the National Council of Teachers of Mathematics, the
746

Mathematical Sciences Education Board, and state-level mathematics curriculum supervisors. To the

extent possible, the group was balanced by race/ethnicity, gender, community type, and region of the

country.

The two-and-one-half day meeting began by briefing panelists on the purpose of the meeting.

They first reviewed the operationalized descriptions of the achievement levels for consistency with the

NAGB policy definitions of Basic, Proficient, and Advanced and with the NAEP Mathematics

Objectives. Next, they reviewed the operationalized descriptions of the achievement levels for qualities

such as within- and across-grade consistency, grade-level appropriateness, and utility for increasing the

public’s understanding of the NAEP mathematics results. Finally, working first in grade level (4, 8, and

12) groups of six panelists each, then as a whole group, panelists revised the operationalized descriptions

to provide more within- and across-grade consistency and to align the language of the description more

closely with the language of the NCTM Standards. Both the original descriptions and the revised

descriptions are included later in this appendix.

On the third day, panelists again split into grade-level groups of six panelists each and reviewed

the possible exemplar items. The task was to select a set of items, for each achievement level for their

grade, that would best communicate to the public the levels of mathematics ability and the types of skills

needed to perform in mathematics at that level.

After selecting sets of items for their grades, the three grade-level groups met as a whole group to

review item selection. During this process, cross-grade items that had been selected as exemplars by two

grade groups (three such items were selected by grade groups 4 and 8) were assigned to one grade by

whole-group consensus. In addition, items were evaluated by the whole group for overall quality. Two

items were rejected by the group during this process due to possible bias. This process yielded 14 items

as recommended exemplars for grade 4, 11 items as recommended exemplars for grade 8, and 14 items as

recommended exemplars for grade 12.

Mapping Panelists’ Ratings to the NAEP Scales

The process of mapping panelists’ ratings to the NAEP scales made significant use of item

response theory (IRT). IRT provides statistically sophisticated methods for determining the expected

performance of examinees on particular test items in terms of an appropriate measurement scale. The

same measurement scale simultaneously describes the characteristics of the test items and the

performance of the examinees. Once the item characteristics are set, it is possible to precisely determine

how examinees are likely to perform on the test items at different points of the measurement scale.

The panelists’ ratings of the NAEP test items were likewise linked, by definition, to the expected

performance of examinees at the theoretical achievement level cut points. It was therefore feasible to use

the IRT item characteristics to calculate the values on the measurement scale corresponding to each

achievement level. This was done by averaging the item ratings over panelists for each achievement level

and then simply using the item characteristics to find the corresponding achievement level cut points on

the IRT measurement scale. This process was repeated for each of the NAEP content strands within each

grade (4, 8, and 12).

In the final stage in the mapping process, the achievement level cut points on the IRT

measurement scale were combined over content strands and rescaled to the NAEP score scale. Weighted

averages of the achievement level cut points were computed. The weighting constants accounted for the

measurement precision of the test items evaluated by the panelists, the proportion of items belonging to
747

each NAEP content strand, and the linear NAEP scale transformation. These weighted averages produced

the final cut points for the Basic, Proficient, and Advanced achievement levels within each grade.

Evaluation of the Mathematics Levels

The 1992 mathematics achievement levels used here to report the 1996 NAEP data were

evaluated under a Congressional mandate by the National Academy of Education (NAE). A series of

research studies were mounted by the NAE (National Academy of Education, 1993) to look at various

aspects of the validity of the level-setting process, and the levels adopted by NAGB. Several of these

studies focused specifically on the mathematics achievement levels, and were conducted for the

Academy by staff at the Learning Research and Development Center at the University of Pittsburgh.

Based on these studies, the Academy’s 1993 policy report concluded that the achievement levels in

mathematics were flawed and should not be continued. The more recent report from the Academy

(National Academy of Education, 1997) concluded that the current achievement levels raised serious

concerns about their reliability and validity, were not reasonable (i.e., were set too high), and in the final

analysis, should be abandoned by the end of the century.

While NAGB did not agree with the earlier policy report and continues to disagree with the more

recent one, and while the Board’s contractor and its technical advisors do not believe the weight of the

evidence supported the NAEP conclusions, the Board believes that standards-based reporting is

responsive to the needs of the users of NAEP data, is an important aspect of the national reform

movement, and assists in making NAEP data more useful and understandable to the public. The Board is

committed to making improvements in the process, and will continue to support further investigation into

the validity of the levels through additional research.

Figure F-1

Final Description of 1992 Mathematics Achievement Levels

GRADE 4

The NAEP content strands include: (1) number sense, properties, and operations; (2) measurement;

(3) geometry; (4) data analysis, statistics, and probability; (5) algebra and functions. (Note: At the fourthgrade level, algebra and functions are treated in informal and exploratory ways, often through the study of

patterns.) Skills are cumulative across levelsfrom Basic to Proficient to Advanced.

BASIC. Fourth-grade students performing at the basic level should show some evidence of understanding

the mathematical concepts and procedures in the five NAEP content strands.

Specifically, fourth graders performing at the basic level should be able to estimate and use basic

facts to perform simple computations with whole numbers; show some understanding of fractions and

decimals; and solve simple real-world problems in all NAEP content strands. Students at this level should

be able to usethough not always accuratelyfour-function calculators, rulers, and geometric shapes.

Their written responses are often minimal and presented without supporting information.

(continued)
Figure F-1 (continued)

Final Description of 1992 Mathematics Achievement Levels

748

PROFICIENT. Fourth-grade students performing at the proficient level should consistently apply

integrated procedural knowledge and conceptual understanding to problem solving in the five NAEP

content strands.

Specifically, fourth graders performing at the proficient level should be able to use whole numbers

to estimate, compute, and determine whether results are reasonable. They should have a conceptual

understanding of fractions and decimals; be able to solve real-world problems in all NAEP content strands;

and use four-function calculators, rulers, and geometric shapes appropriately. Students performing at the

proficient level should employ problem-solving strategies such as identifying and using appropriate

information. Their written solutions should be organized and presented both with supporting information

and explanations of how they were achieved.

ADVANCED. Fourth-grade students performing at the advanced level should apply integrated procedural

knowledge and conceptual understanding to complex and nonroutine real-world problem solving in the five

NAEP content strands.

Specifically, fourth graders performing at the advanced level should be able to solve complex and

nonroutine real-world problems in all NAEP content strands. They should display mastery in the use of

four-function calculators, rulers, and geometric shapes. These students are expected to draw logical

conclusions and justify answers and solution processes by explaining why, as well as how, they were

achieved. They should go beyond the obvious in their interpretations and be able to communicate their

thoughts clearly and concisely.

GRADE 8

NAEP content strands: (1) number sense, properties, and operations; (2) measurement; (3)

geometry; (4) data analysis, statistics, and probability; (5) algebra and functions. Skills are cumulative

across all levelsfrom Basic to Proficient to Advanced.

BASIC. Eighth-grade students performing at the basic level should exhibit evidence of conceptual and

procedural understanding in the five NAEP content strands. This level of performance signifies an

understanding of arithmetic operationsincluding estimationon whole numbers, decimals, fractions, and

percents.

Eighth graders performing at the basic level should complete problems correctly with the help of

structural prompts such as diagrams, charts, and graphs. They should be able to solve problems in all NAEP

content strands through the appropriate selection and use of strategies and technological toolsincluding

calculators, computers, and geometric shapes. Students at this level also should be able to use fundamental

algebraic and informal geometric concepts in problem solving.

As they approach the proficient level, students at the basic level should be able to determine which

of available data are necessary and sufficient for correct solutions and use them in problem solving.

However, these eighth graders show limited skill in communicating mathematically.

(continued)
Figure F-1 (continued)

Final Description of 1992 Mathematics Achievement Levels

749

PROFICIENT. Eighth-grade students performing at the proficient level should apply mathematical

concepts and procedures consistently to complex problems in the five NAEP content strands.

They should be able to conjecture, defend their ideas, and give supporting examples. They should

understand the connections between fractions, percents, decimals, and other mathematical topics such as

algebra and functions. Students at this level are expected to have a thorough understanding of basic-level

arithmetic operationsan understanding sufficient for problem solving in practical situations.

Quantity and spatial relationships in problem solving and reasoning should be familiar to them, and

they should be able to convey underlying reasoning skills beyond the level of arithmetic. They should be

able to compare and contrast mathematical ideas and generate their own examples. These students should

make inferences from data and graphs; apply properties of informal geometry; and accurately use the tools

of technology. Students at this level should understand the process of gathering and organizing data and be

able to calculate, evaluate, and communicate results within the domain of statistics and probability.

ADVANCED. Eighth-grade students at the advanced level should be able to reach beyond the recognition,

identification, and application of mathematical rules in order to generalize and synthesize concepts and

principles in the five NAEP content strands.

They should be able to probe examples and counter examples in order to shape generalizations from

which they can develop models. Eighth graders performing at the advanced level should use number sense

and geometric awareness to consider the reasonableness of an answer. They are expected to use abstract

thinking to create unique problem-solving techniques and explain the reasoning processes underlying their

conclusions.

GRADE 12

NAEP content strands: (1) number sense, properties, and operations; (2) measurement; (3)

geometry; (4) data analysis, statistics, and probability; (5) algebra and functions. Skills are cumulative

across levelsfrom Basic to Proficient to Advanced.

BASIC. Twelfth-grade students at the basic level should demonstrate procedural and conceptual

knowledge in solving problems in the five NAEP content strands.

They should be able to use estimation to verify solutions and determine the reasonableness of

results as applied to real-world problems. They are expected to use algebraic and geometric reasoning

strategies to solve problems. Twelfth graders performing at the basic level should recognize relationships

presented in verbal, algebraic, tabular, and graphical forms; and demonstrate knowledge of geometric

relationships and corresponding measurement skills.

Twelfth graders at the basic level should be able to apply statistical reasoning in the organization

and display of data and in reading tables and graphs. They also should be able to generalize from patterns

and examples in the areas of algebra, geometry, and statistics. At this level, they should use correct

mathematical language and symbols to communicate mathematical relationships and reasoning processes;

and use calculators appropriately to solve problems.
Figure F-1 (continued)

Final Description of 1992 Mathematics Achievement Levels

750

PROFICIENT. Twelfth-grade students at the proficient level should consistently integrate mathematical

concepts and procedures to the solutions of more complex problems in the five NAEP content strands.

Twelfth graders performing at the proficient level should demonstrate an understanding of

algebraic, statistical, and geometric and spatial reasoning. They should be able to perform algebraic

operations involving polynomials; justify geometric relationships; and judge and defend the reasonableness

of answers as applied to real-world situations. These students should be able to analyze and interpret data in

tabular and graphical form; understand and use elements of the function concept in symbolic, graphical, and

tabular form; and make conjectures, defend ideas, and give supporting examples.

ADVANCED. Twelfth-grade students at the advanced level should consistently demonstrate the

integration of procedural and conceptual knowledge and the synthesis of ideas in the five NAEP content

strands.

They should understand the function concept; and be able to compare and apply the numeric,

algebraic, and graphical properties of functions. They should apply their knowledge of algebra, geometry,

and statistics to solve problems in more advanced areas of continuous and discrete mathematics.

Twelfth graders performing at the advanced level should be able to formulate generalizations and

create models through probing examples and counterexamples. They are expected to communicate their

mathematical reasoning through the clear, concise, and correct use of mathematical symbolism and logical

thinking.

Figure F-2

Draft Descriptions of the Achievement Levels

Prepared by the Original Level-Setting Panel

Fourth-Grade Draft Descriptions

BASIC. The Basic level signifies some evidence of conceptual and procedural understanding in the five

NAEP content strands of number sense, properties, and operations; measurement; geometry and spatial

sense; data analysis, statistics, and probability; and algebra and functions. Understanding simple facts and

single-step operations are included at this level, as is the ability to perform simple computations with whole

numbers. This level shows a partial mastery of estimation, basic fractions, and decimals relating to money or

the number line; it shows an ability to solve simple real-world problems involving measurement,

probability, statistics, and geometry. At this level, there is a partial mastery of tools such as four-function

calculators and manipulatives (geometric shapes and rulers). Written responses are often minimal, perhaps

with a partial response and lack of supportive information.

(continued)
Figure F-2 (continued)

Draft Descriptions of the Achievement Levels

Prepared by the Original Level-setting Panel

751

PROFICIENT. The Proficient level signifies consistent demonstration of the integration of procedural

knowledge and conceptual understanding as applied to problem solving in the five NAEP content strands of

number sense, properties, and operations; measurement; geometry and spatial sense; data analysis,

statistics, and probability; and algebra and functions. The Proficient level indicates an ability to perform

computation and estimation with whole numbers, to identify fractions, and to work with decimals involving

money or the number line. Solving real-world problems involving measurement, probability, statistics, and

geometry is an important part of this level. This level signifies the ability to use, as tools, four-function

calculators, rulers, and manipulatives (geometric shapes). It includes the ability to identify and use

pertinent/appropriate information in problem settings. The ability to make connections between and among

skills and concepts emerges at this level. Clear and organized written presentations, with supportive

information, are typical. And, there is an ability to explain how the solution was achieved.

ADVANCED. The Advanced level signifies the integration of procedural knowledge and conceptual

understanding as applied to problem solving in the five NAEP content strands of number sense, properties,

and operations; measurement; geometry and spatial sense; data analysis, statistics, and probability; and

algebra and functions. This is evidenced by divergent and elaborate written responses. The Advanced level

indicates an ability to solve multistep and nonroutine real-world problems involving measurement,

probability, statistics, and geometry, and an ability to perform complex tasks involving multiple steps and

variables. Tools are mastered, including four-function calculators, rulers, and manipulatives (geometric

shapes). This level signifies the ability to apply facts and procedures by explaining why as well as how.

Interpretations extend beyond obvious connections and thoughts are communicated clearly and concisely.

At this level, logical conclusions can be drawn and complete justifications can be provided for answers

and/or solution processes.

Eighth-Grade Draft Descriptions

BASIC. Basic students should begin to describe objects, to process accurately and elaborate relationships,

to compare and contrast, to find patterns, to reason from graphs, and to understand spatial reasoning. This

level of partial mastery signifies an understanding of arithmetic operations on whole numbers, decimals,

fractions, and percents, including estimation. Problems that are already set up are generally solved correctly,

as are one-step problems. However, problems involving the use of available data, and determinations of

what is necessary and sufficient to solve the problem, are generally quite difficult. Students should select

appropriate problem-solving tools, including calculators, computers, and manipulatives (geometric shapes)

to solve problems from the five content strands. Students should also be able to use elementary algebraic

concepts and elementary geometric concepts to solve problems. This level indicates familiarity with the

general characteristics of measurement. Students at this level may demonstrate limited ability to

communicate mathematical ideas.

(continued)
Figure F-2 (continued)

Draft Descriptions of the Achievement Levels

Prepared by the Original Level-setting Panel

752

PROFICIENT. Proficient students apply mathematical concepts consistently to more complex problems.

They should make conjectures, defend their ideas, and give supporting examples. They have developed the

ability to relate the connections between fractions, percents, and decimals, as well as other mathematical

topics. The Proficient level denotes a thorough understanding of the arithmetic operations listed at the Basic

level. This understanding is sufficient to permit applications to problem solving in practical situations.

Quantity and spatial relationships are familiar situations for problem solving and reasoning, and this level

signifies an ability to convey the underlying reasoning skills beyond the level of arithmetic. Ability to

compare and contrast mathematical ideas and generating examples is within the Proficient domain.

Proficient students can make inferences from data and graphs; they understand the process of gathering and

organizing data, calculating and evaluating within the domain of statistics and probability, and

communicating the results. The Proficient level includes the ability to apply the properties of elementary

geometry. Students at this level should accurately use the appropriate tools of technology.

ADVANCED. The Advanced level is characterized by the ability to go beyond recognition, identification,

and application of mathematical rules in order to generalize and synthesize concepts and principles.

Generalization often takes shape through probing examples and counterexamples and can be focused toward

creating models. Mathematical concepts and relationships are frequently communicated with mathematical

language, using symbolic representations where appropriate. Students at the Advanced level consider the

reasonableness of an answer, with both number sense and geometric awareness. Their abstract thinking

ability allows them to create unique problem-solving techniques and explain the reasoning processes they

followed in reaching a conclusion. These students can probe through examples and counterexamples that

allow generalization and description of assumptions with models and elegant mathematical language.

Twelfth-Grade Draft Descriptions

BASIC. This level represents understanding of fundamental algebraic operations with real numbers,

including the ability to solve two-step computational problems. It also signifies an understanding of

elementary geometrical concepts such as area, perimeter, and volume, and the ability to make measurements

of length, weight, capacity, and time. Also included at the Basic level is the ability to comprehend data in

both tabular and graphical form and to translate between verbal, algebraic, and graphical forms of linear

expression. Students at this level should be able to use a calculator appropriately.

PROFICIENT. This level represents mastery of fundamental algebraic operations and concepts with real

numbers, and an understanding of complex numbers. It also represents understanding of polynomials and

their graphs up to the second degree, including conic sections. The elements of plane, solid, and coordinate

geometry should be understood at the Proficient level. The Proficient level includes the ability to apply

concepts and formulas to problem solving. Students at this level should demonstrate critical thinking skills.

The Proficient level also represents the ability to judge the reasonableness of answers and the ability to

analyze and interpret data in both tabular and graphical form. Basic algebraic concepts, measurement, and

constructive geometry concepts are mastered at this level.

ADVANCED. The Advanced level represents mastery of trigonometric, exponential, logarithmic, and

composite functions, zeros and inverses of functions, polynomials of the third degree and higher, rational

functions, and graphs of all of these. In addition, the Advanced level represents mastery of topics in discrete

mathematics including matrices and determinants, sequences and series, and probability and statistics, as

well as topics in analytic geometry. The Advanced level also signifies the ability to successfully apply these

concepts to a variety of problem-solving situations.
753

Figure F-3

Revised Draft Descriptions of the Achievement Levels

Recommended by the Follow-Up Validation Panel

Revised Fourth-Grade Draft Descriptions

BASIC. Basic students exhibit some evidence of conceptual and procedure understanding in the

five NAEP content strands. At the fourth grade level, algebra and functions are treated in informal and

exploratory ways often through the study of patterns. Basic students estimate and use basic facts to perform

simple computations with whole numbers. These students show some understanding of fractions and

decimals. They solve simple real world problems in all areas. These students use, although not always

accurately, four-function calculators, rulers, and geometric shapes. Written responses are often minimal and

lack supporting information.

PROFICIENT. Proficient students consistently integrate procedural knowledge and conceptual

understanding as applied to problem solving in the five NAEP content strands. Using whole numbers they

estimate, compute, and determine whether their results are reasonable. They have a conceptual

understanding of fractions and decimals. Solving real world problems in all areas is important at this level.

Proficient students appropriately use four-function calculators, rulers and geometric shapes. These students

use problem solving strategies such as identifying and using appropriate information. [Problem-solving

strategies include identification and use of appropriate information.] They present organized written

solutions with supporting information and explain how they were achieved.

ADVANCED. Advanced students integrate procedural knowledge and conceptual understanding as

applied to problem solving in the five NAEP content strands. They solve complex and nonroutine real-world

problems in all areas. They have mastered the use of tools such as four-function calculators, rulers and

geometric shapes. Advanced students draw logical conclusions and justify answers and solution processes

by explaining the “why” as well as the “how.” Interpretations extend beyond obvious connections and

thoughts are communicated clearly and concisely.

Revised Eighth-Grade Draft Descriptions

BASIC. Basic students exhibit evidence of conceptual and procedural understanding. These

students compare and contrast, find patterns, reason from graphs, and understand spatial reasoning. This

level of performance signifies an understanding of arithmetic operations, including estimation, on whole

numbers, decimals, fractions, and percents. Students complete problems correctly with the help of structural

prompts such as diagrams, charts, and graphs. As students approach the proficient level, they will solve

problems involving the use of available data, and determine what is necessary and sufficient for a correct

solution. Students use problem solving strategies and select appropriate tools, including calculators,

computers, and manipulatives (geometric shapes) to solve problems from the five content strands. Students

use fundamental algebraic and informal geometric concepts to solve problems. Students at this level

demonstrate limited skills in communicating mathematically.

(continued)
Figure F-3 (continued)

Revised Draft Descriptions of the Achievement Levels

Recommended by the Follow-Up Validation Panel

754 (continued)

PROFICIENT. Proficient students apply mathematical concepts and procedures consistently to

complex problems. They make conjectures, defend their ideas, and give supporting examples. They have

developed the ability to relate the connections between fractions, percents, and decimals, as well as other

mathematical topics, such as algebra and functions. The proficient level denotes a thorough understanding

of the arithmetic operations listed at the basic level. This understanding is sufficient to permit applications

to problem solving in practical situations. Quantity and spatial relationships are familiar situations for

problem solving and reasoning, and students at this level convey the underlying reasoning skills beyond the

level of arithmetic. Proficient students compare and contrast mathematical ideas and generate their own

examples. These students make inferences from data and graphs; they understand the process of gathering

and organizing data, calculating, evaluating, and communicating the results within the domain of statistics

and probability. Proficient students apply the properties of informal geometry, and accurately use the

appropriate tools of technology.

ADVANCED. Advanced students go beyond recognition, identification, and application of

mathematical rules in order to generalize and synthesize concepts and principles. Generalization often takes

shape through probing examples and counter examples and can be used to create models. Mathematical

concepts and relationships are frequently communicated with mathematical language, using symbolic

representations where appropriate. Students at the advanced level consider the reasonableness of an answer,

with both number sense and geometric awareness. Their abstract thinking allows them to create unique

problem solving techniques and explain the reasoning processes they followed in reaching a conclusion.

These students probe examples and counter examples that allow generalization and description of

assumptions with models and elegant mathematical language.

Revised Twelfth-Grade Draft Descriptions

BASIC. Basic students demonstrate procedural and conceptual knowledge in solving problems in

the five NAEP content strands. They use estimation to verify solutions and determine the reasonableness of

the results to real world problems. Algebraic and geometric reasoning strategies are used to solve problems.

These students recognize relationships in verbal, algebraic, tabular, and graphical forms. Basic students

demonstrate knowledge of geometric relationships as well as corresponding measurement skills. Statistical

reasoning is applied to the organization and display of data and to reading tables and graphs. These students

generalize from patterns and examples in the areas of algebra, geometry, and statistics. They communicate

mathematical relationships and reasoning processes with correct mathematical language and symbolic

representations. Calculators are used appropriately to solve problems.

PROFICIENT. Proficient students integrate mathematical concepts and procedures consistently to

more complex problems in the five NAEP content strands. They demonstrate an understanding of algebraic

reasoning, geometric and spatial reasoning, and statistical reasoning as applied to other areas of

mathematics. They perform algebraic operations involving polynomials, justify geometric relationships, and

judge and defend the reasonableness of answers in real world situations. These students analyze and

interpret data in tabular and graphical form. Proficient students understand and use elements of the function

concept in symbolic, graphical and tabular form. They make conjectures, defend their ideas, and give

supporting examples.
Figure F-3 (continued)

Revised Draft Descriptions of the Achievement Levels

Recommended by the Follow-Up Validation Panel

(continued) 755

ADVANCED. Advanced students consistently demonstrate the integration of procedural and

conceptual knowledge, as well as the synthesis of ideas, in the five NAEP content strands. Advanced

students understand the function concept, and they compare and apply the numeric. algebraic, and graphical

properties of functions. They apply and connect their knowledge of algebra, geometry, and statistics to solve

problems in more advanced areas of continuous and discrete mathematics. Advanced students formulate

generalizations using examples and counter examples to create models. In communicating their

mathematical reasoning, these students demonstrate clear, concise, and correct use of mathematical

symbolism and logical thinking.

Figure F-4

Meeting Participants, NAEP Mathematics Achievement Level-Setting

Original Meeting, St. Louis, Missouri, March 20-24, 1992

Marge Blizard

Blizard Professional Cleaning

Franklin, Connecticut

Christopher Chomyak

The Episcopal Church

Calais, Maine

Janet Green

Met Life

Crownsville, Maryland

Mary Norman

Dekalb County Board of

Education

Decatur, Georgia

Janice Wamsley

Alcorn School System

Glen, Mississippi

Ronald Higgins

Walla Walla School District

Walla Walla, Washington

Leona Lee

Baltimore City Public Schools

Baltimore, Maryland

Lisa Bietau

USD383 Manhat. Pub Schools

Manhattan, Kansas

Marsha Davis

Alcorn County Schools

Corinth, Mississippi

Jean Bush Ragin

Patterson High School

Baltimore, Maryland

Bill Oldham

Harding University

Searcy, Arkansas

George Shell

Retired Principal

Draper, Utah

Marsha Stovey

Detroit Public Schools

Detroit, Michigan

Vance Morris

Dekalb County Board of

Education

Atlanta, Georgia

Carol Ballentine

Duval County Schools

Jacksonville, Florida

Tami Harvey, ESD

Audiometric Technician

Burns, Oregon

Laurence Payne

Greater Houston Coalition

 for Educational Excellence

Houston, Texas

Cheryl Yunk

USD 383

Manhattan, Kansas

Kirby Gchachu

Zuni Public School District

Zuni, New Mexico

Corliss Hubert

Rutherford Board of Education

Englewood, New Jersey

Joyce Dunn

Alcorn County Schools

Corinth, Mississippi
756 (continued)

Figure F-4 (continued)

Meeting Participants, NAEP Mathematics Achievement Level-Setting

Original Meeting, St. Louis, Missouri, March 20-24, 1992

Gloria Moran

Williams Junior High

Bridgewater, Massachusetts

Charles Jackson

Blairsville, Pennsylvania

Cassandra Turner

Internal Revenue Service

Miami, Florida

Jack Deal

Bethel Park School District

Pittsburgh, Pennsylvania

Ninfa Rivera

Lyford CISD

Raymondville, Texas

Gerald Zeringue

Garrity Construction Company

Harvey, Louisiana

Linda Brown

Van Zile Elementary School

Detroit, Michigan

Judy Bibb

Lonoke High School

Cabot, Arkansas

David Rank

School District of Greenville

Greenville, South Carolina

John Sweeney

Freed-Haideman University

Henderson, Tennessee

Nancy Pejouhy

Woodstock Union High School

Woodstock, Vermont

Jim Trefzger

Parkland College

Champaign, Illinois

Joanne Greaver

Jefferson Coun. Public Schools

Louisville, Kentucky

Ellie Cucinatto

Bridgewater Public Schools

Bridgewater, Massachusetts

Lillie Carr

Pender County Schools

Teachey, North Carolina

Eric Cain

IBM

Metairie, Louisiana

Phillip Stroup

Butler County MR/DD

Seven Mile, Ohio

Mike Gobel

Walla Walla School District

Walla Walla, Washington

Juanita Tietze

Retired Principal

Canton, Ohio

Norma Newman

Ysleta Independent School

District

El Paso, Texas

William Rickenbach

Bethel Park School District

Bethel Park, Pennsylvania

Violet Cosgrove

Retired

Glen Burnie, Maryland

Danny McDougal

Pre-Mc, Inc.

Allen, Oklahoma

Bill Anderson

Administration Eagle Union

Zionsville, Indiana

Dan Thompson

Thompson Construction

Company

Trinidad, Colorado

Nancy Gallagher

West Penn Power Company

Kittanning, Pennsylvania

William Hawes

The Hawes Company

Tucker, Georgia

Zhining Qin

Minnesota Department of

Education

St. Paul, Minnesota

Charles McGee

Greenville County School

District

Greenville, South Carolina

Barbara Bayne

Greenville County Schools

District

Greenville, South Carolina

Landa McLaurin

Baltimore City Schools

Baltimore, Maryland

Bill Cramer, Jr.

Cramer & Mallon,

Attorneys at Law

Burns, Oregon

Nancy Potempa

St. Xavier University

Mokena, Illinois

Florencetine Jasmin

Baltimore City Public Schools

Baltimore, Maryland
757

Figure F-4 (continued)

Meeting Participants, NAEP Mathematics Achievement Level-Setting

Original Meeting, St. Louis, Missouri, March 20-24, 1992

Florence Kelly

Manville Board of Education

Manville, New Jersey

Philip Brach

University of the District of

Columbia

Washington, D.C.

Larry Brown

Oil industry (Self-Employed)

Allen, Oklahoma

W. Garry Quast

Slippery Rock University

Slippery Rock, Pennsylvania

Carl Springfels

Consultant (Self-Employed)

Miami Shores, Florida

Anna Maria Golan

Santa Ana Unified

Fountain Valley, California

Ricardo Suarez

Lyford CISD

Raymondville, Texas
758

Figure F-5

Meeting Participants, NAEP Mathematics Achievement Level-Setting

Follow-up Validation Meeting, Nantucket, Massachusetts, July 17-19, 1992

Charles Allen

Michigan Department of Education

Lansing, Michigan

Linda Brown

Van Zile Elementary School

Clinton Township, Michigan

Ellie Cucinatto

Bridgewater Public Schools

Bridgewater, Massachusetts

Jack Deal

Bethel Park School District

Pittsburgh, Pennsylvania

Paula Duckett

River Terrace Community School Board

Washington, DC

Edward Esty

SRI International

Washington, D.C.

Barbara Faltz-Jackson

Baltimore Public Schools

Baltimore, Maryland

Joan Ferini-Mundy

University of New Hampshire

Durham, New Hampshire

Marilyn Hala

National Council of Teachers of Mathematics

Washington, D.C.

Florence Kelly

Largo Public Schools

Largo, Florida

Henry Kepner

University of Wisconsin at Milwaukee

Milwaukee, Wisconsin

Charles McGee

Greenville Public Schools

Greenville, South Carolina

Landa McLaurin

Baltimore City Schools

Baltimore, Maryland

Gloria Moran

Williams Junior High School

Bridgewater, Massachusetts

Jo Ann Mosier

Kentucky Department of Education

Frankfort, Kentucky

Mary Norman

DeKalb County Board of Education

Decatur, Georgia

David Rank

Greenville Public Schools

Greenville, South Carolina

Sharon Steglein

Minnesota Department of Education

St. Paul, Minnesota
759

Appendix G

REPORT ON DEVELOPING ACHIEVEMENT LEVEL DESCRIPTIONS

FOR THE 1996 NAEP SCIENCE ASSESSMENT

Mary Lyn Bourque

National Assessment Governing Board

Background

Principle 4 of the Board’s policy states that the Board shall exercise its policy judgment in setting

the levels. That is, the Board’s statutory authority requires that it be the final judge in developing the

achievement levels. In so doing, the Board uses a national consensus process, seeking advice from a broad

audience. The preliminary descriptions, for example, are widely circulated as part of the assessment

framework documents before adoption by the Board. Similarly, in designing the level setting process the

Board solicits broad input from relevant professional organizations, the technical community, and others. 

And after the proposed levels are recommended, they are generally widely circulated for comment and

reflection by stakeholder groups, policymakers, content specialists, and others. 

The Board’s current decision in science considered several evaluative features of the student

performance standards, including the reasonableness of the levels, whether they are valid, and whether they

are likely to be useful to the public in interpreting the NAEP data. These evaluation criteria are part of the

authorizing legislation, and are among the criteria against which the National Academy of Sciences will

judge their merit.

Before adopting achievement levels in a particular subject area, the Board examines relevant extant

data from other sources to inform the decision and to place it in a broader context. In the case of the 1996

science, the Board examined other data sources, including Advanced Placement (AP) science data for the

grade 12 cohort; Third International Mathematics and Science Study (TIMSS) data for grade 8; and

performance standards already set by the Board in reading, mathematics, U.S. history, and geography. They

also examined the individual ratings of panelists who participated in the ACT level setting process.

In the final analysis, the Board adopted cut scores at points on the scale they judged to be

reasonable, taking other available information into account. The interim cut points on the NAEP scale

varied from those generated through the level setting process in non-systematic ways: some were higher

than the recommendations; some were lower. Consequently, the relationships between the interim cut

scores and the content descriptions was not clear.

Therefore, the Board determined that a further examination of the descriptions was necessary, either

to modify those that existed, or to craft new ones that would be more consistent with the policy definitions,

the framework, and the item pool used to measure the 1996 NAEP science performance. In this way, the

Board could have some assurance that the descriptions would be providing a proper interpretation of the

interim cut scores of what students know and can do at the various achievement levels. This report provides

a summary of the process used to develop the descriptions of the levels, as well as the results of this

initiative from a meeting convened in St. Louis, Missouri from June 19 through June 22, 1997.
760

Panel Selection

The plan called for convening about 24 content experts who have knowledge of what students in the

respective grades can do and are capable of doing in elementary, middle school, and secondary school

science. Generally teachers, curriculum specialists, and science educators are in the best position to make

these kinds of judgments. The Board-approved plan also called for some number of noneducators, though a

proportionally smaller number than is typically used in the level setting process. The Advisory Committee

on Education Statistics (ACES) who provides technical advice to the Board, recommended that the grade

level groups include at least 5 panelists (rather than 4 as was suggested in the original plan).

A combination of two sources of experts was used to identify participants: (1) states who had

participated in the 1996 NAEP state assessments; and (2) an American College Testing (ACT) sample

previously identified for the various panels which ACT assembles in conducting its work for the Board

under the achievement level setting contract. Thirty-two state assessment directors were contacted and

encouraged to submit the name of one person who would represent their state at the meeting. Twenty-three

states responded by electing to send a participant, with one state (MA) electing to send two, and another

state (TX) sending three representatives. Participating jurisdictions are shown in Figure G-1.

Figure G-1

Participating Jurisdictions in Developing Achievement Level

Descriptions for the 1996 NAEP Science Assessment

Alabama Georgia Montana Tennessee

Alaska Hawaii Nebraska Texas

Arizona Louisiana Nevada Utah

Connecticut Maine North Dakota Virginia

Delaware Massachusetts New York Washington

District of Columbia Minnesota Rhode Island

Because some states in the initial sample were not able to send a representative, the multiple

participants from a single state were honored. In addition, three noneducator panelists participated, taken

from the sample of nine nominees supplied by ACT. The total number of panelists was 28. Participants

were given the opportunity to select the grade level to which they wished to be assigned in a pre-registration

interview. They were given their first choice in all cases, and randomly assigned to one of the two parallel

groups (either Group A or Group B), ensuring that male participants and noneducators were assigned

throughout the groups (since there were fewer participants in both categories).

Study Design

The plan approved by the Board called for two independent panels to conduct this work. Each

panel was composed of 14 persons, assigned to grade level groups of 4 or 5 persons per grade. All

participants were provided the same training and were asked to complete the same tasks. The design of two

independent panels was for purposes of embedding a validity check into the process, which, if successful,

would provide a built-in replication study. Panelists were asked in the opening plenary session to honor the

independent groups design and to refrain from discussing their work with members of the other group until

it was appropriate to do so, on the last day.
761

The plan was implemented as a multi-step process, with individual judgments being the focus of

Day 2, within-grade consensus the focus of Day 3, and cross-grade, cross-level consistency being the focus

of Day 4. Further, it was made clear to the panelists that this was an empirically-based design insofar as

data based on actual student performances was the basis for developing the descriptions of what students

know and can do at each level.

Materials

Prior to the meeting panelists were sent copies of the 1996 NAEP science assessment framework. 

They referred to this regularly during the course of their work. In addition, on Day 2 they were given the

item anchor book from which they would do all their work. The items in a given grade level appeared in the

anchor book in the order in which they map to the NAEP scale, from easiest to hardest. In addition,

panelists were given an index listing of the items in the same order as the anchor book, copies of the policy

definitions, and forms on which they could submit the work of Day 2 and Day 3.

Training

Panelists were trained simultaneously to insure standardized exposure to all relevant information

needed to accomplish the task. Since all panelists were not equally familiar with NAEP and NAGB, a brief

introduction was provided covering such topics as the scope of the framework, the nature of the item pool, a

non-technical summary of NAEP scaling and item mapping, and instructions on how to read the information

provided in the anchor binders containing all the items, from easiest to hardest.

During the first plenary session, panelists were given an overview of NAEP and NAGB, and had

the tasks of the next three days presented in summary form. The training focused on giving them a level of

comfort with the tasks they were going to be doing, introducing the scope of the 1996 NAEP science

assessment, with a full explanation of the content of the assessment, the vocabulary peculiar to NAEP, e.g.,

"blocks of exercises," and the chronology of the 1996 NAEP assessment. It was important for panelists to

understand where they were coming into the process, that the assessment had already been administered,

scored, scaled, and reported. The achievement levels remained the only unfinished aspect of the science

assessment.

On Day 2 the agenda called for a plenary session to provide specific training in the task of making

independent judgments about the content represented by the items at the grade level. Though not described

in these terms, this activity was designed to provide panelists with first-hand knowledge of the contents of

the NAEP assessments, the kinds of items used, the manner of scoring the items, and the performance of

students on average, at the borderline of Basic, Proficient, and Advanced, and across the levels. The time

spent in reviewing each item without the assistance of other panelists forced each participant to deal with

the notion of item difficulty without being influenced by others, and with the idea of operationalizing the

Board’s policy definitions. The real goal of Day 2, though not formally stated to panelists, was to train them

in the item pool. Requiring each panelist to make an individual judgment about each item was more likely

to have them read and try to understand the cognitive demand of the items than would the simpler task of

reading the items thoughtfully.

Panelists were directed to start with the items in the Basic level, those items falling between the

lower and upper borderline for Basic (according to the interim levels adopted by the Board), and generate

working lists of general statements representing the content of the assessment. They could generate these

lists by clustering similar content together and developing generalized statements which summarized that

content. When they had completed this part of the task, they were asked to compare the results with the
762

policy definition for Basic, i.e., partial mastery of the prerequisite knowledge and skill fundamental for

proficient work. The consistency check between the descriptions and the definitions was encouraged on an

holistic basis, not an item-by-item basis. The question to be answered was, "Does this description, in

general, reflect the policy definition?", not "Does this item, and this item, etc., reflect the policy

definition?" If the panelist was satisfied with the general consistency between the two, they could move on

to the next level, Proficient, and repeat the process; then to Advanced, and repeat the process a third time. 

The product for Day 2 was a working list of statements for each level from each participant. The grade 4

panelists reviewed 226 items in this process; grades 8 and 12 reviewed 312 items each.

During the training panelists were given suggestions for verbs that could be used to indicate

increasing knowledge and skills from one level to the next, as well as common phrases that are also

indicators of hierarchical skill levels and increasing levels of sophistication in the knowledge domain. 

These linguistic features had been used in the past in other anchoring settings and were judged to be useful

in this one as well. Panelists were not required to use these words, but most found them helpful in

completing the task.

Panelists were also trained in how to "read" the data accompanying each item. The scale value for

the item using a response probability (rp) criterion of .65 was explained, as well as the differences between

the "borderline" p-values and the "across the range" p-values. The training also focused on the differences

between a constructed response coded [2] or partial, and those coded [3] or [4], complete.

Finally, panelists were led through a set of about 10 items as a model for the task they were about to

complete, demonstrating how to use the data, judge the difficulty of the items, and write statements using

the linguistic suggestions.

Day 3 was designed to have the grade level panels confer and reach group agreement on the initial

descriptions for Basic, Proficient, and Advanced at grades 4, 8, and 12. As on the previous day, the plenary

session in the morning was focused on the groups’ task. They were instructed to start by reading each

other’s lists from the work of Day 2. They were then to select those statements from the lists for which there

was general group agreement, and to return to the remaining statements where there was lack of agreement

and work them through to inclusion or exclusion, in the end creating a group master working list for Basic. 

They needed to ensure that in general the master list was consistent with the policy definition for Basic,

before they could move on to the Proficient or Advanced levels. Each group worked between the individual

lists created the previous day and flip charts of the master lists. For tracking purposes, groups were

encouraged to list the items that were the underpinnings for their statements, thus creating a "paper trail", as

it were, for the statements in the lists. The product for Day 3 was a working list for the three levels for each

grade level group.

Day 4 training was focused on achieving clarity in the statements, examining consistency across

levels, across grades, and with the policy definitions. Panelists were organized conference style which

supported group discussion and consensus-building. Panelists were asked to track the general statements

found to be inconsistent, as well as the associated items, so that a "paper trail" existed for the content judged

to be irregular or inappropriately placed at a particular level. Such anomalies would be handled at the end

of the work session if necessary.

Results

The results of the initiative can be found in Figures G-2, G-3, and G-4 below. These are the

narrative versions of the results, which have been professionally edited and returned to the panelists for their

concurrence. Table G-1 shows the percentages of panelists who approved the narratives.
763

Figure G-2

1996 NAEP Science Achievement Level Descriptions

Grade 4

Cut Score Content Descriptions*

Basic

138

Students performing at the Basic level demonstrate some of the knowledge and reasoning required for understanding of

the earth, physical, and life sciences at a level appropriate to Grade 4. For example, they can carry out simple

investigations and read uncomplicated graphs and diagrams. Students at this level also show a beginning understanding of

classification, simple relationships, and energy.

Fourth-grade students performing at the Basic level are able to follow simple procedures, manipulate simple materials,

make observations, and record data. They are able to read simple graphs and diagrams and draw reasonable but limited

conclusions based on data provided to them. These students can recognize appropriate experimental designs, although they

are unable to justify their decisions.

When presented with diagrams, students at this level can identify seasons; distinguish between day and night; and place

the position of the Earth, sun, and planets. They are able to recognize major energy sources and simple energy changes. In

addition, they show an understanding of the relationship between sound and vibrations. These students are able to identify

organisms by physical characteristics and group organisms with similar physical features. They can also describe simple

relationships among structure, function, habitat, life cycles, and different organisms.

Proficient

170

Students performing at the Proficient level demonstrate the knowledge and reasoning required for understanding of the

earth, physical, and life sciences at a level appropriate to Grade 4. For example, they understand concepts relating to the

Earth’s features, physical properties, and structure and function. In addition, student can formulate solutions to familiar

problems as well as show a beginning awareness of issues associated with technology.

Fourth-grade students performing at the Proficient level are able to provide an explanation of day and night when given a

diagram. They can recognize major features of the Earth’s surface and the impact of natural forces. They are also able to

recognize water in its various forms in the water cycle and can suggest ways to conserve it. These students recognize that

various materials possess different properties that make them useful. Students at this level are able to explain how

structure and function help living things survive. They have a beginning awareness of the benefits and challenges

associated with technology and recognize some human effects on the environment. They can also make straightforward

predictions and justify their position.

Advanced

204

Students performing at the Advanced level demonstrate a solid understanding of the earth, physical, and life sciences as

well as the ability to apply their understanding to practical situations at a level appropriate to Grade 4. For example, they

can perform and critique simple investigations, make connections from one or more of the sciences to predict or conclude,

and apply fundamental concepts to practical applications.

Fourth-grade students performing at the Advanced level are able to combine information, data, and knowledge from one

or more of the sciences to reach a conclusion or to make a valid prediction. They can also recognize, design, and explain

simple experimental procedures.

Students at this level recognize nonrenewable sources of energy. They also recognize that light and sound travel at

different speeds. These students understand some principles of ecology and are able to compare and contrast life cycles of

various common organisms. In addition, they have a developmental awareness of the benefits and challenges associated

with technology.

*Shaded areas indicate summary of content descriptions.
764

Figure G-3

1996 NAEP Science Achievement Level Descriptions

Grade 8

Cut Score Content Descriptions

Basic

143

Students performing at the Basic level demonstrate some of the knowledge and reasoning required for understanding of

the earth, physical, and life sciences at a level appropriate to Grade 8. For example, they can carry out investigations and

obtain information from graphs, diagrams, and tables. In addition, they demonstrate some understanding of concepts

relating to the solar system and relative motion. Students at this level also have a beginning understanding of cause-andeffect relationships.

Eighth-grade students performing at the Basic level are able to observe, measure, collect, record, and compute data from

investigations. They can read simple graphs and tables and are able to make simple data comparisons. These students are

able to follow directions and use basic science equipment to perform simple experiments. In addition, they have an

emerging ability to design experiments.

Students at this level have some awareness of causal relationships. They recognize the position of planets and their

movement around the sun and know basic weather-related phenomena. These students can explain changes in position and

motion such as the movement of a truck in relation to that of a car. They also have an emerging understanding of the

interrelationships among plants, animals, and the environment.

Proficient

170

Students performing at the Proficient level demonstrate much of the knowledge and many of the reasoning abilities

essential for understanding of the earth, physical, and life sciences at a level appropriate to Grade 8. For example, students

can interpret graphic information, design simple investigations, and explain such scientific concepts as energy transfer.

Students at this level also show an awareness of environmental issues, especially those addressing energy and pollution..

Eighth-grade students performing at the Proficient level are able to create, interpret, and make predictions from charts,

diagrams, and graphs based on information provided to them or from their own investigations. They have the ability to

design an experiment and have an emerging understanding of variables and controls. These students are able to read and

interpret geographic and topographic maps. In addition, they have an emerging ability to use and understand models, can

partially formulate explanations of their understanding of scientific phenomena, and can design plans to solve problems.

Students at this level can begin to identify forms of energy and describe the role of energy transformations in living and 

nonliving systems. They have knowledge of organization, gravity, and motions within the solar system and can identify

some factors that shape the surface of the Earth. These students have some understanding of properties of materials and

have an emerging understanding of the particulate nature of matter, especially the effect of temperature on states of matter.

They also know that light and sound travel at different speeds and can apply their knowledge of force, speed, and motion.

These students demonstrate a developmental understanding of the flow of energy from the sun through living systems,

especially plants. They know that organisms reproduce and that characteristics are inherited from previous generations.

These students also understand that organisms are made up of cells and that cells have subcomponents with different

functions. In addition, they are able to develop their own classification system based on physical characteristics. These

students can list some effects of air and water pollution as well as demonstrate knowledge of the advantages

Advanced

207

Students performing at the Advanced level demonstrate a solid understanding of the earth, physical, and life sciences as

well as the abilities required to apply their understanding in practical situations at a level appropriate to Grade 8. For

example, students perform and critique the design of investigations, relate scientific concepts to each other, explain their

reasoning, and discuss the impact of human activities on the environment.

Eighth-grade students performing at the Advanced level are able to provide an explanation for scientific results. They have

a modest understanding of scale and are able to design a controlled experiment. These students have an understanding of

models as representations of natural systems and can describe energy transfer in living and nonliving systems.

Students at this level are able to understand that present physical clues, including fossils and geological formations, are

indications that the Earth has not always been the same and that the present is a key to understanding the past. They have a

solid knowledge of forces and motions within the solar system and an emerging understanding of atmospheric pressure.

These students can recognize a wide range of physical and chemical properties of matter and some of their interactions and

understand some of the properties of light and sound. Also, they can infer relationships between structure and function.

These students know the differences between plant and animal cells and can apply their knowledge of food as a source of

energy to a practical situation. In addition, they are able to explain the impact of human activities on the environment and

the economy.

*Shaded areas indicate summary of content descriptions.
765

Figure G-4

1996 NAEP Science Achievement Level Descriptions

Grade 12

Cut Score Content Descriptions*

Basic

145

Students performing at the Basic level demonstrate some knowledge and certain reasoning abilities required for

understanding of the earth, physical, and life sciences at a level appropriate to Grade 12. In addition, they demonstrate

knowledge of the themes of science (models, systems, patterns of change) required for understanding the most basic

relationships among the earth, physical, and life sciences. They are able to conduct investigations, critique the design of

investigations, and demonstrate a rudimentary understanding of scientific principles.

Twelfth-grade students performing at the Basic level are able to select and use appropriate simple laboratory equipment

and write down simple procedures that others can follow. They also have a developmental ability to design complex

experiments. These students are able to make classifications based on definitions such as physical properties and

characteristics.

Students at this level demonstrate a rudimentary understanding of basic models and can identify some parts of physical

and biological systems. They are also able to identify some patterns in nature and rates of change over time. These students

have the ability to identify basic scientific facts and terminology and have a rudimentary understanding of the scientific

principles underlying such phenomena as volcanic activity, disease transmission, and energy transformation. In addition,

they have some familiarity with the application of technology.

Proficient

178

Students performing at the Proficient level demonstrate the knowledge and reasoning abilities required for understanding

of the earth, physical, and life sciences at a level appropriate to Grade 12. In addition, they demonstrate knowledge of the

themes of science (models, systems, patterns of change) required for understanding how these themes illustrate essential

relationships among the earth, physical, and life sciences. They are able to analyze data and apply scientific principles to

everyday situations.

Twelfth-grade students performing at the Proficient level are able to demonstrate a working ability to design and conduct

scientific investigations. They are able to analyze data in various forms and utilize information to provide explanations and

to draw reasonable conclusions.

Students at this level have a developmental understanding of both physical and conceptual models and are able to compare

various models. They recognize some inputs and outputs, causes and effects, and interactions of a system. In addition, they

can correlate structure to function for the parts of a system that they can identify. These students also recognize that rate of

change depends on initial conditions and other factors. They are able to apply scientific concepts and principles to

practical applications and solutions for problems in the real world and show a developmental understanding of technology,

its uses, and its applications.

Advanced

210

Students performing at the Advanced level demonstrate the knowledge and reasoning abilities required for a solid

understanding of the earth, physical, and life sciences at a level appropriate to Grade 12. In addition, they demonstrate

knowledge of the themes of science (models, systems, patterns of change) required for integrating knowledge and

understanding of scientific principles from the earth, physical, and life sciences. Students can design investigations that

answer questions about real-world situations and use their reasoning abilities to make predications.

Twelfth-grade students performing at the Advanced level are able to design scientific investigations to solve complex,

real-world situations. They can integrate, interpolate, and extrapolate information embedded in data to draw wellformulated explanations and conclusions. They are also able to use complex reasoning skills to apply scientific knowledge

to make predictions based on conditions, variables, and interactions.

Students at this level recognize the inherent strengths and limitations of models and can revise models based on additional

information. They are able to recognize cause-and-effect relationships within systems and can utilize this knowledge to

make reasonable predictions of future events. These students are able to recognize that patterns can be constant,

exponential, or irregular and can apply this recognition to make predictions. They can also design a technological solution

for a given problem.

*Shaded areas indicate summary of content descriptions.
766

Table G-1

Percentage1

 of Panelists Approving

the Edited Versions of the Descriptions

(Narrative Format)

Grade 4 Grade 8 Grade 12

Group A 100% 100% 67%

Group B 100 100 100

Total 100 100 80

The descriptions have been reviewed by the three science specialists who facilitated the meeting. In

general, there are few differences to be noted between the results for Group A and Group B, though,

according to one reviewer, Group B’s statements tend to be more general than Group A’s. Even though the

descriptions were produced on-site, without benefit of time to be very selective in language and style, and

the opportunity to conduct careful checks across grades and levels, they are remarkably consistent one to

another. However, the consensus among reviewers is that Group A’s descriptions are probably more

consistent both within and across grades than Group B’s, as well as more complete in terms of representing

the items in the pool than Group B’s.

Evaluation

Panelists were asked to complete an evaluation questionnaire at the end of the process. The

purpose of the evaluation was to accomplish two goals: (1) provide feedback regarding the panelists’

opinion about the consistency of the achievement levels descriptions across levels and grades, as well as

between the descriptions and the policy definitions; and (2) provide feedback on the interim cut scores and

the percentage of students at or above the levels. The questions covering the latter were adapted from an

evaluation questionnaire used by ACT in conducting the achievement level setting process. It should be

noted that the Board-approved plan did not allow for providing consequences data to the panelists during

the working sessions. However, consequences data were presented on the evaluation questionnaire during

the debriefing segment of the meeting when it could in no way influence the results.

The first eight questions dealt with consistency. Panelists provided a rating from [1] not at all

consistent, to [5] very consistent. In examining differences between the two parallel groups, [A] and [B],

Group [B] reports slightly more consistency than does Group [A]. Within Group [A], the grade 12 group

reports less consistency in all comparisons than either grades 4 or 8. 

The questionnaire also asked a series of questions which were adapted from the ACT

questionnaires regarding the cut scores adopted on an interim basis, the percentage of students at or above

the levels, and the achievement levels descriptions. Additionally, the questionnaire gave panelists the

opportunity to recommend no changes in the cut scores and thus leaving the percentages of students who

score at or above the levels as presented, or to recommend smaller or larger percentages of students who

 

1

 Percentages based on the total number of respondents, not necessarily the total number of panelists.
767

should score at or above the levels and consequently changing the cut scores. Opinion was split evenly

among those who would not change the cut scores and those who would either raise or lower the cut scores.

The final question asked panelists to take a position regarding the recommendations they wished to

make to the National Assessment Governing Board regarding the achievement level descriptions. The

majority of panelists (between 64% and 71%) recommended that the achievement levels adopted by the

Board on an interim basis, and which were used to develop the achievement levels descriptions, be reported.

Summary and Recommendations

The Board’s plan called for describing the performance of students within the ranges on the NAEP

scale for the achievement levels adopted by the Board on an interim basis. This report provides the results

of the Board’s plan as carried out by this process. These descriptions represent what students know and can

do within the achievement level ranges of Basic, Proficient, and Advanced.

It is recommended that, should the Board adopt the interim levels on a permanent basis for the 1996

NAEP science achievement levels, then these descriptions should be adopted to assist in the proper

interpretation of the NAEP scores. It is also recommended that in the Board’s report on the levels, the

differences between these descriptions on the 1996 NAEP science assessment of what students know and

can do and achievement levels descriptions in other NAEP content areas of what students should know and

be able to do, needs to be made clear to the readers. Additionally, it is recommended that the exemplar

items (in a separate document) which are typical of student performances for each level accompany the cut

scores and descriptions to assist in the proper interpretation of the NAEP scores.

Finally, a word about the Group A and Group B descriptions. The purpose of the parallel working

groups was to provide the Board with some evidence that the resulting descriptions were not idiosyncratic to

a particular set of panelists. The evidence points to the fact that either set of descriptions could be used in

the report. However, since the Group A descriptions seem to be slightly more internally consistent, it is

recommended that Group A’s become the official set, and Group B’s be viewed as the validation set, and not

used to report the achievement levels results.
768
769

Appendix H

THE INFORMATION WEIGHTING ERROR

Susan C. Loomis, Luz Bay, and Wen-Hung Chen

American College Testing

The Error

In the process of recomputing the reading cutscores set in 1992 for the three achievement

levels, an error in the information weighting function was detected. The error affected data for all

achievement levels set in 1992: reading and mathematics. The Muraki information weighting

function published in 1993 was used in the 1994 programs to compute achievement levels, so

only 1992 levels are affected.

The procedures used for 1992 were printed and reported in numerous places. No one had

detected an error. The psychometrician who developed the programs for the 1994 process used

Muraki’s information weighting function because he found it to be more straightforward than the

1992 procedure.

The 1992 equation1

 is as follows:

j

2 2

j

c=1

m

I ( ) = D a

j

θ ∑.

The 1994 equation is as follows:

j

2 2

j

c=1

m

2 I c j jc ( ) = D a [T -T ( )] P ( ) ,

j

θ ∑ θ θ

where T is the expected score for item j or proficiency θ

j

c=1

m

T c jc ( ) = T P ( ) ,

j

θ θ ∑

and Tc is the score assigned to the response category c.

 

1

 Notations for this equation were modified to correspond to those of Equation 2. The reader will need to refer to the

articles (Luecht, 1993, and Muraki, 1993) for a complete explanation of the equations.
770

Analysis of the Error: Magnitude

The differences in achievement levels reported for 1992 and the corrected achievement

levels are due both to the error in item parameters and to the error in information weights. The

cutscores and percentages of students scoring at or above each for each achievement level are

reported in Tables H-1 and H-2. Data in Table H-1 are the previously reported (incorrect) data,

and data in Table H-2 are the corrected data.

Table H-1

Mathematics Cutpoints and Percents At or Above as Reported

Grade Basic Proficient Advanced

4 Cutpoint

%92 Dist

211

 61

248

 18

280

 2

8 Cutpoint

%92 Dist

256

 63

294

 25

331

 4

12 Cutpoint

%92 Dist

287

 64

334

 16

366

 2

Table H-2

Corrected Mathematics Cutpoints and Percents At or Above as Reported

Grade Basic Proficient Advanced

4 Cutpoint

%92 Dist

214

 59

249

 18

282

 2

8 Cutpoint

%92 Dist

262

 58

299

 21

333

 3

12 Cutpoint

%92 Dist

288

 64

336

 15

367

 2

The corrected cutscores are consistently the same or higher than those previously

reported. The maximum difference in cutscores originally reported and the corrected cutscores is

found for grade 8 at the Basic level:

((original cutscore = 256) - (corrected cutscore = 262)) = -6 points.

The differences attributable to each error (parameter estimates and information weighting)

appear to be rather small in most cases.

Table H-3 reports the differences in cutscores due to the two errors, examined one at a

time. Relative to the correct data, the information weighting error generally resulted in a lower

composite cutscore, and the recoding error resulting in incorrect item parameters generally

resulted in an even lower composite cutscore.
771

Table H-3

Composite NAEP Scale Cutpoint Differences in Mathematics Due to Errors

Achievement Level Cutpoint Information Weighting1 Item Parameters2

Grade 4

Basic

Proficient

Advanced

-2

-1

-1

-5

-1

-1

Grade 8

Basic

Proficient

Advanced

-4

-3

-1

-8

-2

-2

Grade 12

Basic

Proficient

Advanced

0

-1

-1

-4

-2

-2

1

 Difference = Incorrect - Correct, based on correct item parameters. If the recoding of data had been correct, the

cutpoints would have been in error by these amounts, due to the incorrect information weighting function.

2

 Difference = Incorrect - Correct, based on correct information weights. If the correct information weighting function

had been used, the cutpoints would have been in error by these amounts due to the recoding error resulting in incorrect

item parameters.

Table H-4 shows comparisons of percentages of students who scored at or above each

achievement level in 1992. The center row for each grade shows student performance relative to

each achievement level in 1992 using both correct item parameters and correct information

weights. The first row for each grade shows student performance relative to achievement levels

computed with the correct item parameters and incorrect information weights. The third row for

each grade shows student performance relative to achievement levels computed with the

incorrect item parameters and the correct information weights.
772

Table H-4

NAEP Mathematics Achievement Levels:

Cutpoints and 1992 Distribution Data

GRADE 4

Cutpoint Percent At or Above Cutpoint

Basic Proficient Advanced Basic Proficient Advanced

Correct Data, Incorrect Weight 212 248 281 62.2 19.7 2.1

Correct Data, Correct Weight 214 249 282 60 18.7 1.9

Incorrect Data, Correct Weight 209 248 281 65.6 19.7 2.1

GRADE 8

Cutpoint Percent At or Above Cutpoint

Basic Proficient Advanced Basic Proficient Advanced

Correct Data, Incorrect Weight 258 296 332 62.6 24.4 3.6

Correct Data, Correct Weight 262 299 333 58.6 21.8 3.3

Incorrect Data, Correct Weight 254 297 331 66.5 23.5 3.8

GRADE 12

Cutpoint Percent At or Above Cutpoint

Basic Proficient Advanced Basic Proficient Advanced

Correct Data, Incorrect Weight 288 335 366 64.8 16.1 1.9

Correct Data, Correct Weight 288 336 367 64.8 15.4 1.8

Incorrect Data, Correct Weight 284 334 364 68.7 16.9 2.3

Analysis of the Error: The Information Weighting Functions

Various analyses were conducted to determine what, if any, general conclusions could be

drawn to help inform users of NAEP achievement levels data about the factors related to

differences in cutscores due to the information weighting error.

Item ratings are collected from two groups of panelists at each grade level. These groups

are called item rating groups, and panelists are assigned to an item rating group so that the two

are as equivalent as possible in terms of panelist type (teacher, career educator, or general public;

gender; race/ethnicity; and region of residence). These item rating groups rate slightly over half

of all items at their grade level. Item rating pools are developed so that the items in each are as

equivalent as possible in terms of item difficulty, item format (multiple choice, short constructed

response, and extended constructed response), test time for the block, and so forth. Item blocks

remain intact for the item rating pools. At least one block (a “common block”) is rated by all

panel members (i.e., both item rating groups, in the grade group).

Item ratings are placed on the NAEP scale by computing a theta value for the

dichotomous items and for the polytomous items in each subscale for each rating group.

Information weights are applied for the polytomous items at the subscale level before computing

the subscale score for both dichotomous and polytomous items.
773

Table H-5 presents the information weights computed for each rating group for the

mathematics NAEP achievement levels. Those data show that there is a consistent pattern of

error caused by the incorrect information function for math. Analyses of the data from 1992 and

1994 Trial State Assessments in reading show the corrected cutscores are consistently neither

higher nor lower as a result of this error, although the impact of the error was generally to

estimate a higher cutscore for the polytomous items. Simulated item parameters were used to

demonstrate that there was no consistent effect of the incorrect information function on weights

used to form the final composite cutpoints.

Analysis of the Effect of Item Discrimination

Figures H-1 through H-3 show graphs for the correct and the incorrect information

functions holding other parameters constant while varying the item discrimination parameter in

the generalized partial credit item response theory (IRT) model. In general, the differences

between correct and incorrect weights increase as item discrimination increases.

These figures show no consistent pattern in the direction (positive or negative) of the

difference between the correct and incorrect information function based on item discrimination.

When the correct information function is greater than the incorrect function, we observe that

information is in the area of maximum information. It is not the case, however, that the correct

function always results in a greater information weight where information is maximized.
774

Table H-5

Comparison Between the Incorrect and Correct Information Weights for 1992 Mathematics Achievement 

Based on Corrected Item Parameters

Basic Proficient 

Grade

Rating

Group Scale Incorrect Correct Diff Incorrect Correct Diff Incorrect 

4 A 1 0.24 0.80 -0.57 0.22 0.70 -0.48 0.14 

3 0.22 0.39 -0.16 0.21 0.33 -0.12 0.21 5 0.34 0.77 -0.42 0.32 0.61 -0.29 0.30 

B 1 0.26 0.66 -0.40 0.25 0.57 -0.32 0.22 

4 0.51 1.10 -0.59 0.49 0.96 -0.47 0.50 

8 A 1 0.72 1.23 -0.51 0.76 1.57 -0.81 0.63 

3 1.05 1.79 -0.74 1.08 2.00 -0.92 1.05 

B 2 0.50 1.25 -0.75 0.47 1.11 -0.64 0.39 

4 1.04 2.85 -1.81 0.94 2.19 -1.24 0.83 5 0.41 1.31 -0.90 0.32 0.92 -0.60 0.27 

12 A 3 0.13 0.29 -0.16 0.13 0.29 -0.16 0.13 

4 0.17 0.37 -0.21 0.16 0.36 -0.20 0.15 5 1.59 4.11 -2.52 1.69 4.82 -3.17 1.67 

B 5 1.41 2.60 -1.19 1.52 3.60 -2.09 1.43 
Figure H-1

Comparison Between the Incorrect and Correct Information Functions

Using Hypothetical Item Parameters a=.2, b=0, d0=0, d1=2, d2=0, d3=-2,

in Case of Four Response Categories

0123456

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 

Theta

I(Information)

DifferenceCorrectIncorrect
776

Figure H-2

Comparison Between the Incorrect and Correct Information Functions

Using Hypothetical Item Parameters a=.5, b=0, d0=0, d1=2, d2=0, d3=-2,

in Case of Four Response Categories

0123456

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 

Theta

I(Information)

DifferenceCorrectIncorrect
Figure H-3

Comparison Between the Incorrect and Correct Information Functions

Using Hypothetical Item Parameters a=1, b=0, d0=0, d1=2, d2=0, d3=-2, 

in Case of Four Response Categories

0123456

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 

Theta

I(Information)

DifferenceCorrectIncorrect
778

Analysis of the Effect of Location Parameters

Figures H-4 through H-6 show the correct and incorrect information weighting functions

for varying location parameters. The location parameters only shift the distribution of

information, and that is the case for both the correct and incorrect information weighting

functions. The amount of difference between the two is unchanged; only the locations change.

Analysis of the Effect of the Threshold Parameters.

If the threshold parameters are close, in terms of the locations of ICCs, the correct

information function will have a high peak. In the area of maximum information, i.e. around the

peak of the distribution, the incorrect information function underestimates information.

As can be seen in Figures H-6 through H-8, when the threshold parameters are relatively

far apart, the information is relatively low and the distribution is multimodal. When the threshold

parameters are closer information is higher and the distribution tends to be more unimodal. As

the threshold parameters move even closer, the difference between the correct and incorrect

information functions increases. As the threshold parameters become closer, the rate at which the

incorrect weighting function increments weights at the peak of the distribution is slower than that

for the correct function. This results in a negative difference between the two functions in the

area of maximum information.

Analysis of the Error: Conclusions

Three general conclusions can be drawn from our analyses:

1. No consistent pattern of over- or underestimation can be predicted from the

error in information weights. Generally, the incorrect information weighting

function results in a lower information weight for 1992 mathematics

assessment items, whereas, it results a higher information weight for 1992

reading assessment items.

2. The difference between the correct and incorrect weights increases as item

discrimination increases.

3. The impact of the incorrect information weighting function on the cutscores

is not consistent. The impact depends upon the location of the cutscore and

the relative weight of the dichotomous items.
Figure H-4

Comparison Between the Incorrect and Correct Information Functions

Using Hypothetical Item Parameters a=1.5, b=0, d0=0, d1=2, d2=0, d3=-2,

in Case of Four Response Categories

0123456

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 

Theta

I(Information)

DifferenceCorrectIncorrect
780

Figure H-5

Comparison Between the Incorrect and Correct Information Functions

Using Hypothetical Item Parameters a=1.5, b=-1, d0=0, d1=2, d2=0, d3=-2,

in Case of Four Response Categoreis

0123456

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 

Theta

I(Information)

DifferenceCorrectIncorrect
Figure H-6

Comparison Between the Incorrect and Correct Information Functions

Using Hypothetical Item Parameters a=1.5, b=1, d0=0, d1=2, d2=0, d3=-2,

in Case of Four Response Categories

0123456

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 

Theta

I(Information)

DifferenceCorrectIncorrect
782

Figure H-7

Comparison Between the Incorrect and Correct Information Functions

Using Hypothetical Item Parameters a=1.5, b=0, d0=0, d1=1, d2=0, d3=-1, 

in Case of Four Response Categories

0123456

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 

Theta

I(Information)

DifferenceCorrectIncorrect
Figure H-8

Comparison Between the Incorrect and Correct Information Functions

Using Hypothetical Item Parameters a=1.5, b=0, d0=0, d1=0.52, d2=0, d3=-0.52, 

in Case of Four Response Categories

0123456

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 

Theta

I(Information)

DifferenceCorrectIncorrect
784
785

Appendix I

CONSTRUCTED-RESPONSE ITEM SCORE STATISTICS

This appendix contains information about the constructed-response items included in the scaling of

data from the 1996 main assessments of mathematics and science and the long-term trend assessments of

reading, writing, and mathematics. There were no constructed-response items included in the scaling for the

1996 long-term trend assessment of science.

The information in the tables includes, for each subject area and grade (in the case of long-term trend,

each age class), the NAEP item numbers for each of the constructed-response items included in scaling, and the

block that contains the item. The tables also indicate the codes from the NAEP database that denote the range

of responses and the correct responses where appropriate. A portion of the responses to the constructedresponse items were scored twice for the purpose of examining rater reliability. For each item, the number of

papers with responses that were scored a second time is listed, along with the percent agreement between raters

and an index of reliability based on those responses. Cohen’s Kappa (Cohen, 1968) is the reliability estimate

used for dichotomized items. For items that are not dichotomized (i.e., polytomous items), the intraclass

correlation coefficient is used as the index of reliability. See Chapter 9 for more information about score

reliability for constructed-response items.
786

Table I-1

Score Range, Percent Agreement, and Cohen’s Kappa1

for the Dichotomously Scored Constructed-Response Mathematics Items

Used in Main Assessment Scaling, Grade 42

Range of Correct

Response Response Sample Percent Cohen’s

 Item Block Codes Codes Size Agreement Kappa

M019701 MF 1-2 2- 1962 98 0.967

M019801 MF 1-3 2-3 1956 94 0.877

M019901 MF 1-3 2-3 1953 98 0.963

M020001 MF 1-2 2- 1952 99 0.989

M020101 MF 1-2 2- 1946 99 0.980

M020201 MF 1-2 2- 1947 95 0.917

M020301 MF 1-4 4- 1948 100 0.993

M020401 MF 1-2 2- 1926 99 0.985

M020501 MF 1-2 2- 1850 99 0.980

M020701 MF 1-4 4- 1672 90 0.758

M039201 MC 1-2 2- 1986 99 0.985

M039301 MC 1-3 3- 1977 100 0.997

M040001 MC 1-3 3- 65463 99 0.980

M040201 MC 1-2 2- 1660 95 0.800

M040301 MI 1-2 2- 1961 98 0.970

M040901 MI 1-2 2- 1955 100 1.000

M043201 MM 1-2 2- 1961 98 0.961

M043301 MM 1-3 3- 1967 99 0.980

M043401 MM 1-4 4- 1938 98 0.964

M043402 MM 1-4 4- 1914 99 0.978

M043403 MM 1-3 3- 1896 100 0.978

M046001 MK 1-5 5- 1973 100 0.991

M046601 MK 1-4 4- 1969 98 0.970

M046801 MK 1-5 5- 1967 99 0.986

M046901 MK 1-5 5- 1952 100 0.999

M047301 MK 1-4 4- 1876 100 0.992

M061901 MJ 1-3 3- 1987 97 0.941

M061902 MJ 1-3 2-3 1971 98 0.966

M061903 MJ 1-2 2- 1960 99 0.974

M061904 MJ 1-3 2-3 1945 99 0.962

M061905 MJ 1-4 4- 1863 97 0.937

M061906 MJ 1-3 3- 1758 98 0.845

M074301 MO 1-2 2- 1973 100 0.992

N277903 MF 1-2 2- 1800 100 0.986

1 Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are dichotomized into

right (as indicated by key) and wrong.

2

 Rescored responses from the national and state assessment samples contributed to these statistics.

3 M040001 was erroneously identified as appearing only in the national assessment; therefore, the item was rescored at a higher

rate.
787

Table I-2

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Mathematics Items

Used in Main Assessment Scaling, Grade 41

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

M041201 MI 1-5 1934 84 0.918

M043501 MM 1-5 1882 91 0.962

M066301 ME 1-3 1971 98 0.971

M066501 ME 1-3 1949 98 0.988

M066601 ME 1-3 1932 92 0.935

M066701 ME 1-3 1883 96 0.973

M066801 ME 1-3 1750 96 0.965

M066901 ME 1-5 1756 92 0.979

M067901 MG 1-3 1952 98 0.987

M068001 MG 1-3 1944 99 0.989

M068002 MG 1-3 1902 98 0.983

M068003 MG 1-3 1679 98 0.978

M068004 MG 1-5 1692 94 0.984

M068701 ML 1-3 1968 99 0.985

M068901 ML 1-3 1815 98 0.979

M069001 ML 1-3 1724 91 0.915

M069101 ML 1-5 1686 94 0.970

M072201 MN 1-3 1952 99 0.995

M072202 MN 1-3 1947 97 0.980

M072401 MN 1-3 1848 94 0.964

M072501 MN 1-3 1636 93 0.929

M072601 MN 1-3 1360 98 0.969

M072701 MN 1-5 1337 94 0.973

M074501 MO 1-3 1969 92 0.932

M074701 MO 1-3 1954 98 0.981

M074801 MO 1-3 1930 99 0.988

M074901 MO 1-3 1886 95 0.967

M075001 MO 1-3 1770 99 0.985

M075101 MO 1-5 1766 90 0.964

1 Rescored responses from the national and state assessment samples contributed to these statistics.
788

Table I-3

Score Range, Percent Agreement, and Cohen’s Kappa1

for the Dichotomously Scored Constructed-Response Mathematics Items

Used in Main Assessment Scaling, Grade 82

Range of Correct

Response Response Sample Percent Cohen’s

 Item Block Codes Codes Size Agreement Kappa

M019701 MF 1-2 2- 1742 100 0.987

M019801 MF 1-3 2-3 1786 98 0.953

M019901 MF 1-3 2-3 1781 99 0.983

M020001 MF 1-2 2- 1787 100 0.995

M020101 MF 1-2 2- 1784 100 0.990

M020201 MF 1-2 2- 1785 97 0.883

M020301 MF 1-4 4- 1784 100 0.998

M020401 MF 1-2 2- 1774 99 0.989

M020501 MF 1-2 2- 1774 100 0.991

M020801 MF 1-6 6- 1745 99 0.976

M020901 MF 1-2 2- 2188 94 0.897

M021001 MF 1-2 2- 1773 100 0.994

M021101 MF 1-3 3- 1741 96 0.920

M021201 MF 1-3 3- 1691 98 0.968

M021301 MF 1-2 2- 1704 98 0.968

M021302 MF 1-2 2- 1632 98 0.951

M046001 MK 1-5 5- 1785 100 0.983

M046601 MK 1-4 4- 1783 98 0.957

M046801 MK 1-5 5- 1784 100 0.988

M046901 MK 1-5 5- 1780 100 0.988

M047301 MK 1-4 4- 1775 100 0.982

M047901 MK 1-3 3- 1669 100 0.991

M050801 MC 1-2 2- 1773 100 0.995

M050901 MC 1-4 4- 1755 100 0.993

M051001 MC 1-2 2- 1708 96 0.939

M051201 MM 1-2 2- 1789 100 0.992

M051301 MM 1-2 2- 1778 100 0.986

M051601 MM 1-2 2- 1781 99 0.974

M052101 MM 1-3 3- 1758 97 0.944

M052401 MI 1-2 2- 1788 96 0.929

M052901 MI 1-2 2- 1766 95 0.898

M053001 MI 1-2 2- 1699 94 0.897

M061901 MJ 1-3 3- 1796 97 0.916

M061902 MJ 1-3 2-3 1788 99 0.952

M061903 MJ 1-2 2- 1804 99 0.925

M061904 MJ 1-3 2-3 1794 99 0.982

 (continued)
789

Table I-3 (continued)

Score Range, Percent Agreement, and Cohen’s Kappa1

for the Dichotomously Scored Constructed-Response Mathematics Items

Used in Main Assessment Scaling, Grade 82

Range of Correct

Response Response Sample Percent Cohen’s

 Item Block Codes Codes Size Agreement Kappa

M061905 MJ 1-4 4- 1602 95 0.896

M061907 MJ 1-3 3- 1793 96 0.906

M061908 MJ 1-3 3- 1720 98 0.904

1 Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are dichotomized into

right (as indicated as key) and wrong.

2

 Rescored responses from the national and state assessment samples contributed to these statistics.

Table I-4

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Mathematics Items

Used in Main Assessment Scaling, Grade 81

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

M051101 MC 1-5 1718 85 0.903

M052201 MM 1-5 1759 93 0.968

M053101 MI 1-5 1698 90 0.937

M066301 ME 1-3 1789 99 0.972

M066501 ME 1-3 2349 96 0.976

M066601 ME 1-3 1770 94 0.958

M067201 ME 1-3 1762 91 0.944

M067501 ME 1-5 1750 94 0.942

M067901 MG 1-3 1785 98 0.991

M068003 MG 1-3 1779 99 0.988

M068005 MG 1-3 1785 98 0.981

M068006 MG 1-3 1741 94 0.950

M068008 MG 1-3 1462 92 0.898

M068201 MG 1-5 1001 88 0.939

M069301 ML 1-3 1818 99 0.992

M069601 ML 1-3 1794 95 0.940

M069701 ML 1-3 1769 100 0.997

M069901 ML 1-3 1514 95 0.953

(continued)
790

Table I-4 (continued)

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Mathematics Items

Used in Main Assessment Scaling, Grade 81

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

M070001 ML 1-4 1528 90 0.955

M072901 MN 1-3 1789 93 0.951

M073401 MN 1-3 1752 97 0.963

M073501 MN 1-3 1622 98 0.978

M073601 MN 1-5 1247 87 0.948

M075301 MO 1-3 1774 98 0.980

M075401 MO 1-3 1774 94 0.942

M075601 MO 1-3 1837 95 0.953

M075801 MO 1-3 1671 90 0.939

M076001 MO 1-5 1506 91 0.947

1

 Rescored responses from the national and state assessment samples contributed to these statistics.

Table I-5

Score Range, Percent Agreement, and Cohen’s Kappa1

for the Dichotomously Scored Constructed-Response Mathematics Items

Used in Main Assessment Scaling, Grade 12

Range of Correct

Response Response Sample Percent Cohen’s

 Item Block Codes Codes Size Agreement Kappa

M019701 MF 1-2 2- 163 99 0.978

M019801 MF 1-3 2-3 168 98 0.957

M019901 MF 1-3 2-3 166 99 0.980

M020001 MF 1-2 2- 167 100 1.000

M020101 MF 1-2 2- 167 99 0.973

M020201 MF 1-2 2- 168 96 0.890

M020301 MF 1-4 4- 170 100 1.000

M020401 MF 1-2 2- 167 100 1.000

M020501 MF 1-2 2- 161 99 0.988

M020801 MF 1-6 6- 165 99 0.986

M020901 MF 1-2 2- 198 93 0.888

M021001 MF 1-2 2- 160 99 0.989

M021101 MF 1-3 3- 154 97 0.940

M021201 MF 1-3 3- 158 98 0.964

M021301 MF 1-2 2- 155 97 0.949

M021302 MF 1-2 2- 143 99 0.984

(continued)
791

Table I-5 (continued)

Score Range, Percent Agreement, and Cohen’s Kappa1

for the Dichotomously Scored Constructed-Response Mathematics Items

Used in Main Assessment Scaling, Grade 12

Range of Correct

Response Response Sample Percent Cohen’s

 Item Block Codes Codes Size Agreement Kappa

M046001 MK 1-5 5- 168 99 0.979

M046601 MK 1-4 4- 165 96 0.906

M046801 MK 1-5 5- 165 100 1.000

M046901 MK 1-5 5- 165 100 1.000

M047301 MK 1-4 4- 163 99 0.969

M047901 MK 1-3 3- 147 99 0.976

M050801 MC 1-2 2- 160 100 1.000

M050901 MC 1-4 4- 156 100 1.000

M051001 MC 1-2 2- 154 82 0.671

M051201 MM 1-2 2- 163 100 1.000

M051301 MM 1-2 2- 163 99 0.967

M051601 MM 1-2 2- 159 99 0.971

M052101 MM 1-3 3- 159 98 0.964

M052401 MI 1-2 2- 164 98 0.959

M052901 MI 1-2 2- 157 94 0.892

M053001 MI 1-2 2- 149 97 0.942

M061901 MJ 1-3 3- 185 98 0.954

M061902 MJ 1-3 2-3 179 98 0.932

M061903 MJ 1-2 2- 181 98 0.920

M061904 MJ 1-3 2-3 183 99 0.990

M061905 MJ 1-4 4- 153 92 0.841

M061907 MJ 1-3 3- 171 97 0.935

M061908 MJ 1-3 3- 175 99 0.958

1 Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are dichotomized into

right (as indicated as key) and wrong.
792

Table I-6

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Mathematics Items

Used in Main Assessment Scaling, Grade 12

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

M051101 MC 1-5 152 80 0.886

M052201 MM 1-5 162 95 0.983

M053101 MI 1-5 149 89 0.913

M066301 ME 1-3 158 99 0.995

M066501 ME 1-3 728 96 0.967

M066601 ME 1-3 154 97 0.979

M067201 ME 1-3 159 90 0.946

M067501 ME 1-5 160 92 0.938

M067901 MG 1-3 170 99 0.995

M068003 MG 1-3 163 99 0.989

M068005 MG 1-3 160 98 0.990

M068006 MG 1-3 156 92 0.933

M068008 MG 1-3 128 93 0.890

M068201 MG 1-5 85 88 0.943

M069301 ML 1-3 185 99 0.993

M069601 ML 1-3 180 93 0.908

M069701 ML 1-3 176 99 0.997

M069901 ML 1-3 148 95 0.954

M070001 ML 1-4 147 90 0.967

M072901 MN 1-3 161 94 0.955

M073401 MN 1-3 159 96 0.955

M073501 MN 1-3 147 100 1.000

M073601 MN 1-5 110 89 0.963

M075301 MO 1-3 165 100 1.000

M075401 MO 1-3 161 96 0.966

M075601 MO 1-3 166 94 0.949

M075801 MO 1-3 151 93 0.959

M076001 MO 1-5 128 91 0.958
793

Table I-7

Score Range, Percent Agreement, and Cohen’s Kappa1

for the Dichotomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 4

Range of Correct

Response Response Sample Percent Cohen’s

 Item Block Codes Codes Size Agreement Kappa

K031004 SC 1-2 2- 678 95 0.872

K031101 SD 1-2 2- 675 97 0.914

K031102 SD 1-2 2- 671 95 0.887

K031103 SD 1-2 2- 658 95 0.896

K031104 SD 1-2 2- 652 98 0.930

K031508 SH 1-2 2- 415 95 0.862

K033001 SK 1-2 2- 469 95 0.865

1 Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are dichotomized into

right (as indicated by key) and wrong. These items are dichotomized into right (as indicated by key) and wrong.

Table I-8

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 4

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

K031001 SC 1-3 759 96 0.927

K031002 SC 1-3 748 88 0.872

K031003 SC 1-3 732 91 0.837

K031005 SC 1-3 655 87 0.846

K031006 SC 1-3 579 93 0.951

K031007 SC 1-3 499 84 0.870

K031105 SD 1-3 633 99 0.978

K031107 SD 1-4 543 91 0.949

K031201 SE 1-3 700 99 0.988

K031202 SE 1-3 698 94 0.869

K031203 SE 1-4 687 96 0.990

K031204 SE 1-5 687 89 0.936

K031210 SE 1-3 481 87 0.886

K031211 SE 1-3 382 88 0.854

K031301 SF 1-4 656 94 0.963

K031302 SF 1-3 552 94 0.854

K031303 SF 1-3 488 93 0.926

(continued)
794

Table I-8 (continued)

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 4

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

K031304 SF 1-3 438 94 0.940

K031309 SF 1-4 617 93 0.957

K031401 SG 1-4 604 88 0.902

K031402 SG 1-3 611 93 0.955

K031403 SG 1-3 604 94 0.934

K031404 SG 1-3 598 95 0.976

K031407 SG 1-3 580 91 0.870

K031408 SG 1-3 561 98 0.989

K031409 SG 1-3 550 95 0.948

K031410 SG 1-3 536 96 0.945

K031501 SH 1-3 631 98 0.984

K031502 SH 1-3 622 94 0.880

K031503 SH 1-3 635 94 0.949

K031505 SH 1-3 622 96 0.970

K031506 SH 1-5 610 88 0.938

K031507 SH 1-3 482 94 0.915

K031509 SH 1-3 633 94 0.944

K031602 SI 1-3 603 98 0.987

K031603 SI 1-3 593 98 0.984

K031604 SI 1-3 594 99 0.991

K031606 SI 1-3 584 96 0.940

K031607 SI 1-4 565 93 0.960

K031608 SI 1-3 518 93 0.895

K031609 SI 1-3 480 96 0.920

K031901 SJ 1-3 463 89 0.929

K032001 SJ 1-3 464 97 0.981

K032501 SJ 1-3 448 95 0.962

K032502 SJ 1-3 440 96 0.969

K032601 SJ 1-3 422 89 0.866

K032602 SJ 1-3 390 91 0.915

K033101 SK 1-3 469 93 0.832

K033501 SK 1-3 448 93 0.894

K033502 SK 1-3 422 90 0.920

K033503 SK 1-3 398 89 0.877

K034001 SL 1-3 459 91 0.847

K034101 SL 1-3 466 89 0.906

KW34101 SL 1-3 464 89 0.936

KX34101 SL 1-3 460 95 0.942

KZ34101 SL 1-3 456 86 0.868

(continued)
795

Table I-8 (continued)

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 4

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

K034802 SM 1-3 509 94 0.927

K034901 SM 1-3 503 96 0.932

K034902 SM 1-3 508 95 0.933

K035201 SM 1-3 477 92 0.939

K035301 SM 1-4 455 94 0.940

K035601 SN 1-3 612 95 0.936

K035801 SN 1-3 626 93 0.954

K035901 SN 1-3 605 97 0.936

K036101 SN 1-3 586 94 0.952

K036301 SN 1-3 474 97 0.961

K037301 SO 1-3 452 97 0.937

K037401 SO 1-3 437 95 0.922

K037501 SO 1-3 441 97 0.884

K037601 SO 1-3 433 93 0.909

K037701 SO 1-4 419 97 0.982

K037702 SO 1-3 359 93 0.915

K038801 ST 1-4 510 97 0.967

K038901 ST 1-4 509 91 0.923

K039201 ST 1-3 464 96 0.937

K039301 ST 1-3 452 94 0.955

K039401 ST 1-4 433 94 0.916

K039801 SU 1-3 454 96 0.961

K039901 SU 1-5 460 91 0.963

K040001 SU 1-3 440 90 0.935

K040301 SU 1-3 382 89 0.879

K040401 SU 1-4 342 95 0.950

K040501 SU 1-3 281 97 0.977
796

Table I-9

Score Range, Percent Agreement, and Cohen’s Kappa1

for the Dichotomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 82

Range of Correct

Response Response Sample Percent Cohen’s

 Item Block Codes Codes Size Agreement Kappa

K038101 SO 1-2 2- 1230 98 0.928

K040806 SE 1-2 2- 792 91 0.750

K040904 SG 1-2 2- 1669 98 0.956

K041101 SG 1-2 2- 1654 98 0.959

K042603 SJ 1-2 2- 1158 95 0.850

K048103 SL 1-2 2- 1077 94 0.869

1 Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are dichotomized into

right (as indicated by key) and wrong.

2

 Rescored responses from the national and state assessment samples contributed to these statistics.

Table I-10

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 81

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

K031301 SF 1-4 1901 98 0.918

K031302 SF 1-3 1876 89 0.797

K031305 SF 1-3 1828 94 0.877

K031306 SF 1-3 1809 98 0.867

K031307 SF 1-3 1693 94 0.938

K031308 SF 1-3 1496 97 0.921

K031309 SF 1-4 1898 97 0.925

K031602 SI 1-3 1684 98 0.983

K031603 SI 1-3 1676 99 0.961

K031604 SI 1-3 1680 100 0.994

K031606 SI 1-3 1676 95 0.945

K031607 SI 1-4 1675 90 0.927

K031608 SI 1-3 1668 92 0.891

K031609 SI 1-3 1667 96 0.958

K031610 SI 1-3 1675 98 0.979

K031611 SI 1-3 1658 97 0.951

K031613 SI 1-3 1602 99 0.961

(continued)
797

Table I-10 (continued)

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 81

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

K035601 SN 1-3 1685 93 0.936

K035801 SN 1-3 1688 94 0.955

K035901 SN 1-3 1683 95 0.961

K036101 SN 1-3 1683 94 0.942

K036301 SN 1-3 1677 95 0.974

K036401 SN 1-3 1670 97 0.955

K036402 SN 1-3 1672 93 0.930

K036403 SN 1-3 1669 92 0.876

K036404 SN 1-3 1666 92 0.796

K036701 SN 1-3 1671 97 0.954

K036801 SN 1-3 1656 97 0.965

K037301 SO 1-3 1271 93 0.947

K037401 SO 1-3 1271 93 0.938

K037501 SO 1-3 1263 96 0.916

K037601 SO 1-3 1269 89 0.926

K037701 SO 1-4 1269 99 0.985

K037703 SO 1-3 1266 91 0.900

K038201 SO 1-3 1225 92 0.849

K038301 SO 1-5 1132 85 0.899

K040601 SC 1-3 2051 99 0.988

K040603 SC 1-3 1832 95 0.956

K040604 SC 1-3 1778 93 0.916

K040605 SC 1-3 1665 95 0.970

K040606 SC 1-4 1544 94 0.959

K040607 SC 1-4 2027 94 0.974

K040608 SC 1-4 2005 91 0.959

K040609 SC 1-4 1978 95 0.980

K040610 SC 1-4 1949 93 0.967

K040701 SD 1-4 1892 93 0.942

K040702 SD 1-4 1889 94 0.955

K040704 SD 1-3 1883 95 0.958

K040705 SD 1-3 1872 93 0.864

K040708 SD 1-3 1855 92 0.911

K040711 SD 1-3 1768 92 0.955

K040713 SD 1-4 1561 84 0.898

K040802 SE 1-3 1775 98 0.980

K040803 SE 1-3 1522 96 0.928

K040804 SE 1-3 1117 94 0.903

(continued)
798

Table I-10 (continued)

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 81

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

K040805 SE 1-3 1030 95 0.922

K040808 SE 1-3 1673 98 0.989

K040901 SG 1-3 1670 96 0.981

K040902 SG 1-3 1672 93 0.946

K040903 SG 1-3 1669 91 0.938

K040905 SG 1-3 1668 92 0.915

K041002 SG 1-3 1665 86 0.910

K041004 SG 1-3 1654 96 0.983

K041201 SG 1-3 1650 91 0.911

K041202 SG 1-3 1508 90 0.811

K041306 SH 1-3 1678 87 0.903

K041307 SH 1-3 1685 91 0.894

K041401 SH 1-3 1668 96 0.958

K041402 SH 1-3 1661 99 0.988

K041403 SH 1-3 1643 94 0.890

K041901 SJ 1-3 1254 95 0.927

K042001 SJ 1-3 1259 91 0.888

K042101 SJ 1-3 1256 98 0.959

K042102 SJ 1-3 1258 97 0.974

K042201 SJ 1-3 1259 97 0.915

K042601 SJ 1-3 1215 89 0.875

K042602 SJ 1-4 1190 94 0.956

K043001 SK 1-3 1259 95 0.919

K043101 SK 1-3 1260 89 0.888

K043102 SK 1-4 1261 85 0.934

K043103 SK 1-3 1254 92 0.818

K043501 SK 1-3 1246 94 0.936

K043601 SK 1-3 1178 89 0.881

K043602 SK 1-3 1119 95 0.877

K043603 SK 1-3 1110 95 0.916

K044101 ST 1-3 1268 98 0.974

K044201 ST 1-3 1270 89 0.904

K044301 ST 1-3 1270 95 0.965

K044401 ST 1-3 1268 85 0.860

K044901 ST 1-3 1256 93 0.916

K045001 ST 1-3 1237 90 0.898

K045101 ST 1-4 1155 88 0.928

K045102 ST 1-4 1092 89 0.907

(continued)
799

Table I-10 (continued)

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 81

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

K045301 SU 1-3 1252 93 0.951

K045601 SU 1-4 1257 96 0.954

K045701 SU 1-3 1259 90 0.921

K045801 SU 1-3 1251 94 0.933

K046301 SU 1-3 1246 93 0.879

K046401 SU 1-3 1218 94 0.905

K046501 SU 1-3 1215 95 0.954

K046601 SU 1-3 1192 91 0.942

K046701 SU 1-4 1137 88 0.873

K047201 SL 1-4 1260 88 0.948

K047301 SL 1-3 1262 96 0.969

K047401 SL 1-3 1254 92 0.942

K047901 SL 1-3 1237 92 0.952

K048001 SL 1-3 1209 99 0.986

K048101 SL 1-4 1156 90 0.896

K048102 SL 1-3 1093 95 0.866

K048601 SM 1-3 1272 93 0.931

K048901 SM 1-3 1278 98 0.985

K049001 SM 1-3 1278 100 0.981

K049301 SM 1-3 1254 98 0.941

K049401 SM 1-3 1234 94 0.928

K049402 SM 1-3 1236 89 0.919

K049403 SM 1-3 1190 88 0.875

K049404 SM 1-4 1156 84 0.833

1

 Rescored responses from the national and state assessment samples contributed to these statistics.
800

Table I-11

Score Range, Percent Agreement, and Cohen’s Kappa1

for the Dichotomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 12

Range of Correct

Response Response Sample Percent Cohen’s

 Item Block Codes Codes Size Agreement Kappa

K040806 SE 1-2 2- 401 86 0.705

K048103 SL 1-2 2- 361 94 0.884

K049708 SF 1-2 2- 690 98 0.951

K051002 SJ 1-2 2- 450 92 0.854

K051003 SJ 1-2 2- 448 96 0.911

K051004 SJ 1-2 2- 444 96 0.915

K051701 SK 1-2 2- 455 96 0.918

K054004 SO 1-2 2- 431 97 0.902

1 Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are dichotomized into

right (as indicated by key) and wrong.

Table I-12

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 12

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

K040802 SE 1-3 660 99 0.987

K040803 SE 1-3 603 97 0.946

K040804 SE 1-3 502 93 0.953

K040805 SE 1-3 471 96 0.959

K040808 SE 1-3 641 98 0.991

K041306 SH 1-3 628 88 0.906

K041307 SH 1-3 624 91 0.921

K041401 SH 1-3 630 93 0.924

K041402 SH 1-3 626 99 0.990

K041403 SH 1-3 614 94 0.897

K041404 SH 1-3 608 95 0.963

K041406 SH 1-3 569 96 0.898

K047201 SL 1-4 459 90 0.959

K047301 SL 1-3 458 96 0.971

K047401 SL 1-3 458 92 0.954

K047901 SL 1-3 447 94 0.962

K048001 SL 1-3 437 98 0.972

(continued)
801

Table I-12 (continued)

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 12

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

K048101 SL 1-4 404 84 0.861

K048102 SL 1-3 364 93 0.850

K048601 SM 1-3 476 92 0.937

K048901 SM 1-3 476 98 0.985

K049001 SM 1-3 476 99 0.940

K049301 SM 1-3 475 96 0.921

K049401 SM 1-3 463 94 0.938

K049402 SM 1-3 457 86 0.894

K049403 SM 1-3 432 87 0.871

K049404 SM 1-4 418 81 0.827

K049501 SC 1-4 752 98 0.970

K049502 SC 1-5 743 92 0.930

K049503 SC 1-3 721 94 0.954

K049504 SC 1-3 699 93 0.946

K049505 SC 1-3 667 93 0.945

K049506 SC 1-4 597 96 0.917

K049601 SD 1-4 701 88 0.894

K049602 SD 1-5 697 96 0.983

K049603 SD 1-5 643 82 0.883

K049604 SD 1-3 510 94 0.949

K049701 SF 1-3 692 99 0.968

K049702 SF 1-3 692 98 0.973

K049703 SF 1-3 681 93 0.940

K049704 SF 1-3 670 95 0.951

K049705 SF 1-4 615 88 0.954

K049706 SF 1-3 586 83 0.874

K049707 SF 1-5 509 84 0.938

K049802 SG 1-3 608 95 0.971

K049804 SG 1-3 609 95 0.954

K049807 SG 1-4 610 90 0.941

K049809 SG 1-3 599 88 0.880

K049810 SG 1-4 592 90 0.960

K049812 SG 1-3 588 95 0.940

K049813 SG 1-3 564 92 0.929

K049814 SG 1-3 550 94 0.932

K049815 SG 1-4 472 91 0.921

K049901 SI 1-3 604 94 0.963

K049902 SI 1-3 601 87 0.910

K049903 SI 1-3 599 90 0.931

K049904 SI 1-3 600 93 0.941

(continued)
802

Table I-12 (continued)

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 12

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

K049907 SI 1-3 593 85 0.916

K049908 SI 1-3 585 97 0.980

K049909 SI 1-4 584 94 0.952

K049911 SI 1-3 560 94 0.955

K049912 SI 1-5 534 81 0.906

K049914 SI 1-3 544 93 0.931

K050401 SJ 1-3 457 93 0.898

K050501 SJ 1-4 453 91 0.962

K050901 SJ 1-3 454 98 0.984

K051001 SJ 1-3 453 93 0.976

K051101 SJ 1-3 434 96 0.948

K051102 SJ 1-3 417 93 0.852

K051201 SJ 1-3 380 92 0.878

K051801 SK 1-3 453 91 0.927

K052301 SK 1-3 448 98 0.983

K052401 SK 1-4 450 90 0.959

K052402 SK 1-3 441 97 0.975

K052501 SK 1-4 441 90 0.959

K052502 SK 1-4 435 91 0.926

K052503 SK 1-3 422 94 0.885

K052901 SN 1-4 641 89 0.952

K053001 SN 1-3 638 92 0.914

K053101 SN 1-3 641 99 0.993

K053102 SN 1-3 636 92 0.944

K053601 SN 1-5 621 91 0.944

K053701 SN 1-3 594 95 0.952

K053801 SN 1-3 582 90 0.900

K053901 SN 1-3 540 94 0.895

K054001 SO 1-4 447 98 0.977

K054002 SO 1-3 444 93 0.915

K054003 SO 1-3 436 99 0.985

K054005 SO 1-3 422 96 0.953

K054006 SO 1-3 416 97 0.969

K054007 SO 1-3 396 85 0.793

K054008 SO 1-4 342 79 0.831

(continued)
803

Table I-12 (continued)

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Science Items

Used in Main Assessment Scaling, Grade 12

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

K057401 ST 1-3 465 92 0.943

K057501 ST 1-3 468 91 0.909

K057601 ST 1-3 469 97 0.970

K057701 ST 1-3 470 98 0.973

K058001 ST 1-3 469 95 0.896

K058201 ST 1-4 446 88 0.925

K058301 ST 1-4 423 86 0.906

K058401 ST 1-3 394 93 0.963

K058501 ST 1-4 364 91 0.929

K059001 SU 1-3 475 89 0.873

K059101 SU 1-3 470 95 0.941

K059201 SU 1-4 474 93 0.974

K059301 SU 1-4 473 99 0.985

K059801 SU 1-3 448 91 0.920

K059901 SU 1-4 444 95 0.964

K060001 SU 1-3 433 92 0.954

K060101 SU 1-4 423 89 0.895

K089811 SG 1-3 587 92 0.911

Table I-13

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Reading Items

Used in Long-Term Trend Assessment Scaling, Age Class 9

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

N001507 BH 1-6 252 92 0.947

N002804 BL 1-5 184 95 0.958

N003104 BM 1-5 177 87 0.879

N003704 BN 1-4 192 92 0.902

N008905 BJ 1-6 235 94 0.972
804

Table I-14

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Reading Items

Used in Long-Term Trend Assessment Scaling, Age Class 13

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

N001507 BH 1-6 285 89 0.901

N001904 BJ 1-5 256 87 0.914

N002302 BK 1-9 0 0 0.000

N002804 BL 1-5 309 90 0.911

N003104 BM 1-5 272 90 0.909

N003704 BN 1-4 236 89 0.889

N004303 BO 1-4 259 91 0.908

N004605 BP 1-5 277 94 0.969

Table I-15

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Reading Items

Used in Long-Term Trend Assessment Scaling, Age Class 17

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

N001507 BH 1-6 244 90 0.929

N001904 BJ 1-5 261 91 0.931

N002302 BK 1-9 0 0 0.000

N002804 BL 1-5 262 90 0.911

N003104 BM 1-5 239 89 0.908

N003704 BN 1-4 219 89 0.908

N004303 BO 1-4 180 83 0.879

N004605 BP 1-5 230 95 0.967

N015905 BQ 1-4 201 89 0.932
805

Table I-16

Score Range, Percent Agreement, and Cohen’s Kappa1

for the Dichotomously Scored Constructed-Response Mathematics Items

Used in Long-Term Trend Assessment Scaling, Age Class 9

Range of Correct

Response Response Sample Percent Cohen’s

 Item Block Codes Codes Size Agreement Kappa

N270001 M1 1-2 1- 618 99 0.972

N270901 M1 1-2 1- 618 100 0.973

N271101 M2 1-2 1- 614 98 0.955

N275401 M2 1-2 1- 614 98 0.951

N276001 M2 1-2 1- 614 99 0.976

N276002 M2 1-2 1- 614 98 0.963

N276021 M3 1-2 1- 575 99 0.979

N276022 M3 1-2 1- 575 99 0.979

N276101 M1 1-2 1- 618 100 0.992

N276801 M1 1-2 1- 618 100 0.961

N276802 M1 1-2 1- 618 100 0.979

N276803 M1 1-2 1- 618 98 0.963

N276821 M3 1-2 1- 575 100 0.978

N276822 M3 1-2 1- 575 99 0.962

N276823 M3 1-2 1- 575 99 0.981

N277501 M2 1-2 1- 614 99 0.967

N277601 M2 1-4 1- 614 99 0.979

N277602 M2 1-5 1- 614 99 0.987

N277603 M2 1-2 1- 614 99 0.971

N277621 M3 1-2 1- 575 99 0.971

N277622 M3 1-2 1- 575 99 0.959

N277623 M3 1-2 1- 575 99 0.949

N284001 M1 1-2 1- 618 99 0.981

N284002 M1 1-2 1- 618 99 0.957

N284021 M3 1-2 1- 575 99 0.960

N284022 M3 1-2 1- 575 99 0.959

N286101 M1 1-2 1- 618 99 0.980

N286102 M2 1-2 1- 614 98 0.966

1 Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are dichotomized into

right (as indicated as key) and wrong.
806

Table I-17

Score Range, Percent Agreement, and Cohen’s Kappa1

for the Dichotomously Scored Constructed-Response Mathematics Items

Used in Long-Term Trend Assessment Scaling, Age Class 13

Range of Correct

Response Response Sample Percent Cohen’s

 Item Block Codes Codes Size Agreement Kappa

N256101 M2 1-2 1- 621 99 0.974

N257601 M1 1-2 1- 643 99 0.976

N263101 M1 1-2 1- 643 99 0.975

N264521 M3 1-2 1- 624 98 0.968

N269201 M2 1-2 1- 621 96 0.857

N275001 M1 1-2 1- 643 99 0.981

N276801 M1 1-2 1- 643 100 0.946

N276802 M1 1-2 1- 643 100 0.928

N276803 M1 1-2 1- 643 98 0.951

N276821 M3 1-2 1- 624 100 1.000

N276822 M3 1-2 1- 624 100 0.901

N276823 M3 1-2 1- 624 100 0.979

N277601 M1 1-4 1- 643 99 0.935

N277602 M1 1-5 1- 643 99 0.981

N277603 M1 1-2 1- 643 99 0.964

N277901 M2 1-2 1- 621 100 0.950

N277902 M2 1-2 1- 621 100 0.907

N277903 M2 1-2 2- 621 100 0.980

N280621 M3 1-2 1- 624 99 0.976

N280622 M3 1-2 1- 624 98 0.968

N280623 M3 1-2 1- 624 99 0.978

N280624 M3 1-2 1- 624 98 0.962

N280625 M3 1-2 1- 624 99 0.986

N280626 M3 1-2 1- 624 98 0.967

N286601 M2 1-2 1- 621 99 0.982

N286602 M2 1-3 1- 621 100 0.994

N286603 M2 1-2 1- 621 99 0.984

1 Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are dichotomized into

right (as indicated as key) and wrong.
807

Table I-18

Score Range, Percent Agreement, and Cohen’s Kappa1

for the Dichotomously Scored Constructed-Response Mathematics Items

Used in Long-Term Trend Assessment Scaling, Age Class 17

Range of Correct

Response Response Sample Percent Cohen’s

 Item Block Codes Codes Size Agreement Kappa

N251101 M1 1-2 1- 616 100 0.996

N255801 M2 1-2 1- 616 100 0.986

N256001 M3 1-2 1- 564 99 0.984

N256101 M1 1-2 1- 616 100 1.000

N259001 M2 1-2 1- 616 100 0.994

N260601 M1 1-2 1- 616 100 1.000

N260801 M2 1-2 1- 616 100 0.997

N263001 M1 1-2 1- 616 100 0.994

N263101 M2 1-2 1- 616 100 0.996

N264301 M1 1-2 1- 616 100 0.992

N264321 M3 1-2 1- 564 99 0.990

N264521 M3 1-2 1- 564 100 0.996

N267921 M3 1-2 1- 564 99 0.985

N276821 M3 1-2 1- 564 100 0.940

N276822 M3 1-2 1- 564 100 1.000

N276823 M3 1-2 1- 564 100 1.000

N278501 M1 1-2 1- 616 100 1.000

N278502 M1 1-2 1- 616 100 0.991

N278503 M1 1-2 1- 616 100 1.000

N280401 M2 1-2 1- 616 100 0.996

N280621 M3 1-2 1- 564 100 1.000

N280622 M3 1-2 1- 564 100 1.000

N280623 M3 1-2 1- 564 100 1.000

N280624 M3 1-2 1- 564 100 0.995

N280625 M3 1-2 1- 564 100 0.996

N280626 M3 1-2 1- 564 100 0.997

N285321 M3 1-2 1- 564 99 0.990

N287301 M1 1-2 1- 616 100 0.997

N287302 M1 1-2 1- 616 100 0.997

1 Cohen’s Kappa is a measure of reliability that is appropriate for items that are dichotomized. These items are dichotomized into

right (as indicated by key) and wrong.
808

Table I-19

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Writing Items

Used in Long-Term Trend Assessment Scaling, Age Class 9

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

N000602 BE 1-3 459 97 0.976

N000902 BG 1-4 522 96 0.967

N001002 BG 1-4 384 94 0.945

N007602 BG 1-4 263 93 0.920

N007608 BV 1-4 159 55 0.819

N014702 BC 1-6 511 96 0.962

N014802 BE 1-4 524 93 0.957

N014808 BE 1-6 366 60 0.863

Table I-20

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Writing Items

Used in Long-Term Trend Assessment Scaling, Age Class 13

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

N000302 BC 1-4 591 92 0.916

N000371 BC 1-6 336 59 0.846

N000402 BD 1-4 566 92 0.911

N000471 BD 1-6 329 53 0.813

N000502 BE 1-4 610 89 0.893

N000602 BE 1-3 591 97 0.958

N000902 BG 1-4 616 93 0.936

N001002 BG 1-4 547 94 0.930
809

Table I-21

Score Range, Percent Agreement, and Intraclass Correlation

for the Polytomously Scored Constructed-Response Writing Items

Used in Long-Term Trend Assessment Scaling, Age Class 17

Range of

Response Sample Percent Intraclass

 Item Block Codes Size Agreement Correlation

N000302 BC 1-4 501 92 0.925

N000371 BC 1-6 324 54 0.815

N000402 BD 1-4 477 92 0.915

N000471 BD 1-4 307 50 0.820

N001002 BG 1-4 473 93 0.931

N018002 BE 1-4 519 86 0.904

N019002 BE 1-4 489 93 0.937

N021002 BG 1-4 527 88 0.912
810
811

Appendix J

DIFFERENTIAL ITEM FUNCTIONING (DIF) RESULTS

Table J-1

1996 Mathematics Items Identified as “C” or “CC” Items in At Least One Comparison1

Item Block Scale Category Grade Comparison

Group

Favored

M077701 MQ7 Estimation C 4 White/Black White

M067501 ME11A Data Analysis, Statistics, and

Probability

CC 8 White/Black White

M072901 MN2A Data Analysis, Statistics, and

Probability

CC 8 White/Hispanic White

M075401 MO3A Measurement CC 8 White/Black White

M075601 M05A Measurement CC 8 White/Black White

M085010 MV10A Theme: Measurement CC 8 White/Hispanic White

M070702 ME2A Algebra and Functions C 12 White/Black Black

M070702 ME2A Algebra and Functions C 12 White/Hispanic Hispanic

M071701 MG8A Algebra and Functions CC 12 White/Black Black

M083701 MS2A Advanced Data Analysis,

Statistics, and Probability

C 12 Male/Female Male

1

 For each grade for which an item was administered, three comparisons were performed: Male/Female, White/Black, and

White/Hispanic.

Table J-2

1996 Science Items Identified as “C” or “CC” Items in At Least One Comparison1

Item Block Scale Category Grade Comparison Group Favored

K031202 SE2A Earth Science CC 4 Male/Female Male

K031203 SE3A Physical Science CC 4 Male/Female Female

K031204 SE4A Physical Science CC 4 Male/Female Female

K031304 SF9A Physical Science CC 4 White/Black White

K031403 SG3A Earth Science CC 4 White/Black White

K032901 SK3 Physical Science C 4 Male/Female Male

K033502 SK10A Life Science CC 4 White/Black Black

K033503 SK11A Life Science CC 4 White/Black Black

K035801 SN5A Physical Science CC 4 Male/Female Male

K036301 SN10A Physical Science CC 4 Male/Female Male

K039101 ST8 Earth Science C 4 Male/Female Male

K039201 ST9A Physical Science CC 4 Male/Female Male

K040001 SU6A Physical Science CC 4 Male/Female Male

K040101 SU7 Physical Science C 4 Male/Female Male

K036101 SN7A Earth Science CC 8 Male/Female Male

K036401 SN10A Earth Science CC 8 Male/Female Male

K037501 SO6A Physical Science CC 8 Male/Female Male

1

 For each grade for which an item was administered, three comparisons were performed: Male/Female, White/Black,

and White/Hispanic.

(continued)
Table J-2 (continued)

1996 Science Items Identified as “C” or “CC” Items in At Least One Comparison1

812

Item Block Scale Category Grade Comparison Group Favored

K037601 SO7A Earth Science CC 8 White/Hispanic White

K038201 SO15A Earth Science CC 8 White/Hispanic Hispanic

K040608 SC2G Physical Science CC 8 White/Black Black

K041202 SG12A Earth Science CC 8 Male/Female Male

K041601 SJ2 Earth Science C 8 Male/Female Male

K042602 SJ15A Life Science CC 8 Male/Female Female

K042701 SK1 Earth Science C 8 Male/Female Male

K043101 SK7A Earth Science CC 8 Male/Female Male

K043301 SK11 Earth Science C 8 Male/Female Male

K044101 ST5A Earth Science CC 8 Male/Female Male

K044901 ST13A Life Science CC 8 White/Black Black

K045001 ST14A Life Science CC 8 White/Black Black

K045001 ST14A Life Science CC 8 White/Hispanic Hispanic

K046701 SU16A Life Science CC 8 White/Hispanic White

K049401 SM13A Earth Science CC 8 White/Black Black

K049401 SM13A Earth Science CC 8 Male/Female Male

K040805 SE4A Earth Science CC 12 White/Hispanic Hispanic

K048901 SM8A Life Science CC 12 Male/Female Female

K048901 SM8A Life Science CC 12 White/Black White

K048901 SM8A Life Science CC 12 White/Hispanic White

K049301 SM12A Life Science CC 12 White/Hispanic Hispanic

K049603 SD3A Physical Science CC 12 Male/Female Female

K049702 SF2A Physical Science CC 12 White/Hispanic Hispanic

K049802 SG2A Earth Science CC 12 White/Black Black

K050501 SJ6A Earth Science CC 12 White/Black White

K052201

SK10

SS10 Life Science C 12 Male/Female Male

K052502 SK15A Life Science CC 12 White/Hispanic Hispanic

K053601 SN13A

SS5A

Life Science CC 12 Male/Female Female

K053701 SN14A Life Science CC 12 Male/Female Male

K054005 SO5A Life Science CC 12 White/Black Black

K058201 ST13A Physical Science CC 12 White/Hispanic White

K058301 ST14A Physical Science CC 12 Male/Female Male

K058401 ST15A Earth Science CC 12 Male/Female Male

K059901 SU14A Earth Science CC 12 White/Black White

1

 For each grade for which an item was administered, three comparisons were performed: Male/Female, White/Black, and

White/Hispanic.
813

Appendix K

CORRECTION OF THE NAEP PROGRAM DOCUMENTATION ERROR

IN THE 1992 STATE MATHEMATICS RESULTS

Frank Jenkins and Ed Kulick

Educational Testing Service

In April 1995, results from the 1994 Trial State Assessment in reading were released as part of

the report 1994 NAEP Reading: A First Look (Williams, Reese, Campbell, Mazzeo, & Phillips, 1995).

Subsequently, ETS/NAEP research scientists discovered an error in the documentation for the ETS

version of the PARSCALE program, which is used to compute NAEP scale score results. The error

affected how omitted responses were treated in the IRT scaling of the extended constructed-response

items that received partial-credit scoring (i.e., could have several partially correct categories) in analyses

of 1992 and 1994 assessment data. The error affected only polytomous items; omitted multiple-choice

and omitted short constructed responses were treated appropriately.

The conventional treatment in NAEP subjects has been to treat omitted responses (blank

responses to an item that are followed by valid responses to items that appear later in the test) as the

lowest possible score category in the production of NAEP scale scores. In contrast, not-reached

responses (blank responses that are not followed by any further student responses) are treated as missing

data. As a result of the documentation error, for a number of the partial credit (or polytomous)

constructed-response items and across several subject areas, all blank responses (both omitted and notreached responses) to affected items were treated as missinga reasonable model for treating omits but

one that does not conform to the conventional practice in NAEP.

The error occurred because of a documentation error in the description of one of the PARSCALE

control parameters, designated as POMIT. The program permits the analyst to choose two different ways

of treating blank responses for partial credit items: (a) as missing data, and (b) as incorrect, i.e. a valid

response falling in the lowest score category. The documentation indicates that by setting POMIT= 1,

the treatment in (a) occurs. By setting POMIT=0 or POMIT=1, the treatment in (b) is supposed to occur.

The POMIT=1 setting is the program default. In reality, POMIT=1 and POMIT= 1 operate equivalently,

treating blank responses as missing data.

The error appears to have been introduced in 1992 when the programs BILOG and PARSCALE

were merged to form the ETS version of PARSCALE. Verification of the accuracy of existing

documentation, modifications to internal program diagnostics, and more systematic testing procedures

for any and all changes to NAEP-related programs were implemented immediately to reduce the

likelihood of experiencing this kind of error in subsequent NAEP cycles.

The PARSCALE documentation error affected a number of the NAEP scales constructed since

1992. Specifically, the 1992 national and state mathematics results were affected by the error. Results

from these two assessments have been released to the public in a number of NAEP publications. The data

has also been available to the public through NCES’s secondary-use data files.
814

NCES and ETS felt that the most technically correct plan of action would be to recalculate all

affected NAEP scales, no matter how slight the change, and to issue revised results. ETS was therefore

instructed by NCES to recalculate all affected scales.

In recomputing the cutpoints for the achievement levels, an additional error (the information

weighting error) was discovered in the procedures used by American College Testing (ACT) in 1992 to

“map” the achievement-level cutpoints onto the NAEP scale. The procedures contained an incorrectly

derived formula. Details can be found in Appendix I of the Technical Report of the NAEP 1994 Trial

State Assessment in Reading (Mazzeo, Allen, & Kline, 1995). ACT used revised procedures with the

correct formula to map the achievement-level cutpoints for the 1994 U.S. history and geography scales

and the 1996 science scales. The error in the procedures affected achievement-level cutpoints for the

1990 mathematics national assessment at all grade levels (grades 4, 8, and 12) and the 1990 Trial State

Assessment in mathematics, which was only at grade 8. This error also affected the 1992 mathematics

results for the nation and the states. The information weighting error added a source of error to results in

addition to the error associated problem with defining omits. Note that the proficiency estimates for 1990

mathematics are correct and did not have to be recalculated; only the cutpoints for the achievement

levels were affected. For this reason, the achievement level almanac for 1990 state mathematics is not

included in this appendix.

The information documenting the original analysis of the 1992 data that appears in the Technical

Report of the NAEP 1992 Trial State Assessment Program in Mathematics (Johnson, Mazzeo, & Kline,

1993) is substantially in agreement with the revised 1994 analysis. The transformation constants for the

revised analysis are provided in Table K-1. The information in the other sections of the technical report

for the 1996 state mathematics assessment refer only to the revised analysis of the 1994 Trial State

Assessment data.

Table K-1

Transformation Constants for the 1992 Trial State Assessment in Mathematics

Grade 4 Grade 8

Scale k1 k2 k1 k2

Numbers and Operations 214.59 34.16 268.76 34.60

Measurement 221.40 33.28 262.82 43.95

Geometry 220.55 28.59 260.44 33.81

Data Analysis, Statistics, and Probability 217.80 32.66 264.58 39.93

Algebra and Functions 217.91 29.00 264.23 36.13

Estimation 205.41 35.52 267.14 28.14

As shown by Tables K-3 through K-5, all jurisdictions had average scores that were adjusted

upward slightly as a result of the revision of results. For grade 4, Tables K-3 and K-4 indicate that

average scores increased from .9 to 1.5 points on the proficiency scale. Accordingly, average scores in all

the percentiles go up in a similar fashion. Since all jurisdictions were affected in a similar manner, there

is little change in the ranking of the jurisdictions. With regard to achievement level results, Tables K-7

and K-8 indicate that although the average scale scores moved up when revised, the percent of students
815

at or above the advanced, proficient and basic achievement levels went down slightly as a result of the

two revisions. For the advanced level, percentages for the jurisdictions went down 0 to .8 percent, while

for the below basic level, percentages went up .6 to 1.7 percent. Since the scale scores uniformly moved

up, the shift in achievement level percentages must be due to the shift upward of the achievement level

cutpoints as result of the information weighting error (see Table K-2). Similar results are evident for

grade 8.

Tables K-5 and K-6 indicate that average scores for jurisdictions went up from .6 to 1 point as a

result of the revision of scale scores, with similar slight upward adjustments occurring at every listed

percentile. This change is somewhat less than what was demonstrated for grade 4 and again the ranking

of the jurisdictions did not show much change. Tables K-9 and K-10 list the original and revised eighthgrade results for the achievement-level percentages. These tables also list values for 1990, since the Trial

State Assessment was administered only at grade 8 and not grade 4 that year. The effects of the revision

of the 1992 results are similar to those for grade 4. Although revised means were greater for every

jurisdiction, the percentage above the three achievement levels were slightly smaller while percentages

below the basic achievement level were slightly larger. As with grade 4, this reflects the upward shift of

the achievement-level cutpoints as a result of the information weighting error (see Table K-2).

For grade 8, there is the question of the effect of the revisions on the 1990 to 1992 trend. All of

the 17 jurisdictions that had significant trends in average scale scores in the original analysis also had

significant trends in the revised analysis. In addition, seven jurisdictions that did not have significant

trends in the original analysis had significant trends in revision. A number of changes in the percent of

students at or above the achievement levels occurred due to the combination of the revision of scale

scores and achievement level cutpoints for the 1992 data. As a result, more differences across the two

years were significant. For 15 jurisdictions, there was an increase in the percent of students at or above

an achievement level from 1990 to 1992 that did not change when the analysis was redone. There were

nine jurisdictions where there was a significant trend upward in the percent of students at or above an

achievement level only after revision. For only one jurisdiction was a trend no longer significant after

revision (the percent at or above advanced for Michigan).

Table K-2

Original and Revised ACT Achievement Level Cut Scores for

1992 Mathematics

Basic Proficient Advanced

Grade 4 Original 211 248 280

Revised 214 249 282

Grade 8 Original 256 294 331

Revised 262 299 333

In summary, while changes in average scale score, in percent of students at or above an

achievement level, and in 1990 to 1992 trends in these statistics did occur, the meaning of the results

were constant for most jurisdictions. An exception was that seven jurisdictions had significant

differences in mean scale score from 1990 to 1992 that did not appear in the 1992 state reports. Also, 10

jurisdictions had changes in the percent of students at or above at least one achievement level.
816

Table K-3

NAEP 1992 Trial State Assessment in Mathematics

Grade 4 Weighted Percentages and Composite Scale Means

Weighted Means, Standard Deviations, and Percentiles

Original Results

 MEAN STD DEV 10TH 25TH 50TH 75TH 90TH

 Alabama 206.9( 1.6) 32.0( 0.6) 165.3( 1.7) 184.2( 1.5) 206.8( 2.4) 230.1( 1.7) 248.8( 1.8)

 Arizona 213.8( 1.1) 31.3( 0.5) 171.8( 2.1) 192.6( 1.7) 215.4( 1.3) 235.6( 1.3) 253.3( 2.3)

 Arkansas 208.7( 0.9) 30.9( 0.6) 167.1( 1.3) 187.4( 1.1) 210.0( 1.3) 230.4( 1.0) 247.9( 1.5)

 California 207.1( 1.6) 36.6( 0.8) 157.6( 2.4) 183.0( 2.7) 209.3( 1.3) 232.7( 1.5) 252.9( 1.9)

 Colorado 219.8( 1.0) 31.2( 0.4) 178.8( 1.7) 199.5( 1.0) 221.0( 1.4) 241.0( 1.4) 259.2( 1.4)

 Connecticut 225.8( 1.2) 32.1( 0.7) 183.6( 2.2) 204.6( 1.6) 227.0( 1.7) 248.2( 1.7) 265.8( 1.3)

 Delaware 216.6( 0.8) 32.4( 0.7) 174.6( 1.5) 193.7( 1.0) 216.4( 1.0) 239.4( 1.5) 258.9( 1.4)

 District of Columbia 191.2( 0.5) 32.8( 0.4) 151.7( 0.7) 169.0( 0.7) 189.5( 0.7) 210.9( 0.9) 233.2( 2.2)

 Florida 212.4( 1.5) 32.6( 0.8) 169.6( 2.5) 191.2( 2.0) 213.6( 1.8) 234.5( 1.4) 253.6( 2.6)

 Georgia 214.3( 1.3) 32.8( 0.6) 171.3( 2.0) 191.8( 1.2) 215.2( 1.3) 237.3( 1.7) 256.8( 2.0)

 Hawaii 212.8( 1.3) 34.0( 0.7) 167.5( 1.7) 190.1( 1.6) 214.1( 1.9) 236.8( 1.2) 256.3( 2.1)

 Idaho 220.3( 1.0) 28.1( 0.5) 183.4( 1.4) 201.7( 2.5) 221.9( 0.9) 240.0( 0.7) 255.6( 1.2)

 Indiana 219.7( 1.1) 28.3( 0.5) 183.6( 1.2) 199.9( 1.7) 219.7( 1.2) 239.3( 1.2) 256.2( 1.0)

 Iowa 229.0( 1.1) 29.6( 0.5) 190.5( 2.2) 210.0( 1.1) 230.5( 0.8) 249.4( 0.7) 265.8( 1.1)

 Kentucky 213.6( 1.0) 29.8( 0.6) 175.8( 1.7) 193.3( 1.1) 213.1( 1.1) 233.9( 0.9) 252.6( 1.7)

 Louisiana 202.8( 1.4) 32.2( 0.9) 160.4( 2.3) 181.3( 1.6) 203.1( 2.0) 225.1( 3.2) 244.0( 1.5)

 Maine 230.7( 1.0) 28.3( 0.7) 194.0( 1.9) 212.2( 1.2) 231.9( 1.8) 250.5( 1.0) 265.1( 1.3)

 Maryland 216.1( 1.3) 35.2( 0.8) 169.5( 2.0) 191.5( 2.6) 217.6( 2.2) 241.1( 1.2) 260.5( 1.6)

 Massachusetts 225.5( 1.2) 31.2( 0.7) 184.4( 1.5) 205.3( 1.5) 227.0( 1.3) 247.1( 1.4) 264.0( 1.1)

 Michigan 218.6( 1.8) 32.9( 1.0) 174.3( 3.3) 197.9( 2.3) 221.1( 1.8) 241.8( 1.5) 258.9( 1.6)

 Minnesota 227.5( 0.9) 31.1( 0.6) 185.9( 3.8) 207.7( 1.2) 229.5( 1.0) 249.4( 0.9) 265.8( 1.0)

 Mississippi 200.1( 1.1) 31.8( 0.6) 159.2( 1.7) 178.3( 1.0) 200.2( 1.3) 222.4( 1.2) 240.7( 1.8)

 Missouri 221.0( 1.2) 30.3( 0.7) 181.6( 2.9) 201.2( 1.4) 221.8( 1.5) 242.1( 1.2) 259.7( 1.4)

 Nebraska 224.2( 1.3) 30.8( 0.6) 183.3( 1.9) 203.8( 1.7) 225.8( 1.1) 245.5( 1.4) 262.4( 1.6)

 New Hampshire 228.6( 1.2) 28.7( 0.5) 191.6( 1.6) 210.0( 1.2) 229.2( 1.4) 248.6( 1.3) 264.6( 2.5)

 New Jersey 226.1( 1.5) 31.4( 0.9) 184.6( 2.8) 205.6( 1.9) 227.9( 1.6) 248.3( 1.1) 265.0( 2.3)

 New Mexico 211.8( 1.5) 30.2( 0.7) 172.5( 2.5) 191.1( 1.8) 212.3( 1.0) 232.2( 1.7) 250.7( 2.5)

 New York 217.2( 1.3) 32.8( 0.9) 173.3( 3.2) 195.9( 1.4) 218.5( 1.8) 240.1( 1.7) 258.4( 1.6)

 North Carolina 211.4( 1.1) 33.1( 0.6) 167.7( 1.6) 188.2( 1.4) 212.7( 1.4) 234.6( 1.3) 253.0( 1.2)

 North Dakota 227.6( 0.8) 26.9( 0.6) 192.8( 2.8) 210.2( 1.9) 228.3( 0.8) 246.3( 0.8) 261.0( 1.1)

 Ohio 217.5( 1.2) 31.4( 0.8) 177.2( 2.7) 196.4( 1.5) 218.0( 1.3) 239.1( 2.3) 257.7( 1.4)

 Oklahoma 219.0( 1.0) 27.4( 0.6) 183.7( 1.5) 200.9( 1.2) 219.2( 1.1) 237.2( 1.4) 253.8( 2.2)

 Pennsylvania 223.2( 1.4) 31.4( 0.7) 181.0( 1.8) 202.5( 1.9) 224.8( 1.8) 245.7( 1.6) 262.0( 2.3)

 Rhode Island 214.0( 1.6) 32.0( 0.9) 171.9( 3.1) 193.2( 2.9) 215.7( 1.8) 235.9( 2.0) 253.7( 2.2)

 South Carolina 211.1( 1.1) 31.8( 0.6) 170.7( 1.3) 189.1( 1.2) 210.3( 1.2) 233.3( 1.4) 252.9( 2.4)

 Tennessee 209.4( 1.4) 30.8( 0.6) 169.3( 2.0) 188.3( 1.9) 210.3( 1.7) 230.8( 1.5) 248.5( 1.9)

 Texas 216.6( 1.3) 31.3( 0.8) 176.6( 2.2) 196.3( 1.5) 217.0( 1.6) 237.9( 1.6) 256.5( 2.7)

 Utah 222.8( 1.0) 29.3( 0.6) 184.8( 1.6) 203.7( 0.9) 223.8( 1.3) 242.8( 0.8) 259.8( 0.9)

 Virginia 219.6( 1.3) 32.6( 0.7) 177.5( 1.4) 197.4( 1.6) 219.8( 1.1) 241.9( 1.8) 261.7( 2.8)

 West Virginia 213.9( 1.1) 30.1( 0.6) 174.9( 1.5) 193.9( 1.4) 214.1( 1.2) 234.3( 1.4) 252.2( 1.8)

 Wisconsin 227.7( 1.1) 29.6( 0.7) 188.5( 2.1) 208.7( 1.0) 229.4( 1.2) 248.3( 1.3) 264.2( 1.3)

 Wyoming 224.2( 1.0) 27.2( 0.5) 189.1( 2.1) 206.8( 1.9) 225.1( 1.2) 242.9( 1.3) 258.2( 1.2)

 Guam 191.1( 0.8) 34.3( 0.6) 147.1( 1.8) 167.5( 0.9) 191.4( 1.1) 213.8( 1.4) 235.3( 1.3)

 Virgin Islands 178.0( 1.2) 28.3( 0.7) 140.4( 1.8) 159.2( 2.2) 178.7( 1.3) 197.2( 2.2) 213.8( 2.4)
817

Table K-4

NAEP 1992 Trial State Assessment in Mathematics

Grade 4 Weighted Percentages and Composite Scale Means

Weighted Means, Standard Deviations, and Percentiles

Revised Results

 MEAN STD DEV 10TH 25TH 50TH 75TH 90TH

 Alabama 208.3( 1.6) 31.3( 0.6) 167.7( 1.4) 186.2( 1.8) 208.4( 2.2) 231.1( 1.5) 249.3( 2.4)

 Arizona 215.3( 1.1) 30.3( 0.5) 174.4( 1.8) 194.6( 2.3) 216.9( 1.3) 236.5( 0.9) 253.4( 1.4)

 Arkansas 210.2( 0.9) 30.1( 0.6) 169.7( 1.5) 189.5( 1.1) 211.5( 1.2) 231.4( 1.1) 248.5( 1.5)

 California 208.4( 1.6) 36.0( 0.8) 159.8( 1.9) 184.6( 2.3) 210.7( 1.5) 233.5( 1.3) 253.1( 2.6)

 Colorado 221.0( 1.0) 30.3( 0.4) 181.2( 1.8) 201.4( 1.0) 222.3( 0.9) 241.7( 1.2) 259.1( 0.8)

 Connecticut 226.8( 1.1) 31.0( 0.7) 185.8( 2.2) 206.4( 1.3) 228.3( 1.5) 248.5( 0.9) 265.3( 1.6)

 Delaware 217.9( 0.8) 31.4( 0.7) 177.0( 1.4) 195.7( 1.3) 217.9( 0.8) 240.1( 1.4) 258.8( 1.4)

 District of Columbia 192.6( 0.5) 32.6( 0.4) 153.2( 0.8) 170.7( 1.1) 191.0( 0.6) 211.8( 1.2) 234.4( 1.4)

 Florida 213.7( 1.5) 31.8( 0.7) 171.7( 2.1) 193.0( 1.8) 215.2( 1.5) 235.3( 1.4) 253.8( 3.1)

 Georgia 215.6( 1.2) 31.9( 0.6) 173.7( 1.7) 193.7( 1.4) 216.7( 1.4) 238.2( 1.4) 256.8( 1.2)

 Hawaii 214.1( 1.3) 33.2( 0.6) 170.0( 1.5) 191.8( 2.3) 215.5( 1.3) 237.5( 1.1) 256.5( 1.9)

 Idaho 221.6( 1.0) 27.2( 0.5) 185.6( 1.9) 203.5( 1.9) 223.3( 1.2) 240.7( 0.7) 255.5( 1.1)

 Indiana 221.0( 1.0) 27.4( 0.5) 186.0( 1.3) 201.7( 1.3) 221.2( 0.9) 240.1( 1.2) 256.3( 1.0)

 Iowa 229.9( 1.0) 28.6( 0.5) 192.7( 2.3) 211.8( 1.4) 231.6( 0.9) 249.7( 0.7) 265.4( 1.5)

 Kentucky 215.0( 1.0) 28.9( 0.6) 178.2( 1.5) 195.2( 1.0) 214.7( 1.0) 234.9( 1.0) 252.8( 1.3)

 Louisiana 204.1( 1.5) 31.8( 1.1) 162.9( 2.5) 183.3( 2.0) 204.8( 1.4) 226.4( 2.4) 244.5( 1.4)

 Maine 231.6( 1.0) 27.4( 0.7) 196.2( 1.9) 213.8( 1.3) 232.8( 1.3) 250.8( 1.0) 264.6( 1.2)

 Maryland 217.3( 1.3) 34.3( 0.8) 171.8( 1.7) 193.4( 2.2) 219.0( 1.5) 242.0( 1.7) 260.5( 1.2)

 Massachusetts 226.6( 1.2) 30.2( 0.6) 186.7( 1.2) 207.2( 1.5) 228.3( 1.4) 247.6( 1.6) 263.6( 1.7)

 Michigan 219.9( 1.7) 31.9( 1.0) 176.8( 3.1) 199.9( 2.8) 222.5( 1.7) 242.5( 1.6) 258.7( 1.6)

 Minnesota 228.5( 0.9) 30.0( 0.5) 188.3( 4.1) 209.4( 1.1) 230.6( 0.9) 249.6( 1.1) 265.3( 1.4)

 Mississippi 201.8( 1.1) 31.1( 0.6) 161.7( 1.5) 180.5( 1.4) 202.1( 1.6) 223.8( 1.2) 241.6( 1.2)

 Missouri 222.2( 1.2) 29.3( 0.7) 184.0( 1.7) 203.1( 1.4) 223.1( 1.4) 242.7( 1.1) 259.5( 1.6)

 Nebraska 225.3( 1.2) 29.7( 0.6) 185.7( 1.8) 205.7( 1.8) 227.0( 1.3) 246.0( 1.3) 262.1( 1.8)

 New Hampshire 229.7( 1.2) 27.6( 0.5) 193.8( 1.1) 211.7( 1.2) 230.3( 1.3) 248.9( 1.3) 264.1( 2.3)

 New Jersey 227.1( 1.5) 30.3( 0.9) 186.9( 3.1) 207.4( 1.7) 229.1( 1.5) 248.7( 1.0) 264.6( 1.9)

 New Mexico 213.3( 1.4) 29.3( 0.6) 175.0( 2.9) 193.2( 2.1) 213.9( 1.2) 233.3( 2.1) 250.9( 1.8)

 New York 218.4( 1.2) 32.0( 0.9) 175.7( 2.2) 197.6( 1.0) 220.0( 1.7) 240.8( 1.6) 258.3( 1.0)

 North Carolina 212.9( 1.1) 32.2( 0.6) 170.4( 1.1) 190.4( 1.4) 214.3( 1.3) 235.5( 1.6) 253.0( 1.2)

 North Dakota 228.7( 0.8) 25.9( 0.5) 195.0( 2.4) 212.0( 1.1) 229.4( 0.8) 246.7( 0.8) 260.8( 1.4)

 Ohio 218.7( 1.2) 30.5( 0.8) 179.5( 1.8) 198.2( 1.5) 219.4( 0.9) 239.8( 1.7) 257.4( 1.1)

 Oklahoma 220.3( 1.0) 26.5( 0.6) 186.0( 1.7) 202.8( 0.9) 220.6( 1.2) 238.0( 1.4) 253.9( 1.9)

 Pennsylvania 224.3( 1.3) 30.4( 0.7) 183.4( 1.9) 204.3( 1.5) 226.0( 1.7) 246.1( 1.8) 261.7( 1.8)

 Rhode Island 215.4( 1.5) 31.0( 0.9) 174.4( 2.8) 195.3( 3.0) 217.3( 1.9) 236.8( 1.7) 253.8( 3.0)

 South Carolina 212.5( 1.1) 31.0( 0.6) 173.0( 1.3) 191.2( 1.1) 211.9( 1.2) 234.2( 1.4) 253.1( 1.8)

 Tennessee 210.9( 1.4) 29.9( 0.6) 171.8( 2.3) 190.5( 2.1) 212.0( 1.6) 231.9( 1.4) 248.8( 1.4)

 Texas 217.9( 1.2) 30.3( 0.8) 179.0( 2.4) 198.3( 1.4) 218.5( 1.5) 238.7( 1.8) 256.5( 2.3)

 Utah 224.0( 1.0) 28.3( 0.6) 187.0( 1.6) 205.6( 1.2) 225.1( 1.3) 243.3( 1.0) 259.8( 1.0)

 Virginia 220.8( 1.3) 31.7( 0.7) 179.8( 1.3) 199.3( 1.5) 221.2( 1.4) 242.5( 1.9) 261.3( 2.2)

 West Virginia 215.3( 1.1) 29.3( 0.5) 177.3( 1.4) 195.8( 1.6) 215.6( 1.1) 235.2( 1.4) 252.3( 1.6)

 Wisconsin 228.7( 1.1) 28.5( 0.7) 190.8( 1.9) 210.4( 0.7) 230.5( 1.3) 248.6( 1.2) 263.6( 1.8)

 Wyoming 225.4( 0.9) 26.2( 0.5) 191.3( 1.6) 208.6( 1.4) 226.4( 1.2) 243.5( 0.9) 258.2( 0.9)

 Guam 192.8( 0.8) 33.7( 0.6) 149.6( 1.4) 169.6( 1.2) 193.2( 1.1) 215.4( 1.3) 236.2( 1.7)

 Virgin Islands 178.9( 1.2) 28.8( 0.7) 140.7( 2.3) 159.9( 2.2) 179.8( 1.7) 198.3( 1.2) 215.0( 2.5)
818

Table K-5

NAEP 1992 Trial State Assessment in Mathematics

Grade 8 Weighted Percentages and Composite Scale Means

Weighted Means, Standard Deviations, and Percentiles

Original Results

 MEAN STD DEV 10TH 25TH 50TH 75TH 90TH

 Alabama 251.3( 1.7) 35.8( 1.2) 205.5( 1.9) 226.8( 1.8) 250.6( 2.0) 275.7( 1.7) 298.8( 2.0)

 Arizona 264.6( 1.3) 32.8( 0.7) 222.1( 1.6) 242.8( 1.3) 265.0( 1.9) 286.9( 1.2) 306.5( 1.3)

 Arkansas 255.4( 1.2) 34.3( 0.6) 210.9( 1.6) 232.9( 1.2) 256.2( 1.2) 278.8( 1.6) 298.6( 1.6)

 California 260.1( 1.7) 38.7( 1.1) 208.8( 2.7) 233.7( 2.6) 261.4( 1.8) 287.9( 1.7) 308.8( 2.5)

 Colorado 271.7( 1.1) 33.1( 0.6) 227.6( 1.6) 250.1( 1.2) 273.3( 1.1) 295.0( 1.2) 313.3( 1.2)

 Connecticut 273.1( 1.1) 36.0( 0.9) 224.3( 2.6) 248.8( 1.7) 275.4( 0.8) 299.2( 1.0) 318.4( 1.4)

 Delaware 262.1( 1.0) 35.7( 0.7) 216.4( 1.8) 238.7( 0.9) 262.4( 1.3) 286.6( 1.5) 307.0( 1.4)

 District of Columbia 233.9( 0.9) 36.5( 1.0) 188.6( 1.0) 208.7( 1.2) 233.3( 1.8) 257.2( 2.8) 280.2( 1.7)

 Florida 259.1( 1.5) 36.8( 0.8) 210.3( 3.0) 234.0( 1.6) 260.1( 2.0) 284.8( 1.7) 306.5( 2.0)

 Georgia 258.5( 1.2) 34.6( 0.6) 213.8( 1.5) 234.7( 1.5) 259.4( 1.3) 282.9( 2.1) 303.1( 1.5)

 Hawaii 256.6( 0.9) 37.8( 0.7) 208.2( 1.5) 230.9( 1.0) 257.5( 1.6) 282.8( 1.0) 304.7( 1.3)

 Idaho 274.4( 0.8) 30.5( 0.5) 234.9( 1.1) 254.5( 0.9) 275.2( 1.2) 295.7( 0.8) 313.0( 1.1)

 Indiana 269.4( 1.2) 33.9( 0.6) 225.3( 1.5) 246.8( 1.2) 269.8( 1.3) 292.7( 1.9) 313.1( 2.9)

 Iowa 282.8( 1.0) 30.0( 0.6) 243.9( 2.1) 262.5( 1.4) 283.7( 1.1) 303.9( 1.5) 320.6( 1.6)

 Kentucky 261.4( 1.1) 34.3( 0.6) 216.4( 1.7) 238.5( 1.6) 262.0( 1.0) 284.6( 1.3) 305.4( 2.8)

 Louisiana 249.1( 1.7) 34.0( 0.9) 204.6( 2.6) 226.4( 2.2) 249.7( 1.6) 272.1( 2.0) 293.0( 1.8)

 Maine 278.0( 1.0) 30.7( 0.8) 239.1( 2.3) 258.2( 1.2) 278.6( 1.1) 299.0( 1.7) 316.4( 1.3)

 Maryland 264.2( 1.3) 39.2( 0.9) 212.9( 1.8) 236.9( 2.3) 265.0( 1.3) 292.4( 1.6) 313.5( 1.6)

 Massachusetts 272.1( 1.1) 34.2( 0.7) 228.6( 1.4) 248.6( 2.2) 272.9( 2.0) 296.8( 1.6) 315.7( 1.7)

 Michigan 266.6( 1.4) 35.4( 0.6) 220.0( 1.4) 243.1( 2.3) 268.3( 1.6) 291.7( 2.9) 311.2( 2.3)

 Minnesota 281.8( 1.0) 31.9( 0.5) 240.0( 1.4) 260.2( 1.4) 282.9( 1.3) 304.3( 1.4) 322.3( 1.4)

 Mississippi 245.5( 1.2) 34.8( 0.6) 200.6( 1.2) 221.3( 1.3) 245.4( 1.2) 270.0( 1.6) 290.9( 2.0)

 Missouri 270.4( 1.2) 32.7( 0.7) 227.9( 2.9) 248.5( 1.8) 271.8( 1.4) 292.8( 1.6) 311.5( 1.3)

 Nebraska 277.0( 1.1) 32.4( 0.6) 234.0( 1.7) 256.4( 1.2) 279.1( 1.4) 299.7( 1.0) 316.9( 1.5)

 New Hampshire 277.6( 1.0) 30.4( 0.7) 237.9( 1.1) 257.7( 0.8) 278.0( 0.9) 298.5( 1.1) 315.9( 2.0)

 New Jersey 271.2( 1.6) 36.0( 0.9) 222.2( 1.9) 247.0( 2.0) 272.9( 1.9) 296.8( 2.2) 317.1( 1.6)

 New Mexico 258.8( 0.9) 32.3( 0.7) 217.1( 2.0) 237.4( 0.9) 259.3( 1.0) 280.9( 1.0) 300.2( 1.3)

 New York 265.7( 2.1) 39.4( 1.3) 213.2( 3.1) 240.9( 2.7) 268.3( 1.8) 292.7( 1.4) 314.5( 2.4)

 North Carolina 257.6( 1.2) 35.4( 0.8) 212.3( 2.6) 233.6( 1.3) 258.5( 1.2) 282.0( 1.4) 302.7( 1.5)

 North Dakota 282.6( 1.2) 28.3( 0.6) 244.9( 1.2) 263.7( 1.4) 284.3( 1.0) 302.0( 1.4) 318.0( 1.7)

 Ohio 267.4( 1.5) 34.4( 0.9) 222.0( 1.9) 244.4( 2.0) 269.1( 1.6) 291.6( 1.4) 310.4( 1.5)

 Oklahoma 267.4( 1.2) 32.3( 0.6) 225.9( 1.3) 247.1( 1.4) 268.3( 1.1) 290.1( 1.4) 307.7( 1.5)

 Pennsylvania 270.7( 1.5) 34.5( 0.9) 225.1( 2.3) 247.9( 1.5) 272.2( 1.4) 295.0( 1.1) 314.5( 1.8)

 Rhode Island 265.1( 0.7) 33.4( 0.5) 220.9( 1.2) 242.8( 1.1) 266.7( 1.2) 288.5( 1.7) 306.9( 1.1)

 South Carolina 260.0( 1.0) 35.3( 0.7) 215.2( 1.3) 235.2( 1.1) 259.2( 1.2) 284.6( 1.7) 306.7( 1.5)

 Tennessee 258.0( 1.4) 33.9( 0.7) 214.0( 2.1) 234.9( 1.5) 258.2( 1.6) 282.1( 1.4) 301.6( 1.5)

 Texas 263.8( 1.3) 37.2( 0.7) 215.8( 2.7) 237.5( 1.2) 263.9( 1.9) 289.4( 2.3) 312.0( 1.5)

 Utah 273.6( 0.7) 32.0( 0.8) 231.6( 1.2) 252.8( 1.7) 274.9( 0.8) 296.1( 1.2) 313.7( 1.2)

 Virginia 267.1( 1.2) 35.4( 0.7) 221.4( 1.5) 243.0( 1.7) 267.4( 1.7) 291.4( 1.6) 313.4( 1.5)

 West Virginia 258.2( 1.0) 31.1( 0.6) 217.8( 1.5) 237.0( 1.0) 258.4( 1.7) 280.5( 1.1) 298.2( 1.8)

 Wisconsin 277.3( 1.5) 33.2( 0.7) 233.1( 2.6) 256.7( 2.1) 279.1( 1.5) 300.6( 1.5) 317.8( 1.4)

 Wyoming 274.4( 0.9) 29.6( 0.5) 236.5( 1.0) 254.1( 1.2) 275.2( 1.2) 294.9( 1.2) 312.1( 1.1)

 Guam 234.3( 1.0) 39.2( 0.8) 183.8( 2.1) 207.1( 1.8) 233.4( 1.6) 261.4( 2.5) 286.2( 3.7)

 Virgin Islands 221.8( 1.1) 30.1( 0.6) 183.1( 1.2) 201.2( 1.6) 221.4( 1.2) 242.1( 1.5) 260.2( 1.6)
819

Table K-6

NAEP 1992 Trial State Assessment in Mathematics

Grade 8 Weighted Percentages and Composite Scale Means

Weighted Means, Standard Deviations, and Percentiles

Revised Results

 MEAN STD DEV 10TH 25TH 50TH 75TH 90TH

 Alabama 252.2( 1.7) 35.5( 1.2) 206.8( 1.8) 227.9( 1.9) 251.6( 1.7) 276.5( 1.7) 299.2( 2.3)

 Arizona 265.4( 1.3) 32.4( 0.6) 223.3( 2.3) 243.9( 1.4) 265.9( 1.8) 287.6( 1.1) 306.8( 1.2)

 Arkansas 256.3( 1.2) 33.9( 0.6) 212.1( 1.7) 234.1( 1.4) 257.2( 1.2) 279.5( 1.4) 299.0( 1.2)

 California 260.9( 1.7) 38.3( 1.1) 210.1( 1.8) 234.8( 2.8) 262.3( 1.4) 288.5( 2.2) 309.0( 2.2)

 Colorado 272.4( 1.0) 32.7( 0.6) 228.8( 1.8) 251.1( 1.1) 274.1( 1.2) 295.4( 1.1) 313.5( 1.1)

 Connecticut 273.7( 1.1) 35.6( 0.9) 225.4( 2.6) 249.8( 1.4) 276.1( 0.9) 299.6( 0.8) 318.4( 1.3)

 Delaware 262.9( 1.0) 35.3( 0.7) 217.7( 2.4) 239.7( 1.2) 263.3( 1.2) 287.2( 1.7) 307.3( 1.5)

 District of Columbia 234.9( 0.9) 36.2( 0.9) 189.9( 1.1) 210.0( 1.2) 234.3( 1.9) 258.1( 2.1) 280.8( 1.6)

 Florida 259.9( 1.5) 36.4( 0.8) 211.5( 2.9) 235.1( 1.7) 260.9( 1.9) 285.5( 1.5) 306.7( 1.8)

 Georgia 259.4( 1.2) 34.3( 0.6) 215.0( 1.4) 235.8( 1.2) 260.3( 1.3) 283.6( 2.1) 303.3( 1.3)

 Hawaii 257.4( 0.9) 37.3( 0.6) 209.4( 2.5) 232.0( 0.9) 258.3( 1.4) 283.4( 1.0) 305.0( 1.6)

 Idaho 275.1( 0.7) 30.1( 0.5) 236.1( 1.1) 255.4( 1.0) 276.0( 1.0) 296.1( 0.9) 313.1( 1.1)

 Indiana 270.1( 1.1) 33.5( 0.6) 226.4( 1.4) 247.8( 1.5) 270.6( 1.5) 293.2( 1.5) 313.4( 3.2)

 Iowa 283.4( 1.0) 29.5( 0.6) 245.0( 2.6) 263.4( 1.2) 284.3( 1.3) 304.2( 1.8) 320.6( 1.3)

 Kentucky 262.2( 1.1) 33.9( 0.6) 217.6( 1.8) 239.5( 1.6) 262.9( 1.1) 285.2( 1.4) 305.6( 2.3)

 Louisiana 250.0( 1.7) 33.7( 0.8) 205.8( 2.7) 227.4( 1.9) 250.6( 1.8) 272.9( 1.8) 293.5( 1.5)

 Maine 278.6( 1.0) 30.3( 0.8) 240.3( 2.0) 259.1( 1.1) 279.4( 1.3) 299.4( 1.5) 316.5( 1.7)

 Maryland 264.8( 1.3) 38.9( 0.9) 214.1( 2.6) 237.9( 2.1) 265.9( 0.9) 292.9( 1.2) 313.6( 1.9)

 Massachusetts 272.8( 1.0) 33.8( 0.7) 229.7( 1.4) 249.6( 2.2) 273.7( 1.3) 297.2( 1.6) 315.7( 1.5)

 Michigan 267.4( 1.4) 35.0( 0.6) 221.2( 1.1) 244.1( 2.1) 269.1( 1.4) 292.3( 2.3) 311.4( 2.2)

 Minnesota 282.4( 1.0) 31.5( 0.5) 241.1( 1.7) 261.1( 1.2) 283.6( 1.4) 304.5( 1.3) 322.2( 1.6)

 Mississippi 246.5( 1.2) 34.5( 0.6) 201.8( 1.1) 222.5( 1.1) 246.4( 1.3) 270.8( 1.5) 291.4( 1.3)

 Missouri 271.1( 1.2) 32.3( 0.7) 229.0( 2.3) 249.6( 1.9) 272.6( 1.3) 293.3( 1.7) 311.6( 1.4)

 Nebraska 277.7( 1.1) 32.0( 0.5) 235.2( 1.8) 257.3( 1.2) 279.8( 1.5) 300.1( 1.3) 317.0( 1.2)

 New Hampshire 278.2( 1.0) 30.0( 0.7) 239.0( 1.3) 258.6( 0.7) 278.7( 0.8) 298.9( 1.3) 316.0( 1.7)

 New Jersey 271.9( 1.6) 35.6( 0.9) 223.4( 2.0) 248.0( 2.3) 273.6( 1.8) 297.2( 2.0) 317.0( 2.1)

 New Mexico 259.6( 0.9) 31.9( 0.7) 218.4( 2.7) 238.5( 1.1) 260.1( 0.9) 281.6( 1.1) 300.6( 1.0)

 New York 266.4( 2.1) 39.0( 1.3) 214.6( 3.3) 241.9( 2.5) 269.2( 1.9) 293.3( 1.5) 314.6( 2.0)

 North Carolina 258.4( 1.2) 35.1( 0.8) 213.4( 2.1) 234.6( 1.8) 259.3( 0.9) 282.6( 1.2) 302.9( 2.0)

 North Dakota 283.2( 1.1) 27.9( 0.6) 246.0( 1.3) 264.6( 1.3) 284.9( 1.0) 302.3( 1.5) 317.9( 1.6)

 Ohio 268.1( 1.5) 34.2( 0.9) 223.1( 2.3) 245.4( 1.8) 269.9( 1.5) 292.2( 1.0) 310.6( 1.6)

 Oklahoma 268.1( 1.1) 31.9( 0.6) 227.0( 1.1) 248.1( 1.3) 269.1( 1.1) 290.7( 1.5) 308.1( 1.3)

 Pennsylvania 271.4( 1.5) 34.1( 0.9) 226.2( 2.5) 248.9( 1.3) 273.0( 1.3) 295.4( 1.2) 314.6( 1.9)

 Rhode Island 265.9( 0.7) 33.1( 0.5) 222.0( 0.9) 243.8( 1.2) 267.6( 1.1) 289.1( 1.5) 307.1( 1.1)

 South Carolina 260.8( 1.0) 34.9( 0.7) 216.4( 1.6) 236.3( 1.2) 260.0( 1.3) 285.2( 1.7) 307.0( 1.7)

 Tennessee 258.8( 1.4) 33.5( 0.7) 215.3( 2.2) 236.0( 1.5) 259.1( 1.6) 282.8( 1.8) 301.9( 1.7)

 Texas 264.6( 1.3) 36.8( 0.7) 217.0( 2.0) 238.6( 1.3) 264.8( 1.6) 290.0( 1.9) 312.0( 2.1)

 Utah 274.3( 0.7) 31.6( 0.8) 232.7( 1.3) 253.8( 1.3) 275.7( 0.9) 296.6( 1.2) 313.9( 1.2)

 Virginia 267.9( 1.2) 35.0( 0.6) 222.5( 1.7) 244.0( 1.7) 268.2( 1.7) 291.9( 1.6) 313.5( 1.8)

 West Virginia 259.1( 1.0) 30.8( 0.6) 219.0( 1.8) 238.1( 1.1) 259.4( 1.4) 281.1( 0.9) 298.6( 1.5)

 Wisconsin 277.9( 1.5) 32.7( 0.7) 234.3( 2.3) 257.6( 2.1) 279.8( 1.3) 300.9( 1.4) 317.9( 1.2)

 Wyoming 275.1( 0.9) 29.2( 0.5) 237.6( 1.1) 255.1( 1.1) 276.0( 1.2) 295.4( 1.0) 312.2( 1.3)

 Guam 235.1( 1.0) 39.2( 0.9) 184.5( 1.9) 208.1( 3.3) 234.3( 1.9) 262.4( 1.7) 286.7( 1.8)

 Virgin Islands 222.8( 1.1) 30.0( 0.6) 184.2( 1.7) 202.2( 1.8) 222.4( 1.5) 243.0( 1.6) 260.9( 2.0)
820

Table K-7

NAEP 1992 Trial State Assessment in Mathematics

Grade 4 Weighted Percentages and Composite Scale Means

Percent of Students At or Above the Achievement Levels

Original Results

 N WEIGHTED PCT [CV] MEAN ADVANCED PRFCIENT BASIC < BASIC

 Alabama 2605 1.9( 0.1) [ 3%] 206.9( 1.6) 0.6( 0.2) 10.5( 1.3) 44.7( 2.2) 55.3( 2.2)

 Arizona 2741 1.8( 0.0) [ 2%] 213.8( 1.1) 1.2( 0.3) 13.5( 0.9) 55.2( 1.7) 44.8( 1.7)

 Arkansas 2621 1.2( 0.0) [ 4%] 208.7( 0.9) 0.6( 0.2) 10.0( 0.8) 48.8( 1.3) 51.2( 1.3)

 California 2412 12.3( 0.3) [ 3%] 207.1( 1.6) 1.6( 0.5) 12.7( 1.2) 48.0( 2.0) 52.0( 2.0)

 Colorado 2906 1.7( 0.1) [ 3%] 219.8( 1.0) 2.1( 0.4) 18.2( 1.1) 62.5( 1.4) 37.5( 1.4)

 Connecticut 2600 1.2( 0.0) [ 3%] 225.8( 1.2) 3.7( 0.6) 25.2( 1.4) 68.8( 1.5) 31.2( 1.5)

 Delaware 2040 0.3( 0.0) [ 0%] 216.6( 0.8) 2.3( 0.4) 17.0( 0.8) 56.2( 1.0) 43.8( 1.0)

 District of Columbia 2399 0.2( 0.0) [ 0%] 191.2( 0.5) 1.0( 0.2) 5.5( 0.3) 24.9( 1.0) 75.1( 1.0)

 Florida 2828 5.4( 0.2) [ 3%] 212.4( 1.5) 1.5( 0.4) 13.5( 1.4) 53.4( 2.0) 46.6( 2.0)

 Georgia 2766 3.4( 0.1) [ 4%] 214.3( 1.3) 1.5( 0.4) 15.7( 1.2) 54.9( 1.7) 45.1( 1.7)

 Hawaii 2625 0.5( 0.0) [ 2%] 212.8( 1.3) 1.5( 0.4) 15.2( 1.0) 53.7( 1.8) 46.3( 1.8)

 Idaho 2784 0.6( 0.0) [ 3%] 220.3( 1.0) 1.1( 0.3) 16.1( 1.1) 64.2( 1.7) 35.8( 1.7)

 Indiana 2593 2.7( 0.1) [ 3%] 219.7( 1.1) 1.5( 0.3) 16.2( 1.1) 61.6( 1.6) 38.4( 1.6)

 Iowa 2770 1.3( 0.0) [ 4%] 229.0( 1.1) 3.1( 0.5) 27.0( 1.3) 74.1( 1.4) 25.9( 1.4)

 Kentucky 2703 1.8( 0.1) [ 4%] 213.6( 1.0) 1.4( 0.5) 12.9( 1.1) 52.7( 1.5) 47.3( 1.5)

 Louisiana 2792 2.2( 0.1) [ 3%] 202.8( 1.4) 0.6( 0.2) 7.9( 0.8) 40.5( 2.0) 59.5( 2.0)

 Maine 1898 0.6( 0.0) [ 3%] 230.7( 1.0) 3.0( 0.6) 28.3( 1.5) 76.3( 1.3) 23.7( 1.3)

 Maryland 2844 2.0( 0.1) [ 3%] 216.1( 1.3) 2.8( 0.4) 18.9( 1.2) 56.8( 1.6) 43.2( 1.6)

 Massachusetts 2549 2.1( 0.1) [ 4%] 225.5( 1.2) 3.1( 0.5) 24.0( 1.5) 69.7( 1.6) 30.3( 1.6)

 Michigan 2412 4.1( 0.1) [ 4%] 218.6( 1.8) 1.7( 0.5) 19.1( 1.7) 62.1( 2.2) 37.9( 2.2)

 Minnesota 2640 2.1( 0.1) [ 4%] 227.5( 0.9) 3.4( 0.5) 26.8( 1.2) 71.8( 1.4) 28.2( 1.4)

 Mississippi 2712 1.4( 0.0) [ 3%] 200.1( 1.1) 0.4( 0.1) 6.5( 0.7) 37.3( 1.3) 62.7( 1.3)

 Missouri 2509 2.1( 0.1) [ 5%] 221.0( 1.2) 1.9( 0.3) 19.3( 1.3) 63.6( 1.6) 36.4( 1.6)

 Nebraska 2327 0.7( 0.0) [ 3%] 224.2( 1.3) 2.6( 0.5) 22.5( 1.7) 68.1( 1.8) 31.9( 1.8)

 New Hampshire 2265 0.5( 0.0) [ 3%] 228.6( 1.2) 3.0( 0.6) 25.7( 1.7) 73.8( 1.6) 26.2( 1.6)

 New Jersey 2231 2.8( 0.1) [ 4%] 226.1( 1.5) 3.2( 0.7) 25.3( 1.6) 69.9( 2.1) 30.1( 2.1)

 New Mexico 2342 0.8( 0.1) [ 6%] 211.8( 1.5) 1.0( 0.4) 11.4( 1.3) 51.6( 1.9) 48.4( 1.9)

 New York 2284 6.7( 0.2) [ 3%] 217.2( 1.3) 2.0( 0.3) 17.4( 1.3) 58.8( 1.9) 41.2( 1.9)

 North Carolina 2884 3.0( 0.1) [ 3%] 211.4( 1.1) 1.6( 0.4) 13.2( 0.9) 52.1( 1.6) 47.9( 1.6)

 North Dakota 2193 0.3( 0.0) [ 4%] 227.6( 0.8) 1.8( 0.3) 22.9( 1.1) 74.2( 1.2) 25.8( 1.2)

 Ohio 2637 4.9( 0.1) [ 3%] 217.5( 1.2) 1.9( 0.3) 16.8( 1.1) 58.8( 1.7) 41.2( 1.7)

 Oklahoma 2254 1.6( 0.0) [ 3%] 219.0( 1.0) 1.2( 0.4) 14.4( 1.1) 61.5( 1.6) 38.5( 1.6)

 Pennsylvania 2740 4.6( 0.2) [ 4%] 223.2( 1.4) 2.6( 0.5) 22.5( 1.5) 66.4( 1.9) 33.6( 1.9)

 Rhode Island 2390 0.4( 0.0) [ 5%] 214.0( 1.6) 1.6( 0.4) 13.8( 1.2) 55.8( 2.2) 44.2( 2.2)

 South Carolina 2771 1.8( 0.1) [ 3%] 211.1( 1.1) 1.2( 0.3) 13.3( 1.1) 49.3( 1.5) 50.7( 1.5)

 Tennessee 2708 2.4( 0.1) [ 3%] 209.4( 1.4) 0.7( 0.2) 10.2( 1.0) 49.0( 2.1) 51.0( 2.1)

 Texas 2623 9.0( 0.3) [ 4%] 216.6( 1.3) 1.8( 0.5) 15.5( 1.3) 57.6( 1.7) 42.4( 1.7)

 Utah 2799 1.3( 0.0) [ 2%] 222.8( 1.0) 1.9( 0.3) 19.5( 1.1) 67.4( 1.6) 32.6( 1.6)

 Virginia 2786 2.8( 0.1) [ 3%] 219.6( 1.3) 3.1( 0.7) 19.4( 1.6) 60.3( 1.4) 39.7( 1.4)

 West Virginia 2786 0.9( 0.0) [ 4%] 213.9( 1.1) 1.3( 0.3) 12.8( 1.0) 54.1( 1.6) 45.9( 1.6)

 Wisconsin 2780 2.1( 0.1) [ 4%] 227.7( 1.1) 3.0( 0.5) 25.3( 1.4) 72.4( 1.3) 27.6( 1.3)

 Wyoming 2605 0.3( 0.0) [ 3%] 224.2( 1.0) 1.5( 0.3) 19.4( 1.2) 70.1( 1.4) 29.9( 1.4)

 Guam 1933 0.1( 0.0) [ 0%] 191.1( 0.8) 0.4( 0.1) 4.9( 0.5) 28.0( 1.2) 72.0( 1.2)

 Virgin Islands 905 0.1( 0.0) [ 0%] 178.0( 1.2) 0.0( 0.0) 0.4( 0.2) 11.8( 1.6) 88.2( 1.6)
821

Table K-8

NAEP 1992 Trial State Assessment in Mathematics

Grade 4 Weighted Percentages and Composite Scale Means

Percent of Students At or Above the Achievement Levels

Revised Results

 N WEIGHTED PCT [CV] MEAN ADVANCED PRFCIENT BASIC < BASIC

 Alabama 2605 1.9( 0.1) [ 3%] 208.3( 1.6) 0.5( 0.1) 10.1( 1.2) 43.0( 2.1) 57.0( 2.1)

 Arizona 2741 1.8( 0.0) [ 2%] 215.3( 1.1) 0.8( 0.2) 13.1( 0.9) 53.5( 1.6) 46.5( 1.6)

 Arkansas 2621 1.2( 0.0) [ 4%] 210.2( 0.9) 0.4( 0.2) 9.7( 0.7) 46.9( 1.5) 53.1( 1.5)

 California 2412 12.3( 0.3) [ 3%] 208.4( 1.6) 1.3( 0.4) 12.4( 1.2) 46.4( 1.9) 53.6( 1.9)

 Colorado 2906 1.7( 0.1) [ 3%] 221.0( 1.0) 1.5( 0.4) 17.5( 1.0) 60.8( 1.4) 39.2( 1.4)

 Connecticut 2600 1.2( 0.0) [ 3%] 226.8( 1.1) 2.9( 0.5) 24.4( 1.4) 67.3( 1.6) 32.7( 1.6)

 Delaware 2040 0.3( 0.0) [ 0%] 217.9( 0.8) 1.8( 0.3) 16.5( 0.9) 54.6( 1.0) 45.4( 1.0)

 District of Columbia 2399 0.2( 0.0) [ 0%] 192.6( 0.5) 0.9( 0.2) 5.5( 0.3) 23.1( 0.9) 76.9( 0.9)

 Florida 2828 5.4( 0.2) [ 3%] 213.7( 1.5) 1.2( 0.3) 13.3( 1.4) 51.6( 1.7) 48.4( 1.7)

 Georgia 2766 3.4( 0.1) [ 4%] 215.6( 1.2) 1.1( 0.3) 15.3( 1.2) 53.1( 1.7) 46.9( 1.7)

 Hawaii 2625 0.5( 0.0) [ 2%] 214.1( 1.3) 1.1( 0.2) 14.6( 0.9) 51.9( 1.8) 48.1( 1.8)

 Idaho 2784 0.6( 0.0) [ 3%] 221.6( 1.0) 0.7( 0.3) 15.7( 1.0) 62.7( 1.7) 37.3( 1.7)

 Indiana 2593 2.7( 0.1) [ 3%] 221.0( 1.0) 1.0( 0.2) 15.6( 1.1) 59.8( 1.7) 40.2( 1.7)

 Iowa 2770 1.3( 0.0) [ 4%] 229.9( 1.0) 2.3( 0.4) 26.0( 1.2) 72.4( 1.5) 27.6( 1.5)

 Kentucky 2703 1.8( 0.1) [ 4%] 215.0( 1.0) 1.0( 0.3) 12.6( 1.2) 50.9( 1.5) 49.1( 1.5)

 Louisiana 2792 2.2( 0.1) [ 3%] 204.1( 1.5) 0.4( 0.2) 7.6( 0.8) 38.8( 2.0) 61.2( 2.0)

 Maine 1898 0.6( 0.0) [ 3%] 231.6( 1.0) 2.4( 0.5) 27.4( 1.5) 74.8( 1.5) 25.2( 1.5)

 Maryland 2844 2.0( 0.1) [ 3%] 217.3( 1.3) 2.2( 0.3) 18.4( 1.2) 55.1( 1.6) 44.9( 1.6)

 Massachusetts 2549 2.1( 0.1) [ 4%] 226.6( 1.2) 2.4( 0.5) 23.3( 1.5) 68.5( 1.6) 31.5( 1.6)

 Michigan 2412 4.1( 0.1) [ 4%] 219.9( 1.7) 1.2( 0.4) 18.5( 1.7) 60.5( 2.2) 39.5( 2.2)

 Minnesota 2640 2.1( 0.1) [ 4%] 228.5( 0.9) 2.5( 0.4) 25.9( 1.3) 70.6( 1.6) 29.4( 1.6)

 Mississippi 2712 1.4( 0.0) [ 3%] 201.8( 1.1) 0.3( 0.1) 6.3( 0.6) 35.8( 1.3) 64.2( 1.3)

 Missouri 2509 2.1( 0.1) [ 5%] 222.2( 1.2) 1.3( 0.3) 18.6( 1.3) 62.1( 1.7) 37.9( 1.7)

 Nebraska 2327 0.7( 0.0) [ 3%] 225.3( 1.2) 2.0( 0.5) 21.8( 1.6) 66.6( 1.8) 33.4( 1.8)

 New Hampshire 2265 0.5( 0.0) [ 3%] 229.7( 1.2) 2.1( 0.4) 24.9( 1.6) 72.3( 1.6) 27.7( 1.6)

 New Jersey 2231 2.8( 0.1) [ 4%] 227.1( 1.5) 2.5( 0.6) 24.6( 1.5) 68.2( 2.1) 31.8( 2.1)

 New Mexico 2342 0.8( 0.1) [ 6%] 213.3( 1.4) 0.6( 0.2) 11.1( 1.3) 49.8( 2.0) 50.2( 2.0)

 New York 2284 6.7( 0.2) [ 3%] 218.4( 1.2) 1.5( 0.3) 17.0( 1.3) 57.0( 1.8) 43.0( 1.8)

 North Carolina 2884 3.0( 0.1) [ 3%] 212.9( 1.1) 1.2( 0.3) 12.7( 0.8) 50.3( 1.6) 49.7( 1.6)

 North Dakota 2193 0.3( 0.0) [ 4%] 228.7( 0.8) 1.3( 0.3) 22.2( 1.1) 72.5( 1.3) 27.5( 1.3)

 Ohio 2637 4.9( 0.1) [ 3%] 218.7( 1.2) 1.4( 0.3) 16.1( 1.2) 57.0( 1.7) 43.0( 1.7)

 Oklahoma 2254 1.6( 0.0) [ 3%] 220.3( 1.0) 0.8( 0.3) 14.0( 1.2) 59.5( 1.7) 40.5( 1.7)

 Pennsylvania 2740 4.6( 0.2) [ 4%] 224.3( 1.3) 2.0( 0.4) 21.8( 1.5) 64.9( 2.0) 35.1( 2.0)

 Rhode Island 2390 0.4( 0.0) [ 5%] 215.4( 1.5) 1.2( 0.4) 13.3( 1.1) 54.2( 2.2) 45.8( 2.2)

 South Carolina 2771 1.8( 0.1) [ 3%] 212.5( 1.1) 0.9( 0.3) 12.8( 1.1) 47.6( 1.7) 52.4( 1.7)

 Tennessee 2708 2.4( 0.1) [ 3%] 210.9( 1.4) 0.5( 0.2) 9.9( 1.0) 47.3( 2.0) 52.7( 2.0)

 Texas 2623 9.0( 0.3) [ 4%] 217.9( 1.2) 1.2( 0.3) 15.0( 1.2) 55.7( 1.6) 44.3( 1.6)

 Utah 2799 1.3( 0.0) [ 2%] 224.0( 1.0) 1.4( 0.3) 19.0( 1.1) 65.6( 1.7) 34.4( 1.7)

 Virginia 2786 2.8( 0.1) [ 3%] 220.8( 1.3) 2.3( 0.5) 18.8( 1.5) 58.6( 1.4) 41.4( 1.4)

 West Virginia 2786 0.9( 0.0) [ 4%] 215.3( 1.1) 1.0( 0.3) 12.2( 0.9) 52.4( 1.5) 47.6( 1.5)

 Wisconsin 2780 2.1( 0.1) [ 4%] 228.7( 1.1) 2.2( 0.4) 24.5( 1.4) 70.9( 1.4) 29.1( 1.4)

 Wyoming 2605 0.3( 0.0) [ 3%] 225.4( 0.9) 1.0( 0.3) 18.7( 1.1) 68.6( 1.4) 31.4( 1.4)

 Guam 1933 0.1( 0.0) [ 0%] 192.8( 0.8) 0.3( 0.2) 4.7( 0.5) 26.3( 1.4) 73.7( 1.4)

 Virgin Islands 905 0.1( 0.0) [ 0%] 178.9( 1.2) 0.0( 0.0) 0.5( 0.2) 10.6( 1.4) 89.4( 1.4)
822

Table K-9

NAEP 1992 Trial State Assessment in Mathematics

Grade 8 Weighted Percentages and Composite Scale Means

Percent of Students At or Above the Achievement Levels

Original Results

 N WEIGHTED PCT [CV] MEAN ADVANCED PRFCIENT BASIC < BASIC

 Alabama 1992 2522 2.0( 0.1) [ 3%] 251.3( 1.7) 1.1( 0.3) 12.3( 1.1) 44.3( 2.0) 55.7( 2.0)

 1990 2531 2.2( 0.1) [ 3%] 252.9( 1.1) 1.1( 0.2) 11.7( 0.8) 47.2( 1.6) 52.8( 1.6)

 Arizona 1992 2617 1.7( 0.1) [ 4%] 264.6( 1.3)> 1.8( 0.4) 18.8( 1.4) 61.4( 1.8)> 38.6( 1.8)<

 1990 2558 1.8( 0.1) [ 3%] 259.6( 1.3) 1.5( 0.4) 16.1( 1.1) 54.8( 1.8) 45.2( 1.8)

 Arkansas 1992 2556 1.2( 0.0) [ 2%] 255.4( 1.2) 1.0( 0.3) 12.9( 1.0) 50.2( 1.7) 49.8( 1.7)

 1990 2669 1.5( 0.0) [ 2%] 256.2( 0.9) 0.9( 0.2) 12.4( 1.0) 51.2( 1.3) 48.8( 1.3)

 California 1992 2516 12.5( 0.3) [ 3%] 260.1( 1.7) 2.6( 0.7) 19.7( 1.4) 55.1( 2.0) 44.9( 2.0)

 1990 2424 14.2( 0.4) [ 3%] 256.3( 1.3) 2.0( 0.4) 15.9( 1.3) 50.9( 1.6) 49.1( 1.6)

 Colorado 1992 2799 1.6( 0.0) [ 2%] 271.7( 1.1)> 2.5( 0.5) 26.2( 1.3)> 69.2( 1.3)> 30.8( 1.3)<

 1990 2675 1.7( 0.0) [ 2%] 267.4( 0.9) 2.2( 0.4) 21.6( 1.0) 64.3( 1.1) 35.7( 1.1)

 Connecticut 1992 2613 1.2( 0.0) [ 3%] 273.1( 1.1)> 4.1( 0.6) 30.1( 1.1)> 68.9( 1.4) 31.1( 1.4)

 1990 2672 1.4( 0.0) [ 3%] 269.9( 1.0) 3.9( 0.4) 26.2( 1.1) 65.9( 1.3) 34.1( 1.3)

 Delaware 1992 1934 0.3( 0.0) [ 0%] 262.1( 1.0) 2.5( 0.4) 18.5( 1.1) 57.0( 1.2) 43.0( 1.2)

 1990 2110 0.3( 0.0) [ 0%] 260.7( 0.9) 2.1( 0.5) 18.6( 0.9) 54.5( 1.3) 45.5( 1.3)

 Dist of Columbia 1992 1816 0.2( 0.0) [ 0%] 233.9( 0.9)> 0.7( 0.2) 5.6( 1.0) 26.0( 1.3)> 74.0( 1.3)<

 1990 2135 0.2( 0.0) [ 0%] 231.4( 0.9) 0.8( 0.2) 3.8( 0.7) 21.1( 1.0) 78.9( 1.0)

 Florida 1992 2549 4.6( 0.2) [ 3%] 259.1( 1.5) 1.8( 0.4) 17.6( 1.3) 54.7( 1.9) 45.3( 1.9)

 1990 2534 5.5( 0.2) [ 3%] 255.3( 1.2) 1.7( 0.4) 14.8( 1.0) 49.2( 1.4) 50.8( 1.4)

 Georgia 1992 2589 3.1( 0.1) [ 4%] 258.5( 1.2) 1.4( 0.3) 15.9( 1.0) 53.5( 1.5) 46.5( 1.5)

 1990 2766 3.7( 0.1) [ 4%] 258.8( 1.3) 2.6( 0.5) 17.2( 1.3) 53.4( 1.5) 46.6( 1.5)

 Hawaii 1992 2454 0.5( 0.0) [ 0%] 256.6( 0.9)> 2.2( 0.4) 16.5( 0.8) 51.2( 1.2)> 48.8( 1.2)<

 1990 2551 0.5( 0.0) [ 0%] 251.0( 0.8) 1.8( 0.3) 14.3( 0.8) 45.3( 1.0) 54.7( 1.0)

 Idaho 1992 2615 0.7( 0.0) [ 2%] 274.4( 0.8)> 2.5( 0.4) 26.9( 1.2) 73.4( 1.1) 26.6( 1.1)

 1990 2716 0.8( 0.0) [ 1%] 271.4( 0.8) 1.5( 0.4) 23.0( 1.4) 70.1( 1.2) 29.9( 1.2)

 Indiana 1992 2659 3.0( 0.1) [ 3%] 269.4( 1.2) 3.0( 0.4) 23.8( 1.3) 65.8( 1.5) 34.2( 1.5)

 1990 2569 3.2( 0.1) [ 3%] 267.3( 1.2) 2.9( 0.6) 20.6( 1.2) 63.2( 1.6) 36.8( 1.6)

 Iowa 1992 2816 1.4( 0.1) [ 4%] 282.8( 1.0)> 4.7( 0.7) 37.2( 1.4)> 81.3( 1.2)> 18.7( 1.2)<

 1990 2474 1.5( 0.1) [ 4%] 278.0( 1.1) 3.8( 0.5) 30.4( 1.5) 76.3( 1.1) 23.7( 1.1)

 Kentucky 1992 2756 1.8( 0.1) [ 3%] 261.4( 1.1)> 1.9( 0.4) 16.7( 1.1) 57.3( 1.3)> 42.7( 1.3)<

 1990 2680 2.1( 0.1) [ 4%] 257.1( 1.2) 1.2( 0.2) 13.5( 0.9) 50.5( 1.8) 49.5( 1.8)

 Louisiana 1992 2582 1.9( 0.1) [ 4%] 249.1( 1.7) 0.5( 0.2) 9.6( 1.2) 42.4( 2.0) 57.6( 2.0)

 1990 2572 2.2( 0.1) [ 4%] 246.4( 1.2) 0.6( 0.2) 7.6( 1.0) 38.6( 1.7) 61.4( 1.7)

 Maine 1992 2464 0.6( 0.0) [ 2%] 278.0( 1.0) 3.7( 0.6) 30.8( 1.9) 77.5( 1.3) 22.5( 1.3)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Maryland 1992 2399 1.9( 0.1) [ 3%] 264.2( 1.3) 3.7( 0.6) 23.8( 1.3) 58.7( 1.5) 41.3( 1.5)

 1990 2794 2.2( 0.0) [ 2%] 260.8( 1.4) 3.1( 0.6) 19.9( 1.2) 55.8( 1.7) 44.2( 1.7)

 Massachusetts 1992 2456 2.1( 0.1) [ 4%] 272.1( 1.1) 3.3( 0.5) 27.9( 1.4) 67.9( 1.5) 32.1( 1.5)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Michigan 1992 2616 4.2( 0.1) [ 2%] 266.6( 1.4) 2.6( 0.5) 23.1( 1.7) 63.2( 1.6) 36.8( 1.6)

 1990 2587 4.8( 0.1) [ 3%] 264.4( 1.2) 2.4( 0.4) 19.7( 1.4) 60.3( 1.4) 39.7( 1.4)

 Minnesota 1992 2471 2.0( 0.1) [ 2%] 281.8( 1.0)> 5.6( 0.7)> 36.7( 1.2)> 78.5( 1.2)> 21.5( 1.2)<

 1990 2584 2.4( 0.1) [ 4%] 275.4( 0.9) 3.7( 0.4) 28.7( 1.2) 73.9( 1.3) 26.1( 1.3)

 Mississippi 1992 2498 1.4( 0.0) [ 3%] 245.5( 1.2) 0.5( 0.2) 8.5( 0.8) 38.3( 1.5) 61.7( 1.5)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Missouri 1992 2666 2.2( 0.0) [ 2%] 270.4( 1.2) 2.7( 0.4) 23.7( 1.3) 67.9( 1.6) 32.1( 1.6)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

1⁄2 INDICATES A SIGNIFICANT INCREASE (OR DECREASE "<") BETWEEN 1990 AND 1992

(continued)
823

Table K-9 (continued)

NAEP 1992 Trial State Assessment in Mathematics

Grade 8 Weighted Percentages and Composite Scale Means

Percent of Students At or Above the Achievement Levels

Original Results

 N WEIGHTED PCT [CV] MEAN ADVANCED PRFCIENT BASIC < BASIC

 Nebraska 1992 2285 0.8( 0.0) [ 3%] 277.0( 1.1) 3.6( 0.5) 31.7( 1.9) 75.4( 1.2) 24.6( 1.2)

 1990 2519 0.9( 0.0) [ 2%] 275.7( 1.0) 3.6( 0.6) 29.9( 1.4) 74.3( 1.1) 25.7( 1.1)

 New Hampshire 1992 2535 0.5( 0.0) [ 2%] 277.6( 1.0)> 3.5( 0.6) 30.0( 1.5)> 76.8( 1.0)> 23.2( 1.0)<

 1990 2568 0.5( 0.0) [ 1%] 273.1( 0.9) 3.2( 0.5) 25.0( 1.2) 71.3( 1.6) 28.7( 1.6)

 New Kersey 1992 2174 3.2( 0.1) [ 4%] 271.2( 1.6) 3.9( 0.6) 27.6( 1.4) 67.3( 1.8) 32.7( 1.8)

 1990 2710 3.5( 0.1) [ 4%] 269.7( 1.1) 3.9( 0.5) 25.3( 1.3) 64.5( 1.6) 35.5( 1.6)

 New Mexico 1992 2561 0.8( 0.0) [ 1%] 258.8( 0.9)> 1.1( 0.3) 13.8( 1.0) 54.1( 1.4) 45.9( 1.4)

 1990 2643 0.8( 0.0) [ 1%] 256.4( 0.7) 1.2( 0.3) 12.8( 0.9) 50.7( 1.3) 49.3( 1.3)

 New York 1992 2158 6.5( 0.2) [ 3%] 265.7( 2.1) 3.7( 0.6) 23.8( 1.6)> 62.5( 2.3) 37.5( 2.3)

 1990 2302 7.7( 0.2) [ 2%] 260.8( 1.4) 3.1( 0.5) 18.8( 1.0) 56.5( 1.7) 43.5( 1.7)

 North Carolina 1992 2769 3.2( 0.1) [ 3%] 257.6( 1.2)> 1.5( 0.3) 15.2( 1.0)> 52.8( 1.5)> 47.2( 1.5)<

 1990 2843 3.6( 0.1) [ 2%] 250.3( 1.1) 0.8( 0.4) 11.3( 0.8) 44.4( 1.4) 55.6( 1.4)

 North Dakota 1992 2314 0.3( 0.0) [ 3%] 282.6( 1.2) 3.8( 0.6) 35.7( 1.7) 82.4( 1.3) 17.6( 1.3)

 1990 2485 0.4( 0.0) [ 4%] 281.1( 1.2) 3.9( 0.6) 33.8( 2.0) 81.0( 1.6) 19.0( 1.6)

 Ohio 1992 2535 5.4( 0.2) [ 4%] 267.4( 1.5) 2.4( 0.5) 22.4( 1.4) 64.3( 2.0) 35.7( 2.0)

 1990 2673 5.6( 0.1) [ 2%] 264.0( 1.0) 2.0( 0.3) 18.5( 1.2) 59.6( 1.4) 40.4( 1.4)

 Oklahoma 1992 2141 1.5( 0.0) [ 3%] 267.4( 1.2)> 1.5( 0.3) 21.1( 1.2)> 65.2( 2.0) 34.8( 2.0)

 1990 2222 1.8( 0.0) [ 2%] 263.2( 1.3) 1.6( 0.5) 16.9( 1.3) 59.2( 1.6) 40.8( 1.6)

 Pennsylvania 1992 2612 4.5( 0.1) [ 3%] 270.7( 1.5) 3.2( 0.7) 25.9( 1.5) 67.2( 1.7) 32.8( 1.7)

 1990 2528 5.4( 0.2) [ 3%] 266.4( 1.6) 2.3( 0.4) 21.4( 1.5) 62.9( 2.0) 37.1( 2.0)

 Rhode Island 1992 2120 0.4( 0.0) [ 0%] 265.1( 0.7)> 1.8( 0.3) 19.7( 1.3) 62.0( 1.2)> 38.0( 1.2)<

 1990 2675 0.4( 0.0) [ 2%] 260.0( 0.6) 1.8( 0.3) 18.3( 1.0) 54.9( 0.9) 45.1( 0.9)

 South Carolina 1992 2625 1.8( 0.0) [ 2%] 260.0( 1.0) 2.1( 0.5) 18.0( 1.1) 53.4( 1.2) 46.6( 1.2)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Tennessee 1992 2485 2.3( 0.1) [ 3%] 258.0( 1.4) 1.2( 0.4) 14.7( 1.2) 52.6( 1.8) 47.4( 1.8)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Texas 1992 2614 8.8( 0.3) [ 3%] 263.8( 1.3)> 3.6( 0.6) 21.2( 1.4)> 57.8( 1.5)> 42.2( 1.5)<

 1990 2542 10.2( 0.3) [ 3%] 258.2( 1.4) 2.0( 0.4) 15.9( 1.0) 52.0( 1.7) 48.0( 1.7)

 Utah 1992 2726 1.2( 0.0) [ 3%] 273.6( 0.7) 2.7( 0.5) 27.2( 1.1) 72.1( 1.3) 27.9( 1.3)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Virginia 1992 2710 2.8( 0.1) [ 4%] 267.1( 1.2) 3.2( 0.5) 22.9( 1.2) 62.3( 1.6) 37.7( 1.6)

 1990 2661 3.0( 0.1) [ 3%] 264.3( 1.5) 4.1( 0.8) 20.7( 1.6) 57.7( 1.6) 42.3( 1.6)

 West Virginia 1992 2690 0.9( 0.0) [ 3%] 258.2( 1.0) 0.7( 0.2) 12.6( 0.9) 52.9( 1.5) 47.1( 1.5)

 1990 2600 1.1( 0.0) [ 3%] 255.9( 1.0) 1.1( 0.2) 12.1( 0.9) 49.3( 1.2) 50.7( 1.2)

 Wisconsin 1992 2814 2.3( 0.1) [ 6%] 277.3( 1.5) 3.9( 0.6) 32.1( 1.4) 75.6( 1.9) 24.4( 1.9)

 1990 2750 2.4( 0.1) [ 3%] 274.5( 1.3) 3.6( 0.5) 28.8( 1.5) 72.3( 1.7) 27.7( 1.7)

 Wyoming 1992 2444 0.3( 0.0) [ 2%] 274.4( 0.9)> 2.4( 0.5) 25.8( 1.0) 73.0( 1.3) 27.0( 1.3)

 1990 2701 0.3( 0.0) [ 1%] 272.1( 0.7) 2.0( 0.3) 23.6( 1.0) 71.1( 1.3) 28.9( 1.3)

 Guam 1992 1496 0.1( 0.0) [ 1%] 234.3( 1.0)> 0.6( 0.2) 7.0( 0.7) 29.5( 1.4) 70.5( 1.4)

 1990 1617 0.1( 0.0) [ 0%] 231.8( 0.7) 0.5( 0.2) 5.2( 0.6) 26.7( 1.0) 73.3( 1.0)

 Virgin Islands 1992 1479 0.1( 0.0) [ 0%] 221.8( 1.1)> 0.0( 0.1) 0.9( 0.3) 12.7( 1.0) 87.3( 1.0)

 1990 1326 0.1( 0.0) [ 0%] 218.7( 0.9) 0.1( 0.1) 0.9( 0.4) 10.4( 1.1) 89.6( 1.1)

 > INDICATES A SIGNIFICANT INCREASE (OR DECREASE "<") BETWEEN 1990 AND 1992
824

Table K-10

NAEP 1992 Trial State Assessment in Mathematics

Grade 8 Weighted Percentages and Composite Scale Means

Percent of Students At or Above the Achievement Levels

Revised Results

 N WEIGHTED PCT [CV] MEAN ADVANCED PRFCIENT BASIC < BASIC

 Alabama 1992 2522 2.0( 0.1) [ 3%] 252.2( 1.7) 0.9( 0.3) 10.2( 0.9) 38.8( 1.9) 61.2( 1.9)

 1990 2531 2.2( 0.1) [ 3%] 252.9( 1.1) 1.0( 0.2) 9.1( 0.7) 40.3( 1.7) 59.7( 1.7)

 Arizona 1992 2617 1.7( 0.1) [ 4%] 265.4( 1.3)> 1.4( 0.3) 15.1( 1.3) 54.7( 1.8)> 45.3( 1.8)<

 1990 2558 1.8( 0.1) [ 3%] 259.6( 1.3) 1.3( 0.4) 12.7( 0.9) 47.5( 1.8) 52.5( 1.8)

 Arkansas 1992 2556 1.2( 0.0) [ 2%] 256.3( 1.2) 0.7( 0.2) 10.0( 0.8) 44.3( 1.8) 55.7( 1.8)

 1990 2669 1.5( 0.0) [ 2%] 256.2( 0.9) 0.7( 0.2) 9.3( 0.7) 43.9( 1.2) 56.1( 1.2)

 California 1992 2516 12.5( 0.3) [ 3%] 260.9( 1.7)> 2.2( 0.7) 16.2( 1.3) 50.4( 1.9) 49.6( 1.9)

 1990 2424 14.2( 0.4) [ 3%] 256.3( 1.3) 1.7( 0.3) 12.5( 1.1) 44.6( 1.7) 55.4( 1.7)

 Colorado 1992 2799 1.6( 0.0) [ 2%] 272.4( 1.0)> 2.1( 0.4) 21.6( 1.2)> 63.9( 1.4)> 36.1( 1.4)<

 1990 2675 1.7( 0.0) [ 2%] 267.4( 0.9) 1.9( 0.4) 16.9( 1.0) 57.5( 1.2) 42.5( 1.2)

 Connecticut 1992 2613 1.2( 0.0) [ 3%] 273.7( 1.1)> 3.2( 0.6) 25.7( 1.1)> 64.4( 1.4) 35.6( 1.4)

 1990 2672 1.4( 0.0) [ 3%] 269.9( 1.0) 3.4( 0.4) 21.7( 0.9) 59.9( 1.4) 40.1( 1.4)

 Delaware 1992 1934 0.3( 0.0) [ 0%] 262.9( 1.0) 2.3( 0.4) 15.1( 1.0) 51.6( 1.2) 48.4( 1.2)

 1990 2110 0.3( 0.0) [ 0%] 260.7( 0.9) 1.9( 0.4) 14.2( 0.8) 47.8( 1.5) 52.2( 1.5)

 Dist of Columbia 1992 1816 0.2( 0.0) [ 0%] 234.9( 0.9)> 0.6( 0.2) 4.4( 0.9) 21.8( 1.1)> 78.2( 1.1)<

 1990 2135 0.2( 0.0) [ 0%] 231.4( 0.9) 0.8( 0.2) 3.1( 0.6) 16.6( 1.0) 83.4( 1.0)

 Florida 1992 2549 4.6( 0.2) [ 3%] 259.9( 1.5)> 1.5( 0.3) 14.6( 1.2) 48.8( 1.9)> 51.2( 1.9)<

 1990 2534 5.5( 0.2) [ 3%] 255.3( 1.2) 1.4( 0.3) 12.0( 0.9) 42.7( 1.4) 57.3( 1.4)

 Georgia 1992 2589 3.1( 0.1) [ 4%] 259.4( 1.2) 1.1( 0.3)< 12.7( 0.9) 47.9( 1.7) 52.1( 1.7)

 1990 2766 3.7( 0.1) [ 4%] 258.8( 1.3) 2.4( 0.4) 13.8( 1.2) 47.2( 1.5) 52.8( 1.5)

 Hawaii 1992 2454 0.5( 0.0) [ 0%] 257.4( 0.9)> 1.8( 0.3) 13.5( 0.7) 46.2( 1.1)> 53.8( 1.1)<

 1990 2551 0.5( 0.0) [ 0%] 251.0( 0.8) 1.6( 0.3) 11.7( 0.7) 40.0( 1.0) 60.0( 1.0)

 Idaho 1992 2615 0.7( 0.0) [ 2%] 275.1( 0.7)> 2.1( 0.3) 21.7( 1.2) 67.9( 1.0)> 32.1( 1.0)<

 1990 2716 0.8( 0.0) [ 1%] 271.4( 0.8) 1.2( 0.3) 17.9( 1.1) 63.4( 1.2) 36.6( 1.2)

 Indiana 1992 2659 3.0( 0.1) [ 3%] 270.1( 1.1) 2.6( 0.4) 19.7( 1.2) 59.9( 1.5) 40.1( 1.5)

 1990 2569 3.2( 0.1) [ 3%] 267.3( 1.2) 2.5( 0.5) 16.6( 1.1) 56.4( 1.5) 43.6( 1.5)

 Iowa 1992 2816 1.4( 0.1) [ 4%] 283.4( 1.0)> 3.9( 0.7) 31.2( 1.3)> 76.4( 1.3)> 23.6( 1.3)<

 1990 2474 1.5( 0.1) [ 4%] 278.0( 1.1) 3.3( 0.5) 25.1( 1.4) 70.0( 1.2) 30.0( 1.2)

 Kentucky 1992 2756 1.8( 0.1) [ 3%] 262.2( 1.1)> 1.6( 0.3) 13.8( 1.1)> 51.2( 1.5)> 48.8( 1.5)<

 1990 2680 2.1( 0.1) [ 4%] 257.1( 1.2) 1.1( 0.3) 10.5( 0.8) 43.0( 1.7) 57.0( 1.7)

 Louisiana 1992 2582 1.9( 0.1) [ 4%] 250.0( 1.7) 0.5( 0.2) 7.2( 1.0) 36.6( 1.9) 63.4( 1.9)

 1990 2572 2.2( 0.1) [ 4%] 246.4( 1.2) 0.5( 0.2) 5.4( 0.6) 31.7( 1.6) 68.3( 1.6)

 Maine 1992 2464 0.6( 0.0) [ 2%] 278.6( 1.0) 3.1( 0.6) 25.5( 1.5) 71.6( 1.3) 28.4( 1.3)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Maryland 1992 2399 1.9( 0.1) [ 3%] 264.8( 1.3)> 3.2( 0.5) 19.9( 1.2) 53.9( 1.4) 46.1( 1.4)

 1990 2794 2.2( 0.0) [ 2%] 260.8( 1.4) 2.6( 0.5) 16.6( 1.2) 49.7( 1.6) 50.3( 1.6)

 Massachusetts 1992 2456 2.1( 0.1) [ 4%] 272.8( 1.0) 2.8( 0.5) 23.3( 1.3) 62.8( 1.5) 37.2( 1.5)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Michigan 1992 2616 4.2( 0.1) [ 2%] 267.4( 1.4) 2.2( 0.4) 18.9( 1.5) 57.9( 1.7) 42.1( 1.7)

 1990 2587 4.8( 0.1) [ 3%] 264.4( 1.2) 2.1( 0.4) 15.8( 1.2) 53.3( 1.7) 46.7( 1.7)

 Minnesota 1992 2471 2.0( 0.1) [ 2%] 282.4( 1.0)> 4.7( 0.6) 31.1( 1.2)> 74.2( 1.3)> 25.8( 1.3)<

 1990 2584 2.4( 0.1) [ 4%] 275.4( 0.9) 3.3( 0.5) 23.3( 1.2) 67.5( 1.1) 32.5( 1.1)

 Mississippi 1992 2498 1.4( 0.0) [ 3%] 246.5( 1.2) 0.3( 0.1) 6.4( 0.7) 33.4( 1.6) 66.6( 1.6)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Missouri 1992 2666 2.2( 0.0) [ 2%] 271.1( 1.2) 2.3( 0.4) 19.5( 1.2) 62.5( 1.6) 37.5( 1.6)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 > INDICATES A SIGNIFICANT INCREASE (OR DECREASE "<") BETWEEN 1990 AND 1992

(continued)
825

Table K-10 (continued)

NAEP 1992 Trial State Assessment in Mathematics

Grade 8 Weighted Percentages and Composite Scale Means

Percent of Students At or Above the Achievement Levels

Revised Results

 N WEIGHTED PCT [CV] MEAN ADVANCED PRFCIENT BASIC < BASIC

 Nebraska 1992 2285 0.8( 0.0) [ 3%] 277.7( 1.1) 2.9( 0.5) 26.3( 1.6) 70.2( 1.3) 29.8( 1.3)

 1990 2519 0.9( 0.0) [ 2%] 275.7( 1.0) 3.3( 0.5) 24.4( 1.2) 68.2( 1.3) 31.8( 1.3)

 New Hampshire 1992 2535 0.5( 0.0) [ 2%] 278.2( 1.0)> 2.8( 0.5) 24.9( 1.4)> 71.2( 1.3)> 28.8( 1.3)<

 1990 2568 0.5( 0.0) [ 1%] 273.1( 0.9) 2.7( 0.5) 20.3( 1.2) 64.7( 1.5) 35.3( 1.5)

 New Jersey 1992 2174 3.2( 0.1) [ 4%] 271.9( 1.6) 3.2( 0.4) 23.7( 1.3) 62.5( 1.9) 37.5( 1.9)

 1990 2710 3.5( 0.1) [ 4%] 269.7( 1.1) 3.4( 0.5) 21.3( 1.1) 58.2( 1.5) 41.8( 1.5)

 New Mexico 1992 2561 0.8( 0.0) [ 1%] 259.6( 0.9)> 0.9( 0.3) 10.9( 0.8) 47.6( 1.3)> 52.4( 1.3)<

 1990 2643 0.8( 0.0) [ 1%] 256.4( 0.7) 1.0( 0.3) 10.2( 0.9) 43.2( 1.2) 56.8( 1.2)

 New York 1992 2158 6.5( 0.2) [ 3%] 266.4( 2.1)> 3.2( 0.5) 20.0( 1.3)> 57.5( 2.2)> 42.5( 2.2)<

 1990 2302 7.7( 0.2) [ 2%] 260.8( 1.4) 2.7( 0.4) 15.3( 0.9) 49.9( 1.7) 50.1( 1.7)

 North Carolina 1992 2769 3.2( 0.1) [ 3%] 258.4( 1.2)> 1.2( 0.3) 12.1( 1.0)> 47.0( 1.4)> 53.0( 1.4)<

 1990 2843 3.6( 0.1) [ 2%] 250.3( 1.1) 0.6( 0.3) 8.7( 0.7) 37.9( 1.4) 62.1( 1.4)

 North Dakota 1992 2314 0.3( 0.0) [ 3%] 283.2( 1.1) 3.1( 0.5) 29.5( 1.6) 77.9( 1.4) 22.1( 1.4)

 1990 2485 0.4( 0.0) [ 4%] 281.1( 1.2) 3.6( 0.6) 27.3( 1.8) 75.4( 1.6) 24.6( 1.6)

 Ohio 1992 2535 5.4( 0.2) [ 4%] 268.1( 1.5)> 1.9( 0.4) 18.1( 1.3) 58.8( 2.1) 41.2( 2.1)

 1990 2673 5.6( 0.1) [ 2%] 264.0( 1.0) 1.7( 0.3) 14.8( 1.1) 52.6( 1.6) 47.4( 1.6)

 Oklahoma 1992 2141 1.5( 0.0) [ 3%] 268.1( 1.1)> 1.1( 0.3) 16.9( 1.1) 59.4( 1.6)> 40.6( 1.6)<

 1990 2222 1.8( 0.0) [ 2%] 263.2( 1.3) 1.3( 0.4) 13.3( 1.2) 52.0( 1.8) 48.0( 1.8)

 Pennsylvania 1992 2612 4.5( 0.1) [ 3%] 271.4( 1.5)> 2.7( 0.5) 21.5( 1.5) 62.1( 1.7) 37.9( 1.7)

 1990 2528 5.4( 0.2) [ 3%] 266.4( 1.6) 2.0( 0.4) 17.2( 1.3) 56.0( 2.0) 44.0( 2.0)

 Rhode Island 1992 2120 0.4( 0.0) [ 0%] 265.9( 0.7)> 1.4( 0.3) 15.7( 1.1) 56.4( 1.2)> 43.6( 1.2)<

 1990 2675 0.4( 0.0) [ 2%] 260.0( 0.6) 1.6( 0.3) 14.5( 0.7) 48.8( 1.0) 51.2( 1.0)

 South Carolina 1992 2625 1.8( 0.0) [ 2%] 260.8( 1.0) 1.7( 0.5) 14.9( 1.0) 47.8( 1.3) 52.2( 1.3)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Tennessee 1992 2485 2.3( 0.1) [ 3%] 258.8( 1.4) 1.0( 0.4) 11.6( 1.0) 46.8( 1.9) 53.2( 1.9)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Texas 1992 2614 8.8( 0.3) [ 3%] 264.6( 1.3)> 3.2( 0.6) 18.1( 1.2)> 52.7( 1.5)> 47.3( 1.5)<

 1990 2542 10.2( 0.3) [ 3%] 258.2( 1.4) 1.6( 0.3) 12.7( 1.1) 45.5( 1.6) 54.5( 1.6)

 Utah 1992 2726 1.2( 0.0) [ 3%] 274.3( 0.7) 2.3( 0.4) 22.3( 1.0) 66.8( 1.2) 33.2( 1.2)

 1990 0 0.0( 0.0) [ 0%] *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0) *****( 0.0)

 Virginia 1992 2710 2.8( 0.1) [ 4%] 267.9( 1.2) 2.7( 0.6) 19.2( 1.1) 56.7( 1.7) 43.3( 1.7)

 1990 2661 3.0( 0.1) [ 3%] 264.3( 1.5) 3.7( 0.8) 17.3( 1.6) 51.6( 1.7) 48.4( 1.7)

 West Virginia 1992 2690 0.9( 0.0) [ 3%] 259.1( 1.0)> 0.6( 0.2) 9.8( 0.8) 46.8( 1.6)> 53.2( 1.6)<

 1990 2600 1.1( 0.0) [ 3%] 255.9( 1.0) 0.9( 0.2) 9.3( 0.8) 41.8( 1.1) 58.2( 1.1)

 Wisconsin 1992 2814 2.3( 0.1) [ 6%] 277.9( 1.5) 3.2( 0.6) 27.0( 1.4) 70.8( 2.1) 29.2( 2.1)

 1990 2750 2.4( 0.1) [ 3%] 274.5( 1.3) 3.1( 0.4) 23.2( 1.4) 65.7( 1.6) 34.3( 1.6)

 Wyoming 1992 2444 0.3( 0.0) [ 2%] 275.1( 0.9)> 1.9( 0.4) 21.0( 1.1) 67.2( 1.3) 32.8( 1.3)

 1990 2701 0.3( 0.0) [ 1%] 272.1( 0.7) 1.7( 0.2) 18.5( 0.9) 63.7( 1.3) 36.3( 1.3)

 Guam 1992 1496 0.1( 0.0) [ 1%] 235.1( 1.0)> 0.5( 0.1) 5.6( 0.6)> 25.3( 1.4) 74.7( 1.4)

 1990 1617 0.1( 0.0) [ 0%] 231.8( 0.7) 0.4( 0.2) 3.8( 0.4) 21.7( 1.0) 78.3( 1.0)

 Virgin Islands 1992 1479 0.1( 0.0) [ 0%] 222.8( 1.1)> 0.0( 0.1) 0.6( 0.3) 9.4( 0.9) 90.6( 0.9)

 1990 1326 0.1( 0.0) [ 0%] 218.7( 0.9) 0.1( 0.1) 0.7( 0.3) 7.6( 1.0) 92.4( 1.0)

 > INDICATES A SIGNIFICANT INCREASE (OR DECREASE "<") BETWEEN 1990 AND 1992
826
827

REFERENCES CITED IN TEXT

Abedi, J. (1996). The Interrater/Test Reliability System (ITRS). Multivariate Behavioral

Research, 31(4), 409-417.

Allen, N. L. (1992). Data analysis for the science assessment. In E. G. Johnson & N. L. Allen, The

NAEP 1990 technical report (No. 21-TR-20). Princeton, NJ: Educational Testing Service, National

Assessment of Educational Progress.

Allen, N. L., & Donoghue, J. R. (1994, April). Differential item functioning based on complex

samples of dichotomous and polytomous items. Paper presented at the annual meeting of the American

Educational Research Association, New Orleans, LA.

Allen, N. L., & Donoghue, J. R. (in press). Applying the Mantel-Haenszel procedure to complex

samples of items. Journal of Educational Measurement.

Allen, N. L., & Isham, S. P. (1994). Data analysis for the science long-term trend assessment. In E.

G. Johnson & J. E. Carlson, The NAEP 1992 technical report. Washington, DC: National Center for

Education Statistics.

Allen, N. L., Jenkins, F., Kulick, E., & Zelenak, C. A. (1997). Technical report of the NAEP 1996

state assessment program in mathematics (Publication No. NCES 97-951). Washington, DC: National

Center for Education Statistics.

Allen, N. L., Kline, D. L., & Zelenak, C. A. (1996). The NAEP 1994 technical report (Publication

No. NCES 97-897). Washington, DC: National Center for Education Statistics.

Andersen, E. B. (1980). Comparing latent distributions. Psychometrika, 45, 121-134.

Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.),

Educational measurement (2nd ed., pp. 508-600). Washington, DC: American Council on Education.

Ballator, N., Farnum, M., & Kaplan, B. (1999). NAEP 1996 trends in writing: fluency and writing

conventions (Publication No. NCES 1999-456). Washington, DC: National Center for Education Statistics.

Beaton, A. E. (1987). Implementing the new design: The NAEP 1983-84 technical report. (No.

15-TR-20) Princeton, NJ: Educational Testing Service, National Assessment of Educational Progress.

Beaton, A.E. (1988). Expanding the new design: The NAEP 1985-86 technical report (No. 17-TR20). Princeton, NJ: Educational Testing Service, National Assessment of Educational Progress.

Beaton, A. E., & Johnson, E. G. (1990). The average response method of scaling. Journal of

Educational Statistics, 15, 9-38.

Beaton, A. E., & Johnson, E. G. (1992). Overview of the scaling methodology used in the National

Assessment. Journal of Educational Measurement, 26(2), 163-175.

Beaton, A. E., & Zwick, R. (1990). The effect of changes in the National Assessment:

Disentangling the NAEP 1985-86 reading anomaly. (No. 17-TR-21) Princeton, NJ: Educational Testing

Service, National Assessment of Educational Progress.
828

Benjamini, Y., & Hochberg, Y. (1994). Controlling the false discovery rate: A practical and

powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57(1), 289-300.

Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in

two or more latent categories. Psychometrika, 37, 29-51.

Bourque, M. L., Champagne, A. B., & Crissman, S. (1997). NAEP 1996 science performance

standards: Achievement results for the nation and the states. Washington, DC: National Assessment

Governing Board.

Bourque, M. L., & Garrison, H. H. (1991). The levels of mathematics achievement: Vol. I,

national and state summaries. Washington, DC: National Assessment Governing Board.

Calderone, J., King, L. M., & Horkay, N. (Eds.). (1997). The NAEP guide: A description of the

content and methods of the 1997 and 1998 assessments (Publication No. NCES 97-990). Washington, DC:

National Center for Education Statistics.

Campbell, J. R., Voelkl, K. E., & Donahue, P. L. (1997). NAEP 1996 trends in academic

progress: Achievement of U.S. students in science, 1969 to 1996; mathematics, 1973 to 1996; reading,

1971 to 1996; and writing, 1984 to 1996 (Publication No. NCES 97-985). Washington, DC: National

Center for Education Statistics.

Carlson, J. E. (1993). Dimensionality of NAEP instruments that incorporate polytomously-scored

items. Paper presented at the annual meeting of the American Educational Research Association, Atlanta,

GA.

Carlson, J. E., & Jirele, T. (1992, April). Dimensionality of 1990 NAEP mathematics data. Paper

presented at the meeting of the American Educational Research Association, San Francisco, CA.

Cochran, W. G. (1977). Sampling techniques. New York, NY: John Wiley & Sons.

Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled

disagreement or partial credit. Psychological Bulletin, 70(4), 213-220.

Curry, L. (1987, April). Group decision process in setting cut-off scores. Paper presented at the

annual meeting of the American Educational Research Association, Washington, DC.

Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data

via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39, 1-38.

Educational Testing Service. (1992). Innovations and ingenuity: A foundation for the future.

Application for cooperative agreement for NAEP (CFDA Number: 84.999E). Princeton, NJ: Author.

Engelen, R. J. H. (1987). Semiparametric estimation in the Rasch model. Research Report 87-1.

Twente, the Netherlands: Department of Education, University of Twente.

Fitzpatrick, A. R. (1989). Social influences in standard-setting: The effects of social interaction on

group judgments. Review of Educational Research, 59, 315-328.
829

Friedman, C. B., & Ho, K. T. (1990, April). Interjudge consensus and intrajudge consistency: Is it

possible to have both on standard setting? Paper presented at the annual meeting of the National Council

for Measurement in Education, Boston, MA.

Hambleton, R. K., & Bourque, M. L. (1991). The levels of mathematics achievement. Vol. II,

technical report. Washington, DC: National Assessment Governing Board.

Hawkins, E., Stancavage, F., & Dossey, J. A. (1998). School policies affecting instruction in

mathematics: Findings from the National Assessment of Educational Progress. Washington, DC: National

Center for Education Statistics.

Hochberg, Y. (1988). A sharper Bonferroni procedure for multiple tests of significance.

Biometrika, 75, 800-802.

Hoijtink, H. (1991). Estimating the parameters of linear models with a latent dependent variable

by nonparametric maximum likelihood. Research Bulletin HB-91-1040-EX. Groningen, The Netherlands:

Psychological Institute, University of Groningen.

Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel

procedure. In H. Wainer & H. I. Braun (Eds.), Test validity. Hillsdale, NJ: Erlbaum.

Huynh, H. (1994). Some technical aspects of standard setting. Paper presented at the Joint

Conference on Standard Setting for Large-Scale Assessments, Washington, D.C.

Huynh, H. (1998, October). On score locations of binary and partial credit items and their

applications to item mapping and criterion-referenced interpretation. Journal of Educational and

Behavioral Statistics, 23, 38-58.

Johnson, E. G. (1990). Data analysis for the writing assessment. In E. G Johnson & R. Zwick

(Eds.), Focusing the new design: the NAEP 1988 technical report (pp. 267-296). Washington, DC:

National Center for Education Statistics, Office of Educational Research and Improvement, U.S.

Department of Education.

Johnson, E. G., & Allen, N. L. (1992). The NAEP 1990 technical report (No. 21-TR-20).

Princeton, NJ: Educational Testing Service, National Assessment of Educational Progress.

Johnson, E. G., & Carlson, J. E. (1994). The NAEP 1992 technical report. Washington, DC:

National Center for Education Statistics.

Johnson, E. G., & King, B. F. (1987). Generalized variance functions for a complex sample

survey. Journal of Official Statistics, 3, 235-250.

Johnson, E. G., & Rust, K. F. (1992). Population inferences and variance estimation for NAEP

data. Journal of Educational Statistics, 17, 175-190.

Johnson, E. G., & Rust, K. F. (1993). Effective degrees of freedom for variance estimates from a

complex sample survey. Paper presented at the American Statistical Association 1993 Annual Meeting,

San Francisco, CA.

Johnson, E. G., & Zwick, R. (1990). Focusing the new design: The NAEP 1988 technical report.

(No. 19-TR-20) Princeton, NJ: Educational Testing Service, National Assessment of Educational Progress.
830

Kaplan, B. A., Beaton, A. E., Johnson, E. G., & Johnson, J. R. (1988). National Assessment of

Educational Progress: 1986 Bridge Studies. Princeton, NJ: Educational Testing Service, National

Assessment of Educational Progress.

Kaplan, D. (1995). The impact of BIB spiraling-induced missing data patterns on goodness-of-fit

tests in factor analysis. Journal of Educational and Behavioral Statistics, 20, 69-82.

Keiser, K. K., Nelson, J. E., Norris, N. A., & Szyszkiewicz, S. (1998). NAEP 1996 science crossstate data compendium for the grade 8 assessment. Washington, DC: National Center for Education

Statistics.

Kish, L. (1965). Survey sampling. New York, NY: John Wiley & Sons.

Kish, L., & Frankel, M. R. (1974). Inference from complex samples. Journal of the Royal

Statistical Society, Series B, 36, 1-22.

Koretz, D., Lewis, E., Skewes-Cox, T., & Burstein, L. (1993). Omitted and not-reached items in

mathematics in the 1990 National Assessment of Educational Progress. CRESST Technical Report

Number 357. Los Angeles, CA: CRESST.

Krewski, D., & Rao, J. N. K. (1981). Inference from stratified samples: Properties of linearization,

jackknife and balanced repeated replication. Annals of Statistics, 9, 1010-1019.

Laird, N. M. (1978). Nonparametric maximum likelihood estimation of a mixing distribution.

Journal of the American Statistical Association, 73, 805-811.

Lindsey, B., Clogg, C. C., & Grego, J. (1991). Semiparametric estimation in the Rasch model and

related exponential response models, including a simple latent class model for item analysis. Journal of the

American Statistical Association, 86, 96-107.

Little, R. J. A., & Rubin, D. B. (1983). On jointly estimating parameters and missing data.

American Statistician, 37, 218-220.

Little, R. J. A., & Rubin, D. B. (1987). Statistical analysis with missing data. New York, NY:

John Wiley & Sons.

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale,

NJ: Lawrence Erlbaum Associates.

Luecht, R. M. (1993, April). Using IRT to improve the standard setting process for dichotomous

and polytomous items. Paper presented at the annual meeting of the National Council on Measurement in

Education, Atlanta, GA.

Mantel, N. (1963). Chi-square tests with one degree of freedom: Extensions of the MantelHaenszel procedure. Journal of the American Statistical Association, 58, 690-700.

Mazzeo, J., Johnson, E. G., Bowker, D., & Fong, Y. F. (1992). The use of collateral information

in proficiency estimation for the Trial State Assessment. Paper presented at the annual meeting of the

American Educational Research Association, San Francisco, CA.

Messick, S. J., Beaton, A. E., & Lord, F. M. (1983). NAEP reconsidered: A new design for a new

era. (NAEP Report 83-1) Princeton, NJ: Educational Testing Service.
831

Mislevy, R. J. (1985). Estimation of latent group effects. Journal of the American Statistical

Association, 80, 993-997.

Mislevy, R. J. (1990). Scaling procedures. In E. G. Johnson and R. Zwick, Focusing the new

design: The NAEP 1988 technical report (No. 19-TR-20). Princeton, NJ: Educational Testing Service,

National Assessment of Educational Progress.

Mislevy, R. J. (1991). Randomization-based inference about latent variables from complex

samples. Psychometrika, 56, 177-196.

Mislevy, R. J., Beaton, A. E., Kaplan, B., & Sheehan, K. M. (1992). Estimating population

characteristics from sparse matrix samples of item responses. Journal of Educational Measurement, 29(2),

133-161.

Mislevy, R. J., & Bock, R. D. (1982). BILOG: Item analysis and test scoring with binary logistic

models [Computer program]. Mooresville, IN: Scientific Software.

Mislevy, R. J., Johnson, E. G., & Muraki, E. (1992). Scaling procedures in NAEP. Journal of

Educational Statistics, 17(2), 131-154.

Mislevy, R. J., & Sheehan, K. M. (1987). Marginal estimation procedures. In A. E. Beaton,

Implementing the new design: The NAEP 1983-84 technical report (No. 15-TR-20). Princeton, NJ:

Educational Testing Service, National Assessment of Educational Progress.

Mislevy, R. J., & Wu, P. K. (1988). Inferring examinee ability when some item responses are

missing (ETS Research Report RR-88-48-ONR). Princeton, NJ: Educational Testing Service.

Mitchell, J. H., Hawkins, E. F., Jakwerth, P., Stancavage, F. B., & Dossey, J. A. (1999). Student

work and teacher practices in mathematics. Washington, DC: National Center for Education Statistics.

Mitchell, J. H., Hawkins, E. F., Stancavage, F., & Dossey, J. A. (1999). Estimation skills,

mathematics-in-context, and advanced skills in mathematics. Washington, DC: National Center for

Education Statistics.

Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied

Psychological Measurement, 16(2), 159-176.

Muraki, E. (1993). Information functions of the generalized partial credit model. Applied

Psychological Measurement, 17(4), 351-362.

Muraki, E., & Bock, R. D. (1991). PARSCALE: Parameter scaling of rating data. Chicago, IL:

Scientific Software, Inc.

Muraki, E., & Bock, R. D. (1997). PARSCALE: IRT item analysis and test scoring for ratingscale data. Chicago, IL: Scientific Software International.

Muthén, B. (1991). Issues in using NAEP mathematics items to study achievement dimensionality,

within-grade differences, and across-grade growth. Report presented to the Design and Analysis

Committee for NAEP.
832

National Academy of Education. (1992). Assessing student achievement in the states. The first

report of the National Academy of Education Panel on the evaluation of the NAEP Trial State Assessment:

1990 Trial State Assessment. Stanford, CA: Author.

National Academy of Education. (1993a). Setting performance standards for student achievement.

Stanford, CA: Author. 

National Academy of Education. (1993b). The trial state assessment: prospects and realities.

Stanford, CA: Author.

National Assessment Governing Board. (1994). Mathematics framework for the 1996 National

Assessment of Educational Progress. Washington, DC: National Assessment Governing Board.

National Assessment Governing Board. (1994). Science framework for the 1996 National

Assessment of Educational Progress. Washington, DC: National Assessment Governing Board.

National Computer Systems. (1996). Report of processing and professional activities. Iowa City,

IA: Author.

O’Reilly, P. E., Zelenak C. A., Rogers, A. M., & Kline, D. L. (1999). NAEP 1996 state

assessment program in mathematics secondary-use data files user guide. Princeton, NJ: Educational

Testing Service, National Assessment of Educational Progress.

O’Reilly, P. E., Zelenak C. A., Rogers, A. M., & Kline, D. L. (1999). NAEP 1996 state

assessment program in science secondary-use data files user guide. Princeton, NJ: Educational Testing

Service, National Assessment of Educational Progress.

O’Sullivan, C. Y., Jerry, L., Ballator, N., and Herr, F. (1997). NAEP 1996 Science State Reports.

Washington, DC: National Center for Education Statistics.

O’Sullivan, C. Y., Reese, C. M., & Mazzeo, J. (1997). NAEP 1996 science report card for the

nation and the states: Findings from the National Assessment of Educational Progress (Publication No.

NCES 97-499). Washington, DC: National Center for Education Statistics.

O’Sullivan, C. Y. & Weiss, A. R. (1999). Student work and teacher practices in science: A report

on what student know and can do. Washington, DC: National Center for Education Statistics.

O’Sullivan, C. Y., Weiss, A. R., & Askew, J. M. (1998). Students learning science: A report on

policies and practices in U.S. schools. Washington, DC: National Center for Education Statistics.

Petersen, N. (1988). DIF procedures for use in statistical analysis. Internal memorandum.

Reese, C. M., Jerry, L., & Ballator, N. (1997). NAEP 1996 mathematics state reports.

Washington, DC: National Center for Education Statistics.

Reese, C. M., Miller, K. E., Mazzeo, J., & Dossey, J. A. (1997). NAEP 1996 mathematics report

card for the nation and the states: Findings from the National Assessment of Educational Progress

(Publication No. NCES 97-488). Washington, DC: National Center for Education Statistics.

Rock, D. A. (1991, November). Subscale dimensionality. Paper presented at the meeting of the

Design and Analysis Committee of the National Assessment of Educational Progress, Washington, DC.
833

Rogers, A. M., Kline, D. L., & Schoeps, T. L. (1999) NAEP 1996 national assessment secondaryuse data files user guide. Washington, DC: National Center for Education Statistics.

Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York, NY: John Wiley

& Sons.

Rubin, D. B. (1991). EM and beyond. Psychometrika, 56, 241-254.

Rust, K. (1985). Variance estimation for complex estimators in sample surveys. Journal of Official

Statistics, 1(4), 381-397.

Rust, K. F., Bethel, J., Burke, J., & Hansen, M. H. (1990). The 1988 National Assessment of

Educational Progress: Sampling and weighting procedures, final report. Rockville, MD: Westat, Inc.

Rust, K. F., Burke, J., & Fahimi, M. (1992). 1990 National Assessment of Educational Progress

sampling and weighting procedures, part 2: National assessment, final report. Rockville, MD: Westat, Inc.

Satterthwaite, F. E. (1941). Synthesis of variance. Psychometrika, 6, 309-316.

Shaffer, J.P.(1994). Multiple hypothesis testing: A review (Tech. Rep. No. 23). Research Triangle

Park, NC: National Institute of Statistical Sciences.

Sheehan, K. M. (1985). MGROUP: Estimation of group effects in multivariate models [Computer

program]. Princeton, NJ: Educational Testing Service.

Shaughnessy, C.A., Nelson, J.E., & Norris, N.A. (1997). NAEP 1996 mathematics cross-state

data compendium for the grade 4 and grade 8 assessment. Washington, DC: National Center for

Education Statistics.

Smith, R. L., & Smith, J. K. (1988). Differential use of item information by judges using Angoff

and Nedelsky procedures. Journal of Educational Measurement, 25, 259-274.

Somes, G. W. (1986). The generalized Mantel-Haenszel statistic. The American Statistician, 40,

106-108.

Stone, C. A., Mislevy, R. J., & Mazzeo, J. (1994, April). Misclassification error and goodness-offit in IRT models. Paper presented at the annual meeting of the American Educational Research

Association, New Orleans, LA.

Tanner, M., & Wong, W. (1987). The calculation of posterior distributions by data augmentation

(with discussion). Journal of the American Statistical Association, 82, 528-550.

Thomas, N. (1993). Asymptotic corrections for multivariate posterior moments with factored

likelihood functions. Journal of Computational and Graphical Statistics, 2, 309-322.

Wainer, H., & Kiely, G. (1987). Item clusters and computerized adaptive testing: A case for

testlets. Journal of Educational Measurement, 24, 185-202.

Wallace, L., & Rust, K. (1999) The 1996 NAEP Sampling and Weighting Report. Rockville, MD:

Westat, Inc.
834

Westat, Inc. (1996). Report on data collection activities for the 1996 National Assessment of

Educational Progress. Rockville, MD: Author.

Winer, B.J. (1962/1971). Statistical Principles in Experimental Design. (2nd ed.). New York:

McGraw-Hill, Inc.

Wingersky, M., Kaplan, B. A., & Beaton, A. E. (1987). Joint estimation procedures. In A. E.

Beaton, Implementing the new design: The NAEP 1983-84 technical report (pp. 285-292). (No. 15-TR-20)

Princeton, NJ: Educational Testing Service, National Assessment of Educational Progress.

Wolter, K. M. (1985). Introduction to variance estimation. New York, NY: John Wiley & Sons.

Yamamoto, K. (1988). Science data analysis. In A. E. Beaton, Expanding the new design: The

NAEP 1985-86 technical report (pp. 243-255). (No. 17-TR-20) Princeton, NJ: Educational Testing

Service, National Assessment of Educational Progress.

Zieky, M. (1993). Practical questions in the use of DIF statistics. In P. W. Holland & H. Wainer

(Eds.), Differential item functioning. Hillsdale, NJ: Lawrence Erlbaum Associates.

Zwick, R. (1986). Assessment of the dimensionality of NAEP year 15 reading data. (ETS

Research Report No. 86-4). Princeton, NJ: Educational Testing Service.

Zwick, R. (1987). Assessing the dimensionality of NAEP reading data. Journal of Educational

Measurement, 24, 293-308.

Zwick, R. (1988). Professionally scored items. Technical memorandum.

Zwick, R., Donoghue, J. R., & Grima, A. (1993). Assessment of differential item functioning for

performance tasks. Journal of Educational Measurement, 30, 233-251.

Zwick, R., & Grima, A. (1991). Policy for differential item functioning (DIF) analysis in NAEP.

Technical memorandum.

Zwinderman, A. H. (1991). Logistic regression Rasch models. Psychometrika, 56, 589-600.

United States

Department of Education

Washington, DC 20208-5653

Official Business

Penalty for Private Use, $300

Postage and Fees Paid

U.S. Department of Education

Permit No. G-17

Standard Mail (B)